While doing some packet drop testing for a pcap script I'm writing with a collegue a work, I hit a strange situation on my Mac where ab would do ~16k HTTP connections really fast, then stop, then timeout.

Turns out this is caused by the lack of available ephemeral port for new sockets because all 16k ports are 'in-use'.

On Linux or FreeBSD one could easily observe that via netstat:

Linux:   netstat -ant | grep TIME_WAIT
FreeBSD: netstat -anfinet | grep TIME_WAIT

On OSX however, the TIME_WAIT state doesn't seem to be reported in netstat's output.

I still haven't found why, hidding this info really slows down troubleshooting issues...

As a workaround, the number of TIME_WAIT connection can be seen with sysctl net.inet.tcp.tw_pcbcount.

To change the number of available ephemeral ports, change the net.inet.ip.portrange.{first, hifirst} sysctls:

sysctl -w net.inet.ip.portrange.hifirst=16384
sysctl -w net.inet.ip.portrange.first=16384

This will allow the OS to use 49151 ports.

The above won't solve the problem however, since ab will still block after 48k connections. ab should be using SO_REUSEADDR to allow for local address resuse when there's already a socket in TIME_WAIT state using the same port.

Some post on the net regarding this issue recommend to lower the Max segment lifetime, while this can be used to test on localhost, it is not recommended to run an internet facing host with a non default msl.