In 2013, Linux kernel version 3.9 added support for the SO_REUSEPORT flag. This flag allows multiple programs (usually instances of the same program) to bind to a single interface and port specification (IPv4 or IPv6) to load balance across multiple processor threads. This flag is not part of the POSIX standard, but was previously implemented by many UNIX systems (including OSX, and the BSDs). Even before supporting this flag, Linux had the capability to do this sort of load balancing, in userspace, and with several substantial advantages in security, flexibility, and performance: using UNIX sockets.

This presentation starts with the basics of binding to ports and client/server communication. This should be detailed enough for the computer literate to understand, but brief. Then I cover the challenges of scaling work across logical processors, including forking, thread pools, and the kernel load balancer via SO_REUSEPORT.

I then explain the file table held in the kernel vs file handles (int) held by programs, and UNIX sockets. The important takeaway is that file handles can be passed from one process to another via UNIX sockets. Passing file handles from one process to another allows the load balancer to be implemented in userspace.

The last segment of the presentation covers some implementation challenges, specifically for an HTTP(S) server. This also highlights some of the advantages. In short, the access control is more fine grained, the EUID of the worker threads need not be the same, as access control is determined by the permissions of the socket. The load balancer can also act as a transparent reverse proxy. Creative load balancing algorithms can be used, vs the round-robin allocator used by the kernel. Neither the balancer nor the workers ever need root, even to serve ports under 1024. And for buzzwords, file handles can be passed between containers. The downside is a small overhead, and a slightly less straightforward implementation.