Sizing the Tunnel Listener Thread Pool¶
The Pool Min Threads and Pool Max Threads fields on the Hub Configuration page size the thread pool that runs the tunnel listener. This page explains what the pool does and how to size it for a large fleet.
What the Pool Actually Does¶
The relationship between "connected sites" and "threads needed" is not 1:1, and getting this right matters when sizing for scale.
Jetty uses NIO selectors for socket I/O. A small fixed set of selector threads (separate from the pool described here) multiplexes every open tunnel, so 100 connected sites and 10 000 connected sites both use the same handful of selector threads. Idle WebSockets do not each pin a pool thread.
The pool is used for the work that runs off the selector thread once a frame arrives:
- Decoding a heartbeat CBOR frame and forwarding the snapshot to the batched status writer.
- Handling an
ADMINframe (revocation notice, cert renewal request, trust-additional, trust-remove). - Servicing a
/registerHTTP request from a newly enrolling QIE. - Forwarding
REQUEST_HEAD/REQUEST_BODYframes into the tunnel multiplexer for a specific site, andRESPONSE_HEAD/RESPONSE_BODYframes back the other way during an active proxy session. - Sending a WebSocket frame on the wire when the write callback fires.
Each of those is short-lived, typically microseconds to a few milliseconds. The dominant load on the pool is concurrent proxy traffic. Heartbeats and ADMIN frames are too cheap to matter on their own.
Sizing for Site Count¶
The relevant question is how much concurrent proxy traffic do you expect, not how many sites are registered.
| Deployment profile | Reasonable Pool Min | Reasonable Pool Max |
|---|---|---|
| Lab / small deployment up to ~50 sites | 4 (default) | 50 (default) |
| 50–250 sites, light operator activity | 4 (default) | 50 (default) |
| 250–1000 sites, occasional operator activity | 8–16 | 100–150 |
| 1000 sites, multiple operators frequently proxying through many sites simultaneously | 16–32 | 150–250 |
A reasonable rule of thumb for Pool Max is (typical concurrent operators) × (average open proxy tabs per operator) × 4. The factor of 4 accommodates GWT-RPC batches that issue several requests in parallel during a single page load.
Should I Raise Pool Min for 1000 Sites?¶
Raising Pool Min above the default mostly helps smooth out workday ramp-up. At the start of business hours when operators sign in and open dashboards, every tab issues a flurry of parallel requests. With only 4 threads warm, the pool has to grow on every burst, paying a few milliseconds of thread-creation per new thread.
Raising Pool Min to roughly 25–50% of Pool Max keeps enough threads warm that a workday burst (under that floor) never incurs creation overhead at all. Memory cost is modest. Each idle JVM thread is roughly 512 KB to 1 MB of committed stack, so 32 warm threads is around 16–32 MB of overhead.
Raising Pool Min is not a way to scale up steady-state heartbeat
throughput. That work is bound by the database write path (the
batched hub_site_status writer), not by the listener pool.
Heartbeats are I/O-bound at the DB, not CPU-bound at the listener.
Why a Ceiling at All¶
The ceiling protects the JVM from runaway thread creation during a load spike or under attack. Without a cap, a flood of requests would allocate threads indefinitely until heap or native memory was exhausted. With a cap, excess work queues and completes a few milliseconds later, degraded but not catastrophic.
If you see pool saturation in practice (proxy responses slow down
without per-site tunnel issues), raise Pool Max in increments of 50
and observe. A JVM thread dump shows whether hub-tunnel
pool threads are all busy.
Changes to either field take effect when the listener restarts: click Save, then Restart Listener.