Skip to content

Hub Key Custody

Two private keys on a Hub carry most of its authority: the internal CA's key and the statement-signing key. This page describes what each key can do, where the Hub keeps it, and what protects it at rest.

The Internal CA Key

The internal CA signs every enrolled site's client certificate, the tunnel listener's server certificate and the statement-signing certificate. Whoever holds its private key can impersonate any registered site to the tunnel listener. Because sites accept any statement-signing certificate this CA issues, the holder can also issue a signing certificate of their own and sign statements that every site with Hub sign-in turned on accepts.

The key is stored in the Hub database, in the Global Settings zone's SSL key table, like any other private key in QIE.

Treat anyone who can read the Hub database, or any backup of it, as holding the internal CA key. The key also travels in every configuration export that includes the Global Settings zone, the nightly backup among them (see Hub Disaster Recovery), so protect those files as you protect the database.

The Statement-Signing Key

The Hub signs every statement it sends a site with this key: each console sign-in, job step, status check, rollback and AI call. A site accepts a statement only when all of the following hold:

  • The certificate in the statement was issued by the CA the site pinned at enrollment, is within its validity period, and is marked for signing Hub statements and not for the tunnel.
  • The statement names that site and the Hub's id, and is for the kind of action the site is asked to take.
  • The statement has not expired and has not been used before.

The key and its certificate are kept in a table of their own in the Hub database, shared by every Hub node, with the key encrypted under the database key. They are not stored with the SSL certificates and are never exported.

The certificate is short-lived. The internal CA issues it for 7 days by default (qie.hubStatementCertValidityDays), and the Hub issues a new key and certificate when 2 days remain (qie.hubStatementCertRenewalDays). It also issues a new one when the CA that issued the current certificate is no longer the Hub's current CA, and when a CA rotation activates a new CA. The Hub checks 30 seconds after it starts and then every hour, and it issues the first certificate itself when it has none. Each new certificate is written to the Hub audit log as STATEMENT_SIGNING_KEY_ISSUED.

A copied signing key stops being useful when its certificate expires. That limit does not apply to anyone who holds the internal CA key, who can issue another certificate at any time.

Each statement expires 60 seconds after it is signed by default (qie.hubStatementExpirySeconds, set on the Hub), and a site allows 120 seconds of difference between its clock and the Hub's (qie.hubStatementClockToleranceSeconds, set on each site). These limits stop a captured statement from being used again later. They do not stop someone holding the key from signing new ones.

The Database Key

The statement-signing key, a job's uploaded configuration file and its key password are encrypted with the database key. A new database gets its own randomly generated database key the first time QIE starts against it. QIE keeps the database key in the database itself, encrypted with an environment key.

The environment key comes from the qie.longKey, qie.longKeystoreLocation or qie.rsaKeystoreLocation Java option when one of them is set. When none is set, QIE uses a key built into the product.

  • With one of the three options set, the environment key is held outside the database. Keep a copy of it with the same care as the database's own credentials: QIE cannot read a database whose database key it cannot decrypt.
  • With none set, treat anyone who can read the Hub database, or any backup of it, as holding the statement-signing key too.

Rotating the Internal CA

Know the CA rotation procedure before you need one. On a confirmed compromise of the CA key, rotate the internal CA with the Stage New CA, Re-issue All Client Certs, Activate New CA and Retire Old CA steps on the Hub Configuration page. The rotation runs with a dual-CA overlap: the Hub trusts both CAs while sites move to client certificates from the new CA, still connected. It therefore does not need a re-enrollment of the whole fleet, only of a site that stays offline for the entire window. Retire Old CA is held back until every connected site has moved. See Rotating the Internal CA.

Activate New CA also issues a new statement-signing certificate under the new CA, so statements are signed under it from that moment. Running jobs pause while a CA rotation is in progress, and new jobs cannot start, until the rotation completes.