Skip to content

Hub Disaster Recovery

A Hub's configuration travels in the same configuration export as an engine's, the nightly backup included, so a Hub that is wiped or lost can be rebuilt by importing an export into a new Hub. The Hub configuration rides under the System Configuration node of the export. The internal CA and server certificates, with their private keys, are in the Global Settings zone, so a restore needs both. See Backup Considerations for the backup itself.

What the Export Carries

Listener configuration: the tunnel listener's port, bind address and host name, its thread pool, idle timeout and TLS settings, and the certificate lifetimes and bundle expiry. It records which certificates are in use, including those of a CA rotation in progress and its phase. It also carries the Hub's id and the Approval Threshold (sites) and Statement Retention (days) settings.

Sites: every registered site, with its identifier, subdomain, display name, description, enabled state, client certificate, enrollment time and the bundle it enrolled with.

Tags: every tag with its name, description and UUID, All sites included, and the sites carrying each tag other than All sites.

Role grants: every grant a role holds, with the grant (Console, Jobs or MCP) and the tag it names.

Revoked certificates: the Hub's revocation list.

Bundles: every issued enrollment bundle, with whether it was consumed or canceled and when it expires.

MCP settings: the settings on the Hub's Model Context Protocol (MCP) page, Enable the MCP Server included. Their certificates travel as references to Global Settings certificates.

The Hub's users and roles travel with System Configuration as on any QIE. Grants name roles by their role ID, so restore the roles with them.

What an Import Restores

The Hub configuration is restored only when System Configuration is selected for import.

Listener configuration: the file's values replace the Hub's, and its certificate references are matched to the certificates imported into Global Settings. The Hub's id is taken from the file whenever the file has one. The approval threshold and the statement retention are taken from the file as they are, empty values included.

Sites: added or updated by site identifier. A site the Hub has and the file does not is left in place.

Tags, the sites carrying them, role grants and revoked certificates: added or updated, never removed.

  • A tag is matched by its UUID. A tag in the file whose name another tag here already holds, under a different UUID, is skipped with its sites and grants, and the Hub log records a warning.
  • The built-in All sites tag keeps its own name and description, and gets the grants the file gives it.
  • A site tag or grant that names a site or tag the import did not find is skipped with a warning.
  • A certificate already revoked here stays as it is.

Bundles: a bundle new to this Hub is added as the file describes it, with the tags the enrolling site is given. A bundle this Hub already has keeps its signed details, and stays consumed or canceled if it is so here, whatever the file says.

MCP settings: the file's settings replace the Hub's, Enable the MCP Server included, and the MCP listener is rebuilt on every Hub node. A certificate the import does not find among the Global Settings certificates is left empty. An import that changes the Resource URL in force revokes every MCP sign-in, as saving the MCP page does.

Changes to the Job Settings

When an import changes the Approval Threshold (sites) or the Statement Retention (days), the Hub writes a JOB_SETTINGS_RESTORED event to the Hub audit log. The event names the person who ran the import and gives both values before and after. An import that leaves both as they were writes no event.

What Never Travels

These are runtime records and logs. They are never exported, and a restored Hub starts without them:

  • Statements: the Hub's record of every statement it signed.
  • Jobs: every job with its steps, per-site results and uploaded configuration file.
  • Queued session endings: requests to end a person's Hub console sessions that sites have not yet acknowledged.
  • The statement-signing key and certificate. A restored Hub issues a new one from its internal CA. Sites accept it because the CA travels with Global Settings.
  • MCP records: registered AI clients, sign-ins and tokens, operation records, and the MCP audit log.
  • The Hub audit log: the hub_audit_entry table and hub-audit.log.
  • Live state: site status, open proxy sessions, the login throttle and the queue of events between Hub nodes.

What that means after a restore:

  • Each site records the Hub's id when it enrolls, or from the first statement it accepts, and refuses statements from a Hub with a different id. A Hub rebuilt with a different id has its statements refused by every site that recorded the old one.
  • No earlier job can be rolled back from the restored Hub, because its record did not travel.
  • When a person loses access at the Hub, their Hub console sessions are ended only at sites where a console sign-in statement for them is on file. After a restore, that means only the sites where they signed in through the Hub since the restore. Session endings that were still queued are lost.
  • Every person approves their AI client at the Hub again.
  • The audit history before the restore is only in the copies you shipped off the old Hub's host (see Hub Audit Log).