Cluster and Failover Issues

OptimiDoc supports clustered deployments for high availability and load distribution. Understanding what the cluster shares automatically (the database, and therefore jobs, users and rules) versus what has to converge separately (the encryption key and the IPP queue definitions) is the key to diagnosing cluster problems. This article explains the cluster model, then works through the failures you are most likely to meet — a node going inactive, a new node failing to join, encrypted data becoming unreadable after a server move, and IPP queues not appearing on a node. Menu paths refer to the React web console.

How an OptimiDoc Cluster Works

• Server nodes share one database. All Server-type nodes connect to the same database, so jobs, users, devices, rules and most configuration are automatically consistent across them — there is nothing to "distribute." If a node cannot see the same data as the others, that is a database-connectivity problem, not a synchronisation problem.

• Exactly one node is the Master. The first active Server node claims the Master flag (shown as a crown badge in Diagnostics > Cluster Nodes); you can reassign it with the Set as Master action (Server-type nodes only). Because the flag lives in the shared database, there is no split-brain — only one node is ever master. The master runs the singleton responsibilities: scheduled/automatic reports, mailbox polling and device monitoring.

• Printing Nodes (OPN) register and heartbeat. A Printing Node registers with the cluster and sends a heartbeat every 5 minutes; a separate 5-minute timer retries registration if needed. Its liveness is shown by the Wi-Fi icon and Last Activity in Cluster Nodes.

• Every node needs the Master Encryption Key (MEK). Encrypted secrets (credentials, destination secrets) can only be read by a node that holds the MEK. New nodes obtain it during bootstrap (cause 2); the MEK is bound to the machine (cause 3).

• IPP queue definitions converge by pull. Named IPP queue attribute sets are not in the database — they converge across nodes on a pull schedule (cause 5).

Symptoms

• A node shows inactive (Wi-Fi-off icon, stale Last Activity) in Diagnostics > Cluster Nodes.

• A newly added node cannot join the cluster or fails to log in.

• After moving or rebuilding a server, credentials and destination secrets fail to decrypt, or the console shows an encryption-recovery prompt.

• Scheduled reports, mailbox polling or device monitoring have stopped (no active master).

• A printing node is missing IPP queues that exist on the server.

• Devices or print clients cannot reach a node by name after a failover.

Common Causes and Solutions

1. Node Shows Inactive — Network or Database

Cause: A node's Last Activity is stale because it cannot reach the cluster or, for a Server node, the shared database.

Diagnostic steps:

1. Open Diagnostics > Cluster Nodes and note which node is inactive and its Type (Server or Printing Node).

2. From that node, confirm network reachability to the other nodes and, for a Server node, to the database server.

3. Check the node's own logs for database connection errors.

Solution:

• Restore connectivity. Ensure firewalls allow the OptimiDoc API traffic (80/443) between nodes bidirectionally.

• For a Server node, verify its database connection string points to the shared database and that SQL Server accepts the connection — a Server node that cannot reach the database cannot participate.

• For a Printing Node, confirm it can reach the Server API; it re-registers automatically on its 5-minute timer once connectivity returns.

2. New Node Cannot Join (Bootstrap and Peer Authentication)

Cause: A newly installed node cannot complete the join/bootstrap. Joining is a guarded, two-step exchange, and both steps have prerequisites.

How joining works:

• The node must already exist as an Active ClusterNode row before it can bootstrap.

• Bootstrap calls the master at api/v3/cluster-bootstrap/delegate-auth (a delegated username/password login) and then api/v3/cluster-bootstrap/exchange-mek (to receive the encryption key). A node that has no MEK yet logs in via this master delegation and is handed the master's key.

• Both endpoints are protected by cluster peer authentication, an IP allow-list of active nodes: a request from an unknown IP is rejected with 403 (loopback is allowed).

Solution:

• Register the node so it exists as an Active entry in Diagnostics > Cluster Nodes first.

• Ensure the new node's IP is that of an active cluster node (so it passes peer authentication) and that it can reach the master's API.

• Check the logs for cluster bootstrap: delegate-auth / exchange-mek messages, which pinpoint which step failed.

3. Encrypted Data Unreadable After a Server Move or Rebuild

Cause: This is the single most common cluster/migration failure. The Master Encryption Key is bound to the machine (DPAPI, LocalMachine scope). If you move OptimiDoc to new hardware or rebuild the OS, the new machine cannot unwrap the MEK, so all encrypted secrets (stored credentials, destination secrets) become unreadable — even though the database came across intact.

Solution:

• Recover with the recovery key. Upload the .odrk recovery key file on the public recovery page at /recover-encryption to restore the MEK on the new server. The recovery key is derived (HKDF) from the licence and a hardware key and is downloaded in advance from the Diagnostics > Encryption settings page ("Download Recovery Key"). Download and store the .odrk file before any migration.

• Or re-join the cluster. If the server is re-joining an existing cluster that still has a healthy master, it can obtain the MEK again through the bootstrap exchange in cause 2.

• See Migration to a New Server for the full migration procedure, including moving the encryption key.

4. Master Responsibilities Have Stopped

Cause: Scheduled/automatic reports, mailbox polling or device monitoring run only on the master. If there is no active master (for example the master node is down), these singleton tasks stop even though pull printing continues on the remaining nodes.

Solution:

1. Open Diagnostics > Cluster Nodes and confirm a Server node holds the Master crown badge and is active.

2. If the master is down or the badge is on an offline node, use Set as Master on a healthy Server node to move the role (the action is available on Server-type nodes only).

3. Confirm the automatic reports, mailbox and monitoring functions resume on the new master.

5. IPP Queues Not Converging on a Node

Cause: Named IPP queue attribute sets are distributed by a pull mechanism, not the shared database, so a node can briefly lag behind after a queue change.

How convergence works:

• Server nodes converge at application start and whenever a change is notified (there is no polling timer between servers).

• Printing nodes pull on their 5-minute heartbeat: if the cluster reports a newer IPP configuration timestamp than the node last applied, the node fetches and applies each queue.

• The node's "last applied" marker only advances when all queues have been applied successfully; a partial failure is retried on the next heartbeat, so convergence is self-healing.

Solution:

• After changing IPP queues, allow up to one heartbeat (about five minutes) for printing nodes to catch up, or restart the node to force an immediate pull.

• If a node never converges, check that it can reach the server holding the freshest configuration over HTTPS/HTTP. See Driverless Print (IPP) and Print Nodes Configuration.

6. Devices or Clients Cannot Reach a Node (LocalDns)

Cause: OptimiDoc substitutes each node's configured Local DNS name into the URLs it hands out to devices and clients — the print client, scan-delivery callbacks and download links in e-mails. If a node's Local DNS is unset or points to a name the device/client cannot resolve, those callbacks fail even though the node itself is healthy.

Solution:

• Set each node's Local DNS (in the node's cluster configuration) to a hostname that the devices and client machines can resolve and route to.

• Placing a load balancer or DNS alias in front of the Server nodes remains a valid way to give devices a single, failover-friendly address — ensure the name it uses matches what nodes advertise.

7. Version Mismatch After a Partial Upgrade

Cause: An upgrade was applied to some nodes but not all, so nodes run different OptimiDoc versions against the same schema.

Solution:

• Bring all nodes to the same OptimiDoc version; the Version column in Diagnostics > Cluster Nodes shows each node's version at a glance.

• Restart each node after upgrading. Keep clocks reasonably synchronised (via NTP) so that heartbeat and IPP timestamps compare correctly across nodes.

Cluster Health Checklist

In Summary

Because Server nodes share one database, jobs, users and rules never need manual distribution — so "node is out of sync" almost always means that node cannot reach the database or the network. The two failures unique to clusters are encryption and IPP queues: the machine-bound MEK makes the .odrk recovery key essential before any server move, and IPP queue sets converge by pull within a heartbeat. Keep exactly one active master for the scheduled/mailbox/monitoring tasks, and set each node's Local DNS to a name devices can actually resolve.

Related articles

• Failover Architecture

• Cluster Nodes

• Cluster Configuration

• Migration to a New Server

• Print Nodes Configuration

• Database and Performance Issues

• Troubleshooting Guide