On this page

Configuration

Everything that is set per node: the HA block of webserver.json, the shared address, the front door, and what to check on components and on a Webserver-Front.

Cluster key

Generate one key and use it on every node. It authenticates the nodes to each other.


webserver ha genkey

webserver.json

On the first Webserver (the one that holds your existing data):


{
  "HTTPS": "true",
  "PortAPI": "8050",
  "PortWEB": "9090",
  "AdvertiseAddress": "mugnsoft.example.com",
  "HA": {
    "NodeID": "node1",
    "Bind": "10.0.0.11:7001",
    "Key": "<the cluster key>",
    "Bootstrap": "true",
    "Peers": ["10.0.0.12:7001", "10.0.0.13:7001"]
  }
}

On the second Webserver: the same block with its own NodeID, Bind and Peers, and without Bootstrap.

On the witness: its own NodeID, Bind and Peers, and "Witness": "true".

Setting Meaning
NodeID Unique name of the node. Never change it.
Bind host:port of the cluster channel on this node
Advertise What the other nodes must dial, when it differs from Bind
Key The cluster key, identical on every node
Bootstrap "true" on the first node only. Ignored once the cluster exists.
Peers Cluster channel of the other nodes (see below)
Witness "true" on the witness
AdvertiseAddress The shared name or virtual IP that components are told to use (outside the HA block)

Peers: the other nodes

Each node lists the other nodes, not itself. With three nodes, each one lists two:

Node Bind Peers
node 1 10.0.0.11:7001 10.0.0.12:7001, 10.0.0.13:7001
node 2 10.0.0.12:7001 10.0.0.11:7001, 10.0.0.13:7001
witness 10.0.0.13:7001 10.0.0.11:7001, 10.0.0.12:7001

Peers is only used when a node starts, to find the cluster. Listing the node’s own address as well does no harm.

Start order

  1. Start the first Webserver. It becomes active with its existing data.
  2. Start the second Webserver, then the witness. Each copies the data from the active node, then joins. What a joining node held before is kept under dbs/backup/pre-join-<date>.

Front door

Point the virtual IP or load balancer at the two Webservers, on the WEB and API ports, with this health check:

Check Active node Standby
GET /ha/health 200 503

The health check is what steers clients away from a node that is down. A request that reaches a standby is not lost either: the standby passes it to the active node.

Shared address on every Webserver

Set the same AdvertiseAddress in webserver.json on both Webservers. It is the address handed to every component that is set up: by the guided tour, by the configuration builder, at registration. It is read from each node’s own file; the cluster does not copy it.

Situation What happens What the cluster panel shows
No AdvertiseAddress on the active Webserver Components set up now are told to reach the active node’s own address. They lose the Webserver when another node takes over. Check needed: “No shared address is set”, with the name of the active server. Each Webserver also writes a warning in its log at start-up.
A standby has a different AdvertiseAddress, or none Nothing yet. Components added after that node takes over would be told to reach another address. Check needed: “Shared address: <node> is not set up with the same address as the active server”, naming the server to fix.

In both cases: set the same AdvertiseAddress on the server named, then restart it (one server at a time).

The witness serves no component and needs no AdvertiseAddress.

Components after the move to a cluster

Moving a standalone Webserver to a cluster is not an IP migration: no component has to be registered again, and no migrate command is needed. The component records, their certificates and their tokens are copied to every node, so the Webserver that takes over reaches them at once.

Component What it holds about the Webserver What to do
Sentinel agent The address it pushes its data to Nothing. When the active Webserver starts with AdvertiseAddress, it tells each enabled agent the shared address. The agent switches only after it has checked that it reaches that address, so it must resolve the name and reach the API port. An agent too old to be told (the log says so) is upgraded, or saved once from the Components page.
Probe (Monitor) Nothing at run time: the Webserver calls it Nothing. Use the shared address the next time you run register.
Integrator webserverUrl: the link placed in notifications Set webserverUrl to https://<shared address>:9090 in integrator.json and restart it. Otherwise the links keep opening the first node.
New components The address given at registration Register them with the shared address (--webserver <shared address>:8050).

Webserver-Front with a cluster

A Webserver-Front is fed exactly as with a standalone Webserver: the active node pushes to it, one way. The standby nodes stay silent. When another node takes over, it starts pushing and first sends everything again, so the Front catches up on what the previous node had not yet sent.

Things to know:

Point Why it matters
Install the replication material on both Webservers ReplKey and the pki/ files (client certificate and key, signing key, CA) are set in webserver.json and stored on disk. The cluster does not copy them. A node that takes over without them serves normally but cannot push to the Front, which then shows stale data.
Keep the clocks in sync (NTP) The Front keeps whichever copy carries the most recent time. If the clock of the node that takes over is behind, the Front ignores its updates until it catches up.
The Front settings are shared Whether replication is enabled, the Front address and the cadence are saved in the UI and copied to every node. Nothing to repeat per node.
Test a takeover once After installing the cluster, stop the active Webserver and check that the Front’s freshness indicator keeps moving.

The cluster panel checks this for you: when a Front is declared and a Webserver’s replication key or certificates are missing, invalid or different from the active node’s, the panel shows Check needed and names the server, before any failover happens. If the active Webserver itself cannot feed the Front, the panel shows Front not updated.

The witness never pushes to the Front and needs none of this.

Snapshot frequency

To keep its journal of changes short, the cluster regularly saves a complete copy of its data and discards the journal before it. On a large installation raise the number of changes between two copies: in the cluster panel, under Advanced settings and activity. The panel tells you how often that is at the current pace. The value applies to every node, without a restart.

See also

Translations

License