High Availability

To start an ArcadeDB server with HA enabled, at minimum you need to:

Example starting a 3-node cluster on a single host (different ports):

The Raft gRPC port is 2434 by default and is configurable via arcadedb.ha.raftPort. The HTTP port in each entry is required so that replicas can forward non-idempotent requests to the leader over HTTP.

By default, each node identifies its slot in arcadedb.ha.serverList from the numeric suffix of arcadedb.server.name (for example, ArcadeDB_0 maps to the first entry, ArcadeDB_1 to the second, and so on). Display names shown in logs and Studio are then synthesized from the local node’s prefix. This positional convention works well for Kubernetes StatefulSets but is awkward when nodes have human-readable names such as frankfurt, london, nyc.

You can add an optional name@ prefix to any entry in arcadedb.ha.serverList to give each peer a stable, human-readable name:

When peer names are configured, each node identifies its slot by matching arcadedb.server.name against the configured peer names first; if no match is found, it falls back to the legacy prefix_N / prefix-N suffix resolution. This means:

The Raft peer ID is still derived from the address (e.g. 10.0.0.1_2434), so changing or omitting peer names does not affect Raft identity stability.

ArcadeDB uses a leader/replica model with Raft consensus. At any time one server holds leadership and accepts writes; the others are replicas that serve reads and stand by for failover.

Each server persists its own Raft log segments under <rootPath>/raft-storage/. The log is used for recovery after restart and to replicate state to peers that fall behind.

Any read (query) can execute on any server in the cluster. All writes must go through the leader: a replica that receives a write transparently forwards it to the current leader via HTTP.

Internally, every committed write follows a replicate-first, commit-after three-phase commit:

If Phase 2 times out or fails, Phase 3 never runs: no local writes, no divergence. Multiple concurrent transactions are batched into fewer Raft round-trips via group commit (HA_RAFT_GROUP_COMMIT_BATCH_SIZE).

A quorum is the number of peers that must acknowledge a write before it commits. Configure it via arcadedb.ha.quorum:

If the configured quorum is not met within arcadedb.ha.quorumTimeout milliseconds, the transaction is rolled back and an error is returned to the client.

When a read runs on a replica, ArcadeDB offers three consistency levels, configurable per-server via arcadedb.ha.readConsistency:

Clients communicate the consistency contract through three HTTP headers:

The Raft group-commit queue is bounded (arcadedb.ha.groupCommitQueueSize, default 10000 pending transactions). When a write arrives but the queue is full, the server waits up to arcadedb.ha.groupCommitOfferTimeout ms (default 100) for a slot, and if still full throws ReplicationQueueFullException — a NeedRetryException that clients automatically retry with backoff. This protects the server from OOM under sustained write overload and lets clients back off gracefully.

Set arcadedb.ha.serverRole=replica to pin a node as a permanent replica. The node’s Raft priority is set to 0 at startup so it is never elected leader, while still receiving all writes and serving reads. Useful for read-scale deployments or for cross-DC witnesses that should not take over as leader.

If the leader becomes unreachable, replicas start a new Raft election. A replica with an up-to-date log is elected as the new leader and the cluster resumes serving writes. Pre-vote prevents partitioned nodes from triggering disruptive elections.

Common causes of leader unavailability include:

When a replica rejoins after being offline, it catches up via the Raft log automatically; if it has fallen behind past the log purge boundary, the leader streams a full database snapshot over HTTP and the replica installs it atomically.

By default, when an HA cluster forms for the first time and one or more peers already hold a non-empty database on local disk, the peers do not download the database over HTTP from a single source. Instead, every peer reports a (fingerprint, lastTxId) tuple per database, the cluster elects the peer with the highest lastTxId as the source via leadership transfer, and any peer whose fingerprint matches the source bootstraps locally in seconds with zero bytes transferred.

This is the recommended path for scaling a single-instance ArcadeDB deployment up to a multi-node HA cluster after a large dataset has already been imported. Operator workflow:

Time-to-cluster goes from "minutes-to-hours of HTTP snapshot transfer per replica" to "seconds, because everyone already has the bytes". For the Kubernetes recipe, see Pre-staging the database on every pod.

The bootstrap protocol handles the following cases automatically:

The committed bootstrap baseline (fingerprint and lastTxId per database) is visible in the response to GET /api/v1/cluster and in the Studio cluster dashboard.

Cluster membership and leadership can be changed at runtime through dedicated REST endpoints. All endpoints require authentication as the root user.

Example: add a new peer at runtime.

Adding a peer with a human-readable name (from v26.5.1):

ArcadeDB Studio exposes a Cluster tab (visible when HA is enabled) that displays the current leader, peer roles, Raft term and commit index, per-follower replication lag, and provides buttons for leadership transfer, peer add/remove, and database verification. See Studio.

For Kubernetes deployments using the official Helm chart — including the StatefulSet + headless Service pattern, auto-join on scale-up, and the preStop auto-leave hook — see High Availability on Kubernetes in the Kubernetes guide.

The following settings control HA behavior. A complete list of all HA parameters is available in Server Settings.