PTP² Mesh: Self-Healing Timing Topology Across the Timebeat Agent Fleet

What Mesh is for — the problem with classical BMCA

IEEE 1588's Best Master Clock Algorithm (BMCA) is the standard mechanism for selecting which clock on a PTP network is currently authoritative. It works, and it has been the foundation of PTP-based redundancy for two decades. But it has three limitations that matter in modern deployments.

First, BMCA is hierarchical. One clock is the Grandmaster; others are Boundary or Slave. When the Grandmaster fails, the BMCA election picks a successor — but that election is a transition event, not a smooth redistribution of load. Downstream clients may see a step or a sync interruption during the election, depending on how quickly the secondary's clock state can be substituted for the primary's.

Second, BMCA priorities are largely static. A network operator sets priority1 and priority2 on each clock, and the election resolves in their favour based on those priorities plus clock quality fields. There is no runtime notion of current load or current capacity — a clock advertising itself as Grandmaster-eligible is either elected or not, regardless of whether it's currently serving 10 clients or 1,000.

Third, BMCA does not route across network segments. Each PTP domain runs its own BMCA. If you want a single authoritative clock across multiple L3 segments or across a WAN, BMCA alone doesn't provide that — you need boundary clocks and careful network design, and the end result is a hierarchical topology that is brittle in specific ways.

What Mesh adds — peer-to-peer, seat-based, self-healing

PTP² Mesh is a peer-to-peer overlay that runs between Timebeat Agents — it is not visible to downstream PTP clients. Each Agent in the Mesh connects to other Agents using standard peer-discovery mechanisms (mDNS on the LAN, DHT across segmented networks or the internet), and the Agents coordinate among themselves who is currently serving which downstream clients.

The coordination uses a seat-based capacity model. Each Agent advertises two numbers: seats_to_offer (how many downstream clients it can serve time to) and seats_to_fill (how many upstream sources it wants to consume time from). The Mesh topology is then computed as a bipartite matching — clients with seats_to_fill get matched to Agents with available seats_to_offer, and the matching re-computes when anything changes.

When an Agent degrades — its GNSS fails, its advertised capability drops, or it stops responding to Mesh heartbeats — the Mesh detects this in under a second and re-matches the downstream clients that were being served by that Agent to other Agents with available seats. Downstream clients see no step change because they were not talking directly to the degraded Agent — they were talking to the Mesh virtual endpoint, which is stable regardless of which backing Agent is currently responding.

Peer discovery — mDNS and DHT

Mesh supports two peer discovery mechanisms depending on the network topology.

• ●mDNS (same LAN). On a flat L2 network, Agents announce themselves via mDNS and discover other Mesh members automatically without any configuration. Ideal for rack-scale or cabinet-scale deployments where all the Agents share a broadcast domain. Zero configuration, zero DNS.
• ●DHT (distributed hash table). For segmented networks, multiple data centres, or across the internet, Agents use a DHT with a configurable seed list of bootstrap peers. One or more Agents' DHT addresses are shared out-of-band (TLS-authenticated, typically), and new Agents bootstrap into the DHT by connecting to the seeds and learning the rest of the mesh from there. Suitable for multi-site topologies where mDNS can't traverse.
• ●Static peer configuration. A third option for highly controlled environments is to configure peer lists statically in the Agent's yaml. This is the most deterministic option, suitable for regulated deployments where the peer set is fixed by policy and discovery is not wanted.

Seat model and reservations

The seat model is how Mesh allocates capacity. Each Agent advertises seats_to_offer and seats_to_fill; the Mesh computes a matching; the matching determines which Agent serves which downstream clients.

Reservations add a qualification to the matching. A tier-1 bank's trading servers are not the same as a developer's dev/staging cluster, and the Mesh should prefer to match the important clients to the best Agents even when capacity is tight. Reservations express this as a preference score — clients with higher preference_score get priority access to Agent capacity when seats are constrained. Optional domain filters let reservations apply only to specific PTP domains.

In a multi-tenant deployment (a managed service provider hosting timing for multiple client firms), reservations are how the MSP guarantees SLAs to each tenant. Tenant A's preference_score is set high enough that A's clients always get served first when capacity is constrained; Tenant B's clients get what's left. At normal capacity all clients are served equally; under stress, the preference ordering kicks in.

Capabilities — declaring what each Agent is good for

Each Agent in the Mesh advertises its capabilities. The canonical capability is oscillator quality — an Agent with a Rubidium Black+ oscillator advertises itself as offering high-quality oscillator (hqosc-1500 for 1.5 microsecond drift per 24 hours), which the Mesh uses as input to the source-selection cost model.

The cost model weights peer-to-peer paths based on what the Mesh knows about each Agent. Base hop cost, SWTS cost (software timestamping penalty), HWTS cost (hardware timestamping credit), and an 'is-better factor' that determines how much better a new source must be before the Mesh switches to it. The sum of these produces a per-path cost, and the Mesh selects the lowest-cost path from each client to a viable time source.

Operationally, this means the Mesh naturally routes clients to the best available Agent for their needs — high-priority clients get the Rubidium-backed Agents, cost-sensitive clients get OCXO Agents that are closer geographically, and the routing re-computes as conditions change.

What Mesh failover looks like in practice

A useful mental model is the three-unit Shelf deployment. Three Timebeat Agents, each on its own Open Time Appliance unit, are configured as a Mesh. Each Agent has its own GNSS input, its own PPS output, its own PTP Transparent Clock. Downstream clients connect to the Mesh — they see one virtual PTP grandmaster IP, served by whichever Agent the Mesh is currently routing through.

Under normal conditions, the Mesh load-balances downstream clients across the three Agents. Each Agent is serving roughly a third of the downstream traffic. The Mesh heartbeats are healthy, the cost model has stable values, the routing is steady.

If Agent A's GNSS receiver loses lock — antenna fault, localised jamming, roof cable issue — Agent A's Clock Ensemble detects the degradation and down-weights its GNSS input. Agent A's advertised capability drops; the Mesh sees this in under a second and starts routing new downstream connections to Agents B and C. Existing connections on Agent A either continue (if Agent A is still serving acceptable time from its remaining inputs) or migrate (if Agent A's Ensemble output degrades past a threshold). Within a few seconds, the load is rebalanced 0 / 50 / 50 between B and C, Agent A is still alive and listening on the Mesh, and downstream clients have seen no step change.

When Agent A's GNSS recovers, the inverse process runs. Agent A's capability climbs, the Mesh routes new connections back to it, the load drifts back towards 33 / 33 / 33. The full event — from degradation through recovery — is recorded in Sync Insight's telemetry as a series of weight and routing events, all timestamped and attested.

When to use Mesh vs. alternatives

Mesh is not always the right answer. A single-site deployment with one Agent and one time source has nothing to Mesh with. A multi-site deployment where each site has its own self-contained grandmaster topology may not need Mesh at the inter-site level. The value of Mesh comes from coordination between Agents that want to share load and survive failure together.

• ●Mesh is the right answer for: multi-unit Shelf deployments (3-Agent topology in a single rack), multi-site federations where Agents should coordinate across WAN, managed service provider platforms serving multiple tenants, AI / HPC clusters with many servers requiring a unified time view, broadcast production facilities with multiple grandmasters.
• ●Mesh is overkill for: single-site single-Agent deployments, small enterprise campuses with a single NTP replacement, dev/staging environments, deployments where the time source is already highly reliable and business continuity requirements are modest.
• ●Mesh is the wrong answer for: environments where the peer list is classified or policy-restricted such that peer discovery is prohibited (use static peer configuration or skip Mesh entirely), networks where peer-to-peer traffic is blocked by firewall rules that cannot be changed (Mesh requires Agent-to-Agent connectivity).

Debug and operations

Mesh exposes its internal state through Sync Insight telemetry. Operators can see the current peer list, each peer's advertised capabilities, the current routing table (which clients are served by which Agent), the recent heartbeat history, and the cost-model inputs. Mesh debug mode adds additional granularity — per-decision log output that shows exactly why the Mesh selected one path over another, with the cost values that drove the decision.

Latency analysis is an opt-in mode that records per-peer latency distributions over time. Useful for tuning the hop-cost model in WAN deployments where network latency varies by time of day or by peer. The analysis output is a histogram per peer that the operator can inspect in Grafana.

Static peer-ID keypath is the recommended configuration for regulated deployments. A fixed keypath means the Agent presents the same peer identity across restarts, which makes audit trails simpler — all events from a given Agent roll up under a stable identity rather than appearing as events from a succession of ephemeral identities.