Building a Redundant Grandmaster Topology: A/B/C Timing Without the Rack Footprint

The problem with the single-unit grandmaster topology

Most financial institutions — and most enterprise IT organisations that have quietly inherited a grandmaster over the years — are running a timing topology that was designed for a regulatory environment that no longer exists. A single grandmaster clock, typically OCXO-grade, typically purchased at the last hardware refresh, was a defensible choice when NTP at the microsecond level was sufficient and DORA had not yet come into force. It is no longer a defensible choice, and the reasons are structural rather than cosmetic.

The first structural problem is the single point of failure. One grandmaster means one GNSS antenna, one oscillator, one chassis, one power supply and one network uplink. Any of them can fail, and when any of them does, every downstream device that was taking PTP from that grandmaster loses its reference simultaneously. The Best Master Clock Algorithm will elect a fallback if one is configured — but the fallback is usually either a slave OCXO in a nearby switch (minutes of holdover, not hours) or a secondary unit in a different rack with its own shared failure modes. Either way, the transition is not a smooth redistribution of load; it is an event.

The second is holdover. A typical OCXO-grade incumbent specifies ±1.5 microseconds of drift over 24 hours under ideal conditions. In practice, with thermal cycling, oscillator aging and the occasional power event, the effective holdover to MiFID II RTS 25 tolerance (100 microseconds for HFT) is under four hours. A GNSS denial event that runs overnight — a jammer left on in a nearby car park, a roof antenna dislodged in bad weather, an ionospheric scintillation event during a geomagnetic storm — exits the tolerance window before the onsite team gets to their desks.

The third is monitoring. Most incumbent timing platforms emit log files. Log files are not auditable ICT records in the DORA Article 9 sense: they are generated by the firm's own systems, they can be modified after the fact, and they record what the clock claimed to be doing at an interval rather than proving what the clock was actually doing at the moment a specific trade was timestamped. The gap between a log file and continuous, independently-checkable UTC verification is the difference between 'we have logs' and 'we can prove it' — and DORA is moving the regulatory line towards the latter.

What 'A/B/C' actually means in a rack unit

The Open Time Appliance (OTA) is a self-contained grandmaster clock. Each unit has its own u-blox LEA-F9T multi-band GNSS receiver, its own oscillator (selectable from Quartz OCXO, Rubidium Black or Rubidium Black+), its own IEEE 1588 Transparent Clock switch, and its own Timebeat Agent pre-installed on a Linux stack. A single unit is roughly a third of a rack unit in physical footprint.

Three OTA units mount side-by-side in a single 1RU rack kit — the configuration commonly referred to as The Shelf. Each of the three units on The Shelf operates completely independently of the others at the hardware level. A failure on unit A — its antenna, its oscillator, its power feed — does not propagate to units B or C. Each has its own cable feeding its own roof-mounted GNSS antenna. Each has its own power input. Each streams its own telemetry.

This is what 'A/B/C redundancy' looks like in practice. Not two units in different rooms with a manual cutover procedure, not a primary and a hot-standby with BMCA priority tiebreaks, but three active grandmasters in the same rack unit, each one a complete failure domain. The space cost of moving from one active grandmaster to three is zero.

PTP² Mesh — how The Shelf operates as one logical clock

Three independent grandmasters in the same rack would be a management burden if each one served downstream clients independently — clients would have to be configured against one primary and two secondaries, BMCA would take over, and failovers would become manual cutovers by another name. PTP² Mesh solves this. It is a self-healing topology layer that runs across all three Shelf units and makes them appear to downstream slaves as a single, continuously-available grandmaster.

All three units on the Shelf run in active-active configuration. Downstream PTP slaves connect to whichever unit is currently the best master under the Mesh's cost model. If unit A loses its GNSS antenna, Mesh detects the degradation in under one second — the unit downgrades its advertised capability, and downstream traffic re-weights towards units B and C without a manual failover, without a configuration change, and without downstream devices noticing a step change in their clock state. The transition is smooth because unit B and C were already serving time; they simply take on additional load.

Peer discovery on the Mesh uses mDNS for same-LAN nodes and a DHT with configurable seed list for private or segmented networks. Capacity is modelled explicitly — each node advertises seats_to_offer (how many downstream clients it can serve) and seats_to_fill (how many upstream sources it consumes from). Nodes can declare their oscillator capabilities, and Mesh's hop-cost model weighs source selection by a combination of cumulative error-of-source, software versus hardware timestamping, and a base hop cost that biases towards shorter chains. Latency analysis can be enabled as a diagnostic mode.

The practical upshot is that a Shelf looks and behaves like a single virtual grandmaster to the rest of the network. The A/B/C redundancy is internal to the Shelf. Downstream clients do not need to be reconfigured to participate in it.

Clock Ensemble — multi-source fusion on each unit

Each unit on the Shelf runs Clock Ensemble, which fuses multiple inputs into a single disciplined output. The inputs are: the unit's own GNSS receiver (primary reference under normal conditions), PTP feeds from the other two Shelf units (so each unit can cross-check against its peers), and any upstream PTP feeds configured at the site (for example, a White Rabbit reference feeding into a financial venue as a backup signal). Ensemble uses weighted averaging to produce a fused clock output that is more stable than any single source.

This is the same principle used by the BIPM to produce UTC itself: an ensemble of atomic clocks distributed across national metrology labs is statistically more stable than any individual clock, because uncorrelated noise averages out and anomalies from any one clock are outvoted. A Shelf with three GNSS receivers and cross-connected PTP feeds applies the principle at the site level. A unit that loses GNSS continues to be disciplined by the ensemble of the remaining two units and any upstream references, with no phase step visible to downstream applications.

The operational payoff is that the Shelf survives fault modes that would dismount a single grandmaster cleanly. A GNSS jamming incident that affects all three antennas (same building, same interference source) is detected by Ensemble — each unit sees its GNSS quality degrade in parallel, the Ensemble weights down GNSS contribution, and the Shelf falls back on the inter-unit cross-feed and any configured upstream PTP references. Downstream clients see a clean transition rather than a step change.

Oscillator tier per unit — matching cost to holdover requirement

The three units on the Shelf can be configured with different oscillator tiers. Each OTA ships with one of three oscillator grades, and the choice determines how long the unit holds accurate time when GNSS is unavailable. The tiers are: Quartz OCXO at £2,495 per unit, Rubidium Black at £6,500, and Rubidium Black+ at £7,795. The Shelf configurations publish at £7,485 for three OCXOs, £19,500 for three Rubidium Black, and £23,385 for three Rubidium Black+.

For finance venues deploying into a colocation site with potentially GNSS-denied scenarios under consideration — a jammer from a passing vehicle, a misconfigured adjacent transmitter, a rooftop construction event — the Rubidium Black+ configuration is the default. Sub-120 nanoseconds of drift over 24 hours gives headroom of more than 800× the MiFID II RTS 25 tolerance. The same configuration powers the enterprise hardware brief's recommendation for critical national infrastructure and defence deployments.

For enterprise IT deployments replacing an ageing NTP appliance, the OCXO tier is usually the right starting point. A site with reliable GNSS, a supportive thermal environment, and no formal holdover tolerance beyond 'the cluster keeps running during a typical outage' does not need Rubidium. OCXO aging compensation is active from day one, warm-up to specification completes in eight minutes to ±10 ppb, and the hardware cost differential funds the Sync Insight monitoring that turns clock drift from an unknown unknown into a visible, alertable, historically queryable parameter.

Rubidium Black sits in between. 24-hour holdover of under 500 nanoseconds suits telecom fronthaul, broadcast ST 2110 facilities, distributed-database environments and multi-site enterprises with longer outage tolerance. Power draw is 6 W max at startup (versus 20 W max for Black+ and 1.5–3.5 W for the OCXO), which matters when the Shelf is deployed in a street cabinet or space-constrained telecom site.

GNSS antenna diversity as a first-class design principle

Grandmaster redundancy without antenna diversity is not redundancy. If two grandmasters share a single rooftop antenna via a splitter, a cable cut or connector corrosion takes both out simultaneously. If two grandmasters each have their own antenna but the antennas are mounted 30 centimetres apart on the same mast, a localised RF fault (a car-park jammer, a nearby transmitter, a construction-crane EMI event) hits both at once. Physical separation at the antenna head is the only way to guarantee independent GNSS inputs.

Each OTA ships with its own u-blox LEA-F9T multi-band receiver and its own antenna input. A three-unit Shelf deployment has three independent cable runs to three independent antenna mounts. Best practice places the antennas in different roof positions where possible — separating them enough that a localised interferer cannot coherently dominate all three signal paths. Where rooftop geometry permits, one of the three can be mounted on a different building face or even a different building entirely, giving physical diversity against hyper-local RF.

The LEA-F9T operates at 2.5 nanoseconds differential and 5 nanoseconds absolute to UTC. Frequency bands are configurable — L1 plus L2 or L1 plus L5 — so the receiver mitigates ionospheric delay independently rather than relying on a single-frequency model. All four major constellations (GPS, Galileo, GLONASS, BeiDou) can be tracked simultaneously, with continuous multi-constellation integrity monitoring. Tracking sensitivity of −167 dBm and a cold-start time of 24 seconds across all constellations gives the Shelf a practical resilience profile against the kinds of GNSS degradation that a single-band, single-constellation incumbent would not survive.

Compliance posture — DORA, MiFID II and the evidence question

The single largest reason financial institutions are replacing their incumbent timing hardware in 2026 is not raw accuracy — most incumbents meet MiFID II RTS 25's 100-microsecond tolerance comfortably under normal conditions. It is evidence. DORA Article 9 requires accurate and complete ICT system records, Article 10 requires continuous monitoring and anomaly detection, and Article 11 requires documented business continuity arrangements. Those requirements, taken together, invalidate most single-unit log-file-based topologies.

A Shelf deployment addresses each article directly. Article 9: UTC Verification continuously checks the clock state of each unit every second against the full set of telemetry metrics and the unit's configuration, producing a continuous, auditable record of UTC traceability that a regulator can verify directly. Article 10: Sync Insight streams 167 telemetry fields per cycle per unit to Elasticsearch or Grafana — real-time offset, GNSS health, Allan deviation, steering algorithm state, PTP packet counters, per-satellite quality, jamming and spoofing flags. Article 11: the three-unit active-active topology is documented, testable (simulate a GNSS denial on one unit and watch PTP² Mesh redistribute load in real time), and continuously monitored.

ESMA published enforcement actions for MiFID II clock synchronisation failures range from €25,000 to €500,000 per breach, with systematic failures triggering action under the 10% of annual turnover ceiling under Article 70. DORA exposure is framed similarly — up to 2% of annual global turnover for repeated ICT failures under Article 50. The incumbent hardware conversation is not about raw accuracy; it is about the exposure of a topology that cannot produce the evidence that DORA and RTS 25 demand.

The enterprise IT case — no DORA, but the same structural problem

Enterprise IT organisations are not subject to DORA. They are subject to SOC 2 CC6.1 and ISO 27001 Annex A, both of which include time synchronisation controls. The typical Type II audit finding reads 'evidence of continuous monitoring of time synchronisation controls was not provided', and it shows up on report after report because the incumbent NTP appliance was not designed to produce that evidence. Sync Insight deploys on each Open Time Appliance out of the box, streams 1-second resolution telemetry into Grafana, and retains 12 months of audit-ready history that the auditor can query directly.

The operational case for the enterprise is separate from the audit case and sometimes stronger. Distributed databases — CockroachDB, YugabyteDB, Spanner, Cassandra — rely on loose bounds on clock skew between nodes to serialise transactions correctly. etcd and Kubernetes scheduling behave pathologically when nodes drift apart. Log correlation during a P1 incident becomes impossible when server clocks have drifted a few hundred milliseconds over the course of the day. An Open Time Appliance deployed as the site grandmaster, with PTP or hardware-timestamped NTP distribution to each node, turns this from a recurring class of problem into a monitored, alertable, historical parameter.

For a single-site enterprise with under 100 downstream devices, one Open Time Appliance (OCXO tier, £2,495) replaces the ageing NTP appliance directly. GNSS antenna mounts on the roof in a 30-minute install. Sync Insight Professional at £799/month covers full monitoring. Total Year 1 cost comes in under £12,500. For multi-site deployments, a Shelf per data centre with PTP distribution to branch offices gives the same monitored, redundant topology at each location. The Sync Insight Enterprise tier at £2,495/month auto-unlocks at 167 devices — which most multi-site deployments reach within 6–12 months.

5-year total cost of ownership — the numerical case

The capital cost of The Shelf with 3× Rubidium Black+ is £23,385. The incumbent equivalent — two single-unit grandmasters for A/B redundancy — typically lands at around £16,000 at market rates for similar oscillator-class hardware, or £8,000 for a single unit. On the CapEx line alone, the Shelf looks more expensive. The TCO picture inverts over five years because of three recurring costs that most procurement models understate.

Hardware support contracts on the incumbent typically run at 15–20% of CapEx annually — standard for proprietary timing hardware with a closed update process. Open Time Appliances run Linux with the Timebeat Agent; hardware support indexes at around 8% of CapEx because the platform is open, field-serviceable, and does not depend on a single vendor's firmware roadmap. Over five years, on a dual-unit incumbent at £16,000 CapEx, that is roughly £14,400 in support; on a three-unit Shelf at £23,385, approximately £9,350.

Engineering FTE overhead is the second line. A proprietary vendor platform with no API, no Prometheus exposition and no Grafana integration requires manual log extraction, manual compliance report preparation, and manual incident investigation. A rule of thumb from the enterprise briefs is 120 hours per year per site of engineering time managing the incumbent platform, against 40 hours for a Sync Insight-monitored Shelf. At a blended £75/hour, that is a five-year delta of £30,000 per site in favour of the Shelf.

Refresh cycle is the third. A monolithic incumbent chassis is typically replaced at end-of-oscillator-life — 5 years, full replacement cost. The OTA is modular: the GNSS receiver and oscillator are field-replaceable without a chassis swap. Refresh cost indexes at around 50% of initial CapEx on a 7-year cycle rather than 85% on a 5-year cycle. Across a 5-year TCO window, the incumbent hits a full refresh cost in Year 5 that the Shelf does not.

Deployment pattern — parallel run, then cutover

The recommended deployment pattern for a Shelf replacing an incumbent is parallel operation rather than direct cutover. The Shelf is installed in the timing cabinet alongside the existing grandmaster. Sync Insight is configured to monitor both — the incumbent via whatever telemetry it exposes (SNMP, syslog, log scrape), the Shelf natively. Downstream PTP slaves continue to consume time from the incumbent during the parallel run. The Shelf runs silent.

For 30 to 90 days, Sync Insight records a full telemetry comparison. Allan deviation trending on both the incumbent and the Shelf. GNSS event history on both. Holdover simulation — deliberately disconnect the Shelf antenna for a controlled window and observe Clock Ensemble behaviour. Incumbent failover test — trigger the incumbent's own redundancy mechanism and observe the downstream response. At the end of the window, the team has a documented, evidenced performance comparison and a clear decision point.

Cutover happens when the team is ready. Downstream PTP configuration is updated to point at the Shelf's virtual grandmaster address (PTP² Mesh abstracts the three-unit topology behind a single logical endpoint). The incumbent is typically retained for 60 to 90 days as a passive fallback before decommissioning, which gives a safety margin if the production team wants to revert for any reason. No cutover window, no blast radius event, no regulator-facing risk.

What this topology does not replace

The Shelf is a venue or site grandmaster. It is not a final-hop timing card for individual trading servers, and it is not a field-deployable portable time source. Finance venues deploying for HFT latency — colo servers that need sub-50-nanosecond card-to-UTC accuracy — typically combine a Shelf deployment (as the site-wide traceable UTC reference) with Open TimeCard Minis in the PCIe bay of each trading server (as the per-server precision endpoint). The Shelf provides the traceable time; the cards provide the timestamping precision at the point of trade.

Defence, mobile infrastructure and edge-of-network deployments where power, space and GNSS availability are constrained use different products in the TimeBeat range — the Open Time Appliance Mini PT and Lite PT are field-deployable variants with integrated battery and lower-power GNSS receivers. The Shelf is a rack-mount, mains-powered, controlled-environment product. It solves the venue grandmaster problem specifically.

Clock distribution beyond the first PTP hop is also out of scope for The Shelf as a product. The OTA's integrated Transparent Clock switch handles local distribution within the timing cabinet. Beyond that — multi-campus PTP, WAN time distribution, cross-datacentre timing — depends on your network topology and may involve Timebeat PTP² Mesh nodes at each site, White Rabbit fibre links, or boundary clocks in your existing network switches. The Shelf is the reference; the distribution is a separate design conversation.

What to do next

The natural starting point is not buying hardware. The natural starting point is deploying Sync Insight alongside your existing incumbent — PAYG at £1.12 per device per day for up to 20 devices, no procurement approval needed at that scale, live in under an hour. Within a few days of telemetry you will see your actual holdover behaviour, your GNSS quality trend, whether your secondary has been running as passive for months without anyone noticing, and how your incumbent's Allan deviation compares to the specifications you assumed were still valid.

Once the telemetry has done the work, the hardware conversation is not a cold pitch. It is a natural next step — here is the single point of failure Sync Insight just documented, here is the holdover gap we quantified in a simulated GNSS outage, here is the Shelf configuration that closes both. The procurement case writes itself because the numbers are the prospect's own data, not vendor claims. Contact sales@timebeat.app for a Shelf technical demonstration, or to start the parallel-deployment pilot.