Redundant Data Sources

The mechanism: multiple retrievals per measure#

A measure can carry more than one retrieval configuration — each against its own connection, each effective from its own date. The engine uses whichever retrieval covers the period being produced.

That makes the designed use temporal handover: the data lived in system A until a date, and in system B after it.

Water Volume (hourly)
+-- Retrieval 1: SQL table  Water_History      — effective 2011-01-01
+-- Retrieval 2: Historian  020-FQ-001.PV      — effective 2021-08-25

The classic applications, all from real sites:

• Historian migration — old historian for history, new one from cutover day; one measure, seamless chart across the boundary
• Pre-historian bootstrap — years of manual/SQL history, live tags thereafter
• Tag replacement — a tag decommissioned and replaced mid-life (also solvable by re-pointing the single retrieval, but multiple retrievals preserve the ability to re-backfill the old range from the old source)

The operational rule that catches people: the boundary date is a hard wall. The engine will not fill a post-boundary period from the pre-boundary source, and vice versa. The reported symptom is exactly what the mechanism implies — values complete up to the date, null after it — which means the new retrieval's own configuration (tag, connection, permissions) is where to look, per the ordinary missing-data procedure.

What it is not: automatic failover#

There is no "try source A; if it fails, read source B for the same period". When the ticket behind this article asked for redundancy against a flaky Canary link, the accurate answer was that Infohub does not arbitrate between two live sources per period — and the workarounds each cost something:

Lean on retry instead. For transient source outages, Infohub's normal behaviour already is the failover: the period stays unfilled, refresh offsets re-attempt it, and the value lands when the source returns. If data eventually arrives without hands, the "redundancy" requirement is often already met — what looked like a failover need was a latency need. Size the refresh offset to the outage profile (Backfilling and Recalculation covers the offset-vs-backfill split).

Model the arbitration explicitly. Where two live sources genuinely both exist (redundant historians), retrieve two measures — one per source — and put the preference logic in a calculated measure:

Measure_primary.Quality >= 192 && Measure_primary.Value != null
    ? Measure_primary.Value
    : Measure_secondary.Value

Costs: double retrieval load, a third measure, and the arbitration rule is yours to defend (quality thresholds, disagreement handling). Benefits: the logic is visible, chartable (you can see which source fed each period), and auditable — which implicit failover never is.

Fix it upstream. Redundant historians usually exist because the collection layer already does failover. If the source pair presents one virtual endpoint (historian mirroring, an OPC UA redundant server pair), Infohub needs only one connection and the problem stays where it belongs.

Choosing, quickly#

If you use the arbitration pattern#

Three disciplines keep it honest:

1. Alarm on the arbitration, not just the output — a measure counting periods served by the secondary, with a limit. Silent failover is how a dead primary goes unnoticed for a quarter.
2. Verify both sources agree while both are healthy (a difference calc with a limit); divergence is instrumentation drift you want surfaced, not averaged away.
3. Document which measure is canonical for downstream consumers — the arbitrated measure, never a source measure, or restatement behaviour becomes source-dependent.

See also#

• How Infohub Retrieves Data
• Retrieved Measures
• Choosing a Data Source
• Diagnosing Missing Data