Cloud Time Series and IoT Platforms

The connectors#

For the exact property set each declares, see the Connection Property Reference.

What they have in common#

Calls cost money#

This is the difference from on-premise sources that most changes how you should model.

Cloud services charge — per query, per byte scanned, per byte egressed, or per compute-second. An Infohub model that would be unremarkable against a local historian can produce a surprising invoice against a cloud warehouse.

Do the arithmetic before deploying: measures × retrievals per day × cost per retrieval. Four hundred measures retrieved hourly is nearly ten thousand queries a day.

Ways to keep it sane:

• Retrieve at the interval the data actually changes. Asking for minutely data from a service updated hourly costs sixty times as much for no additional information.
• Return one row set covering many values where the service allows it, rather than one query per measure.
• Use caching where the connector offers it, so several measures share one response.
• Keep concurrency modest. Parallelism increases cost rate, not total, but it also trips throttling.

Latency is higher and less predictable#

A local historian answers in milliseconds; a cloud service answers in hundreds of milliseconds, sometimes seconds, and occasionally not at all.

Practical consequences:

• Expect retrieval to take meaningfully longer, and do not conclude the Data Engine is slow.
• Set timeouts that are realistic rather than optimistic.
• Expect occasional failures as normal, not exceptional, and guard calculations so a failed retrieval yields no value rather than a wrong one.

Credentials expire and rotate#

Cloud credentials — access keys, client secrets, API tokens — have lifetimes and organisational rotation policies.

An Infohub connection holds whatever it was given. When the credential is rotated elsewhere, Infohub does not know, and retrieval stops. Record which cloud credentials Infohub depends on, and include Infohub in whatever process rotates them.

Network path and TLS#

These connections leave your network. That means outbound HTTPS from the Platform host — which may require a firewall rule or a proxy — and correct certificate trust on that machine, not on the modeller's workstation.

Per-connector notes#

Snowflake#

A SQL warehouse, so configuration resembles the SQL connectors: you write a query and Infohub substitutes [PeriodStart] and [PeriodEnd].

The performance advice is different from on-premise SQL, though. Snowflake charges for data scanned, so a query that scans a large table cheaply in wall-clock terms may be expensive in billing terms. Use its clustering and partitioning so that a period-bounded query reads only the relevant micro-partitions, and confirm with whoever owns the account that your query pattern is acceptable.

InfluxDB#

Purpose-built for time series, so period-bounded queries are natural and efficient.

Two things to get right: retention policies, since data may have been downsampled or expired before the period you are asking for; and precision, because Influx timestamps can be seconds, milliseconds, microseconds, or nanoseconds, and a mismatch puts every value in the wrong place.

AWS Timestream#

Managed time series with a query language and its own memory-versus-magnetic storage tiers. Queries against magnetic storage are slower and priced differently from recent data in the memory tier, so retrieving a long history behaves differently from retrieving yesterday.

Standard AWS considerations apply: an IAM identity with least privilege, and a region that matches where the data lives — a cross-region query adds both latency and egress cost.

Siemens Mindsphere and Netilion#

Platform APIs rather than databases, so they behave like the Web Service connector in character: authenticated HTTP, rate limits, and provider-defined response shapes.

Expect to work from the provider's documentation for asset and variable identifiers, and expect those identifiers to be opaque strings rather than readable tag names. Record which platform identifier corresponds to which piece of equipment somewhere outside Infohub as well as in the model, because the mapping is not self-evident from the identifier.

Apache Kafka#

A stream, not a store, which makes it the same shape of decision as MQTT: a consumer receives what is published while it is connected.

Kafka retains messages for a configured period, so there is more tolerance for a short outage than with MQTT — but retention is finite and is a broker setting, not something Infohub controls. Confirm the retention period, and understand that an outage longer than it means permanent loss.

Note that Infohub also has a Kafka consumer for publishing outward. The data source subscribes; the consumer publishes.

Telegraf#

A metrics collection agent. The connector declares very little configuration, which reflects its narrow purpose. Where you need broader coverage of what Telegraf gathers, consider having Telegraf write into a store Infohub reads — InfluxDB being the natural pairing — rather than reading the agent directly.

eWon and Thin Manager#

Device and infrastructure connectors. Both are operational-monitoring sources rather than process historians: connectivity, device health, session information. Useful for monitoring the estate that keeps the plant running, and generally modelled as a small number of measures rather than a large chain.

Modelling advice#

Aggregate early. Where a cloud source can return an interval aggregate cheaply, that is one of the few good reasons to retrieve at an interval rather than at Tag. Transferring every raw point out of a cloud service to aggregate locally may cost more than it is worth.

Consider a local staging store. For a large model against an expensive cloud source, having the cloud service write to a local database that Infohub reads can be both cheaper and faster. It adds a component to maintain — weigh that against the query bill.

Treat gaps as normal. Guard calculations, and do not build alerting that fires on every missed cloud retrieval, or the alerts will be ignored within a week.

Diagnosing#

See also#

• Choosing a Data Source
• SQL Databases
• MQTT and Sparkplug B
• Connection Property Reference