shelly-exporter/docs/architecture.md

Architecture and adding products

Design goals

The exporter separates transport, product decoding, Prometheus collection, configuration, and HTTP serving so additional Shelly models do not require changes throughout the application.

The Gen1 implementation uses a superset response structure with optional fields. This matches the Shelly API, where /status contains common fields plus product- and mode-specific blocks.

Request flow

flowchart LR
    P[Prometheus] -->|GET /metrics| H[HTTP server]
    H --> R[Prometheus registry]
    R --> G[Collector group]
    G -->|concurrent| C1[Device collector A]
    G -->|concurrent| C2[Device collector B]
    C1 --> S1[Status client]
    C2 --> S2[Status client]
    S1 -->|GET /status| D1[Shelly device A]
    S2 -->|GET /status| D2[Shelly device B]

There is no background polling loop or value cache. Each /metrics request produces one status request per configured device.

Package responsibilities

Package/file	Responsibility
main.go	Parses -config, installs signal handling, and starts the application.
configuration	Strict YAML loading, defaults, and product-independent validation.
application	Creates clients and collectors, owns the Prometheus registry, and serves HTTP endpoints.
shelly	Gen1 HTTP /status client, timeout, Basic Authentication, status checking, and JSON decoding.
collector/factory.go	Normalizes product identifiers and maps them to collector constructors.
collector/group.go	Presents all device collectors as one registered collector and runs them concurrently.
collector/base.go	Shared shelly_up, request duration, and request error metrics.
collector/gen1_status.go	Optional superset of known Gen1 /status response blocks.
collector/gen1_collector.go	Descriptor definitions and conversion from Gen1 status data to Prometheus metrics.

Startup lifecycle

1. configuration.Load reads and strictly parses YAML.
2. Product-independent validation checks the listener, timeout, unique names, and URLs.
3. application.New creates one HTTP status client per configured device.
4. collector.NewDeviceCollector normalizes the configured product and resolves its factory.
5. All device collectors are wrapped in collector.Group and registered once in a private Prometheus registry.
6. /metrics and /-/healthy handlers are installed.
7. Application.Run opens the listener and serves until cancellation or a server error.

A private registry prevents application metrics from being mixed with process-global collectors registered by dependencies.

Collection behavior

collector.Group.Collect starts one goroutine per configured device and waits for all of them. Each Gen1 collector then:

1. calls StatusClient.GetStatus;
2. emits scrape health, duration, and cumulative error count;
3. stops for that device if HTTP or JSON decoding failed;
4. emits common status metrics;
5. processes relays, inputs, meters, EM meters, rollers, lights, thermostats, and sensors;
6. skips every absent optional field.

The base error counter uses an atomic integer because Prometheus can execute overlapping collections. Device-specific values are const metrics built from the latest response and do not retain mutable state.

Why status fields use pointers

In JSON, a missing field and a present zero value have different meanings. For example:

{}

means that a capability may not exist, while:

{"power": 0}

means that the device supports power measurement and currently reports zero watts.

Numeric and boolean status fields therefore use pointers in gen1Status. Collection helpers emit a metric only for a non-nil pointer. The custom numberOrBool decoder handles the Gen1 overpower field because different product families represent it as either a boolean state or a numeric threshold.

Metric conventions

• Use the shelly_ namespace.
• Use Prometheus base-unit suffixes such as _seconds, _bytes, _watts, and _volts.
• Use _total only for counters.
• All product metrics include device; indexed response arrays additionally include index.
• Boolean values are gauges with 0 or 1.
• Text is exposed through labels on an _info gauge with value 1.
• Reuse an existing metric when a new product has the same semantic value.
• Avoid labels containing frequently changing or unbounded data.
• Preserve native cumulative device values instead of accumulating scrape deltas in exporter memory.

Energy counter rules

Gen1 uses two different cumulative representations:

• meters[].total is watt-minutes and is divided by 60 before being exported as shelly_meter_energy_watt_hours_total.
• emeters[].total and total_returned are already Wh and are exported unchanged.

Both are Prometheus counters. Reboots or explicit device resets can decrease them; Prometheus counter functions handle the reset.

Extending the Gen1 schema

Use this path when a product already supports GET /status but returns a field the generic collector does not understand.

1. Capture an official response fixture

Use the official Shelly documentation and, when available, a real device response. Remove passwords and private network identifiers before committing fixtures.

2. Add optional response fields

Extend gen1_status.go. Use pointers for scalar fields:

type gen1Status struct {
    // Existing fields...
    Voltage *float64 `json:"voltage"`
}

For a nested or repeated block, introduce a dedicated type rather than an anonymous map. A typed structure catches incompatible API changes during JSON decoding.

3. Register descriptors

Add each metric once in registerDescriptors:

c.add("supply_voltage_volts", "Device supply voltage in volts.")

add automatically prefixes the name with shelly_ and prepends the device label. Pass only additional label names.

4. Emit only present values

Use the existing helpers:

c.optionalGauge(channel, "supply_voltage_volts", status.Voltage)
c.optionalBool(channel, "feature_enabled", status.FeatureEnabled)

Use counter only for a cumulative, monotonically increasing source value:

if status.Events != nil && *status.Events >= 0 {
    c.counter(channel, "events_total", *status.Events)
}

5. Register the product identifier

If the new model is known to use this schema, append its normalized identifier to gen1Products. Unlisted compatible devices can already use product: gen1 without registration.

6. Add fixture-based tests

Add a representative JSON fixture in the collector tests and gather through prometheus.NewPedanticRegistry. Test:

• expected metric values and labels;
• metric type, particularly counters;
• unit conversions;
• valid zero values;
• absence of a metric when the source field is absent;
• inconsistent JSON forms if the device family has firmware variations.

Run:

gofmt -w collector
go test ./...
go vet ./...
go build ./...

Adding a different protocol generation

Do not model a Gen2/Gen3 RPC response as a Gen1 /status response. The current shelly.StatusClient always appends /status and implements Basic Authentication, so supporting an RPC generation requires a transport extension as well as a collector.

A clean implementation should:

1. define a product-independent request/client interface or a protocol client factory;
2. choose the protocol client from product metadata during application construction;
3. implement the RPC endpoint and its authentication scheme separately;
4. add a collector with generation-appropriate response structures;
5. register product identifiers through the existing collector factory;
6. retain the shared collector group and scrape health metrics.

This keeps generation-specific endpoint paths and authentication out of the application HTTP server.