This commit is contained in:
@@ -0,0 +1,189 @@
|
||||
# Architecture and adding products
|
||||
|
||||
## Design goals
|
||||
|
||||
The exporter separates transport, product decoding, Prometheus collection, configuration, and HTTP serving so additional Shelly models do not require changes throughout the application.
|
||||
|
||||
The Gen1 implementation uses a superset response structure with optional fields. This matches the Shelly API, where `/status` contains common fields plus product- and mode-specific blocks.
|
||||
|
||||
## Request flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
P[Prometheus] -->|GET /metrics| H[HTTP server]
|
||||
H --> R[Prometheus registry]
|
||||
R --> G[Collector group]
|
||||
G -->|concurrent| C1[Device collector A]
|
||||
G -->|concurrent| C2[Device collector B]
|
||||
C1 --> S1[Status client]
|
||||
C2 --> S2[Status client]
|
||||
S1 -->|GET /status| D1[Shelly device A]
|
||||
S2 -->|GET /status| D2[Shelly device B]
|
||||
```
|
||||
|
||||
There is no background polling loop or value cache. Each `/metrics` request produces one status request per configured device.
|
||||
|
||||
## Package responsibilities
|
||||
|
||||
| Package/file | Responsibility |
|
||||
| --- | --- |
|
||||
| [`main.go`](../main.go) | Parses `-config`, installs signal handling, and starts the application. |
|
||||
| [`configuration`](../configuration/configuration.go) | Strict YAML loading, defaults, and product-independent validation. |
|
||||
| [`application`](../application/application.go) | Creates clients and collectors, owns the Prometheus registry, and serves HTTP endpoints. |
|
||||
| [`shelly`](../shelly/client.go) | Gen1 HTTP `/status` client, timeout, Basic Authentication, status checking, and JSON decoding. |
|
||||
| [`collector/factory.go`](../collector/factory.go) | Normalizes product identifiers and maps them to collector constructors. |
|
||||
| [`collector/group.go`](../collector/group.go) | Presents all device collectors as one registered collector and runs them concurrently. |
|
||||
| [`collector/base.go`](../collector/base.go) | Shared `shelly_up`, request duration, and request error metrics. |
|
||||
| [`collector/gen1_status.go`](../collector/gen1_status.go) | Optional superset of known Gen1 `/status` response blocks. |
|
||||
| [`collector/gen1_collector.go`](../collector/gen1_collector.go) | Descriptor definitions and conversion from Gen1 status data to Prometheus metrics. |
|
||||
|
||||
## Startup lifecycle
|
||||
|
||||
1. `configuration.Load` reads and strictly parses YAML.
|
||||
2. Product-independent validation checks the listener, timeout, unique names, and URLs.
|
||||
3. `application.New` creates one HTTP status client per configured device.
|
||||
4. `collector.NewDeviceCollector` normalizes the configured product and resolves its factory.
|
||||
5. All device collectors are wrapped in `collector.Group` and registered once in a private Prometheus registry.
|
||||
6. `/metrics` and `/-/healthy` handlers are installed.
|
||||
7. `Application.Run` opens the listener and serves until cancellation or a server error.
|
||||
|
||||
A private registry prevents application metrics from being mixed with process-global collectors registered by dependencies.
|
||||
|
||||
## Collection behavior
|
||||
|
||||
`collector.Group.Collect` starts one goroutine per configured device and waits for all of them. Each Gen1 collector then:
|
||||
|
||||
1. calls `StatusClient.GetStatus`;
|
||||
2. emits scrape health, duration, and cumulative error count;
|
||||
3. stops for that device if HTTP or JSON decoding failed;
|
||||
4. emits common status metrics;
|
||||
5. processes relays, inputs, meters, EM meters, rollers, lights, thermostats, and sensors;
|
||||
6. skips every absent optional field.
|
||||
|
||||
The base error counter uses an atomic integer because Prometheus can execute overlapping collections. Device-specific values are const metrics built from the latest response and do not retain mutable state.
|
||||
|
||||
## Why status fields use pointers
|
||||
|
||||
In JSON, a missing field and a present zero value have different meanings. For example:
|
||||
|
||||
```json
|
||||
{}
|
||||
```
|
||||
|
||||
means that a capability may not exist, while:
|
||||
|
||||
```json
|
||||
{"power": 0}
|
||||
```
|
||||
|
||||
means that the device supports power measurement and currently reports zero watts.
|
||||
|
||||
Numeric and boolean status fields therefore use pointers in `gen1Status`. Collection helpers emit a metric only for a non-nil pointer. The custom `numberOrBool` decoder handles the Gen1 `overpower` field because different product families represent it as either a boolean state or a numeric threshold.
|
||||
|
||||
## Metric conventions
|
||||
|
||||
- Use the `shelly_` namespace.
|
||||
- Use Prometheus base-unit suffixes such as `_seconds`, `_bytes`, `_watts`, and `_volts`.
|
||||
- Use `_total` only for counters.
|
||||
- All product metrics include `device`; indexed response arrays additionally include `index`.
|
||||
- Boolean values are gauges with `0` or `1`.
|
||||
- Text is exposed through labels on an `_info` gauge with value `1`.
|
||||
- Reuse an existing metric when a new product has the same semantic value.
|
||||
- Avoid labels containing frequently changing or unbounded data.
|
||||
- Preserve native cumulative device values instead of accumulating scrape deltas in exporter memory.
|
||||
|
||||
### Energy counter rules
|
||||
|
||||
Gen1 uses two different cumulative representations:
|
||||
|
||||
- `meters[].total` is watt-minutes and is divided by 60 before being exported as `shelly_meter_energy_watt_hours_total`.
|
||||
- `emeters[].total` and `total_returned` are already Wh and are exported unchanged.
|
||||
|
||||
Both are Prometheus counters. Reboots or explicit device resets can decrease them; Prometheus counter functions handle the reset.
|
||||
|
||||
## Extending the Gen1 schema
|
||||
|
||||
Use this path when a product already supports `GET /status` but returns a field the generic collector does not understand.
|
||||
|
||||
### 1. Capture an official response fixture
|
||||
|
||||
Use the official Shelly documentation and, when available, a real device response. Remove passwords and private network identifiers before committing fixtures.
|
||||
|
||||
### 2. Add optional response fields
|
||||
|
||||
Extend [`gen1_status.go`](../collector/gen1_status.go). Use pointers for scalar fields:
|
||||
|
||||
```go
|
||||
type gen1Status struct {
|
||||
// Existing fields...
|
||||
Voltage *float64 `json:"voltage"`
|
||||
}
|
||||
```
|
||||
|
||||
For a nested or repeated block, introduce a dedicated type rather than an anonymous map. A typed structure catches incompatible API changes during JSON decoding.
|
||||
|
||||
### 3. Register descriptors
|
||||
|
||||
Add each metric once in `registerDescriptors`:
|
||||
|
||||
```go
|
||||
c.add("supply_voltage_volts", "Device supply voltage in volts.")
|
||||
```
|
||||
|
||||
`add` automatically prefixes the name with `shelly_` and prepends the `device` label. Pass only additional label names.
|
||||
|
||||
### 4. Emit only present values
|
||||
|
||||
Use the existing helpers:
|
||||
|
||||
```go
|
||||
c.optionalGauge(channel, "supply_voltage_volts", status.Voltage)
|
||||
c.optionalBool(channel, "feature_enabled", status.FeatureEnabled)
|
||||
```
|
||||
|
||||
Use `counter` only for a cumulative, monotonically increasing source value:
|
||||
|
||||
```go
|
||||
if status.Events != nil && *status.Events >= 0 {
|
||||
c.counter(channel, "events_total", *status.Events)
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Register the product identifier
|
||||
|
||||
If the new model is known to use this schema, append its normalized identifier to `gen1Products`. Unlisted compatible devices can already use `product: gen1` without registration.
|
||||
|
||||
### 6. Add fixture-based tests
|
||||
|
||||
Add a representative JSON fixture in the collector tests and gather through `prometheus.NewPedanticRegistry`. Test:
|
||||
|
||||
- expected metric values and labels;
|
||||
- metric type, particularly counters;
|
||||
- unit conversions;
|
||||
- valid zero values;
|
||||
- absence of a metric when the source field is absent;
|
||||
- inconsistent JSON forms if the device family has firmware variations.
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
gofmt -w collector
|
||||
go test ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
```
|
||||
|
||||
## Adding a different protocol generation
|
||||
|
||||
Do not model a Gen2/Gen3 RPC response as a Gen1 `/status` response. The current `shelly.StatusClient` always appends `/status` and implements Basic Authentication, so supporting an RPC generation requires a transport extension as well as a collector.
|
||||
|
||||
A clean implementation should:
|
||||
|
||||
1. define a product-independent request/client interface or a protocol client factory;
|
||||
2. choose the protocol client from product metadata during application construction;
|
||||
3. implement the RPC endpoint and its authentication scheme separately;
|
||||
4. add a collector with generation-appropriate response structures;
|
||||
5. register product identifiers through the existing collector factory;
|
||||
6. retain the shared collector group and scrape health metrics.
|
||||
|
||||
This keeps generation-specific endpoint paths and authentication out of the application HTTP server.
|
||||
Reference in New Issue
Block a user