This commit is contained in:
@@ -0,0 +1,88 @@
|
||||
# Architecture and Adding Collectors
|
||||
|
||||
## Collection flow
|
||||
|
||||
```text
|
||||
config.yaml
|
||||
|
|
||||
v
|
||||
Application -> PveApiClient -> Proxmox VE API host(s)
|
||||
| |
|
||||
| +-> liveness checks, round-robin selection, short API cache
|
||||
v
|
||||
PveMetricsManager -> enabled collectors -> Prometheus registry -> /metrics
|
||||
|
|
||||
+-> TTL cleanup for stale label sets
|
||||
```
|
||||
|
||||
## Packages
|
||||
|
||||
| Package | Responsibility |
|
||||
| --- | --- |
|
||||
| `application` | Loads configuration, creates dependencies, registers `/metrics`, and starts the HTTP server. |
|
||||
| `configuration` | YAML models and startup validation. |
|
||||
| `proxmox` | HTTP client, Proxmox-specific API methods, response models, and numeric state conversion. |
|
||||
| `metrics` | Collector implementations, periodic scheduling, Prometheus metric definitions, and TTL cleanup. |
|
||||
| `utils` | Shared formatting helpers. |
|
||||
|
||||
## API client
|
||||
|
||||
`PveApiClient` wraps the generic `ApiClient` with typed methods such as cluster
|
||||
status, node status, guest status, storage, disk, and ZFS calls.
|
||||
|
||||
The generic client:
|
||||
|
||||
- normalizes API hosts so a trailing slash is optional;
|
||||
- checks every host at startup and every five seconds;
|
||||
- selects reachable hosts in round-robin order;
|
||||
- caches a successful method/path response briefly to avoid duplicate API calls;
|
||||
- uses API-token authorization and a ten-second HTTP timeout;
|
||||
- accepts self-signed certificates by disabling TLS verification.
|
||||
|
||||
Configured hosts must represent the same cluster because cached results are
|
||||
shared by method and path rather than host.
|
||||
|
||||
## Metrics manager
|
||||
|
||||
`PveMetricsManager` creates only the collectors enabled in configuration. It
|
||||
runs one full collection immediately, then repeats it every
|
||||
`proxmox.interval` seconds. Collectors execute sequentially.
|
||||
|
||||
Each collector implements:
|
||||
|
||||
```go
|
||||
type PveMetricsCollector interface {
|
||||
CollectMetrics() error
|
||||
GetName() string
|
||||
}
|
||||
```
|
||||
|
||||
Successful execution time is recorded in
|
||||
`pve_metrics_collection_latency_ms`. A collector error is logged and the
|
||||
manager continues with the next collector.
|
||||
|
||||
## TTL metrics
|
||||
|
||||
Most project metrics use `TTLGaugeVec`, a wrapper around Prometheus `GaugeVec`.
|
||||
Every label set records its last update. `TTLRegistry` checks registered metrics
|
||||
every five seconds and removes series that have not been updated for five
|
||||
minutes.
|
||||
|
||||
This prevents deleted guests or storage resources from remaining indefinitely,
|
||||
while allowing short API failures to preserve the last known values.
|
||||
|
||||
## Adding a collector
|
||||
|
||||
1. Add typed response models and API methods under `proxmox`.
|
||||
2. Add a collector implementing `PveMetricsCollector` under `metrics`.
|
||||
3. Create metrics with `NewTTLGaugeVec`, choose stable labels, and register each
|
||||
metric with the shared `TTLRegistry`.
|
||||
4. Add a boolean switch to `PveMetricsConfiguration` and `config.example.yaml`.
|
||||
5. Register the collector in `NewPveMetricsManager`.
|
||||
6. Add parsing, state-mapping, and topology tests as appropriate.
|
||||
7. Document the switch, labels, units, and state values in
|
||||
[configuration.md](configuration.md) and [metrics.md](metrics.md).
|
||||
|
||||
Avoid labels containing changing messages, timestamps, or other unbounded
|
||||
values. For recursive resources such as ZFS, include a stable path so repeated
|
||||
component names remain distinguishable.
|
||||
Reference in New Issue
Block a user