Updated docs

Use base units, export cumulative values as _total counters, update docs, dashboards and rules.

.gitea/workflows/docker-tag-build.yml

@@ -38,5 +38,6 @@ jobs:
           docker buildx build \
             --platform linux/amd64 \
             -t "${IMAGE_NAME}:${IMAGE_TAG}" \
+            -t "${IMAGE_NAME}:latest" \
             --push \
-            .
+            .

README.md

@@ -1,121 +1,154 @@
 # PVE Exporter
 
-Proxmox Virtual Environment Prometheus metrics exporter.
+Prometheus exporter for Proxmox Virtual Environment. It periodically reads the
+Proxmox VE HTTP API and exposes cluster, node, guest, storage, disk, ZFS,
+subscription, and SDN metrics through a single `/metrics` endpoint.
 
-## Overview
-
-PVE Exporter is a tool that collects metrics from a Proxmox Virtual Environment cluster and exposes them for Prometheus to scrape. This exporter supports gathering metrics for cluster state, LXC containers, QEMU virtual machines, physical disks, ZFS pools, node storage, node status, node subscription details, and software-defined networking (SDN).
+The exporter supports standalone nodes and multi-node clusters. Multiple API
+hosts can be configured for endpoint failover within one Proxmox cluster.
 
 ## Features
 
-- Collects various metrics from Proxmox VE to monitor the health and performance of your virtual environment.
-- Supports multi-node clusters for high availability.
-- Securely uses Proxmox API tokens with minimal permissions.
+- Standalone-node and cluster mode detection, node count, and quorum state.
+- Node CPU, memory, load, root filesystem, uptime, time, and system information.
+- QEMU virtual machine and LXC container state and resource metrics.
+- Physical disk health, wearout, and size metrics.
+- Recursive ZFS pool, vdev, cache, and leaf-device state and error metrics.
+- Storage capacity and availability, subscription, and SDN state metrics.
+- Selectively enabled collectors with automatic cleanup of stale label sets.
+- Multiple API endpoints with periodic liveness checks and round-robin requests.
+- Proxmox API token authentication using the read-only `PVEAuditor` role.
 
-## Prerequisites
+## Quick start
 
-- Docker installed on your machine.
-- Access to a Proxmox VE instance with API tokens configured.
-
-## Configuration
-
-### Proxmox API Token
-
-When generating the token, make sure to assign the 'PVEAuditor' permission to both the user and the API token. For security reasons, assign only the 'PVEAuditor' role to limit permissions appropriately.
-
-```yaml
-# Proxmox API token configuration.
-token:
-  tokenId: "your-token-id"
-  secret: "your-secret"
-```
-
-### Proxmox API Hosts
-
-If you are running a multi-node cluster, add multiple API hosts to ensure high availability of metrics. Note that this configuration is not intended for gathering metrics from multiple PVE clusters. For multiple PVE clusters, deploy a separate exporter instance for each cluster.
-
-```yaml
-# Proxmox API hosts.
-hosts:
-  - "https://host1.example.com"
-  - "https://host2.example.com"
-```
-
-### Metrics Configuration
-
-Configure which metrics to collect by enabling or disabling specific metric types.
-
-```yaml
-# Proxmox metrics configuration.
-metrics:
-  clusterState: true   # Enable collection of cluster state metrics.
-  ltc: true            # Enable collection of LXC container metrics.
-  qemu: true           # Enable collection of QEMU virtual machine metrics.
-  disk: true           # Enable collection of physical disk metrics.
-  zfs: true            # Enable collection of ZFS pool metrics.
-  storage: true        # Enable collection of node storage metrics.
-  nodeStatus: true     # Enable collection of node status metrics.
-  subscription: true   # Enable collection of node subscription details.
-  sdn: true            # Enable collection of software-defined network (SDN) metrics.
-```
-
-The cluster state collector exposes `pve_cluster_mode`, where `1` means that
-Proxmox VE is configured as a cluster and `0` means that it is a standalone node.
-
-The `pve_node_subscription_status` metric uses `0` when no subscription is
-configured, `1` for an active subscription, and `2` for an expired or otherwise
-unusable subscription.
-
-The ZFS collector discovers all pools on every node and exports their recursive
-topology with `cluster`, `node`, `pool`, `component`, `path`, and `leaf` labels:
-
-- `pve_node_zfs_state` (`0=UNKNOWN`, `1=ONLINE`, `2=DEGRADED`, `3=FAULTED`,
-  `4=OFFLINE`, `5=REMOVED`, `6=UNAVAIL`)
-- `pve_node_zfs_read_errors`
-- `pve_node_zfs_write_errors`
-- `pve_node_zfs_checksum_errors`
-
-## Build
-
-To build the Docker image for PVE Exporter, use the following command:
+Copy the example configuration and set the Proxmox API hosts and token:
 
 ```sh
-docker build --rm -t gitea.lostak.dev/lostakj/pve-exporter:version .
+cp config.example.yaml config.yaml
 ```
 
-Replace `version` with the appropriate version tag you want to use.
+```yaml
+host: 0.0.0.0
+port: 9090
+logLevel: 4
 
-## Run
+proxmox:
+  token:
+    tokenId: monitoring@pve!exporter
+    secret: change-me
 
-To run the Docker image, use the following command:
+  hosts:
+    - https://pve01.example.com:8006
+    - https://pve02.example.com:8006
+
+  interval: 15
+
+  metrics:
+    clusterState: true
+    nodeStatus: true
+    qemu: true
+    lxc: true
+    disk: true
+    zfs: true
+    storage: true
+    subscription: true
+    sdn: true
+```
+
+Assign the `PVEAuditor` role to both the Proxmox user and API token. All entries
+under `hosts` must belong to the same cluster. IP addresses and DNS names are
+supported, with or without a trailing slash.
+
+Build and start the exporter:
 
 ```sh
-docker run --rm -d -p 9090:9090 --name pve-exporter gitea.lostak.dev/lostakj/pve-exporter:version
+go build -o pve-exporter .
+./pve-exporter -config config.yaml
 ```
 
-Replace `version` with the appropriate version tag you used during the build.
+The exporter exposes Prometheus metrics at:
 
-## Usage
+```text
+http://localhost:9090/metrics
+```
 
-Once the Docker container is running, the exporter will be available on port `9090`. You can configure Prometheus to scrape metrics from the exporter by adding a new scrape configuration to your Prometheus configuration file.
+## Documentation
 
-Example Prometheus scrape configuration:
+- [Documentation index](docs/README.md)
+- [Configuration reference](docs/configuration.md)
+- [Metrics reference](docs/metrics.md)
+- [Prometheus alerting rules](docs/alerting.md)
+- [Operations and troubleshooting](docs/operations.md)
+- [Architecture and adding collectors](docs/architecture.md)
+
+## Prometheus configuration
 
 ```yaml
 scrape_configs:
-  - job_name: 'pve-exporter'
+  - job_name: pve-exporter
     static_configs:
-      - targets: ['localhost:9090']
+      - targets: [pve-exporter:9090]
 ```
 
+Metrics are collected in the background according to `proxmox.interval`.
+Prometheus scrapes the most recently collected values and does not trigger a new
+request to Proxmox VE.
+
+An example alert group for node, guest, storage, subscription, disk, ZFS, SDN,
+and capacity conditions is available in
+[`examples/pve-exporter.rules.yml`](examples/pve-exporter.rules.yml). See the
+[alerting guide](docs/alerting.md) for the `rule_files` configuration and
+collector requirements.
+
+## Docker
+
+Pull the latest published image from the Gitea container registry:
+
+```sh
+docker pull gitea.lostak.dev/lostakj/pve-exporter:latest
+```
+
+Or build it locally:
+
+```sh
+docker build --rm -t pve-exporter:latest .
+```
+
+Run the published image with a local configuration file:
+
+```sh
+docker run --rm -p 9090:9090 \
+  -v "$PWD/config.yaml:/opt/config.yaml:ro" \
+  gitea.lostak.dev/lostakj/pve-exporter:latest
+```
+
+Example with Docker Compose:
+
+```yaml
+services:
+  pve-exporter:
+    image: gitea.lostak.dev/lostakj/pve-exporter:latest
+    ports:
+      - "9090:9090"
+    volumes:
+      - ./config.yaml:/opt/config.yaml:ro
+    restart: unless-stopped
+```
+
+## Scope
+
+One exporter instance monitors one standalone Proxmox VE installation or one
+cluster. Configure multiple `hosts` only for redundant access to that same
+cluster. Deploy a separate exporter instance for every additional cluster.
+
+The published workflow currently builds the container image for `linux/amd64`.
+The API client accepts self-signed Proxmox certificates because TLS certificate
+verification is disabled; use trusted networks and protect the API token.
+
 ## License
 
-This project is licensed under the MIT License. See the LICENSE.txt file for details.
+This project is licensed under the [MIT License](LICENSE).
 
 ## Contributing
 
-Contributions are welcome! Please open an issue or submit a pull request if you have any improvements or bug fixes.
-
-## Support
-
-If you encounter any issues or have questions, please open an issue on the project's GitHub repository.
+Contributions and bug reports are welcome through the project issue tracker.

config.example.yaml

@@ -22,7 +22,7 @@ proxmox:
   #       For multiple PVE clusters, deploy a separate exporter instance for each cluster.
   hosts: []
 
-  # Scrape interval in seconds.
+  # Metrics collection interval in seconds.
   interval: 15
 
   # Proxmox metrics configuration.

configuration/configuration.go

@@ -63,6 +63,10 @@ func (c *Configuration) Validate() error {
 		if !(u.Scheme == "http" || u.Scheme == "https") {
 			return fmt.Errorf("PVE host '%s' must be protocol type of HTTP or HTTPS.", host)
 		}
+
+		if u.Hostname() == "" {
+			return fmt.Errorf("PVE host '%s' must contain an IP address or DNS name.", host)
+		}
 	}
 
 	// Validate PVE token

configuration/configuration_test.go

@@ -0,0 +1,32 @@
+package configuration
+
+import "testing"
+
+func TestConfigurationValidateAcceptsIPAndDNSPveHosts(t *testing.T) {
+	tests := []struct {
+		name string
+		host string
+	}{
+		{name: "IP address", host: "https://192.168.0.10:8006"},
+		{name: "DNS name", host: "https://pve.example.com:8006"},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			configuration := Configuration{
+				Host: "0.0.0.0",
+				PVE: PveConfiguration{
+					Hosts: []string{tt.host},
+					Token: PveTokenConfiguration{
+						TokenId: "token",
+						Secret:  "secret",
+					},
+				},
+			}
+
+			if err := configuration.Validate(); err != nil {
+				t.Fatalf("Validate() rejected %s host %q: %v", tt.name, tt.host, err)
+			}
+		})
+	}
+}

docs/README.md

@@ -0,0 +1,27 @@
+# PVE Exporter Documentation
+
+PVE Exporter periodically collects monitoring data from the Proxmox VE HTTP API
+and stores the latest values in the Prometheus registry exposed at `/metrics`.
+
+## Guides
+
+- [Configuration reference](configuration.md) describes every YAML setting,
+  API token permissions, and collector switch.
+- [Metrics reference](metrics.md) lists exported metrics, labels, units, and
+  numeric state mappings.
+- [Prometheus alerting](alerting.md) provides a ready-to-use rule group,
+  installation example, and collector requirements.
+- [Operations and troubleshooting](operations.md) covers native and container
+  deployment, endpoint failover, logging, security, and common errors.
+- [Architecture and adding collectors](architecture.md) explains the collection
+  flow and how the main packages fit together.
+
+## Important behavior
+
+- One exporter instance is intended for one standalone node or one PVE cluster.
+- Multiple configured API hosts provide redundant access to that same cluster.
+- Collection runs every `proxmox.interval` seconds independently of Prometheus
+  scrapes.
+- Collector label sets that stop receiving updates are removed after five
+  minutes.
+- The only application HTTP endpoint is `/metrics`.

docs/alerting.md

@@ -0,0 +1,70 @@
+# Prometheus Alerting
+
+The repository includes a ready-to-use example rule group in
+[`examples/pve-exporter.rules.yml`](../examples/pve-exporter.rules.yml). It
+covers node, guest, storage, subscription, physical disk, ZFS, SDN, clock-skew,
+and memory-overcommit conditions.
+
+## Enable the rule file
+
+Make the rule file available to Prometheus and reference it from
+`prometheus.yml`:
+
+```yaml
+rule_files:
+  - /etc/prometheus/rules/pve-exporter.rules.yml
+
+scrape_configs:
+  - job_name: pve-exporter
+    static_configs:
+      - targets: [pve-exporter:9090]
+```
+
+For a containerized Prometheus deployment, mount the example at the configured
+path:
+
+```yaml
+services:
+  prometheus:
+    image: prom/prometheus:latest
+    volumes:
+      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
+      - ./examples/pve-exporter.rules.yml:/etc/prometheus/rules/pve-exporter.rules.yml:ro
+```
+
+Validate the file before reloading Prometheus:
+
+```sh
+promtool check rules examples/pve-exporter.rules.yml
+```
+
+## Collector requirements
+
+Prometheus evaluates a rule only when its referenced time series exist. Enable
+the matching exporter collectors:
+
+| Alerts | Required collector setting |
+| --- | --- |
+| Node filesystem, state, CPU, clock skew, memory overcommit | `metrics.nodeStatus` |
+| Guest CPU and memory overcommit | `metrics.qemu` and/or `metrics.lxc` |
+| Storage state and capacity | `metrics.storage` |
+| Subscription expiration | `metrics.subscription` |
+| Physical disk health and wear | `metrics.disk` |
+| ZFS state and errors | `metrics.zfs` |
+| SDN state | `metrics.sdn` |
+
+The default thresholds and `for` durations are operational examples. Review
+them against the size, workload, redundancy, and maintenance policy of your
+environment before enabling notifications.
+
+Three rules require deployment-specific review:
+
+- `PveUnexpectedStandaloneMode` should be removed or disabled when standalone
+  PVE operation is expected.
+- `PveNodeSubscriptionMissing` is informational and should be removed when
+  unsubscribed nodes are intentional.
+- `PveExporterDown` expects the Prometheus scrape job to be named
+  `pve-exporter`; adjust its `job` matcher if a different name is used.
+
+Every rule uses the `cluster` label. Standalone installations are supported and
+use the label value `Standalone node - <node>`.

docs/architecture.md

@@ -0,0 +1,96 @@
+# Architecture and Adding Collectors
+
+## Collection flow
+
+```text
+config.yaml
+    |
+    v
+Application -> PveApiClient -> Proxmox VE API host(s)
+    |               |
+    |               +-> liveness checks, round-robin selection, short API cache
+    v
+PveMetricsManager -> enabled collectors -> Prometheus registry -> /metrics
+                              |
+                              +-> TTL cleanup for stale label sets
+```
+
+## Packages
+
+| Package | Responsibility |
+| --- | --- |
+| `application` | Loads configuration, creates dependencies, registers `/metrics`, and starts the HTTP server. |
+| `configuration` | YAML models and startup validation. |
+| `proxmox` | HTTP client, Proxmox-specific API methods, response models, and numeric state conversion. |
+| `metrics` | Collector implementations, periodic scheduling, Prometheus metric definitions, and TTL cleanup. |
+| `utils` | Shared formatting helpers. |
+
+## API client
+
+`PveApiClient` wraps the generic `ApiClient` with typed methods such as cluster
+status, node status, guest status, storage, disk, and ZFS calls.
+
+The generic client:
+
+- normalizes API hosts so a trailing slash is optional;
+- checks every host at startup and every five seconds;
+- selects reachable hosts in round-robin order;
+- caches a successful method/path response briefly to avoid duplicate API calls;
+- uses API-token authorization and a ten-second HTTP timeout;
+- accepts self-signed certificates by disabling TLS verification.
+
+Configured hosts must represent the same cluster because cached results are
+shared by method and path rather than host.
+
+## Metrics manager
+
+`PveMetricsManager` creates only the collectors enabled in configuration. It
+runs one full collection immediately, then repeats it every
+`proxmox.interval` seconds. Collectors execute sequentially.
+
+Each collector implements:
+
+```go
+type PveMetricsCollector interface {
+    CollectMetrics() error
+    GetName() string
+}
+```
+
+Successful execution time is recorded in
+`pve_metrics_collection_duration_seconds`. A collector error is logged and the
+manager continues with the next collector.
+
+## TTL metrics
+
+Gauges use `TTLGaugeVec`, a wrapper around Prometheus `GaugeVec`. Cumulative
+values use `TTLCounterVec`, which exports the absolute value reported by the PVE
+API as a Prometheus counter (`GaugeVec` cannot be used because a counter name
+must carry a `_total` suffix, and `CounterVec` cannot be used because it only
+supports `Inc`/`Add`). Every label set records its last update. `TTLRegistry`
+checks registered metrics every five seconds and removes series that have not
+been updated for five minutes.
+
+This prevents deleted guests or storage resources from remaining indefinitely,
+while allowing short API failures to preserve the last known values.
+
+## Adding a collector
+
+1. Add typed response models and API methods under `proxmox`.
+2. Add a collector implementing `PveMetricsCollector` under `metrics`.
+3. Create metrics with `NewTTLGaugeVec` or, for cumulative values, with
+   `NewTTLCounterVec`, choose stable labels, and register each metric with the
+   shared `TTLRegistry`.
+4. Add a boolean switch to `PveMetricsConfiguration` and `config.example.yaml`.
+5. Register the collector in `NewPveMetricsManager`.
+6. Add parsing, state-mapping, and topology tests as appropriate.
+7. Document the switch, labels, units, and state values in
+   [configuration.md](configuration.md) and [metrics.md](metrics.md).
+
+Metric names must follow the Prometheus naming conventions; they are verified by
+`TestPveMetricNamesFollowPrometheusConventions`, which lints every registered
+metric with `promlint`.
+
+Avoid labels containing changing messages, timestamps, or other unbounded
+values. For recursive resources such as ZFS, include a stable path so repeated
+component names remain distinguishable.

docs/configuration.md

@@ -0,0 +1,113 @@
+# Configuration Reference
+
+The exporter reads `config.yaml` from the current working directory by default.
+Use `-config` to select a different file:
+
+```sh
+./pve-exporter -config /etc/pve-exporter/config.yaml
+```
+
+## Complete example
+
+```yaml
+host: 0.0.0.0
+port: 9090
+logLevel: 4
+
+proxmox:
+  token:
+    tokenId: monitoring@pve!exporter
+    secret: change-me
+
+  hosts:
+    - https://192.168.0.10:8006
+    - https://pve02.example.com:8006/
+
+  interval: 15
+
+  metrics:
+    clusterState: true
+    nodeStatus: true
+    qemu: true
+    lxc: true
+    disk: true
+    zfs: true
+    storage: true
+    subscription: true
+    sdn: true
+```
+
+## Server settings
+
+| Setting | Type | Description |
+| --- | --- | --- |
+| `host` | IP address | Address on which the exporter HTTP server listens, for example `0.0.0.0` or `127.0.0.1`. DNS names are not accepted here. |
+| `port` | integer | Exporter HTTP port. Prometheus metrics are served at `/metrics`. |
+| `logLevel` | integer | Logrus level: `0=Panic`, `1=Fatal`, `2=Error`, `3=Warn`, `4=Info`, `5=Debug`, `6=Trace`. |
+
+## Proxmox token
+
+| Setting | Type | Description |
+| --- | --- | --- |
+| `proxmox.token.tokenId` | string | Full Proxmox API token ID, normally `user@realm!token-name`. |
+| `proxmox.token.secret` | string | API token secret. Treat the configuration file as sensitive. |
+
+Create a dedicated monitoring user and token, then assign the read-only
+`PVEAuditor` role to both the user and the token. The role supplies the audit
+permissions used by cluster, node, storage, disk, ZFS, guest, and subscription
+API calls.
+
+## API hosts and failover
+
+`proxmox.hosts` is a non-empty list of Proxmox API base URLs. Both IP addresses
+and DNS names are supported:
+
+```yaml
+proxmox:
+  hosts:
+    - https://192.168.0.10:8006
+    - https://pve02.example.com:8006/
+```
+
+A trailing slash is optional and normalized automatically. Every host must use
+`http` or `https` and must belong to the same PVE cluster. The exporter checks
+endpoint liveness every five seconds and distributes API requests across the
+currently reachable endpoints.
+
+Do not combine unrelated clusters in one list. Their API responses share the
+same short-lived client cache and Prometheus registry. Run a separate exporter
+for each cluster.
+
+## Collection interval
+
+`proxmox.interval` is the positive number of seconds between collection cycles.
+The first cycle runs during application startup. Prometheus scrapes only the
+values already present in the registry, so its scrape interval does not control
+the PVE API request rate.
+
+## Collector switches
+
+All switches are booleans. Omitted switches default to `false`.
+
+| Setting | Collector | Main metric prefix |
+| --- | --- | --- |
+| `clusterState` | Cluster mode, node count, and quorum | `pve_cluster_` |
+| `nodeStatus` | Node health and resource usage | `pve_node_` |
+| `qemu` | QEMU virtual machines | `pve_vm_` |
+| `lxc` | LXC containers | `pve_ct_` |
+| `disk` | Physical disk health and metadata | `pve_node_disk_` |
+| `zfs` | ZFS pool topology, state, and errors | `pve_node_zfs_` |
+| `storage` | Enabled PVE storage status and capacity | `pve_storage_` |
+| `subscription` | Node subscription information | `pve_node_subscription_` |
+| `sdn` | Software-defined network state | `pve_sdn_` |
+
+The exporter-level `pve_metrics_collection_duration_seconds` summary is always
+registered.
+
+## Validation notes
+
+- `host` must be a literal IP address.
+- At least one Proxmox API host is required.
+- API host URLs require an `http` or `https` scheme and an IP or DNS hostname.
+- Token ID and secret cannot be empty.
+- Keep `proxmox.interval` greater than zero and `logLevel` between `0` and `6`.

docs/metrics.md

@@ -0,0 +1,204 @@
+# Metrics Reference
+
+All exporter-specific metric names start with `pve_`. The Go Prometheus client
+also exposes its standard `go_*`, `process_*`, and HTTP handler metrics.
+
+Metric names follow the [Prometheus naming conventions](https://prometheus.io/docs/practices/naming/):
+base units only (seconds, bytes, ratios) and a `_total` suffix on every metric
+exported as a counter. Counters hold the absolute value reported by the PVE API
+and reset when the guest or node restarts, so query them with `rate()` or
+`increase()`. All remaining metrics are gauges.
+
+## Common labels and lifecycle
+
+Most resource metrics use these labels:
+
+| Label | Description |
+| --- | --- |
+| `cluster` | PVE cluster name. A standalone installation uses `Standalone node - <node>`. |
+| `node` | Proxmox node name. |
+| `vmid` | QEMU VM or LXC container ID. |
+| `name` | Guest name. |
+
+Metric label sets are retained for five minutes after their last update and are
+then removed. Consequently, metrics available only while a guest is running can
+remain visible for up to five minutes after it stops.
+
+## Exporter
+
+| Metric | Labels | Description |
+| --- | --- | --- |
+| `pve_metrics_collection_duration_seconds` | `collector` | Summary of successful collector execution time in seconds. |
+
+## Cluster state
+
+Enabled with `metrics.clusterState`. Labels: `cluster`.
+
+| Metric | Description |
+| --- | --- |
+| `pve_cluster_mode` | `1` when PVE is configured as a cluster, `0` for a standalone node. A configured one-node cluster still reports `1`. |
+| `pve_cluster_nodes` | Number of nodes reported by the cluster status record. Standalone mode reports `0`. |
+| `pve_cluster_quorate` | Cluster quorum state: `1` quorate, `0` not quorate or standalone. |
+
+## Node status
+
+Enabled with `metrics.nodeStatus`. Unless noted otherwise, labels are `cluster`
+and `node`.
+
+| Metric | Description |
+| --- | --- |
+| `pve_node_state` | Node online state: `1` online, `0` offline. |
+| `pve_node_uptime_seconds` | Node uptime in seconds. |
+| `pve_node_cpus` | Number of logical CPUs. |
+| `pve_node_cpu_usage_ratio` | CPU usage reported by PVE as a ratio between `0` and `1`. |
+| `pve_node_memory_total_bytes` | Total memory in bytes. |
+| `pve_node_memory_used_bytes` | Used memory in bytes. |
+| `pve_node_memory_free_bytes` | Free memory in bytes. |
+| `pve_node_ksm_bytes` | Memory shared by Kernel Same-page Merging in bytes. |
+| `pve_node_cgroup_mode` | PVE cgroup mode. |
+| `pve_node_load1` | One-minute load average. |
+| `pve_node_load5` | Five-minute load average. |
+| `pve_node_load15` | Fifteen-minute load average. |
+| `pve_node_rootfs_free_bytes` | Root filesystem free bytes. |
+| `pve_node_rootfs_used_bytes` | Root filesystem used bytes. |
+| `pve_node_rootfs_total_bytes` | Root filesystem total bytes. |
+| `pve_node_rootfs_avail_bytes` | Root filesystem bytes available to unprivileged processes. |
+| `pve_node_time_seconds` | Node UTC Unix timestamp. |
+| `pve_node_localtime_seconds` | Node local Unix timestamp. |
+| `pve_node_cpu_info` | Constant `1` carrying `flags`, `cores`, `model`, `sockets`, `cpus`, and `hvm` labels. |
+| `pve_node_system_info` | Constant `1` carrying `kversion`, `pveversion`, `machine`, `sysname`, and `release` labels. |
+
+## Physical disks
+
+Enabled with `metrics.disk`. Labels are `cluster`, `node`, `wwn`, `type`,
+`model`, `serial`, `vendor`, `used`, and `osd_id`.
+
+| Metric | Description |
+| --- | --- |
+| `pve_node_disk_healthy` | SMART health state: `1` for `OK` or `PASSED`, otherwise `0`. |
+| `pve_node_disk_wearout_percent` | Device wearout percentage when supplied by the PVE API. |
+| `pve_node_disk_size_bytes` | Physical disk size in bytes. |
+
+## ZFS
+
+Enabled with `metrics.zfs`. The exporter discovers all pools on every node,
+retrieves each pool detail, and recursively flattens pool, vdev, section, cache,
+and leaf-device entries.
+
+Labels are `cluster`, `node`, `pool`, `component`, `path`, and `leaf`.
+`component` is the current topology entry, `path` identifies its complete
+hierarchy, and `leaf` is `true` for a device entry.
+
+| Metric | Description |
+| --- | --- |
+| `pve_node_zfs_state` | Numeric component state: `0=UNKNOWN`, `1=ONLINE`, `2=DEGRADED`, `3=FAULTED`, `4=OFFLINE`, `5=REMOVED`, `6=UNAVAIL`. |
+| `pve_node_zfs_read_errors_total` | ZFS read error count reported for the component. |
+| `pve_node_zfs_write_errors_total` | ZFS write error count reported for the component. |
+| `pve_node_zfs_checksum_errors_total` | ZFS checksum error count reported for the component. |
+
+ZFS section entries that do not contain a counter do not produce a false zero
+series. A counter explicitly returned as zero is exported normally.
+
+Example alert expressions:
+
+```promql
+pve_node_zfs_state != 1
+```
+
+```promql
+(pve_node_zfs_read_errors_total > 0)
+or (pve_node_zfs_write_errors_total > 0)
+or (pve_node_zfs_checksum_errors_total > 0)
+```
+
+## PVE storage
+
+Enabled with `metrics.storage`. Disabled storages are skipped. Labels are
+`cluster`, `node`, `storage`, `type`, `content`, and `shared`.
+
+| Metric | Description |
+| --- | --- |
+| `pve_storage_up` | Storage active state: `1` active, `0` inactive. |
+| `pve_storage_total_bytes` | Total storage capacity in bytes. |
+| `pve_storage_avail_bytes` | Available storage capacity in bytes. |
+| `pve_storage_used_bytes` | Used storage capacity in bytes. |
+
+## Node subscriptions
+
+Enabled with `metrics.subscription`. The common labels are `cluster` and
+`node`; `pve_node_subscription_info` also has `productname` and `serverid`.
+
+| Metric | Description |
+| --- | --- |
+| `pve_node_subscription_info` | Constant `1` carrying subscription product and server identifiers. |
+| `pve_node_subscription_status` | `0` for API status `notfound`, `1` for `active`, `2` for expired or otherwise unusable states such as `invalid`, `suspended`, and `new`. |
+| `pve_node_subscription_registration_timestamp_seconds` | Subscription registration date as a Unix timestamp, when available. |
+| `pve_node_subscription_next_due_timestamp_seconds` | Next due date as a Unix timestamp, when available. |
+| `pve_node_subscription_sockets` | Number of covered sockets reported by PVE. |
+
+The PVE API cannot distinguish a node that never had a subscription from one
+whose key was removed; both appear as `notfound` and therefore use value `0`.
+
+## SDN
+
+Enabled with `metrics.sdn`. Labels are `cluster`, `node`, `sdn`, and `sdn_id`.
+
+| Metric | Description |
+| --- | --- |
+| `pve_sdn_state` | `1` when the SDN resource status is `ok`, otherwise `0`. |
+
+PVE 8 reports SDN zones as `sdn` cluster resources, PVE 9 reports them as
+`network` resources (zones and fabrics). Both are exported as `pve_sdn_state`;
+the `sdn` label holds the zone or fabric name and `sdn_id` the resource ID,
+which differs between the two formats (`sdn/<node>/<zone>` versus
+`network/<node>/<network-type>/<name>`).
+
+## LXC containers
+
+Enabled with `metrics.lxc`. Templates are skipped. Labels are `cluster`,
+`node`, `vmid`, and `name`.
+
+| Metric | Description |
+| --- | --- |
+| `pve_ct_state` | `1` running, `0` stopped. |
+| `pve_ct_uptime_seconds` | Uptime in seconds; updated only while running. |
+| `pve_ct_cpus` | Configured CPU count. |
+| `pve_ct_cpu_usage_ratio` | CPU usage reported by PVE as a ratio between `0` and `1`; updated only while running. |
+| `pve_ct_memory_total_bytes` | Configured memory limit in bytes. |
+| `pve_ct_memory_used_bytes` | Used memory in bytes; updated only while running. |
+| `pve_ct_network_receive_bytes_total` | Received bytes since container start; updated only while running. |
+| `pve_ct_network_transmit_bytes_total` | Transmitted bytes since container start; updated only while running. |
+| `pve_ct_disk_read_bytes_total` | Disk bytes read; updated only while running. |
+| `pve_ct_disk_write_bytes_total` | Disk bytes written; updated only while running. |
+| `pve_ct_disk_used_bytes` | Used root disk bytes; updated only while running. |
+| `pve_ct_disk_size_bytes` | Configured root disk size in bytes. |
+| `pve_ct_swap_used_bytes` | Used swap in bytes; updated only while running. |
+
+## QEMU virtual machines
+
+Enabled with `metrics.qemu`. Templates are skipped. Base labels are `cluster`,
+`node`, `vmid`, and `name`.
+
+| Metric | Extra label | Description |
+| --- | --- | --- |
+| `pve_vm_state` | none | `1` running, `0` stopped. |
+| `pve_vm_uptime_seconds` | none | Uptime in seconds; updated only while running. |
+| `pve_vm_agent` | none | QEMU guest agent state reported by PVE; updated only while running. |
+| `pve_vm_cpus` | none | Configured CPU count. |
+| `pve_vm_cpu_usage_ratio` | none | CPU usage reported by PVE as a ratio between `0` and `1`; updated only while running. |
+| `pve_vm_memory_total_bytes` | none | Configured maximum memory in bytes. |
+| `pve_vm_memory_used_bytes` | none | Used memory in bytes; updated only while running. |
+| `pve_vm_disk_used_bytes` | none | Root disk usage reported by PVE. |
+| `pve_vm_disk_size_bytes` | none | Configured maximum root disk size in bytes. |
+| `pve_vm_network_receive_bytes_total` | `interface` | Bytes received by a virtual interface. |
+| `pve_vm_network_transmit_bytes_total` | `interface` | Bytes transmitted by a virtual interface. |
+| `pve_vm_disk_read_operations_total` | `device` | Successful read operations. |
+| `pve_vm_disk_write_operations_total` | `device` | Successful write operations. |
+| `pve_vm_disk_read_bytes_total` | `device` | Bytes read from the block device. |
+| `pve_vm_disk_write_bytes_total` | `device` | Bytes written to the block device. |
+| `pve_vm_disk_failed_read_operations_total` | `device` | Failed read operations. |
+| `pve_vm_disk_failed_write_operations_total` | `device` | Failed write operations. |
+| `pve_vm_disk_read_time_seconds_total` | `device` | Total block-device read time in seconds. |
+| `pve_vm_disk_write_time_seconds_total` | `device` | Total block-device write time in seconds. |
+
+Interface and block-device metrics are collected only for running VMs.

docs/operations.md

@@ -0,0 +1,138 @@
+# Operations and Troubleshooting
+
+## Native execution
+
+Build and run from the repository root:
+
+```sh
+go build -o pve-exporter .
+./pve-exporter -config config.yaml
+```
+
+The default configuration path is `config.yaml`. The HTTP listener is formed
+from the top-level `host` and `port` settings. Only `/metrics` is registered.
+
+Verify the exporter locally:
+
+```sh
+curl http://localhost:9090/metrics
+```
+
+## Docker
+
+Pull and run the published image:
+
+```sh
+docker pull gitea.lostak.dev/lostakj/pve-exporter:latest
+docker run --rm -p 9090:9090 \
+  -v "$PWD/config.yaml:/opt/config.yaml:ro" \
+  gitea.lostak.dev/lostakj/pve-exporter:latest
+```
+
+The image contains the example configuration at `/opt/config.yaml`, but that
+file has no API hosts or real credentials. Mount a configured file at the same
+path for normal use.
+
+## Prometheus
+
+When Prometheus and the exporter share a container network:
+
+```yaml
+scrape_configs:
+  - job_name: pve-exporter
+    scrape_interval: 30s
+    static_configs:
+      - targets: [pve-exporter:9090]
+```
+
+The Prometheus scrape interval and `proxmox.interval` are independent. A scrape
+does not contact PVE; it reads the latest values collected in the background.
+
+For alerting, load
+[`examples/pve-exporter.rules.yml`](../examples/pve-exporter.rules.yml) through
+Prometheus `rule_files`. The [alerting guide](alerting.md) lists the required
+collectors and a container mount example.
+
+## API endpoint availability
+
+At startup and every five seconds, the exporter requests `api2/json/` from each
+configured PVE host. Requests are distributed round-robin across endpoints that
+passed the latest check.
+
+Use multiple hosts only for nodes in the same cluster:
+
+```yaml
+proxmox:
+  hosts:
+    - https://192.168.0.10:8006
+    - https://pve02.example.com:8006
+```
+
+IP and DNS hosts work with or without a trailing slash. If every endpoint is
+unavailable, collectors log `All API endpoints are unreachable.` and retain
+their previous series until the five-minute stale-metric timeout removes them.
+
+## Logging
+
+Set `logLevel` between `0` and `6`:
+
+| Value | Level |
+| --- | --- |
+| `0` | Panic |
+| `1` | Fatal |
+| `2` | Error |
+| `3` | Warn |
+| `4` | Info |
+| `5` | Debug |
+| `6` | Trace |
+
+`Info` is suitable for normal operation. `Debug` or `Trace` can help diagnose
+endpoint availability and individual collector execution.
+
+## Common problems
+
+### Authentication failed
+
+Check that `tokenId` contains the full `user@realm!token-name` identifier and
+that `secret` contains the token secret rather than the user password. Assign
+the `PVEAuditor` role to both the user and token.
+
+### Permission or empty cluster-status errors
+
+The `/cluster/status` endpoint is used by every collector to identify nodes and
+the cluster label. Ensure the token can audit `/` and the relevant nodes. The
+exporter reports a specific permission hint when the endpoint returns an empty
+array.
+
+### No metrics from a collector
+
+Confirm that its boolean switch under `proxmox.metrics` is `true`. Some series
+are intentionally conditional:
+
+- VM and LXC templates are skipped.
+- Runtime guest metrics are updated only while the guest is running.
+- Disabled PVE storages are skipped.
+- Optional ZFS counters are omitted when the API does not return them.
+- Subscription dates are omitted when they are unavailable.
+
+### ZFS collection errors
+
+Enable `proxmox.metrics.zfs`, confirm that the node has ZFS utilities and pools,
+and verify that the token can call both `/nodes/{node}/disks/zfs` and each pool
+detail endpoint.
+
+### Metrics remain after a resource disappears
+
+Collector metrics use a five-minute TTL. A guest, disk, storage, or ZFS topology
+entry that disappears can therefore remain visible for up to five minutes.
+
+## Security notes
+
+- Keep `config.yaml` readable only by the exporter account because it contains
+  the API token secret.
+- The HTTP endpoint has no authentication; bind to a protected interface or
+  restrict access with a firewall or reverse proxy.
+- The PVE client currently disables TLS certificate verification to support the
+  default self-signed certificates. Use the exporter only across trusted
+  networks or a protected tunnel.
+- Prefer the read-only `PVEAuditor` role and do not use a root token.

examples/pve-exporter.rules.yml

@@ -0,0 +1,526 @@
+groups:
+  - name: proxmox.rules
+    rules:
+      - alert: PveNodeFilesystemFillingUp
+        expr: |-
+          (
+            pve_node_rootfs_avail_bytes / pve_node_rootfs_total_bytes * 100 < 15
+          and
+            predict_linear(pve_node_rootfs_avail_bytes[6h], 24 * 60 * 60) < 0
+          )
+        for: 1h
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox root filesystem is predicted to run out of space within 24 hours.
+          description: >-
+            Filesystem on node {{ $labels.node }} in the {{ $labels.cluster }}
+            cluster has only {{ printf "%.2f" $value }}% available space left
+            and is filling up.
+
+      - alert: PveGuestCpuSaturated
+        expr: |-
+          pve_vm_cpu_usage_ratio > 0.95 or pve_ct_cpu_usage_ratio > 0.95
+        for: 1h
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox guest CPU has been saturated for more than an hour.
+          description: >-
+            Guest {{ $labels.name }} with ID {{ $labels.vmid }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster has sustained
+            CPU usage above 95%.
+
+      - alert: PveStorageDown
+        expr: |-
+          pve_storage_up == 0
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox storage is down.
+          description: >-
+            Storage {{ $labels.storage }} on node {{ $labels.node }} in the
+            {{ $labels.cluster }} cluster is down. This can severely impact
+            running guests.
+
+      - alert: PveStorageFillingUp
+        expr: |-
+          (
+            pve_storage_avail_bytes / pve_storage_total_bytes * 100 < 15
+          and
+            predict_linear(pve_storage_avail_bytes[6h], 24 * 60 * 60) < 0
+          )
+        for: 1h
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox storage is predicted to run out of space within 24 hours.
+          description: >-
+            Storage {{ $labels.storage }} on node {{ $labels.node }} in the
+            {{ $labels.cluster }} cluster has only {{ printf "%.2f" $value }}%
+            available space left and is filling up.
+
+      - alert: PveNodeDown
+        expr: |-
+          pve_node_state == 0
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox node is down.
+          description: >-
+            Node {{ $labels.node }} in the {{ $labels.cluster }} cluster is down.
+
+      - alert: PveNodeCpuSaturated
+        expr: |-
+          pve_node_cpu_usage_ratio > 0.95
+        for: 1h
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox node CPU has been saturated for more than an hour.
+          description: >-
+            CPU usage on node {{ $labels.node }} in the {{ $labels.cluster }}
+            cluster has remained above 95%. This can affect guests running on
+            the node.
+
+      - alert: PveSdnDown
+        expr: |-
+          pve_sdn_state == 0
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox software-defined network is down.
+          description: >-
+            SDN resource {{ $labels.sdn }} on node {{ $labels.node }} in the
+            {{ $labels.cluster }} cluster is down.
+
+      - alert: PveClusterClockSkew
+        expr: |-
+          max by (cluster) (pve_node_localtime_seconds)
+          - min by (cluster) (pve_node_localtime_seconds) > 15
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox cluster clock skew is too high.
+          description: >-
+            Clock skew between nodes in the {{ $labels.cluster }} cluster is
+            greater than 15 seconds. Cluster functions may be affected.
+
+      - alert: PveNodeSubscriptionExpiringSoon
+        expr: |-
+          (
+            pve_node_subscription_next_due_timestamp_seconds > time()
+          and
+            pve_node_subscription_next_due_timestamp_seconds < time() + (30 * 24 * 60 * 60)
+          )
+          and on (cluster, node)
+            (pve_node_subscription_status == 1)
+        for: 15m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox node subscription expires in less than 30 days.
+          description: >-
+            Subscription for node {{ $labels.node }} in the
+            {{ $labels.cluster }} cluster expires at
+            {{ $value | humanizeTimestamp }}.
+
+      - alert: PveNodeDiskFailed
+        expr: |-
+          pve_node_disk_healthy == 0
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox node disk failed.
+          description: |-
+            A disk on node {{ $labels.node }} in the {{ $labels.cluster }} cluster is in a failed state.
+            WWN: {{ $labels.wwn }}
+            Serial: {{ $labels.serial }}
+            Model: {{ $labels.model }}
+
+            Immediate device replacement is required to ensure safe operation of the node.
+
+      - alert: PveNodeDiskWearout
+        expr: |-
+          pve_node_disk_wearout_percent < 5
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox node disk has less than 5% wear life remaining.
+          description: |-
+            A disk on node {{ $labels.node }} in the {{ $labels.cluster }} cluster is showing significant wear and may fail soon.
+            WWN: {{ $labels.wwn }}
+            Serial: {{ $labels.serial }}
+            Model: {{ $labels.model }}
+            Current wear level: {{ $value }}%
+
+            Replace the disk to reduce the risk of data loss.
+
+      - alert: PveNodeZfsPoolUnknown
+        expr: |-
+          pve_node_zfs_state == 0
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox ZFS pool component state is unknown.
+          description: >-
+            ZFS component {{ $labels.path }} in pool {{ $labels.pool }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster has been in
+            the UNKNOWN state for at least five minutes.
+
+      - alert: PveNodeZfsPoolDegraded
+        expr: |-
+          pve_node_zfs_state == 2
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox ZFS pool component is degraded.
+          description: >-
+            ZFS component {{ $labels.path }} in pool {{ $labels.pool }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster has been
+            DEGRADED for at least five minutes.
+
+      - alert: PveNodeZfsPoolFaulted
+        expr: |-
+          pve_node_zfs_state == 3
+        for: 5m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox ZFS pool component has faulted.
+          description: >-
+            ZFS component {{ $labels.path }} in pool {{ $labels.pool }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster has been
+            FAULTED for at least five minutes.
+
+      - alert: PveNodeZfsPoolOffline
+        expr: |-
+          pve_node_zfs_state == 4
+        for: 5m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox ZFS pool component is offline.
+          description: >-
+            ZFS component {{ $labels.path }} in pool {{ $labels.pool }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster has been
+            OFFLINE for at least five minutes.
+
+      - alert: PveNodeZfsPoolRemoved
+        expr: |-
+          pve_node_zfs_state == 5
+        for: 5m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox ZFS pool component was removed.
+          description: >-
+            ZFS component {{ $labels.path }} in pool {{ $labels.pool }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster has been
+            REMOVED for at least five minutes.
+
+      - alert: PveNodeZfsPoolUnavailable
+        expr: |-
+          pve_node_zfs_state == 6
+        for: 5m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox ZFS pool component is unavailable.
+          description: >-
+            ZFS component {{ $labels.path }} in pool {{ $labels.pool }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster has been
+            UNAVAIL for at least five minutes.
+
+      - alert: PveNodeZfsReadErrors
+        expr: |-
+          pve_node_zfs_read_errors_total > 0
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox ZFS component reports read errors.
+          description: >-
+            ZFS component {{ $labels.path }} in pool {{ $labels.pool }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster reports
+            {{ $value | humanize }} read errors.
+
+      - alert: PveNodeZfsWriteErrors
+        expr: |-
+          pve_node_zfs_write_errors_total > 0
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox ZFS component reports write errors.
+          description: >-
+            ZFS component {{ $labels.path }} in pool {{ $labels.pool }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster reports
+            {{ $value | humanize }} write errors.
+
+      - alert: PveNodeZfsChecksumErrors
+        expr: |-
+          pve_node_zfs_checksum_errors_total > 0
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox ZFS component reports checksum errors.
+          description: >-
+            ZFS component {{ $labels.path }} in pool {{ $labels.pool }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster reports
+            {{ $value | humanize }} checksum errors.
+
+      - alert: PveNodeMemoryOvercommit
+        expr: |-
+          (
+            sum by (cluster, node) (
+              (
+                pve_vm_memory_total_bytes
+                and on (cluster, node, vmid, name) (pve_vm_state == 1)
+              )
+              or
+              (
+                pve_ct_memory_total_bytes
+                and on (cluster, node, vmid, name) (pve_ct_state == 1)
+              )
+            )
+            / pve_node_memory_total_bytes * 100 - 100
+          ) > 20
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox node memory is overcommitted by more than 20%.
+          description: >-
+            Node {{ $labels.node }} in the {{ $labels.cluster }} cluster has
+            {{ printf "%.2f" $value }}% more memory allocated to running guests
+            than is physically available. Concurrent guest memory demand can
+            exhaust node memory and disrupt services.
+
+      - alert: PveClusterQuorumLost
+        expr: |-
+          (pve_cluster_mode == 1)
+          and on (cluster)
+          (pve_cluster_quorate == 0)
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox cluster has lost quorum.
+          description: >-
+            The {{ $labels.cluster }} cluster has not been quorate for at least
+            one minute. Cluster operations and guest availability may be
+            affected.
+
+      - alert: PveUnexpectedStandaloneMode
+        expr: |-
+          pve_cluster_mode == 0
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox is running in standalone mode.
+          description: >-
+            {{ $labels.cluster }} has reported standalone mode for at least five
+            minutes. Disable this rule when standalone operation is expected.
+
+      - alert: PveNodeMemoryHigh
+        expr: |-
+          pve_node_memory_used_bytes / pve_node_memory_total_bytes > 0.90
+        for: 15m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox node memory usage is high.
+          description: >-
+            Node {{ $labels.node }} in the {{ $labels.cluster }} cluster has
+            used {{ $value | humanizePercentage }} of its memory for at least
+            15 minutes.
+
+      - alert: PveNodeFilesystemLowSpace
+        expr: |-
+          pve_node_rootfs_avail_bytes / pve_node_rootfs_total_bytes < 0.10
+        for: 5m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox root filesystem has less than 10% space available.
+          description: >-
+            The root filesystem on node {{ $labels.node }} in the
+            {{ $labels.cluster }} cluster has only
+            {{ $value | humanizePercentage }} available space left.
+
+      - alert: PveStorageLowSpace
+        expr: |-
+          pve_storage_avail_bytes / pve_storage_total_bytes < 0.10
+        for: 5m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox storage has less than 10% space available.
+          description: >-
+            Storage {{ $labels.storage }} on node {{ $labels.node }} in the
+            {{ $labels.cluster }} cluster has only
+            {{ $value | humanizePercentage }} available space left.
+
+      - alert: PveNodeSubscriptionExpired
+        expr: |-
+          pve_node_subscription_status == 2
+        for: 15m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox node subscription has expired or is unusable.
+          description: >-
+            Subscription for node {{ $labels.node }} in the
+            {{ $labels.cluster }} cluster is expired, invalid, suspended, or
+            otherwise unusable.
+
+      - alert: PveNodeSubscriptionMissing
+        expr: |-
+          pve_node_subscription_status == 0
+        for: 15m
+        labels:
+          severity: info
+        annotations:
+          summary: Proxmox node has no subscription.
+          description: >-
+            Node {{ $labels.node }} in the {{ $labels.cluster }} cluster reports
+            no subscription. Disable this rule when an unsubscribed node is
+            expected.
+
+      - alert: PveNodeDiskWearoutWarning
+        expr: |-
+          (pve_node_disk_wearout_percent < 20)
+          and
+          (pve_node_disk_wearout_percent >= 5)
+        for: 15m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox node disk has less than 20% wear life remaining.
+          description: |-
+            A disk on node {{ $labels.node }} in the {{ $labels.cluster }} cluster is approaching its wear limit.
+            WWN: {{ $labels.wwn }}
+            Serial: {{ $labels.serial }}
+            Model: {{ $labels.model }}
+            Current wear level: {{ $value }}%
+
+      - alert: PveGuestMemoryHigh
+        expr: |-
+          pve_vm_memory_used_bytes / pve_vm_memory_total_bytes > 0.90
+          or
+          pve_ct_memory_used_bytes / pve_ct_memory_total_bytes > 0.90
+        for: 15m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox guest memory usage is high.
+          description: >-
+            Guest {{ $labels.name }} with ID {{ $labels.vmid }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster has used
+            {{ $value | humanizePercentage }} of its configured memory for at
+            least 15 minutes.
+
+      - alert: PveGuestDiskFillingUp
+        expr: |-
+          pve_vm_disk_used_bytes / pve_vm_disk_size_bytes > 0.90
+          or
+          pve_ct_disk_used_bytes / pve_ct_disk_size_bytes > 0.90
+        for: 15m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox guest root disk usage is high.
+          description: >-
+            Guest {{ $labels.name }} with ID {{ $labels.vmid }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster has used
+            {{ $value | humanizePercentage }} of its configured root disk.
+
+      - alert: PveContainerSwapUsageHigh
+        expr: |-
+          pve_ct_swap_used_bytes > 256 * 1024 * 1024
+        for: 15m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox container swap usage is high.
+          description: >-
+            Container {{ $labels.name }} with ID {{ $labels.vmid }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster has used
+            {{ $value | humanize1024 }}B of swap for at least 15 minutes.
+
+      - alert: PveVmDiskErrorsIncreasing
+        expr: |-
+          increase(pve_vm_disk_failed_read_operations_total[10m]) > 0
+          or
+          increase(pve_vm_disk_failed_write_operations_total[10m]) > 0
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: Proxmox virtual machine reports new disk I/O failures.
+          description: >-
+            VM {{ $labels.name }} with ID {{ $labels.vmid }} on node
+            {{ $labels.node }} in the {{ $labels.cluster }} cluster reports new
+            failed I/O operations on device {{ $labels.device }}.
+
+      - alert: PveNodeClockDrift
+        expr: |-
+          abs(pve_node_time_seconds - time()) > 60
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: Proxmox node clock differs from Prometheus by more than one minute.
+          description: >-
+            The clock on node {{ $labels.node }} in the {{ $labels.cluster }}
+            cluster differs from the Prometheus server by
+            {{ $value | humanizeDuration }}.
+
+      - alert: PveNodeRestarted
+        expr: |-
+          resets(pve_node_uptime_seconds[15m]) > 0
+        for: 0m
+        labels:
+          severity: info
+        annotations:
+          summary: Proxmox node restarted recently.
+          description: >-
+            Uptime for node {{ $labels.node }} in the {{ $labels.cluster }}
+            cluster was reset during the last 15 minutes.
+
+      - alert: PveCollectorSlow
+        expr: |-
+          rate(pve_metrics_collection_duration_seconds_sum[5m])
+          / rate(pve_metrics_collection_duration_seconds_count[5m]) > 5
+        for: 10m
+        labels:
+          severity: warning
+        annotations:
+          summary: PVE exporter collector is slow.
+          description: >-
+            Collector {{ $labels.collector }} has taken an average of
+            {{ printf "%.1f" $value }} s per successful run for at least ten
+            minutes.
+
+      - alert: PveExporterDown
+        expr: |-
+          up{job="pve-exporter"} == 0
+        for: 2m
+        labels:
+          severity: critical
+        annotations:
+          summary: PVE Exporter is unreachable.
+          description: >-
+            Prometheus has been unable to scrape PVE Exporter instance
+            {{ $labels.instance }} for at least two minutes.

metrics/pve_metrics_manager.go

@@ -23,8 +23,8 @@ type PveMetricsManager struct {
 	collectors []PveMetricsCollector // Metrics collector instances.
 	registry   *TTLRegistry          // Registry which handles autoamtic dangling metrics deletion.
 
-	latencySummary *prometheus.SummaryVec // Collection latency summary.
-	interval       int                    // Collection interval.
+	durationSummary *prometheus.SummaryVec // Collection duration summary.
+	interval        int                    // Collection interval.
 
 	stop chan struct{} // Stop channel which is used in ticker.
 }
@@ -80,10 +80,10 @@ func NewPveMetricsManager(apiClient *proxmox.PveApiClient, conf *configuration.P
 		c.RegisterCollector(NewPveVirtualMachineCollector(apiClient, c.registry))
 	}
 
-	// Metrics collection latency summary.
-	c.latencySummary = promauto.NewSummaryVec(prometheus.SummaryOpts{
-		Name: "pve_metrics_collection_latency_ms",
-		Help: "Summary of metrics collection latency milliseconds from PVE API.",
+	// Metrics collection duration summary.
+	c.durationSummary = promauto.NewSummaryVec(prometheus.SummaryOpts{
+		Name: "pve_metrics_collection_duration_seconds",
+		Help: "Summary of the PVE API metrics collection duration in seconds.",
 	}, []string{"collector"})
 
 	c.registry.StartCleanupLoop(5 * time.Second)
@@ -102,7 +102,7 @@ func (c *PveMetricsManager) collectMetrics() {
 		} else {
 			latency := time.Since(start)
 			log.Tracef("Finished collecting '%s' metrics after %s.", collector.GetName(), utils.HumanDuration(latency))
-			c.latencySummary.With(prometheus.Labels{"collector": collector.GetName()}).Observe(float64(latency.Milliseconds()))
+			c.durationSummary.With(prometheus.Labels{"collector": collector.GetName()}).Observe(latency.Seconds())
 		}
 	}
 }

metrics/pve_metrics_naming_test.go

@@ -0,0 +1,72 @@
+package metrics
+
+import (
+	"testing"
+
+	"github.com/prometheus/client_golang/prometheus"
+	"github.com/prometheus/client_golang/prometheus/testutil/promlint"
+)
+
+// Verifies that all exported PVE metrics follow the Prometheus naming conventions
+// (base units, '_total' suffix on counters, '_info' suffix on info metrics, ...)
+// so that tools such as promtool or Grafana do not report naming problems.
+func TestPveMetricNamesFollowPrometheusConventions(t *testing.T) {
+	registry := prometheus.NewRegistry()
+
+	// Collectors register their metrics on the default registerer, so it is
+	// temporarily replaced by a dedicated registry.
+	defaultRegisterer := prometheus.DefaultRegisterer
+	prometheus.DefaultRegisterer = registry
+	defer func() { prometheus.DefaultRegisterer = defaultRegisterer }()
+
+	// Collector constructors only create metrics, the API client is used during
+	// collection only, so a nil client is enough here.
+	ttlRegistry := NewTTLRegistry()
+	NewPveClusterStateCollector(nil, ttlRegistry)
+	NewPveNodeStatusCollector(nil, ttlRegistry)
+	NewPveSubscriptionCollector(nil, ttlRegistry)
+	NewPveNodeDiskCollector(nil, ttlRegistry)
+	NewPveNodeZfsCollector(nil, ttlRegistry)
+	NewPveSdnCollector(nil, ttlRegistry)
+	NewPveStorageCollector(nil, ttlRegistry)
+	NewPveContainerCollector(nil, ttlRegistry)
+	NewPveVirtualMachineCollector(nil, ttlRegistry)
+
+	// Metric vectors are exported only once they hold a label set.
+	for _, metric := range ttlRegistry.metrics {
+		switch m := metric.(type) {
+		case *TTLGaugeVec:
+			m.With(emptyLabels(m.labelNames)).Set(0)
+		case *TTLCounterVec:
+			m.With(emptyLabels(m.labelNames)).Set(0)
+		default:
+			t.Fatalf("Unknown TTL metric type %T.", metric)
+		}
+	}
+
+	families, err := registry.Gather()
+	if err != nil {
+		t.Fatalf("Unable to gather metrics. Error: %s.", err)
+	}
+	if len(families) != len(ttlRegistry.metrics) {
+		t.Fatalf("Gathered %d metric families but %d metrics are registered.", len(families), len(ttlRegistry.metrics))
+	}
+
+	problems, err := promlint.NewWithMetricFamilies(families).Lint()
+	if err != nil {
+		t.Fatalf("Unable to lint metrics. Error: %s.", err)
+	}
+
+	for _, problem := range problems {
+		t.Errorf("Metric '%s' violates the Prometheus naming conventions: %s.", problem.Metric, problem.Text)
+	}
+}
+
+// emptyLabels builds a label set with all given label names set to an empty value.
+func emptyLabels(labelNames []string) prometheus.Labels {
+	labels := prometheus.Labels{}
+	for _, name := range labelNames {
+		labels[name] = ""
+	}
+	return labels
+}

metrics/pve_node_container_collector.go

@@ -21,11 +21,11 @@ type PveContainerCollector struct {
 	memBytes     *TTLGaugeVec // Container memory in bytes prometheus gauge.
 	memBytesUsed *TTLGaugeVec // Container memory usage in bytes prometheus gauge.
 
-	netReceive  *TTLGaugeVec // Container network RX in bytes prometheus gauge.
-	netTransmit *TTLGaugeVec // Container network TX in bytes prometheus gauge.
+	netReceive  *TTLCounterVec // Container received network traffic in bytes prometheus counter.
+	netTransmit *TTLCounterVec // Container transmitted network traffic in bytes prometheus counter.
 
-	diskWrite *TTLGaugeVec // Container disk written in bytes prometheus gauge.
-	diskRead  *TTLGaugeVec // Container disk read in bytes prometheus gauge.
+	diskWrite *TTLCounterVec // Container disk written in bytes prometheus counter.
+	diskRead  *TTLCounterVec // Container disk read in bytes prometheus counter.
 
 	disk    *TTLGaugeVec // Container disk space usage in bytes prometheus gauge.
 	diskMax *TTLGaugeVec // Container disk size in bytes prometheus gauge.
@@ -51,8 +51,8 @@ func NewPveContainerCollector(apiClient *proxmox.PveApiClient, registry *TTLRegi
 	// Container uptime.
 	c.uptime = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_ct_uptime",
-			Help: "Container uptime.",
+			Name: "pve_ct_uptime_seconds",
+			Help: "Container uptime in seconds.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
 		5*time.Minute,
@@ -62,7 +62,7 @@ func NewPveContainerCollector(apiClient *proxmox.PveApiClient, registry *TTLRegi
 	// Container CPU count.
 	c.cpu = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_ct_cpu_count",
+			Name: "pve_ct_cpus",
 			Help: "Container CPU count.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
@@ -73,8 +73,8 @@ func NewPveContainerCollector(apiClient *proxmox.PveApiClient, registry *TTLRegi
 	// Container CPU usage.
 	c.cpuUsage = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_ct_cpu_usage",
-			Help: "Container CPU usage.",
+			Name: "pve_ct_cpu_usage_ratio",
+			Help: "Container CPU usage ratio (0-1).",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
 		5*time.Minute,
@@ -84,7 +84,7 @@ func NewPveContainerCollector(apiClient *proxmox.PveApiClient, registry *TTLRegi
 	// Container memory total.
 	c.memBytes = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_ct_mem_total_bytes",
+			Name: "pve_ct_memory_total_bytes",
 			Help: "Container total memory in bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
@@ -95,7 +95,7 @@ func NewPveContainerCollector(apiClient *proxmox.PveApiClient, registry *TTLRegi
 	// Container memory usage.
 	c.memBytesUsed = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_ct_mem_used_bytes",
+			Name: "pve_ct_memory_used_bytes",
 			Help: "Container used memory in bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
@@ -104,10 +104,10 @@ func NewPveContainerCollector(apiClient *proxmox.PveApiClient, registry *TTLRegi
 	c.registry.Register(c.memBytesUsed)
 
 	// Container network RX.
-	c.netReceive = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_ct_network_in_bytes",
-			Help: "Container network RX bytes.",
+	c.netReceive = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_ct_network_receive_bytes_total",
+			Help: "Container received network traffic in bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
 		5*time.Minute,
@@ -115,10 +115,10 @@ func NewPveContainerCollector(apiClient *proxmox.PveApiClient, registry *TTLRegi
 	c.registry.Register(c.netReceive)
 
 	// Container network TX.
-	c.netTransmit = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_ct_network_out_bytes",
-			Help: "Container network TX bytes.",
+	c.netTransmit = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_ct_network_transmit_bytes_total",
+			Help: "Container transmitted network traffic in bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
 		5*time.Minute,
@@ -126,10 +126,10 @@ func NewPveContainerCollector(apiClient *proxmox.PveApiClient, registry *TTLRegi
 	c.registry.Register(c.netTransmit)
 
 	// Container disk written.
-	c.diskWrite = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_ct_disk_wr_bytes",
-			Help: "Container disk written bytes.",
+	c.diskWrite = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_ct_disk_write_bytes_total",
+			Help: "Container written bytes to disk.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
 		5*time.Minute,
@@ -137,10 +137,10 @@ func NewPveContainerCollector(apiClient *proxmox.PveApiClient, registry *TTLRegi
 	c.registry.Register(c.diskWrite)
 
 	// Container disk read.
-	c.diskRead = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_ct_disk_rd_bytes",
-			Help: "Container disk read bytes.",
+	c.diskRead = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_ct_disk_read_bytes_total",
+			Help: "Container read bytes from disk.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
 		5*time.Minute,
@@ -150,8 +150,8 @@ func NewPveContainerCollector(apiClient *proxmox.PveApiClient, registry *TTLRegi
 	// Container disk size.
 	c.disk = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_ct_disk_usage_bytes",
-			Help: "Container disk read bytes.",
+			Name: "pve_ct_disk_used_bytes",
+			Help: "Container used disk space in bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
 		5*time.Minute,

metrics/pve_node_disk_collector.go

@@ -38,13 +38,13 @@ func NewPveNodeDiskCollector(apiClient *proxmox.PveApiClient, registry *TTLRegis
 	// Node disk wearout.
 	c.wearout = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_node_disk_wearout",
-			Help: "Node disk wearout percent.",
+			Name: "pve_node_disk_wearout_percent",
+			Help: "Node disk wearout in percent (0-100).",
 		},
 		[]string{"cluster", "node", "wwn", "type", "model", "serial", "vendor", "used", "osd_id"},
 		5*time.Minute,
 	)
-	c.registry.Register(c.healthy)
+	c.registry.Register(c.wearout)
 
 	// Node disk size in bytes.
 	c.sizeBytes = NewTTLGaugeVec(

metrics/pve_node_sdn_collector.go

@@ -53,7 +53,7 @@ func (c *PveSdnCollector) CollectMetrics() error {
 				labels := prometheus.Labels{
 					"cluster": cluster.GetClusterName(),
 					"node":    node.Name,
-					"sdn":     sdn.SDN,
+					"sdn":     sdn.GetName(),
 					"sdn_id":  sdn.ID,
 				}
 

metrics/pve_node_status_collector.go

@@ -55,8 +55,8 @@ func NewPveNodeStatusCollector(apiClient *proxmox.PveApiClient, registry *TTLReg
 	// Node uptime.
 	c.uptime = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_node_uptime",
-			Help: "Node uptime.",
+			Name: "pve_node_uptime_seconds",
+			Help: "Node uptime in seconds.",
 		},
 		[]string{"cluster", "node"},
 		5*time.Minute,
@@ -66,7 +66,7 @@ func NewPveNodeStatusCollector(apiClient *proxmox.PveApiClient, registry *TTLReg
 	// Node cpu count.
 	c.cpus = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_node_cpu_count",
+			Name: "pve_node_cpus",
 			Help: "Node CPU count.",
 		},
 		[]string{"cluster", "node"},
@@ -77,8 +77,8 @@ func NewPveNodeStatusCollector(apiClient *proxmox.PveApiClient, registry *TTLReg
 	// Node CPU usage.
 	c.cpuUsage = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_node_cpu_usage",
-			Help: "Cluster node CPU usage %.",
+			Name: "pve_node_cpu_usage_ratio",
+			Help: "Node CPU usage ratio (0-1).",
 		},
 		[]string{"cluster", "node"},
 		5*time.Minute,
@@ -220,8 +220,8 @@ func NewPveNodeStatusCollector(apiClient *proxmox.PveApiClient, registry *TTLReg
 	// Node CPU info.
 	c.cpuInfo = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_node_cpuinfo",
-			Help: "Node CPU info.",
+			Name: "pve_node_cpu_info",
+			Help: "Node CPU information.",
 		},
 		[]string{"cluster", "node", "flags", "cores", "model", "sockets", "cpus", "hvm"},
 		5*time.Minute,
@@ -231,8 +231,8 @@ func NewPveNodeStatusCollector(apiClient *proxmox.PveApiClient, registry *TTLReg
 	// Node system info metrics.
 	c.systemInfo = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_node_systeminfo",
-			Help: "Node system info.",
+			Name: "pve_node_system_info",
+			Help: "Node system information.",
 		},
 		[]string{"cluster", "node", "kversion", "pveversion", "machine", "sysname", "release"},
 		5*time.Minute,
@@ -242,8 +242,8 @@ func NewPveNodeStatusCollector(apiClient *proxmox.PveApiClient, registry *TTLReg
 	// Node time info.
 	c.time = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_node_time",
-			Help: "Node time.",
+			Name: "pve_node_time_seconds",
+			Help: "Node UTC time as a unix timestamp in seconds.",
 		},
 		[]string{"cluster", "node"},
 		5*time.Minute,
@@ -253,8 +253,8 @@ func NewPveNodeStatusCollector(apiClient *proxmox.PveApiClient, registry *TTLReg
 	// Node localtime info.
 	c.localTime = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_node_localtime",
-			Help: "Node localtime.",
+			Name: "pve_node_localtime_seconds",
+			Help: "Node local time as a unix timestamp in seconds.",
 		},
 		[]string{"cluster", "node"},
 		5*time.Minute,

metrics/pve_node_subscription_collector.go

@@ -49,8 +49,8 @@ func NewPveSubscriptionCollector(apiClient *proxmox.PveApiClient, registry *TTLR
 	// Node subscription registration date.
 	c.regDate = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_node_subscription_regdate",
-			Help: "Node subscription registration date.",
+			Name: "pve_node_subscription_registration_timestamp_seconds",
+			Help: "Node subscription registration date as a unix timestamp in seconds.",
 		},
 		[]string{"cluster", "node"},
 		5*time.Minute,
@@ -60,8 +60,8 @@ func NewPveSubscriptionCollector(apiClient *proxmox.PveApiClient, registry *TTLR
 	// Node subscription next due date.
 	c.nextDueDate = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_node_subscription_nextduedate",
-			Help: "Node subscription next due date.",
+			Name: "pve_node_subscription_next_due_timestamp_seconds",
+			Help: "Node subscription next due date as a unix timestamp in seconds.",
 		},
 		[]string{"cluster", "node"},
 		5*time.Minute,

metrics/pve_node_virtual_machine_collector.go

@@ -26,20 +26,20 @@ type PveVirtualMachineCollector struct {
 	diskMax *TTLGaugeVec // Virtual machine disk size in bytes prometheus gauge.
 	swap    *TTLGaugeVec // Virtual machine swap usage in bytes prometheus gauge.
 
-	netReceive  *TTLGaugeVec // Virtual machine network receive in bytes prometheus gauge.
-	netTransmit *TTLGaugeVec // Virtual machine network transmit in bytes prometheus gauge.
+	netReceive  *TTLCounterVec // Virtual machine received network traffic in bytes prometheus counter.
+	netTransmit *TTLCounterVec // Virtual machine transmitted network traffic in bytes prometheus counter.
 
-	diskReadOps  *TTLGaugeVec // Virtual machine disk read ops prometheus gauge.
-	diskWriteOps *TTLGaugeVec // Virtual machine disk write ops prometheus gauge.
+	diskReadOps  *TTLCounterVec // Virtual machine disk read operations prometheus counter.
+	diskWriteOps *TTLCounterVec // Virtual machine disk write operations prometheus counter.
 
-	diskReadBytes  *TTLGaugeVec // Virtual machine disk read bytes prometheus gauge.
-	diskWriteBytes *TTLGaugeVec // Virtual machine disk write bytes prometheus gauge.
+	diskReadBytes  *TTLCounterVec // Virtual machine disk read bytes prometheus counter.
+	diskWriteBytes *TTLCounterVec // Virtual machine disk written bytes prometheus counter.
 
-	diskReadTimeNs  *TTLGaugeVec // Virtual machine disk read time total prometheus gauge.
-	diskWriteTimeNs *TTLGaugeVec // Virtual machine disk write time total prometheus gauge.
+	diskReadTime  *TTLCounterVec // Virtual machine disk read time total in seconds prometheus counter.
+	diskWriteTime *TTLCounterVec // Virtual machine disk write time total in seconds prometheus counter.
 
-	diskFailedReadOps  *TTLGaugeVec // Virtual machine disk failed read ops prometheus gauge.
-	diskFailedWriteOps *TTLGaugeVec // Virtual machine disk failed write ops prometheus gauge.
+	diskFailedReadOps  *TTLCounterVec // Virtual machine failed disk read operations prometheus counter.
+	diskFailedWriteOps *TTLCounterVec // Virtual machine failed disk write operations prometheus counter.
 
 	agent *TTLGaugeVec // Virtual machine agent enabled prometheus gauge.
 }
@@ -63,8 +63,8 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	// Virtual machine uptime.
 	c.uptime = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_vm_uptime",
-			Help: "Virtual machine uptime.",
+			Name: "pve_vm_uptime_seconds",
+			Help: "Virtual machine uptime in seconds.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
 		5*time.Minute,
@@ -85,7 +85,7 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	// Virtual machine CPU count.
 	c.cpu = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_vm_cpu_count",
+			Name: "pve_vm_cpus",
 			Help: "Virtual machine CPU count.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
@@ -96,8 +96,8 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	// Virtual machine CPU usage.
 	c.cpuUsage = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_vm_cpu_usage",
-			Help: "Virtual machine CPU usage.",
+			Name: "pve_vm_cpu_usage_ratio",
+			Help: "Virtual machine CPU usage ratio (0-1).",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
 		5*time.Minute,
@@ -107,7 +107,7 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	// Virtual machine memory total.
 	c.memBytes = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_vm_mem_total_bytes",
+			Name: "pve_vm_memory_total_bytes",
 			Help: "Virtual machine total memory in bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
@@ -118,7 +118,7 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	// Virtual machine memory usage.
 	c.memBytesUsed = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_vm_mem_used_bytes",
+			Name: "pve_vm_memory_used_bytes",
 			Help: "Virtual machine used memory in bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
@@ -129,8 +129,8 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	// Virtual machine disk size.
 	c.disk = NewTTLGaugeVec(
 		prometheus.GaugeOpts{
-			Name: "pve_vm_disk_usage_bytes",
-			Help: "Virtual machine disk read bytes.",
+			Name: "pve_vm_disk_used_bytes",
+			Help: "Virtual machine used disk space in bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name"},
 		5*time.Minute,
@@ -149,10 +149,10 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	c.registry.Register(c.diskMax)
 
 	// Virtual machine network receive bytes.
-	c.netReceive = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_vm_network_in_bytes",
-			Help: "Virtual machine network receive in bytes.",
+	c.netReceive = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_vm_network_receive_bytes_total",
+			Help: "Virtual machine received network traffic in bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name", "interface"},
 		5*time.Minute,
@@ -160,10 +160,10 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	c.registry.Register(c.netReceive)
 
 	// Virtual machine network transmit bytes.
-	c.netTransmit = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_vm_network_out_bytes",
-			Help: "Virtual machine network transmit in bytes.",
+	c.netTransmit = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_vm_network_transmit_bytes_total",
+			Help: "Virtual machine transmitted network traffic in bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name", "interface"},
 		5*time.Minute,
@@ -171,10 +171,10 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	c.registry.Register(c.netTransmit)
 
 	// Virtual machine disk read ops.
-	c.diskReadOps = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_vm_disk_rd_operations",
-			Help: "Virtual machine disk read ops.",
+	c.diskReadOps = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_vm_disk_read_operations_total",
+			Help: "Virtual machine disk read operations.",
 		},
 		[]string{"cluster", "node", "vmid", "name", "device"},
 		5*time.Minute,
@@ -182,10 +182,10 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	c.registry.Register(c.diskReadOps)
 
 	// Virtual machine disk write ops.
-	c.diskWriteOps = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_vm_disk_wr_operations",
-			Help: "Virtual machine disk write ops.",
+	c.diskWriteOps = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_vm_disk_write_operations_total",
+			Help: "Virtual machine disk write operations.",
 		},
 		[]string{"cluster", "node", "vmid", "name", "device"},
 		5*time.Minute,
@@ -193,9 +193,9 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	c.registry.Register(c.diskWriteOps)
 
 	// Virtual machine disk read bytes.
-	c.diskReadBytes = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_vm_disk_rd_bytes",
+	c.diskReadBytes = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_vm_disk_read_bytes_total",
 			Help: "Virtual machine disk read bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name", "device"},
@@ -204,9 +204,9 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	c.registry.Register(c.diskReadBytes)
 
 	// Virtual machine disk write bytes.
-	c.diskWriteBytes = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_vm_disk_wr_bytes",
+	c.diskWriteBytes = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_vm_disk_write_bytes_total",
 			Help: "Virtual machine disk write bytes.",
 		},
 		[]string{"cluster", "node", "vmid", "name", "device"},
@@ -215,10 +215,10 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	c.registry.Register(c.diskWriteBytes)
 
 	// Virtual machine failed disk read ops.
-	c.diskFailedReadOps = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_vm_disk_failed_rd_ops",
-			Help: "Virtual machine failed disk read ops.",
+	c.diskFailedReadOps = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_vm_disk_failed_read_operations_total",
+			Help: "Virtual machine failed disk read operations.",
 		},
 		[]string{"cluster", "node", "vmid", "name", "device"},
 		5*time.Minute,
@@ -226,37 +226,37 @@ func NewPveVirtualMachineCollector(apiClient *proxmox.PveApiClient, registry *TT
 	c.registry.Register(c.diskFailedReadOps)
 
 	// Virtual machine failed disk write ops.
-	c.diskFailedWriteOps = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_vm_disk_failed_wr_ops",
-			Help: "Virtual machine failed disk write ops.",
+	c.diskFailedWriteOps = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_vm_disk_failed_write_operations_total",
+			Help: "Virtual machine failed disk write operations.",
 		},
 		[]string{"cluster", "node", "vmid", "name", "device"},
 		5*time.Minute,
 	)
 	c.registry.Register(c.diskFailedWriteOps)
 
-	// Virtual machine disk read time total nanoseconds.
-	c.diskReadTimeNs = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_vm_disk_rd_time_total_ns",
-			Help: "Virtual machine disk read time total in nanoseconds.",
+	// Virtual machine disk read time total seconds.
+	c.diskReadTime = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_vm_disk_read_time_seconds_total",
+			Help: "Virtual machine disk read time total in seconds.",
 		},
 		[]string{"cluster", "node", "vmid", "name", "device"},
 		5*time.Minute,
 	)
-	c.registry.Register(c.diskReadTimeNs)
+	c.registry.Register(c.diskReadTime)
 
-	// Virtual machine disk write time total nanoseconds.
-	c.diskWriteTimeNs = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_vm_disk_wr_time_total_ns",
-			Help: "Virtual machine disk write time total in nanoseconds.",
+	// Virtual machine disk write time total seconds.
+	c.diskWriteTime = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_vm_disk_write_time_seconds_total",
+			Help: "Virtual machine disk write time total in seconds.",
 		},
 		[]string{"cluster", "node", "vmid", "name", "device"},
 		5*time.Minute,
 	)
-	c.registry.Register(c.diskWriteTimeNs)
+	c.registry.Register(c.diskWriteTime)
 
 	return &c
 }
@@ -336,8 +336,8 @@ func (c *PveVirtualMachineCollector) CollectMetrics() error {
 					c.diskFailedReadOps.With(labels).Set(float64(value.FailedRdOperations))
 					c.diskFailedWriteOps.With(labels).Set(float64(value.FailedWrOperations))
 
-					c.diskReadTimeNs.With(labels).Set(float64(value.RdTotalTimeNs))
-					c.diskWriteTimeNs.With(labels).Set(float64(value.WrTotalTimeNs))
+					c.diskReadTime.With(labels).Set(float64(value.RdTotalTimeNs) / float64(time.Second))
+					c.diskWriteTime.With(labels).Set(float64(value.WrTotalTimeNs) / float64(time.Second))
 				}
 			}
 		}

metrics/pve_node_zfs_collector.go

@@ -15,9 +15,9 @@ type PveNodeZfsCollector struct {
 	registry  *TTLRegistry
 
 	state          *TTLGaugeVec
-	readErrors     *TTLGaugeVec
-	writeErrors    *TTLGaugeVec
-	checksumErrors *TTLGaugeVec
+	readErrors     *TTLCounterVec
+	writeErrors    *TTLCounterVec
+	checksumErrors *TTLCounterVec
 }
 
 // zfsMetricComponent is a flattened entry from the recursive ZFS topology.
@@ -50,9 +50,9 @@ func NewPveNodeZfsCollector(apiClient *proxmox.PveApiClient, registry *TTLRegist
 	)
 	c.registry.Register(c.state)
 
-	c.readErrors = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_node_zfs_read_errors",
+	c.readErrors = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_node_zfs_read_errors_total",
 			Help: "ZFS pool component read error count.",
 		},
 		componentLabelNames,
@@ -60,9 +60,9 @@ func NewPveNodeZfsCollector(apiClient *proxmox.PveApiClient, registry *TTLRegist
 	)
 	c.registry.Register(c.readErrors)
 
-	c.writeErrors = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_node_zfs_write_errors",
+	c.writeErrors = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_node_zfs_write_errors_total",
 			Help: "ZFS pool component write error count.",
 		},
 		componentLabelNames,
@@ -70,9 +70,9 @@ func NewPveNodeZfsCollector(apiClient *proxmox.PveApiClient, registry *TTLRegist
 	)
 	c.registry.Register(c.writeErrors)
 
-	c.checksumErrors = NewTTLGaugeVec(
-		prometheus.GaugeOpts{
-			Name: "pve_node_zfs_checksum_errors",
+	c.checksumErrors = NewTTLCounterVec(
+		prometheus.CounterOpts{
+			Name: "pve_node_zfs_checksum_errors_total",
 			Help: "ZFS pool component checksum error count.",
 		},
 		componentLabelNames,

metrics/pve_ttl_metrics.go

@@ -23,6 +23,7 @@ type TTLMetric interface {
 // it is automatically removed from the underlying GaugeVec.
 type TTLGaugeVec struct {
 	gaugeVec   *prometheus.GaugeVec // Underlying Prometheus GaugeVec.
+	labelNames []string             // Label names of the underlying GaugeVec.
 	ttl        time.Duration        // Duration after which an unused label set is considered stale.
 	lastUpdate sync.Map             // Map storing last update time for each label set (key is a sorted labels string).
 }
@@ -31,8 +32,9 @@ type TTLGaugeVec struct {
 // The underlying GaugeVec is registered using promauto.
 func NewTTLGaugeVec(opts prometheus.GaugeOpts, labelNames []string, ttl time.Duration) *TTLGaugeVec {
 	return &TTLGaugeVec{
-		gaugeVec: promauto.NewGaugeVec(opts, labelNames),
-		ttl:      ttl,
+		gaugeVec:   promauto.NewGaugeVec(opts, labelNames),
+		labelNames: labelNames,
+		ttl:        ttl,
 	}
 }
 
@@ -137,3 +139,118 @@ func (r *TTLRegistry) StartCleanupLoop(interval time.Duration) {
 		}
 	}()
 }
+
+// TTLCounterVec exposes cumulative values as Prometheus counters while keeping
+// the same "set an absolute value" usage as TTLGaugeVec. The PVE API reports
+// counters as absolute totals, which prometheus.CounterVec cannot express
+// (it only supports Inc/Add), so the metrics are emitted as constant metrics
+// of type counter. Label sets not updated within the TTL are dropped.
+type TTLCounterVec struct {
+	desc       *prometheus.Desc            // Metric descriptor.
+	labelNames []string                    // Label names in exposition order.
+	ttl        time.Duration               // Duration after which an unused label set is considered stale.
+	mu         sync.RWMutex                // Guards values.
+	values     map[string]*ttlCounterEntry // Current value per label set.
+}
+
+// ttlCounterEntry holds the current value of a single label set.
+type ttlCounterEntry struct {
+	labelValues []string  // Label values in exposition order.
+	value       float64   // Current counter value.
+	lastUpdate  time.Time // Time of the last update.
+}
+
+// TTLCounter is a handle to a single label set of a TTLCounterVec.
+type TTLCounter struct {
+	vec   *TTLCounterVec
+	entry *ttlCounterEntry
+}
+
+// Set stores the current absolute value of the counter.
+func (c *TTLCounter) Set(value float64) {
+	c.vec.mu.Lock()
+	defer c.vec.mu.Unlock()
+	c.entry.value = value
+}
+
+// NewTTLCounterVec creates a new TTLCounterVec and registers it using promauto.
+func NewTTLCounterVec(opts prometheus.CounterOpts, labelNames []string, ttl time.Duration) *TTLCounterVec {
+	c := &TTLCounterVec{
+		desc: prometheus.NewDesc(
+			prometheus.BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
+			opts.Help,
+			labelNames,
+			opts.ConstLabels,
+		),
+		labelNames: labelNames,
+		ttl:        ttl,
+		values:     make(map[string]*ttlCounterEntry),
+	}
+	prometheus.MustRegister(c)
+	return c
+}
+
+// With returns the counter handle for the given label set and records the current
+// time as the last update for those labels.
+func (t *TTLCounterVec) With(labels prometheus.Labels) *TTLCounter {
+	key := labelsKey(labels)
+
+	t.mu.Lock()
+	defer t.mu.Unlock()
+
+	entry, ok := t.values[key]
+	if !ok {
+		labelValues := make([]string, len(t.labelNames))
+		for i, name := range t.labelNames {
+			labelValues[i] = labels[name]
+		}
+		entry = &ttlCounterEntry{labelValues: labelValues}
+		t.values[key] = entry
+	}
+	entry.lastUpdate = time.Now()
+
+	return &TTLCounter{vec: t, entry: entry}
+}
+
+// Delete removes the metric associated with the given label set.
+func (t *TTLCounterVec) Delete(labels prometheus.Labels) bool {
+	key := labelsKey(labels)
+
+	t.mu.Lock()
+	defer t.mu.Unlock()
+
+	if _, ok := t.values[key]; !ok {
+		return false
+	}
+	delete(t.values, key)
+	return true
+}
+
+// Cleanup deletes all label sets that have not been updated within the TTL duration.
+func (t *TTLCounterVec) Cleanup() {
+	now := time.Now()
+
+	t.mu.Lock()
+	defer t.mu.Unlock()
+
+	for key, entry := range t.values {
+		if now.Sub(entry.lastUpdate) > t.ttl {
+			delete(t.values, key)
+		}
+	}
+}
+
+// Describe implements prometheus.Collector.
+func (t *TTLCounterVec) Describe(ch chan<- *prometheus.Desc) {
+	ch <- t.desc
+}
+
+// Collect implements prometheus.Collector.
+func (t *TTLCounterVec) Collect(ch chan<- prometheus.Metric) {
+	t.mu.RLock()
+	defer t.mu.RUnlock()
+
+	for _, entry := range t.values {
+		ch <- prometheus.MustNewConstMetric(t.desc, prometheus.CounterValue, entry.value, entry.labelValues...)
+	}
+}

proxmox/api_client.go

@@ -7,6 +7,7 @@ import (
 	"fmt"
 	"net"
 	"net/http"
+	"strings"
 	"sync"
 	"time"
 
@@ -55,7 +56,7 @@ func NewApiClient(endpoints []string, tokenId string, secret string, checkInterv
 	// Prepare API endpoints.
 	for _, endpoint := range endpoints {
 		apiEndpoint := ApiEndpoint{
-			host:  endpoint,
+			host:  normalizeApiHost(endpoint),
 			alive: false,
 		}
 		instance.endpoints = append(instance.endpoints, &apiEndpoint)
@@ -86,6 +87,11 @@ func NewApiClient(endpoints []string, tokenId string, secret string, checkInterv
 	return &instance
 }
 
+// normalizeApiHost ensures API paths can be appended to either IP- or DNS-based hosts.
+func normalizeApiHost(host string) string {
+	return strings.TrimRight(strings.TrimSpace(host), "/") + "/"
+}
+
 // Check endpoint liveness state.
 func (instance *ApiClient) checkEndpointsLiveness() {
 	// We want to make sure other routines won't make any requests until we have checked for alive connections.

proxmox/api_client_test.go

@@ -0,0 +1,49 @@
+package proxmox
+
+import (
+	"testing"
+	"time"
+)
+
+func TestNewApiClientNormalizesEndpointHosts(t *testing.T) {
+	tests := []struct {
+		name string
+		host string
+		want string
+	}{
+		{
+			name: "IP without trailing slash",
+			host: "https://192.168.0.10:8006",
+			want: "https://192.168.0.10:8006/",
+		},
+		{
+			name: "IP with trailing slash",
+			host: "https://192.168.0.10:8006/",
+			want: "https://192.168.0.10:8006/",
+		},
+		{
+			name: "DNS without trailing slash",
+			host: "https://pve.example.com:8006",
+			want: "https://pve.example.com:8006/",
+		},
+		{
+			name: "DNS with trailing slash",
+			host: "https://pve.example.com:8006/",
+			want: "https://pve.example.com:8006/",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			client := NewApiClient([]string{tt.host}, "token", "secret", time.Second)
+			defer client.httpClient.CloseIdleConnections()
+
+			if got := client.endpoints[0].host; got != tt.want {
+				t.Fatalf("normalized endpoint host = %q, want %q", got, tt.want)
+			}
+			if got := client.endpoints[0].host + "api2/json/"; got != tt.want+"api2/json/" {
+				t.Fatalf("liveness URL = %q, want %q", got, tt.want+"api2/json/")
+			}
+		})
+	}
+}

proxmox/model.go

@@ -39,7 +39,7 @@ type PveClusterStatus struct {
 
 // PveResource represents a generic PVE resource object.
 type PveResource struct {
-	Type   string `mapstructure:"type"`   // Type of resource (e.g., "lxc", "qemu", "node", "storage", "sdn").
+	Type   string `mapstructure:"type"`   // Type of resource (e.g., "lxc", "qemu", "node", "storage", "sdn", "network").
 	Node   string `mapstructure:"node"`   // Node where the resource is located.
 	Status string `mapstructure:"status"` // Status of the resource (e.g., "running", "stopped", "online", "available").
 	ID     string `mapstructure:"id"`     // Unique identifier for the resource.
@@ -106,9 +106,12 @@ type PveStorageResource struct {
 }
 
 // PveSdnResource represents a PVE software-defined network (SDN) resource.
+// PVE 8 reports these as "sdn" resources, PVE 9 as "network" resources.
 type PveSdnResource struct {
 	PveResource
-	SDN string `mapstructure:"sdn"` // Name of the SDN.
+	SDN         string `mapstructure:"sdn"`          // Name of the SDN (only "sdn" resources).
+	Network     string `mapstructure:"network"`      // Name of the network entity (only "network" resources).
+	NetworkType string `mapstructure:"network-type"` // Type of the network entity, e.g. "zone" or "fabric" (only "network" resources).
 }
 
 // PVE resources.
@@ -455,6 +458,15 @@ func (r *PveResources) FindNodeSDN(nodeName string) *[]PveSdnResource {
 	return &sdns
 }
 
+// GetName returns the name of an SDN resource, regardless of whether it was
+// reported as an "sdn" (PVE 8) or a "network" (PVE 9) resource.
+func (r *PveSdnResource) GetName() string {
+	if r.SDN != "" {
+		return r.SDN
+	}
+	return r.Network
+}
+
 // GetStatusNumeric returns the numeric status of an SDN resource.
 // Returns 1 if the status is "ok", otherwise returns 0.
 func (r *PveSdnResource) GetStatusNumeric() float64 {

proxmox/model_test.go

@@ -3,6 +3,8 @@ package proxmox
 import (
 	"encoding/json"
 	"testing"
+
+	"github.com/mitchellh/mapstructure"
 )
 
 func TestPveClusterStatusGetClusterModeNumeric(t *testing.T) {
@@ -112,3 +114,63 @@ func TestPveZfsPoolStatusUnmarshal(t *testing.T) {
 		t.Fatal("missing cache counters must stay absent instead of becoming zero metrics")
 	}
 }
+
+func TestPveSdnResourceGetName(t *testing.T) {
+	tests := []struct {
+		name     string
+		resource PveSdnResource
+		want     string
+	}{
+		{
+			name:     "sdn resource",
+			resource: PveSdnResource{SDN: "localnetwork"},
+			want:     "localnetwork",
+		},
+		{
+			name:     "network resource",
+			resource: PveSdnResource{Network: "localnetwork", NetworkType: "zone"},
+			want:     "localnetwork",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			if got := tt.resource.GetName(); got != tt.want {
+				t.Fatalf("GetName() = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}
+
+func TestPveSdnResourceDecodeNetwork(t *testing.T) {
+	// Resource object as reported by PVE 9 for SDN entities.
+	obj := map[string]interface{}{
+		"id":           "network/pve1/zone/localnetwork",
+		"type":         "network",
+		"node":         "pve1",
+		"status":       "ok",
+		"network":      "localnetwork",
+		"network-type": "zone",
+	}
+
+	var resource PveSdnResource
+	if err := mapstructure.Decode(obj, &resource); err != nil {
+		t.Fatalf("Decode() error = %v", err)
+	}
+	if err := mapstructure.Decode(obj, &resource.PveResource); err != nil {
+		t.Fatalf("Decode() error = %v", err)
+	}
+
+	if got, want := resource.GetName(), "localnetwork"; got != want {
+		t.Fatalf("GetName() = %v, want %v", got, want)
+	}
+	if got, want := resource.NetworkType, "zone"; got != want {
+		t.Fatalf("NetworkType = %v, want %v", got, want)
+	}
+	if got, want := resource.ID, "network/pve1/zone/localnetwork"; got != want {
+		t.Fatalf("ID = %v, want %v", got, want)
+	}
+	if got, want := resource.GetStatusNumeric(), float64(1); got != want {
+		t.Fatalf("GetStatusNumeric() = %v, want %v", got, want)
+	}
+}

proxmox/pve_api_client.go

@@ -64,18 +64,18 @@ func (instance *PveApiClient) GetClusterStatus() (*PveClusterStatus, error) {
 		switch obj["type"] {
 		case "cluster":
 			if err := mapstructure.Decode(obj, &cluster); err != nil {
-				log.Errorf("Unable to decode cluster status object. Error:", err)
+				log.Errorf("Unable to decode cluster status object. Error: %s", err)
 				continue
 			}
 		case "node":
 			var node PveNodeStatus
 			if err := mapstructure.Decode(obj, &node); err != nil {
-				log.Errorf("Unable to decode node status object. Error:", err)
+				log.Errorf("Unable to decode node status object. Error: %s", err)
 				continue
 			}
 			cluster.NodeStatuses = append(cluster.NodeStatuses, node)
 		default:
-			log.Errorf("Unable to decode cluster status object. Unknown type:", obj["type"])
+			log.Warnf("Skipping cluster status object of unknown type '%v'.", obj["type"])
 		}
 	}
 
@@ -106,60 +106,60 @@ func (instance *PveApiClient) GetClusterResources() (*PveResources, error) {
 		case "lxc":
 			var resource PveLxcResource
 			if err := mapstructure.Decode(obj, &resource); err != nil {
-				log.Errorf("Unable to decode cluster resource object. Error:", err)
+				log.Errorf("Unable to decode cluster resource object. Error: %s", err)
 				continue
 			}
 			if err := mapstructure.Decode(obj, &resource.PveResource); err != nil {
-				log.Errorf("Unable to decode cluster resource object. Error:", err)
+				log.Errorf("Unable to decode cluster resource object. Error: %s", err)
 				continue
 			}
 			resources.CTs = append(resources.CTs, resource)
 		case "qemu":
 			var resource PveQemuResource
 			if err := mapstructure.Decode(obj, &resource); err != nil {
-				log.Errorf("Unable to decode cluster resource object. Error:", err)
+				log.Errorf("Unable to decode cluster resource object. Error: %s", err)
 				continue
 			}
 			if err := mapstructure.Decode(obj, &resource.PveResource); err != nil {
-				log.Errorf("Unable to decode cluster resource object. Error:", err)
+				log.Errorf("Unable to decode cluster resource object. Error: %s", err)
 				continue
 			}
 			resources.VMs = append(resources.VMs, resource)
 		case "node":
 			var resource PveNodeResource
 			if err := mapstructure.Decode(obj, &resource); err != nil {
-				log.Errorf("Unable to decode cluster resource object. Error:", err)
+				log.Errorf("Unable to decode cluster resource object. Error: %s", err)
 				continue
 			}
 			if err := mapstructure.Decode(obj, &resource.PveResource); err != nil {
-				log.Errorf("Unable to decode cluster resource object. Error:", err)
+				log.Errorf("Unable to decode cluster resource object. Error: %s", err)
 				continue
 			}
 			resources.Nodes = append(resources.Nodes, resource)
 		case "storage":
 			var resource PveStorageResource
 			if err := mapstructure.Decode(obj, &resource); err != nil {
-				log.Errorf("Unable to decode cluster resource object. Error:", err)
+				log.Errorf("Unable to decode cluster resource object. Error: %s", err)
 				continue
 			}
 			if err := mapstructure.Decode(obj, &resource.PveResource); err != nil {
-				log.Errorf("Unable to decode cluster resource object. Error:", err)
+				log.Errorf("Unable to decode cluster resource object. Error: %s", err)
 				continue
 			}
 			resources.Storages = append(resources.Storages, resource)
-		case "sdn":
+		case "sdn", "network":
 			var resource PveSdnResource
 			if err := mapstructure.Decode(obj, &resource); err != nil {
-				log.Errorf("Unable to decode cluster resource object. Error:", err)
+				log.Errorf("Unable to decode cluster resource object. Error: %s", err)
 				continue
 			}
 			if err := mapstructure.Decode(obj, &resource.PveResource); err != nil {
-				log.Errorf("Unable to decode cluster resource object. Error:", err)
+				log.Errorf("Unable to decode cluster resource object. Error: %s", err)
 				continue
 			}
 			resources.SDNs = append(resources.SDNs, resource)
 		default:
-			log.Errorf("Unable to decode cluster resource object. Unknown type:", obj["type"])
+			log.Warnf("Skipping cluster resource object of unknown type '%v'.", obj["type"])
 		}
 	}