Align metric names with Prometheus naming conventions
Use base units, export cumulative values as _total counters, update docs, dashboards and rules.
This commit is contained in:
+103
-43
@@ -3,6 +3,12 @@
|
||||
All exporter-specific metric names start with `pve_`. The Go Prometheus client
|
||||
also exposes its standard `go_*`, `process_*`, and HTTP handler metrics.
|
||||
|
||||
Metric names follow the [Prometheus naming conventions](https://prometheus.io/docs/practices/naming/):
|
||||
base units only (seconds, bytes, ratios) and a `_total` suffix on every metric
|
||||
exported as a counter. Counters hold the absolute value reported by the PVE API
|
||||
and reset when the guest or node restarts, so query them with `rate()` or
|
||||
`increase()`. All remaining metrics are gauges.
|
||||
|
||||
## Common labels and lifecycle
|
||||
|
||||
Most resource metrics use these labels:
|
||||
@@ -22,7 +28,7 @@ remain visible for up to five minutes after it stops.
|
||||
|
||||
| Metric | Labels | Description |
|
||||
| --- | --- | --- |
|
||||
| `pve_metrics_collection_latency_ms` | `collector` | Summary of successful collector execution time in milliseconds. |
|
||||
| `pve_metrics_collection_duration_seconds` | `collector` | Summary of successful collector execution time in seconds. |
|
||||
|
||||
## Cluster state
|
||||
|
||||
@@ -42,9 +48,9 @@ and `node`.
|
||||
| Metric | Description |
|
||||
| --- | --- |
|
||||
| `pve_node_state` | Node online state: `1` online, `0` offline. |
|
||||
| `pve_node_uptime` | Node uptime in seconds. |
|
||||
| `pve_node_cpu_count` | Number of logical CPUs. |
|
||||
| `pve_node_cpu_usage` | CPU usage reported by PVE. |
|
||||
| `pve_node_uptime_seconds` | Node uptime in seconds. |
|
||||
| `pve_node_cpus` | Number of logical CPUs. |
|
||||
| `pve_node_cpu_usage_ratio` | CPU usage reported by PVE as a ratio between `0` and `1`. |
|
||||
| `pve_node_memory_total_bytes` | Total memory in bytes. |
|
||||
| `pve_node_memory_used_bytes` | Used memory in bytes. |
|
||||
| `pve_node_memory_free_bytes` | Free memory in bytes. |
|
||||
@@ -57,10 +63,10 @@ and `node`.
|
||||
| `pve_node_rootfs_used_bytes` | Root filesystem used bytes. |
|
||||
| `pve_node_rootfs_total_bytes` | Root filesystem total bytes. |
|
||||
| `pve_node_rootfs_avail_bytes` | Root filesystem bytes available to unprivileged processes. |
|
||||
| `pve_node_time` | Node UTC Unix timestamp. |
|
||||
| `pve_node_localtime` | Node local Unix timestamp. |
|
||||
| `pve_node_cpuinfo` | Constant `1` carrying `flags`, `cores`, `model`, `sockets`, `cpus`, and `hvm` labels. |
|
||||
| `pve_node_systeminfo` | Constant `1` carrying `kversion`, `pveversion`, `machine`, `sysname`, and `release` labels. |
|
||||
| `pve_node_time_seconds` | Node UTC Unix timestamp. |
|
||||
| `pve_node_localtime_seconds` | Node local Unix timestamp. |
|
||||
| `pve_node_cpu_info` | Constant `1` carrying `flags`, `cores`, `model`, `sockets`, `cpus`, and `hvm` labels. |
|
||||
| `pve_node_system_info` | Constant `1` carrying `kversion`, `pveversion`, `machine`, `sysname`, and `release` labels. |
|
||||
|
||||
## Physical disks
|
||||
|
||||
@@ -70,7 +76,7 @@ Enabled with `metrics.disk`. Labels are `cluster`, `node`, `wwn`, `type`,
|
||||
| Metric | Description |
|
||||
| --- | --- |
|
||||
| `pve_node_disk_healthy` | SMART health state: `1` for `OK` or `PASSED`, otherwise `0`. |
|
||||
| `pve_node_disk_wearout` | Device wearout percentage when supplied by the PVE API. |
|
||||
| `pve_node_disk_wearout_percent` | Device wearout percentage when supplied by the PVE API. |
|
||||
| `pve_node_disk_size_bytes` | Physical disk size in bytes. |
|
||||
|
||||
## ZFS
|
||||
@@ -86,9 +92,9 @@ hierarchy, and `leaf` is `true` for a device entry.
|
||||
| Metric | Description |
|
||||
| --- | --- |
|
||||
| `pve_node_zfs_state` | Numeric component state: `0=UNKNOWN`, `1=ONLINE`, `2=DEGRADED`, `3=FAULTED`, `4=OFFLINE`, `5=REMOVED`, `6=UNAVAIL`. |
|
||||
| `pve_node_zfs_read_errors` | ZFS read error count reported for the component. |
|
||||
| `pve_node_zfs_write_errors` | ZFS write error count reported for the component. |
|
||||
| `pve_node_zfs_checksum_errors` | ZFS checksum error count reported for the component. |
|
||||
| `pve_node_zfs_read_errors_total` | ZFS read error count reported for the component. |
|
||||
| `pve_node_zfs_write_errors_total` | ZFS write error count reported for the component. |
|
||||
| `pve_node_zfs_checksum_errors_total` | ZFS checksum error count reported for the component. |
|
||||
|
||||
ZFS section entries that do not contain a counter do not produce a false zero
|
||||
series. A counter explicitly returned as zero is exported normally.
|
||||
@@ -100,9 +106,9 @@ pve_node_zfs_state != 1
|
||||
```
|
||||
|
||||
```promql
|
||||
(pve_node_zfs_read_errors > 0)
|
||||
or (pve_node_zfs_write_errors > 0)
|
||||
or (pve_node_zfs_checksum_errors > 0)
|
||||
(pve_node_zfs_read_errors_total > 0)
|
||||
or (pve_node_zfs_write_errors_total > 0)
|
||||
or (pve_node_zfs_checksum_errors_total > 0)
|
||||
```
|
||||
|
||||
## PVE storage
|
||||
@@ -126,8 +132,8 @@ Enabled with `metrics.subscription`. The common labels are `cluster` and
|
||||
| --- | --- |
|
||||
| `pve_node_subscription_info` | Constant `1` carrying subscription product and server identifiers. |
|
||||
| `pve_node_subscription_status` | `0` for API status `notfound`, `1` for `active`, `2` for expired or otherwise unusable states such as `invalid`, `suspended`, and `new`. |
|
||||
| `pve_node_subscription_regdate` | Subscription registration date as a Unix timestamp, when available. |
|
||||
| `pve_node_subscription_nextduedate` | Next due date as a Unix timestamp, when available. |
|
||||
| `pve_node_subscription_registration_timestamp_seconds` | Subscription registration date as a Unix timestamp, when available. |
|
||||
| `pve_node_subscription_next_due_timestamp_seconds` | Next due date as a Unix timestamp, when available. |
|
||||
| `pve_node_subscription_sockets` | Number of covered sockets reported by PVE. |
|
||||
|
||||
The PVE API cannot distinguish a node that never had a subscription from one
|
||||
@@ -155,16 +161,16 @@ Enabled with `metrics.lxc`. Templates are skipped. Labels are `cluster`,
|
||||
| Metric | Description |
|
||||
| --- | --- |
|
||||
| `pve_ct_state` | `1` running, `0` stopped. |
|
||||
| `pve_ct_uptime` | Uptime in seconds; updated only while running. |
|
||||
| `pve_ct_cpu_count` | Configured CPU count. |
|
||||
| `pve_ct_cpu_usage` | CPU usage reported by PVE; updated only while running. |
|
||||
| `pve_ct_mem_total_bytes` | Configured memory limit in bytes. |
|
||||
| `pve_ct_mem_used_bytes` | Used memory in bytes; updated only while running. |
|
||||
| `pve_ct_network_in_bytes` | Received bytes since container start; updated only while running. |
|
||||
| `pve_ct_network_out_bytes` | Transmitted bytes since container start; updated only while running. |
|
||||
| `pve_ct_disk_rd_bytes` | Disk bytes read; updated only while running. |
|
||||
| `pve_ct_disk_wr_bytes` | Disk bytes written; updated only while running. |
|
||||
| `pve_ct_disk_usage_bytes` | Used root disk bytes; updated only while running. |
|
||||
| `pve_ct_uptime_seconds` | Uptime in seconds; updated only while running. |
|
||||
| `pve_ct_cpus` | Configured CPU count. |
|
||||
| `pve_ct_cpu_usage_ratio` | CPU usage reported by PVE as a ratio between `0` and `1`; updated only while running. |
|
||||
| `pve_ct_memory_total_bytes` | Configured memory limit in bytes. |
|
||||
| `pve_ct_memory_used_bytes` | Used memory in bytes; updated only while running. |
|
||||
| `pve_ct_network_receive_bytes_total` | Received bytes since container start; updated only while running. |
|
||||
| `pve_ct_network_transmit_bytes_total` | Transmitted bytes since container start; updated only while running. |
|
||||
| `pve_ct_disk_read_bytes_total` | Disk bytes read; updated only while running. |
|
||||
| `pve_ct_disk_write_bytes_total` | Disk bytes written; updated only while running. |
|
||||
| `pve_ct_disk_used_bytes` | Used root disk bytes; updated only while running. |
|
||||
| `pve_ct_disk_size_bytes` | Configured root disk size in bytes. |
|
||||
| `pve_ct_swap_used_bytes` | Used swap in bytes; updated only while running. |
|
||||
|
||||
@@ -176,23 +182,77 @@ Enabled with `metrics.qemu`. Templates are skipped. Base labels are `cluster`,
|
||||
| Metric | Extra label | Description |
|
||||
| --- | --- | --- |
|
||||
| `pve_vm_state` | none | `1` running, `0` stopped. |
|
||||
| `pve_vm_uptime` | none | Uptime in seconds; updated only while running. |
|
||||
| `pve_vm_uptime_seconds` | none | Uptime in seconds; updated only while running. |
|
||||
| `pve_vm_agent` | none | QEMU guest agent state reported by PVE; updated only while running. |
|
||||
| `pve_vm_cpu_count` | none | Configured CPU count. |
|
||||
| `pve_vm_cpu_usage` | none | CPU usage reported by PVE; updated only while running. |
|
||||
| `pve_vm_mem_total_bytes` | none | Configured maximum memory in bytes. |
|
||||
| `pve_vm_mem_used_bytes` | none | Used memory in bytes; updated only while running. |
|
||||
| `pve_vm_disk_usage_bytes` | none | Root disk usage reported by PVE. |
|
||||
| `pve_vm_cpus` | none | Configured CPU count. |
|
||||
| `pve_vm_cpu_usage_ratio` | none | CPU usage reported by PVE as a ratio between `0` and `1`; updated only while running. |
|
||||
| `pve_vm_memory_total_bytes` | none | Configured maximum memory in bytes. |
|
||||
| `pve_vm_memory_used_bytes` | none | Used memory in bytes; updated only while running. |
|
||||
| `pve_vm_disk_used_bytes` | none | Root disk usage reported by PVE. |
|
||||
| `pve_vm_disk_size_bytes` | none | Configured maximum root disk size in bytes. |
|
||||
| `pve_vm_network_in_bytes` | `interface` | Bytes received by a virtual interface. |
|
||||
| `pve_vm_network_out_bytes` | `interface` | Bytes transmitted by a virtual interface. |
|
||||
| `pve_vm_disk_rd_operations` | `device` | Successful read operations. |
|
||||
| `pve_vm_disk_wr_operations` | `device` | Successful write operations. |
|
||||
| `pve_vm_disk_rd_bytes` | `device` | Bytes read from the block device. |
|
||||
| `pve_vm_disk_wr_bytes` | `device` | Bytes written to the block device. |
|
||||
| `pve_vm_disk_failed_rd_ops` | `device` | Failed read operations. |
|
||||
| `pve_vm_disk_failed_wr_ops` | `device` | Failed write operations. |
|
||||
| `pve_vm_disk_rd_time_total_ns` | `device` | Total block-device read time in nanoseconds. |
|
||||
| `pve_vm_disk_wr_time_total_ns` | `device` | Total block-device write time in nanoseconds. |
|
||||
| `pve_vm_network_receive_bytes_total` | `interface` | Bytes received by a virtual interface. |
|
||||
| `pve_vm_network_transmit_bytes_total` | `interface` | Bytes transmitted by a virtual interface. |
|
||||
| `pve_vm_disk_read_operations_total` | `device` | Successful read operations. |
|
||||
| `pve_vm_disk_write_operations_total` | `device` | Successful write operations. |
|
||||
| `pve_vm_disk_read_bytes_total` | `device` | Bytes read from the block device. |
|
||||
| `pve_vm_disk_write_bytes_total` | `device` | Bytes written to the block device. |
|
||||
| `pve_vm_disk_failed_read_operations_total` | `device` | Failed read operations. |
|
||||
| `pve_vm_disk_failed_write_operations_total` | `device` | Failed write operations. |
|
||||
| `pve_vm_disk_read_time_seconds_total` | `device` | Total block-device read time in seconds. |
|
||||
| `pve_vm_disk_write_time_seconds_total` | `device` | Total block-device write time in seconds. |
|
||||
|
||||
Interface and block-device metrics are collected only for running VMs.
|
||||
|
||||
## Migration from earlier releases
|
||||
|
||||
All metric names were aligned with the Prometheus naming conventions. The table
|
||||
below maps the previous names to the current ones. Dashboards, recording rules,
|
||||
and alerting rules built against the old names have to be updated; the example
|
||||
dashboards and rules in `examples/` already use the new names.
|
||||
|
||||
| Previous name | Current name | Note |
|
||||
| --- | --- | --- |
|
||||
| `pve_ct_cpu_usage` | `pve_ct_cpu_usage_ratio` | Values unchanged, they are already a ratio between `0` and `1`. |
|
||||
| `pve_ct_cpu_count` | `pve_ct_cpus` | Renamed only. |
|
||||
| `pve_ct_disk_rd_bytes` | `pve_ct_disk_read_bytes_total` | Type changed to counter. |
|
||||
| `pve_ct_disk_usage_bytes` | `pve_ct_disk_used_bytes` | Renamed only. |
|
||||
| `pve_ct_disk_wr_bytes` | `pve_ct_disk_write_bytes_total` | Type changed to counter. |
|
||||
| `pve_ct_mem_total_bytes` | `pve_ct_memory_total_bytes` | Renamed only. |
|
||||
| `pve_ct_mem_used_bytes` | `pve_ct_memory_used_bytes` | Renamed only. |
|
||||
| `pve_ct_network_in_bytes` | `pve_ct_network_receive_bytes_total` | Type changed to counter. |
|
||||
| `pve_ct_network_out_bytes` | `pve_ct_network_transmit_bytes_total` | Type changed to counter. |
|
||||
| `pve_ct_uptime` | `pve_ct_uptime_seconds` | Renamed only. |
|
||||
| `pve_metrics_collection_latency_ms` | `pve_metrics_collection_duration_seconds` | Values converted from milliseconds to seconds. |
|
||||
| `pve_node_cpuinfo` | `pve_node_cpu_info` | Renamed only. |
|
||||
| `pve_node_cpu_usage` | `pve_node_cpu_usage_ratio` | Values unchanged, they are already a ratio between `0` and `1`. |
|
||||
| `pve_node_cpu_count` | `pve_node_cpus` | Renamed only. |
|
||||
| `pve_node_disk_wearout` | `pve_node_disk_wearout_percent` | Renamed only. |
|
||||
| `pve_node_localtime` | `pve_node_localtime_seconds` | Renamed only. |
|
||||
| `pve_node_subscription_nextduedate` | `pve_node_subscription_next_due_timestamp_seconds` | Renamed only. |
|
||||
| `pve_node_subscription_regdate` | `pve_node_subscription_registration_timestamp_seconds` | Renamed only. |
|
||||
| `pve_node_systeminfo` | `pve_node_system_info` | Renamed only. |
|
||||
| `pve_node_time` | `pve_node_time_seconds` | Renamed only. |
|
||||
| `pve_node_uptime` | `pve_node_uptime_seconds` | Renamed only. |
|
||||
| `pve_node_zfs_checksum_errors` | `pve_node_zfs_checksum_errors_total` | Type changed to counter. |
|
||||
| `pve_node_zfs_read_errors` | `pve_node_zfs_read_errors_total` | Type changed to counter. |
|
||||
| `pve_node_zfs_write_errors` | `pve_node_zfs_write_errors_total` | Type changed to counter. |
|
||||
| `pve_vm_cpu_usage` | `pve_vm_cpu_usage_ratio` | Values unchanged, they are already a ratio between `0` and `1`. |
|
||||
| `pve_vm_cpu_count` | `pve_vm_cpus` | Renamed only. |
|
||||
| `pve_vm_disk_failed_rd_ops` | `pve_vm_disk_failed_read_operations_total` | Type changed to counter. |
|
||||
| `pve_vm_disk_failed_wr_ops` | `pve_vm_disk_failed_write_operations_total` | Type changed to counter. |
|
||||
| `pve_vm_disk_rd_bytes` | `pve_vm_disk_read_bytes_total` | Type changed to counter. |
|
||||
| `pve_vm_disk_rd_operations` | `pve_vm_disk_read_operations_total` | Type changed to counter. |
|
||||
| `pve_vm_disk_rd_time_total_ns` | `pve_vm_disk_read_time_seconds_total` | Values converted from nanoseconds to seconds, type changed to counter. |
|
||||
| `pve_vm_disk_usage_bytes` | `pve_vm_disk_used_bytes` | Renamed only. |
|
||||
| `pve_vm_disk_wr_bytes` | `pve_vm_disk_write_bytes_total` | Type changed to counter. |
|
||||
| `pve_vm_disk_wr_operations` | `pve_vm_disk_write_operations_total` | Type changed to counter. |
|
||||
| `pve_vm_disk_wr_time_total_ns` | `pve_vm_disk_write_time_seconds_total` | Values converted from nanoseconds to seconds, type changed to counter. |
|
||||
| `pve_vm_mem_total_bytes` | `pve_vm_memory_total_bytes` | Renamed only. |
|
||||
| `pve_vm_mem_used_bytes` | `pve_vm_memory_used_bytes` | Renamed only. |
|
||||
| `pve_vm_network_in_bytes` | `pve_vm_network_receive_bytes_total` | Type changed to counter. |
|
||||
| `pve_vm_network_out_bytes` | `pve_vm_network_transmit_bytes_total` | Type changed to counter. |
|
||||
| `pve_vm_uptime` | `pve_vm_uptime_seconds` | Renamed only. |
|
||||
|
||||
Cumulative metrics are now exported as Prometheus counters instead of gauges,
|
||||
which is what `rate()` and `increase()` expect and what removes the
|
||||
"metric might not be a counter" hint in Grafana.
|
||||
|
||||
Reference in New Issue
Block a user