139 lines
4.1 KiB
Markdown
139 lines
4.1 KiB
Markdown
# Operations and Troubleshooting
|
|
|
|
## Native execution
|
|
|
|
Build and run from the repository root:
|
|
|
|
```sh
|
|
go build -o pve-exporter .
|
|
./pve-exporter -config config.yaml
|
|
```
|
|
|
|
The default configuration path is `config.yaml`. The HTTP listener is formed
|
|
from the top-level `host` and `port` settings. Only `/metrics` is registered.
|
|
|
|
Verify the exporter locally:
|
|
|
|
```sh
|
|
curl http://localhost:9090/metrics
|
|
```
|
|
|
|
## Docker
|
|
|
|
Pull and run the published image:
|
|
|
|
```sh
|
|
docker pull gitea.lostak.dev/lostakj/pve-exporter:latest
|
|
docker run --rm -p 9090:9090 \
|
|
-v "$PWD/config.yaml:/opt/config.yaml:ro" \
|
|
gitea.lostak.dev/lostakj/pve-exporter:latest
|
|
```
|
|
|
|
The image contains the example configuration at `/opt/config.yaml`, but that
|
|
file has no API hosts or real credentials. Mount a configured file at the same
|
|
path for normal use.
|
|
|
|
## Prometheus
|
|
|
|
When Prometheus and the exporter share a container network:
|
|
|
|
```yaml
|
|
scrape_configs:
|
|
- job_name: pve-exporter
|
|
scrape_interval: 30s
|
|
static_configs:
|
|
- targets: [pve-exporter:9090]
|
|
```
|
|
|
|
The Prometheus scrape interval and `proxmox.interval` are independent. A scrape
|
|
does not contact PVE; it reads the latest values collected in the background.
|
|
|
|
For alerting, load
|
|
[`examples/pve-exporter.rules.yml`](../examples/pve-exporter.rules.yml) through
|
|
Prometheus `rule_files`. The [alerting guide](alerting.md) lists the required
|
|
collectors and a container mount example.
|
|
|
|
## API endpoint availability
|
|
|
|
At startup and every five seconds, the exporter requests `api2/json/` from each
|
|
configured PVE host. Requests are distributed round-robin across endpoints that
|
|
passed the latest check.
|
|
|
|
Use multiple hosts only for nodes in the same cluster:
|
|
|
|
```yaml
|
|
proxmox:
|
|
hosts:
|
|
- https://192.168.0.10:8006
|
|
- https://pve02.example.com:8006
|
|
```
|
|
|
|
IP and DNS hosts work with or without a trailing slash. If every endpoint is
|
|
unavailable, collectors log `All API endpoints are unreachable.` and retain
|
|
their previous series until the five-minute stale-metric timeout removes them.
|
|
|
|
## Logging
|
|
|
|
Set `logLevel` between `0` and `6`:
|
|
|
|
| Value | Level |
|
|
| --- | --- |
|
|
| `0` | Panic |
|
|
| `1` | Fatal |
|
|
| `2` | Error |
|
|
| `3` | Warn |
|
|
| `4` | Info |
|
|
| `5` | Debug |
|
|
| `6` | Trace |
|
|
|
|
`Info` is suitable for normal operation. `Debug` or `Trace` can help diagnose
|
|
endpoint availability and individual collector execution.
|
|
|
|
## Common problems
|
|
|
|
### Authentication failed
|
|
|
|
Check that `tokenId` contains the full `user@realm!token-name` identifier and
|
|
that `secret` contains the token secret rather than the user password. Assign
|
|
the `PVEAuditor` role to both the user and token.
|
|
|
|
### Permission or empty cluster-status errors
|
|
|
|
The `/cluster/status` endpoint is used by every collector to identify nodes and
|
|
the cluster label. Ensure the token can audit `/` and the relevant nodes. The
|
|
exporter reports a specific permission hint when the endpoint returns an empty
|
|
array.
|
|
|
|
### No metrics from a collector
|
|
|
|
Confirm that its boolean switch under `proxmox.metrics` is `true`. Some series
|
|
are intentionally conditional:
|
|
|
|
- VM and LXC templates are skipped.
|
|
- Runtime guest metrics are updated only while the guest is running.
|
|
- Disabled PVE storages are skipped.
|
|
- Optional ZFS counters are omitted when the API does not return them.
|
|
- Subscription dates are omitted when they are unavailable.
|
|
|
|
### ZFS collection errors
|
|
|
|
Enable `proxmox.metrics.zfs`, confirm that the node has ZFS utilities and pools,
|
|
and verify that the token can call both `/nodes/{node}/disks/zfs` and each pool
|
|
detail endpoint.
|
|
|
|
### Metrics remain after a resource disappears
|
|
|
|
Collector metrics use a five-minute TTL. A guest, disk, storage, or ZFS topology
|
|
entry that disappears can therefore remain visible for up to five minutes.
|
|
|
|
## Security notes
|
|
|
|
- Keep `config.yaml` readable only by the exporter account because it contains
|
|
the API token secret.
|
|
- The HTTP endpoint has no authentication; bind to a protected interface or
|
|
restrict access with a firewall or reverse proxy.
|
|
- The PVE client currently disables TLS certificate verification to support the
|
|
default self-signed certificates. Use the exporter only across trusted
|
|
networks or a protected tunnel.
|
|
- Prefer the read-only `PVEAuditor` role and do not use a root token.
|