Monitor a Server with node_exporter
Install Prometheus node_exporter as a hardened systemd service, configure scraping, and set up Grafana dashboards for full server observability.
Before you start
- ▸A running Prometheus instance (version 2.x) on this or another host
- ▸sudo / root access on the server being monitored
- ▸wget and tar installed for the binary installation method
- ▸Grafana connected to Prometheus if you want the dashboard step
Node Exporter is the standard Prometheus agent for hardware and OS metrics: CPU, memory, disk I/O, network, filesystem usage, and dozens more. Once it is running and Prometheus is scraping it, you have a real-time and historical view of every server in your fleet. This guide walks through installing node_exporter as a hardened systemd service, wiring it into Prometheus, and loading a production-quality Grafana dashboard.
Create a Dedicated System User
Node Exporter does not need root privileges. Run it as a locked-down system account so a compromise of the process cannot write to arbitrary paths.
sudo useradd --system --no-create-home --shell /usr/sbin/nologin node_exporter
Install Node Exporter
Option A: Distribution Package (Quick)
Debian/Ubuntu and Fedora ship node_exporter, but the packaged version often lags behind upstream. It is fine for non-critical use.
# Debian / Ubuntu
sudo apt install prometheus-node-exporter
# Fedora / RHEL 9+ (EPEL required on RHEL)
sudo dnf install golang-github-prometheus-node-exporter
# Arch
sudo pacman -S prometheus-node-exporter
Package installs drop a systemd unit automatically. Skip to the Verify the Service section if you go this route.
Option B: Official Binary (Recommended)
Grab the latest stable release from github.com/prometheus/node_exporter/releases. At the time of writing the current stable is 1.8.x. Adjust the version variable below as needed.
NODE_EXPORTER_VERSION="1.8.1"
ARCH="linux-amd64" # use linux-arm64 for ARM servers
wget -q "https://github.com/prometheus/node_exporter/releases/download/v${NODE_EXPORTER_VERSION}/node_exporter-${NODE_EXPORTER_VERSION}.${ARCH}.tar.gz"
tar xzf "node_exporter-${NODE_EXPORTER_VERSION}.${ARCH}.tar.gz"
sudo cp "node_exporter-${NODE_EXPORTER_VERSION}.${ARCH}/node_exporter" /usr/local/bin/
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter
Verify the binary is intact before continuing.
node_exporter --version
# Output will show version, build date, and Go version — it will vary.
Create the systemd Unit
Write a hardened service file. The security directives (ProtectSystem, PrivateTmp, NoNewPrivileges) limit blast radius if the process is ever exploited.
sudo tee /etc/systemd/system/node_exporter.service <<'EOF'
[Unit]
Description=Prometheus Node Exporter
Documentation=https://github.com/prometheus/node_exporter
After=network-online.target
Wants=network-online.target
[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter \
--collector.systemd \
--collector.processes \
--web.listen-address=":9100"
Restart=on-failure
RestartSec=5s
# Hardening
ProtectSystem=full
ProtectHome=true
PrivateTmp=true
NoNewPrivileges=true
CapabilityBoundingSet=
[Install]
WantedBy=multi-user.target
EOF
The --collector.systemd flag exposes unit states. --collector.processes exposes per-process stats. Both require the node_exporter user to be able to read /proc, which the system user can do by default. If you do not want those collectors, remove those flags.
sudo systemctl daemon-reload
sudo systemctl enable --now node_exporter
Verify the Service
sudo systemctl status node_exporter
You should see active (running). Then confirm the metrics endpoint is live:
curl -s http://localhost:9100/metrics | head -20
You will see lines like # HELP node_cpu_seconds_total ... followed by metric samples. If the output is empty or the connection is refused, check the journal:
journalctl -u node_exporter -n 50 --no-pager
Open the Firewall Port (If Applicable)
Port 9100 only needs to be reachable from your Prometheus server, not the open internet. Restrict accordingly.
# firewalld (Fedora / RHEL / Rocky)
sudo firewall-cmd --permanent --add-rich-rule='
rule family="ipv4"
source address="192.168.1.50/32"
port protocol="tcp" port="9100" accept'
sudo firewall-cmd --reload
# ufw (Ubuntu / Debian)
sudo ufw allow from 192.168.1.50 to any port 9100 proto tcp
Replace 192.168.1.50 with the actual IP of your Prometheus host.
Configure Prometheus to Scrape node_exporter
On your Prometheus server, edit prometheus.yml. The default scrape interval is 15 s; 60 s is fine for most servers and saves storage.
sudo nano /etc/prometheus/prometheus.yml
Add a new job under scrape_configs:
scrape_configs:
# ... existing jobs above ...
- job_name: 'node'
scrape_interval: 60s
static_configs:
- targets:
- '192.168.1.10:9100' # web server
- '192.168.1.11:9100' # database server
labels:
env: 'production'
For large fleets, replace static_configs with file_sd_configs or a service-discovery block (Consul, EC2, etc.). For a handful of servers, static targets are perfectly fine.
Reload Prometheus without downtime:
sudo systemctl reload prometheus
# Prometheus must have been started with --web.enable-lifecycle, otherwise:
sudo systemctl restart prometheus
Check that the target appears healthy in the Prometheus UI at http://<prometheus-host>:9090/targets. The state column should show UP.
Useful Grafana Dashboards
If you are running Grafana connected to Prometheus, import one of the following community dashboards. Go to Dashboards → Import and paste the ID.
| ID | Name | Best For |
|---|---|---|
| 1860 | Node Exporter Full | Most comprehensive single-host view; CPU, RAM, disk, network, systemd |
| 405 | Node Exporter Server Metrics | Compact summary, good for overview rows in multi-host dashboards |
| 11074 | Node Exporter for Prometheus — EN | Clean layout, modern panel types, works well with Grafana 10+ |
Dashboard 1860 is the community standard. After import, set the job variable to node (matching your job_name above) and the instance variable to the target you want to inspect.
Troubleshooting
node_exporter fails to start: permission denied on /proc/N/...
The --collector.processes collector reads /proc/<pid> entries owned by other users. On some hardened kernels (kernel.dmesg_restrict, hidepid=2 mount option on /proc), this is blocked. Either remove --collector.processes from ExecStart or add the node_exporter user to the group that has hidepid access (usually proc or adm — check your distro docs).
Target shows DOWN in Prometheus
Run curl http://<target-ip>:9100/metrics from the Prometheus host. If it times out, the firewall is blocking the connection. If it returns an error page, node_exporter is not listening on that interface — check --web.listen-address; use :9100 (all interfaces) rather than 127.0.0.1:9100 if Prometheus is on a separate machine.
Metrics are stale or missing after reboot
Confirm the service is enabled, not just started: systemctl is-enabled node_exporter should print enabled. If it prints disabled, run sudo systemctl enable node_exporter.
Frequently asked questions
- Does node_exporter need to run as root?
- No. Almost all collectors work as a regular system user. The exceptions are a few hardware-level collectors (RAPL energy, IPMI) that need elevated access. For standard server monitoring a non-privileged user is correct and safer.
- How do I expose node_exporter over TLS?
- Pass --web.config.file=/etc/node_exporter/web.yml to ExecStart and populate that YAML file with tls_server_config containing cert_file and key_file paths. Prometheus then needs tls_config in its scrape job to trust the certificate.
- Can I run node_exporter on the same host as Prometheus?
- Yes. Use --web.listen-address=127.0.0.1:9100 in that case and target 127.0.0.1:9100 in prometheus.yml. You can skip the firewall rule entirely since traffic stays on loopback.
- How many metrics does node_exporter expose by default?
- Typically 800–1200 time series per host depending on the number of CPUs, disks, and network interfaces. You can reduce cardinality by disabling unneeded collectors with --no-collector.<name> flags.
- What is the difference between node_exporter and the Prometheus agent built into some distros?
- Some distro packages bundle node_exporter under a different binary name or service name (e.g., prometheus-node-exporter on Debian). The underlying code is the same project; the difference is just packaging and the version may lag upstream by several months.
Related guides
Back Up Linux with Borg or restic
Set up encrypted, deduplicated backups with BorgBackup or restic: local and remote repos, retention pruning, restoring files, and systemd timer scheduling.
How to Check Disk Health with SMART
Learn to use smartctl to read SMART attributes, run drive self-tests, and identify early warning signs of HDD and SSD failure before data loss occurs.
Debug systemd Units that Won't Start
Learn a repeatable workflow to debug systemd services that won't start: status output, journalctl, systemd-analyze verify, and safe override.conf patches.
Linux Server Disaster Recovery Checklist
A practical Linux server disaster recovery checklist: what to back up, RTO/RPO planning, immutable off-site copies, automated restore drills, and verification.