Prometheus collects metrics; Grafana visualizes them. That's the entire model. You run Prometheus as a time-series database that scrapes HTTP endpoints on a schedule, stores the data, and exposes a query API. Grafana points at that API and renders panels. This tutorial walks you through both, end to end.
Install Prometheus
Download the latest release from prometheus.io/download. Untar it into /opt/prometheus or /usr/local/prometheus—choose once and stick with it.
cd /opt
sudo tar xzf prometheus-*.tar.gz
sudo mv prometheus-* prometheus
sudo useradd --no-create-home --shell /bin/false prometheus
sudo chown -R prometheus:prometheus /opt/prometheus
Prometheus needs a config file. Create /opt/prometheus/prometheus.yml:
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
This tells Prometheus to scrape itself every 15 seconds. The scrape_interval is how often Prometheus polls targets; evaluation_interval controls how often it evaluates alerting rules (not needed yet, but set it anyway).
Gotcha: If you're behind a proxy, add proxy_url to scrape_configs. If you're not, skip it—it's a common mistake to add it anyway.
Run Prometheus as a service
Create /etc/systemd/system/prometheus.service:
[Unit]
Description=Prometheus
After=network.target
[Service]
Type=simple
User=prometheus
Group=prometheus
ExecStart=/opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml --storage.tsdb.path=/opt/prometheus/data
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target
Enable and start it:
sudo systemctl daemon-reload
sudo systemctl enable prometheus
sudo systemctl start prometheus
Verify: curl http://localhost:9090/api/v1/query?query=up. You should see JSON back. If you see connection refused, check systemctl status prometheus and journalctl -u prometheus -n 20.
Install Grafana
Grafana runs on port 3000 by default. Download from grafana.com/grafana/download, or use your package manager:
sudo apt-get install -y grafana-server # Debian/Ubuntu
Or:
sudo dnf install grafana # RHEL/CentOS
Start it:
sudo systemctl enable grafana-server
sudo systemctl start grafana-server
Navigate to http://localhost:3000. Default credentials are admin/admin. Change the password immediately.
Connect Grafana to Prometheus
In Grafana, go to Configuration (gear icon) → Data Sources. Click "Add data source." Select Prometheus. Set the URL to http://localhost:9090 (or your Prometheus IP if remote). Click "Save & Test." You should see green text: "Data source is working."
Gotcha: If you get a connection error, check that Prometheus is actually running and that your firewall or security group allows port 9090 from the Grafana host. If Grafana is containerized and Prometheus is on the host, use http://host.docker.internal:9090 on Mac or http://172.17.0.1:9090 on Linux.
Create your first dashboard
Go to Dashboards (home icon) → Create → Dashboard. Click "Add a new panel." In the query editor, type a metric name. Start simple: up. This is a built-in metric Prometheus exports for every target it scrapes. A value of 1 means the target is up; 0 means down.
Click "Run queries." You should see a line graph. Change the panel title (top left) to "Prometheus Up Status." Click "Apply."
Add another panel. Query rate(prometheus_http_requests_total[5m]). This shows the rate of HTTP requests to Prometheus over the last 5 minutes. Prometheus exports this metric automatically. Set the legend to {{handler}} so you can see which endpoints are being hit.
Click the save icon (disk) at the top. Name your dashboard "System Overview." Save it.
Add a real target
To make this useful, add a second target. The Node Exporter exports system metrics (CPU, memory, disk, network). Download it from prometheus.io/download#node_exporter.
cd /opt
sudo tar xzf node_exporter-*.tar.gz
sudo mv node_exporter-* node_exporter
sudo useradd --no-create-home --shell /bin/false node_exporter
sudo chown -R node_exporter:node_exporter /opt/node_exporter
Create /etc/systemd/system/node_exporter.service:
[Unit]
Description=Node Exporter
After=network.target
[Service]
Type=simple
User=node_exporter
Group=node_exporter
ExecStart=/opt/node_exporter/node_exporter
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target
Start it:
sudo systemctl daemon-reload
sudo systemctl enable node_exporter
sudo systemctl start node_exporter
Verify: curl http://localhost:9100/metrics. You should see lines like node_cpu_seconds_total, node_memory_MemFree_bytes, etc.
Now add it to Prometheus. Edit /opt/prometheus/prometheus.yml:
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
- job_name: 'node'
static_configs:
- targets: ['localhost:9100']
Reload Prometheus without restarting:
curl -X POST http://localhost:9090/-/reload
Gotcha: The reload endpoint is disabled by default if Prometheus is started without --web.enable-lifecycle. If the reload fails, restart the service instead: sudo systemctl restart prometheus.
Go back to Grafana. In your dashboard, add a panel with the query node_memory_MemAvailable_bytes. Set the unit to "bytes (IEC)" in the field options. You now have memory free on your system.
Trade-offs and when this breaks
Prometheus stores data in-memory and on disk. The default retention is 15 days. If you scrape many targets or have a short interval, disk usage grows fast. Tune scrape_interval and retention in the systemd service: --storage.tsdb.retention.time=30d.
Grafana queries are synchronous. If Prometheus is slow or down, your dashboards hang. Add a second Prometheus instance and configure Grafana to query both, or use Prometheus federation to aggregate metrics upstream.
Metrics are pull-based. Prometheus must reach your targets. If a target is behind a NAT or firewall, scraping fails. Use the Pushgateway for those cases (not covered here, but it's a separate binary that receives metrics via HTTP POST). If you're also running containerized workloads locally, this guide on Kubernetes local development with Minikube covers a complementary setup worth reading alongside.
Dashboards are JSON. If you delete one, it's gone unless you have backups. Export important dashboards regularly or store them in Git. You can record your dashboard walkthroughs for teammates using one of the best screen recording software for PC free options available.
One-line takeaway
Prometheus + Grafana is a pull-based metrics pipeline: install both, point Grafana at Prometheus, add targets to prometheus.yml, and query metrics in dashboard panels.