Skip to main content

Set Up Prometheus

Prometheus is the first piece we start. It stores the numbers: CPU, memory, connections and later the k6 results.

Table of Contents

Why​

Prometheus is the "memory" in The Grafana Stack. It keeps metrics over time so Grafana can draw them later.

We start it by hand with docker run, one piece at a time. This way you see what each piece needs, and you know where to look when something breaks.

How: start the playground​

The Pizza Playground must be running before the monitoring starts. Start it from its own folder:

cd /path/to/pizza-playground
docker compose up -d --wait

Then run everything below from the folder that contains monitoring/ (the root of this repo).

Expected: docker compose up ends with all playground containers Healthy.

How: find the playground network​

The monitoring containers join the playground's network. This lets them reach db, api and the others by name.

The network name comes from the running playground api container, so it works whatever folder the playground was cloned into.

NET=$(docker ps --filter label=com.docker.compose.service=api --format '{{.Networks}}' | head -1)
PROJECT=$(docker ps --filter label=com.docker.compose.service=api --format '{{.Label "com.docker.compose.project"}}' | head -1)
[ -n "$NET" ] && [ -n "$PROJECT" ] || echo "Playground is not running: start it first (docker compose up -d --wait in its folder)"
echo "$NET" "$PROJECT"

Expected:

pizza-playground_default pizza-playground

PROJECT is the playground's compose project name. Alloy uses it on page 07 to ignore containers from other apps on your machine.

How: the config​

Prometheus reads this file. It scrapes itself and Alloy's own health. Container, Postgres and k6 metrics arrive by remote write, so they are not listed.

# Prometheus scrapes itself and Alloy's own health metrics.
# Container, Postgres and k6 metrics arrive by remote write, so they are not listed here.
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ["localhost:9090"]
  - job_name: alloy
    static_configs:
      - targets: ["alloy:12345"]

The file is monitoring/sample/prometheus/prometheus.yml. The docker run below mounts it into the container.

How: start Prometheus​

docker run -d --name prometheus --network "$NET" -p 9090:9090 \
  -v "$PWD/monitoring/sample/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro" \
  prom/prometheus:v3.15.0 \
  --config.file=/etc/prometheus/prometheus.yml --web.enable-remote-write-receiver

--web.enable-remote-write-receiver lets other tools push metrics into Prometheus. Alloy and k6 both do this.

How: check it​

curl -s localhost:9090/-/ready

Expected:

Prometheus Server is Ready.

Now open http://localhost:9090/targets. You should see two targets:

  • prometheus is UP.
  • alloy is DOWN. This is expected. Alloy does not exist until Set Up Alloy.

Tips​

  • Prometheus has no volume here. Its data lives inside the container. docker stop and docker start keep it, but docker rm erases all history.
  • NET and PROJECT are lost when you open a new terminal. Run the lookup again.
  • If NET is empty, the playground is not running. Start it first.
  • alloy stays DOWN until page 07. Do not try to fix it now.

Next: Set Up Loki.