$linuxjunkies
>

Build a Woodpecker CI Pipeline End-to-End

Set up a Woodpecker CI server and agent with Docker Compose, then build a real pipeline with registry secrets, BuildKit layer caching, and SSH deploy steps.

IntermediateUbuntuDebianFedoraArch12 min readUpdated June 7, 2026

Before you start

  • Docker Engine 24+ with the Compose plugin installed (docker compose version)
  • A domain name with DNS pointing at the host and a working reverse proxy for TLS termination
  • An OAuth application created on your forge (GitHub, Gitea, Forgejo, or GitLab) with the callback URL set
  • Root or sudo access on the host and port 443 open inbound

Woodpecker CI is a lightweight, self-hosted continuous integration server that runs pipelines defined in a simple YAML file committed to your repository. It splits into two components: a server (manages state, UI, API) and one or more agents (run the actual pipeline steps inside containers). This guide walks through standing up both on a single host with Docker Compose, wiring up a real pipeline with secrets, container layer caching, and a deploy step.

Prerequisites and Architecture

Woodpecker delegates authentication entirely to a forge — GitHub, Gitea, Forgejo, GitLab, or Bitbucket. You must create an OAuth application on your forge before the server will start. The agent connects to the server over gRPC and pulls jobs; the server never needs to reach the agent directly, so the agent can sit behind NAT.

  • A Linux host with Docker Engine 24+ and the Compose plugin (docker compose, not the old docker-compose).
  • A domain name pointing at the host, with TLS handled by a reverse proxy (Caddy, Traefik, nginx). Woodpecker itself speaks plain HTTP internally.
  • An OAuth application registered on your forge. For GitHub: Settings → Developer settings → OAuth Apps → New OAuth App. Set the callback URL to https://ci.example.com/authorize.
  • Port 443 open inbound (for the UI/API) and port 9000 reachable internally (agent ↔ server gRPC).

Step 1 — Write the Docker Compose File

Create a working directory and drop in a compose.yaml. The server and agent share a user-defined network so the agent can resolve woodpecker-server by name.

mkdir -p /opt/woodpecker && cd /opt/woodpecker
cat > compose.yaml <<'EOF'
services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:latest
    restart: always
    ports:
      - "8000:8000"   # HTTP – expose via reverse proxy only
      - "9000:9000"   # gRPC for agents
    volumes:
      - woodpecker-server-data:/var/lib/woodpecker
    environment:
      WOODPECKER_OPEN: "false"
      WOODPECKER_HOST: "https://ci.example.com"
      WOODPECKER_GITHUB: "true"
      WOODPECKER_GITHUB_CLIENT: "${GITHUB_CLIENT_ID}"
      WOODPECKER_GITHUB_SECRET: "${GITHUB_CLIENT_SECRET}"
      WOODPECKER_AGENT_SECRET: "${AGENT_SECRET}"
      WOODPECKER_DATABASE_DRIVER: sqlite3
      WOODPECKER_DATABASE_DATASOURCE: /var/lib/woodpecker/woodpecker.sqlite

  woodpecker-agent:
    image: woodpeckerci/woodpecker-agent:latest
    restart: always
    depends_on:
      - woodpecker-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      WOODPECKER_SERVER: "woodpecker-server:9000"
      WOODPECKER_AGENT_SECRET: "${AGENT_SECRET}"
      WOODPECKER_BACKEND: docker
      WOODPECKER_MAX_WORKFLOWS: 4

volumes:
  woodpecker-server-data:

networks:
  default:
    name: woodpecker
EOF

Step 2 — Create the Environment File

Never put secrets in compose.yaml. Use a .env file that Docker Compose reads automatically. Generate a strong agent secret with openssl.

AGENT_SECRET=$(openssl rand -hex 32)
cat > .env <

If you are using Gitea or Forgejo instead of GitHub, replace the WOODPECKER_GITHUB* variables with WOODPECKER_GITEA=true, WOODPECKER_GITEA_URL, WOODPECKER_GITEA_CLIENT, and WOODPECKER_GITEA_SECRET.

Step 3 — Configure a Reverse Proxy (Caddy Example)

Caddy is the fastest path to automatic TLS. Add a block to your Caddyfile:

ci.example.com {
    reverse_proxy localhost:8000
}
systemctl reload caddy

For nginx, proxy http://127.0.0.1:8000 and add the standard X-Forwarded-* headers. The gRPC port (9000) does not need to be exposed publicly unless you run agents on other hosts.

Step 4 — Start the Stack and Activate Your First Repo

docker compose up -d
docker compose logs -f

Navigate to https://ci.example.com. Log in with your forge account. Woodpecker redirects through the OAuth flow and drops you into the dashboard. Click + Add repository, find your repo, and enable it. Woodpecker installs a webhook automatically.

Step 5 — Write Your First Pipeline

Woodpecker looks for .woodpecker.yaml (or .woodpecker/ directory) at the repository root. Each step runs in its own container sequentially unless you set depends_on for parallelism.

cat > .woodpecker.yaml <<'EOF'
steps:
  - name: lint
    image: golangci/golangci-lint:v1.57
    commands:
      - golangci-lint run ./...

  - name: test
    image: golang:1.22-alpine
    commands:
      - go test -race ./...

  - name: build-image
    image: woodpeckerci/plugin-docker-buildx:latest
    settings:
      repo: ghcr.io/yourorg/yourapp
      tags: latest,${CI_COMMIT_SHA:0:8}
      registry: ghcr.io
      username:
        from_secret: ghcr_user
      password:
        from_secret: ghcr_token
      cache_from: ghcr.io/yourorg/yourapp:cache
      cache_to: ghcr.io/yourorg/yourapp:cache
    when:
      branch: main
EOF

The cache_from / cache_to pair pushes BuildKit layer cache to the registry so subsequent builds skip unchanged layers. On a cold cache the first build is normal speed; warm builds can cut image build time by 60–80% for typical Go or Node apps.

Step 6 — Add Pipeline Secrets

Secrets are stored encrypted in the Woodpecker database and injected at runtime. Never hard-code credentials in .woodpecker.yaml.

Add secrets via the UI: Repository → Settings → Secrets → Add secret, or use the CLI:

# Install the CLI (single binary)
curl -Lo woodpecker-cli https://github.com/woodpecker-ci/woodpecker/releases/latest/download/woodpecker-cli_linux_amd64
chmod +x woodpecker-cli && mv woodpecker-cli /usr/local/bin/woodpecker
export WOODPECKER_SERVER=https://ci.example.com
export WOODPECKER_TOKEN=

woodpecker secret add \
  --repository yourorg/yourapp \
  --name ghcr_user \
  --value yourorg

woodpecker secret add \
  --repository yourorg/yourapp \
  --name ghcr_token \
  --value ghp_xxx...

Secrets are available to pipeline steps by referencing from_secret: secret_name inside a settings block, or as environment variables with environment: MY_VAR: from_secret: my_secret. Woodpecker masks secret values in logs automatically.

Step 7 — Add a Deploy Step

A deploy step runs only on the main branch after a successful image build. This example SSHes into a remote host and runs a docker compose pull && docker compose up -d:

# Append to .woodpecker.yaml
cat >> .woodpecker.yaml <<'EOF'

  - name: deploy
    image: appleboy/drone-ssh:latest
    settings:
      host:
        from_secret: deploy_host
      username:
        from_secret: deploy_user
      key:
        from_secret: deploy_ssh_key
      script:
        - cd /opt/myapp
        - docker compose pull
        - docker compose up -d --remove-orphans
    when:
      branch: main
      event: push
EOF

Add the three secrets (deploy_host, deploy_user, deploy_ssh_key) via the CLI or UI as shown in Step 6. The appleboy/drone-ssh plugin is compatible with Woodpecker because Woodpecker is a Drone fork and shares the plugin protocol.

Step 8 — Verify the Pipeline

Push a commit to your repository. The forge fires a webhook, Woodpecker queues the pipeline, and the agent picks it up within seconds.

# Watch agent logs in real time
docker compose logs -f woodpecker-agent

You should see the agent claim the job, pull the step images, execute each step, and report back. In the UI, navigate to Pipelines and click the run to see per-step logs and timing.

Troubleshooting

Agent won't connect to server

Confirm both containers are on the same Docker network and the AGENT_SECRET value in .env matches exactly. Check docker compose logs woodpecker-server for gRPC handshake errors. If you changed the secret after first start, restart both services.

Webhook 400 or not firing

Woodpecker registers the webhook at the forge using the WOODPECKER_HOST value. If that URL is unreachable from the internet (or from Gitea on the same host), webhooks silently fail. Test reachability: curl -I https://ci.example.com/healthz.

Docker-in-Docker permission errors

The agent mounts /var/run/docker.sock. On SELinux hosts (Fedora, RHEL, Rocky) you may need to add the :z relabelling option to the volume mount, or run setsebool -P container_manage_cgroup on. On AppArmor systems (Ubuntu), the default Docker profile is usually sufficient.

Registry cache push fails

BuildKit cache requires the registry credentials to have write access. Confirm the ghcr_token secret has the write:packages scope. If the cache tag doesn't exist yet, the first push creates it automatically — this is expected and not an error.

tested on:Ubuntu 24.04Debian 12Fedora 40Rocky 9

Frequently asked questions

Can I run multiple agents for parallel pipelines?
Yes. Add additional agent service blocks to compose.yaml (or run agents on separate hosts) pointing at the same server and agent secret. Each agent handles up to WOODPECKER_MAX_WORKFLOWS concurrent pipelines, and the server distributes jobs across all connected agents.
Does Woodpecker support matrix builds?
Yes, using the matrix key at the top of .woodpecker.yaml. Define axes such as go_version: [1.21, 1.22] and Woodpecker spawns a separate pipeline run for each combination, all reported under the same commit.
How do I persist the SQLite database across container recreations?
The compose.yaml above mounts a named volume (woodpecker-server-data) at /var/lib/woodpecker. As long as you do not run docker compose down -v, the database and all secrets survive restarts and image upgrades.
Should I use PostgreSQL instead of SQLite for production?
SQLite is fine for teams up to roughly 20 users with moderate pipeline volume. For larger installations or high concurrency, set WOODPECKER_DATABASE_DRIVER=postgres and provide WOODPECKER_DATABASE_DATASOURCE with a Postgres connection string, then add a postgres service to compose.yaml.
Is the appleboy/drone-ssh plugin safe to use with Woodpecker?
Woodpecker is a fork of Drone and shares the plugin environment variable protocol, so Drone-compatible plugins work without modification. Vet any third-party plugin image as you would any container you run with elevated credentials — prefer pinned image digests over floating tags in production.

Related guides