Build a Woodpecker CI Pipeline End-to-End
Set up a Woodpecker CI server and agent with Docker Compose, then build a real pipeline with registry secrets, BuildKit layer caching, and SSH deploy steps.
Before you start
- ▸Docker Engine 24+ with the Compose plugin installed (docker compose version)
- ▸A domain name with DNS pointing at the host and a working reverse proxy for TLS termination
- ▸An OAuth application created on your forge (GitHub, Gitea, Forgejo, or GitLab) with the callback URL set
- ▸Root or sudo access on the host and port 443 open inbound
Woodpecker CI is a lightweight, self-hosted continuous integration server that runs pipelines defined in a simple YAML file committed to your repository. It splits into two components: a server (manages state, UI, API) and one or more agents (run the actual pipeline steps inside containers). This guide walks through standing up both on a single host with Docker Compose, wiring up a real pipeline with secrets, container layer caching, and a deploy step.
Prerequisites and Architecture
Woodpecker delegates authentication entirely to a forge — GitHub, Gitea, Forgejo, GitLab, or Bitbucket. You must create an OAuth application on your forge before the server will start. The agent connects to the server over gRPC and pulls jobs; the server never needs to reach the agent directly, so the agent can sit behind NAT.
- A Linux host with Docker Engine 24+ and the Compose plugin (
docker compose, not the olddocker-compose). - A domain name pointing at the host, with TLS handled by a reverse proxy (Caddy, Traefik, nginx). Woodpecker itself speaks plain HTTP internally.
- An OAuth application registered on your forge. For GitHub: Settings → Developer settings → OAuth Apps → New OAuth App. Set the callback URL to
https://ci.example.com/authorize. - Port 443 open inbound (for the UI/API) and port 9000 reachable internally (agent ↔ server gRPC).
Step 1 — Write the Docker Compose File
Create a working directory and drop in a compose.yaml. The server and agent share a user-defined network so the agent can resolve woodpecker-server by name.
mkdir -p /opt/woodpecker && cd /opt/woodpecker
cat > compose.yaml <<'EOF'
services:
woodpecker-server:
image: woodpeckerci/woodpecker-server:latest
restart: always
ports:
- "8000:8000" # HTTP – expose via reverse proxy only
- "9000:9000" # gRPC for agents
volumes:
- woodpecker-server-data:/var/lib/woodpecker
environment:
WOODPECKER_OPEN: "false"
WOODPECKER_HOST: "https://ci.example.com"
WOODPECKER_GITHUB: "true"
WOODPECKER_GITHUB_CLIENT: "${GITHUB_CLIENT_ID}"
WOODPECKER_GITHUB_SECRET: "${GITHUB_CLIENT_SECRET}"
WOODPECKER_AGENT_SECRET: "${AGENT_SECRET}"
WOODPECKER_DATABASE_DRIVER: sqlite3
WOODPECKER_DATABASE_DATASOURCE: /var/lib/woodpecker/woodpecker.sqlite
woodpecker-agent:
image: woodpeckerci/woodpecker-agent:latest
restart: always
depends_on:
- woodpecker-server
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
WOODPECKER_SERVER: "woodpecker-server:9000"
WOODPECKER_AGENT_SECRET: "${AGENT_SECRET}"
WOODPECKER_BACKEND: docker
WOODPECKER_MAX_WORKFLOWS: 4
volumes:
woodpecker-server-data:
networks:
default:
name: woodpecker
EOF
Step 2 — Create the Environment File
Never put secrets in compose.yaml. Use a .env file that Docker Compose reads automatically. Generate a strong agent secret with openssl.
AGENT_SECRET=$(openssl rand -hex 32)
cat > .env <
If you are using Gitea or Forgejo instead of GitHub, replace the WOODPECKER_GITHUB* variables with WOODPECKER_GITEA=true, WOODPECKER_GITEA_URL, WOODPECKER_GITEA_CLIENT, and WOODPECKER_GITEA_SECRET.
Step 3 — Configure a Reverse Proxy (Caddy Example)
Caddy is the fastest path to automatic TLS. Add a block to your Caddyfile:
ci.example.com {
reverse_proxy localhost:8000
}
systemctl reload caddy
For nginx, proxy http://127.0.0.1:8000 and add the standard X-Forwarded-* headers. The gRPC port (9000) does not need to be exposed publicly unless you run agents on other hosts.
Step 4 — Start the Stack and Activate Your First Repo
docker compose up -d
docker compose logs -f
Navigate to https://ci.example.com. Log in with your forge account. Woodpecker redirects through the OAuth flow and drops you into the dashboard. Click + Add repository, find your repo, and enable it. Woodpecker installs a webhook automatically.
Step 5 — Write Your First Pipeline
Woodpecker looks for .woodpecker.yaml (or .woodpecker/ directory) at the repository root. Each step runs in its own container sequentially unless you set depends_on for parallelism.
cat > .woodpecker.yaml <<'EOF'
steps:
- name: lint
image: golangci/golangci-lint:v1.57
commands:
- golangci-lint run ./...
- name: test
image: golang:1.22-alpine
commands:
- go test -race ./...
- name: build-image
image: woodpeckerci/plugin-docker-buildx:latest
settings:
repo: ghcr.io/yourorg/yourapp
tags: latest,${CI_COMMIT_SHA:0:8}
registry: ghcr.io
username:
from_secret: ghcr_user
password:
from_secret: ghcr_token
cache_from: ghcr.io/yourorg/yourapp:cache
cache_to: ghcr.io/yourorg/yourapp:cache
when:
branch: main
EOF
The cache_from / cache_to pair pushes BuildKit layer cache to the registry so subsequent builds skip unchanged layers. On a cold cache the first build is normal speed; warm builds can cut image build time by 60–80% for typical Go or Node apps.
Step 6 — Add Pipeline Secrets
Secrets are stored encrypted in the Woodpecker database and injected at runtime. Never hard-code credentials in .woodpecker.yaml.
Add secrets via the UI: Repository → Settings → Secrets → Add secret, or use the CLI:
# Install the CLI (single binary)
curl -Lo woodpecker-cli https://github.com/woodpecker-ci/woodpecker/releases/latest/download/woodpecker-cli_linux_amd64
chmod +x woodpecker-cli && mv woodpecker-cli /usr/local/bin/woodpecker
export WOODPECKER_SERVER=https://ci.example.com
export WOODPECKER_TOKEN=
woodpecker secret add \
--repository yourorg/yourapp \
--name ghcr_user \
--value yourorg
woodpecker secret add \
--repository yourorg/yourapp \
--name ghcr_token \
--value ghp_xxx...
Secrets are available to pipeline steps by referencing from_secret: secret_name inside a settings block, or as environment variables with environment: MY_VAR: from_secret: my_secret. Woodpecker masks secret values in logs automatically.
Step 7 — Add a Deploy Step
A deploy step runs only on the main branch after a successful image build. This example SSHes into a remote host and runs a docker compose pull && docker compose up -d:
# Append to .woodpecker.yaml
cat >> .woodpecker.yaml <<'EOF'
- name: deploy
image: appleboy/drone-ssh:latest
settings:
host:
from_secret: deploy_host
username:
from_secret: deploy_user
key:
from_secret: deploy_ssh_key
script:
- cd /opt/myapp
- docker compose pull
- docker compose up -d --remove-orphans
when:
branch: main
event: push
EOF
Add the three secrets (deploy_host, deploy_user, deploy_ssh_key) via the CLI or UI as shown in Step 6. The appleboy/drone-ssh plugin is compatible with Woodpecker because Woodpecker is a Drone fork and shares the plugin protocol.
Step 8 — Verify the Pipeline
Push a commit to your repository. The forge fires a webhook, Woodpecker queues the pipeline, and the agent picks it up within seconds.
# Watch agent logs in real time
docker compose logs -f woodpecker-agent
You should see the agent claim the job, pull the step images, execute each step, and report back. In the UI, navigate to Pipelines and click the run to see per-step logs and timing.
Troubleshooting
Agent won't connect to server
Confirm both containers are on the same Docker network and the AGENT_SECRET value in .env matches exactly. Check docker compose logs woodpecker-server for gRPC handshake errors. If you changed the secret after first start, restart both services.
Webhook 400 or not firing
Woodpecker registers the webhook at the forge using the WOODPECKER_HOST value. If that URL is unreachable from the internet (or from Gitea on the same host), webhooks silently fail. Test reachability: curl -I https://ci.example.com/healthz.
Docker-in-Docker permission errors
The agent mounts /var/run/docker.sock. On SELinux hosts (Fedora, RHEL, Rocky) you may need to add the :z relabelling option to the volume mount, or run setsebool -P container_manage_cgroup on. On AppArmor systems (Ubuntu), the default Docker profile is usually sufficient.
Registry cache push fails
BuildKit cache requires the registry credentials to have write access. Confirm the ghcr_token secret has the write:packages scope. If the cache tag doesn't exist yet, the first push creates it automatically — this is expected and not an error.
Frequently asked questions
- Can I run multiple agents for parallel pipelines?
- Yes. Add additional agent service blocks to compose.yaml (or run agents on separate hosts) pointing at the same server and agent secret. Each agent handles up to WOODPECKER_MAX_WORKFLOWS concurrent pipelines, and the server distributes jobs across all connected agents.
- Does Woodpecker support matrix builds?
- Yes, using the matrix key at the top of .woodpecker.yaml. Define axes such as go_version: [1.21, 1.22] and Woodpecker spawns a separate pipeline run for each combination, all reported under the same commit.
- How do I persist the SQLite database across container recreations?
- The compose.yaml above mounts a named volume (woodpecker-server-data) at /var/lib/woodpecker. As long as you do not run docker compose down -v, the database and all secrets survive restarts and image upgrades.
- Should I use PostgreSQL instead of SQLite for production?
- SQLite is fine for teams up to roughly 20 users with moderate pipeline volume. For larger installations or high concurrency, set WOODPECKER_DATABASE_DRIVER=postgres and provide WOODPECKER_DATABASE_DATASOURCE with a Postgres connection string, then add a postgres service to compose.yaml.
- Is the appleboy/drone-ssh plugin safe to use with Woodpecker?
- Woodpecker is a fork of Drone and shares the plugin environment variable protocol, so Drone-compatible plugins work without modification. Vet any third-party plugin image as you would any container you run with elevated credentials — prefer pinned image digests over floating tags in production.
Related guides
Configure Prometheus Alertmanager
Configure Prometheus Alertmanager with routing trees, receivers, inhibition rules, grouping, Go templates, and PagerDuty/Slack on-call integrations.
Build an Intranet Server on Linux
Set up a complete small-office intranet on one Linux box: Nginx web server, dnsmasq local DNS, Samba file sharing, and a Wiki.js team wiki.
Build an nftables Firewall Script
Build a complete nftables firewall from scratch: tables, chains, sets, default-deny input policy, service allowlisting, and persistent systemd configuration.
Caddy as a Reverse Proxy
Set up Caddy as a reverse proxy with automatic HTTPS, load balancing, WebSocket passthrough, reusable snippets, and header control — no certbot required.