cloud-init Explained (with worked examples)

cloud-init is the industry-standard tool for bootstrapping cloud instances. The moment a new VM starts, cloud-init reads vendor-supplied metadata and your user-data script, then configures the machine: sets hostnames, installs packages, writes files, adds SSH keys, runs commands. Understanding how it works — and how to debug it when it doesn't — saves hours of head-scratching when your instance comes up wrong at 2 AM.

How cloud-init Works

cloud-init runs in four ordered stages during early boot, tied to systemd targets:

• generator – determines whether cloud-init should run at all
• local (cloud-init-local.service) – finds the datasource, applies network config
• network (cloud-init.service) – runs modules that need network access
• config and final (cloud-config.service, cloud-final.service) – runs the bulk of user-data modules

Each stage only runs once per instance unless you explicitly clean the cache. This is intentional — you don't want your bootstrap script re-running on every reboot.

Datasources

A datasource is how cloud-init talks to the hypervisor or cloud API to get instance metadata and user-data. The datasource is auto-detected, but you can pin it for faster boot times.

• EC2 – polls http://169.254.169.254/latest/ (IMDSv1/v2)
• Hetzner – polls http://169.254.169.254/hetzner/v1/ and also reads a config drive
• DigitalOcean – polls http://169.254.169.254/metadata/v1/
• NoCloud – reads from a local ISO or seed directory; useful for VMs and testing
• ConfigDrive – OpenStack-style attached virtual disk containing metadata

To pin a datasource and skip detection (speeds up boot by several seconds):

sudo nano /etc/cloud/cloud.cfg.d/99-datasource.cfg

# /etc/cloud/cloud.cfg.d/99-datasource.cfg
datasource_list: [ Hetzner ]
# or: [ Ec2 ], [ DigitalOcean ], [ NoCloud, None ]

User-Data Formats

The first line of your user-data determines how cloud-init interprets the rest of the payload.

Cloud Config (YAML)

The most common format. Must start with #cloud-config. This is not a shell script — it's a declarative YAML document interpreted by cloud-init modules.

#cloud-config
hostname: web01
package_update: true
package_upgrade: true
packages:
  - nginx
  - ufw
  - fail2ban

users:
  - name: deploy
    groups: sudo
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - ssh-ed25519 AAAAC3Nza... you@workstation

write_files:
  - path: /etc/motd
    content: |
      Bootstrapped by cloud-init. Do not edit manually.
    permissions: '0644'

runcmd:
  - systemctl enable --now nginx
  - ufw allow 22
  - ufw allow 80
  - ufw allow 443
  - ufw --force enable

Shell Script

Starts with a shebang. Runs as root during the final stage. Useful for imperative logic that YAML can't express cleanly.

#!/bin/bash
set -euo pipefail
apt-get update -y
apt-get install -y caddy
systemctl enable --now caddy
echo "Bootstrap complete" >> /var/log/my-setup.log

Use set -euo pipefail so failures are not silently swallowed.

MIME Multi-part

Combines multiple formats in one payload. Useful when you want both a cloud-config and a shell script.

#!/usr/bin/env python3
import sys
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

combined = MIMEMultipart()

cloud_config = """#cloud-config
hostname: web01
packages: [nginx]
"""

shell_script = """#!/bin/bash
echo 'post-config hook' >> /var/log/init.log
"""

for content, subtype in [(cloud_config, 'cloud-config'), (shell_script, 'x-shellscript')]:
    part = MIMEText(content, subtype, sys.getdefaultencoding())
    combined.attach(part)

print(combined.as_string())

Run the script and pipe its output as your user-data when creating the instance.

Key Modules

Modules are the individual tasks cloud-init runs. They are listed under cloud_init_modules, cloud_config_modules, and cloud_final_modules in /etc/cloud/cloud.cfg. You rarely need to edit this file directly — the cloud-config keys map to modules automatically.

• users-groups – creates users, sets shells and sudo rules
• write-files – places files on disk before other modules run
• package-update-upgrade-install – handles packages, package_update, package_upgrade
• runcmd – runs a list of shell commands late in the final stage
• bootcmd – runs commands very early, on every boot (use carefully)
• ntp – configures NTP servers
• puppet, ansible, chef – configuration management integration

Worked Example: Hetzner Cloud

On Hetzner, user-data is supplied via their web console, CLI tool hcloud, or Terraform provider. The Hetzner datasource is detected automatically, but you can verify it after first boot.

# Create a server with user-data from a local file
hcloud server create \
  --name web01 \
  --image ubuntu-24.04 \
  --type cx22 \
  --user-data-from-file ./cloud-config.yaml \
  --ssh-key my-key

After the server starts, SSH in and check the datasource that was used:

sudo cloud-init status --long

Output will show datasource: DataSourceHetzner and the overall status (done or error).

Worked Example: AWS EC2

EC2 supports IMDSv2 (token-based). cloud-init handles this transparently from version 21.3 onward. Supply user-data with the AWS CLI:

aws ec2 run-instances \
  --image-id ami-0c55b159cbfafe1f0 \
  --instance-type t3.micro \
  --user-data file://cloud-config.yaml \
  --key-name my-key \
  --security-group-ids sg-0abc123

To retrieve the instance's own metadata from inside the VM (useful for debugging):

# IMDSv2 — get a token first
TOKEN=$(curl -sX PUT "http://169.254.169.254/latest/api/token" \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/meta-data/instance-id

Worked Example: DigitalOcean

DigitalOcean calls user-data "User Data" in the droplet creation UI. Via doctl:

doctl compute droplet create web01 \
  --image ubuntu-24-04-x64 \
  --size s-1vcpu-1gb \
  --region fra1 \
  --user-data-file ./cloud-config.yaml \
  --ssh-keys $(doctl compute ssh-key list --no-header --format ID)

DO's metadata endpoint does not require a token:

curl -s http://169.254.169.254/metadata/v1/id

Debugging cloud-init

When something goes wrong, these are the steps to take in order.

Check Overall Status

sudo cloud-init status --long

Read the Logs

# Structured, most useful
sudo cat /var/log/cloud-init.log

# Output of runcmd and scripts
sudo cat /var/log/cloud-init-output.log

cloud-init-output.log captures stdout/stderr from your runcmd entries and shell scripts. If a command silently failed, look here first.

Inspect What Was Received

# The raw user-data cloud-init received
sudo cat /var/lib/cloud/instance/user-data.txt

# Decoded and split parts (for multi-part payloads)
ls /var/lib/cloud/instance/scripts/

Validate a Cloud Config Before Deploying

sudo cloud-init schema --config-file ./cloud-config.yaml

This checks your YAML against the cloud-init schema and catches typos in module names or wrong value types before you waste a server creation.

Re-run cloud-init for Testing

Clean the instance cache and re-run — only do this on a disposable test VM, never on production.

sudo cloud-init clean --logs --reboot

The instance will reboot and cloud-init will run again as if it were a fresh boot.

Analyze Boot Timing

sudo cloud-init analyze show
sudo cloud-init analyze blame

These show how long each module took to run — useful if your instance is taking 90 seconds to boot because package_upgrade is running on every start (a config mistake).

Common Pitfalls

• YAML indentation errors — always validate with cloud-init schema or an online YAML linter before deploying.
• runcmd vs bootcmd — bootcmd runs on every boot; runcmd runs once. Mixing them up causes repeated destructive actions.
• Package installation during network stage — if your datasource takes too long, the network may not be fully up. Use package_update: true rather than raw apt-get in bootcmd.
• User-data size limits — EC2 caps user-data at 16 KB. Use x-include-url to pull a larger config from S3 or a URL if needed.
• Default user conflicts — if you redefine the ubuntu or debian default user under users:, you must include - default as the first list item to preserve it, or that user vanishes.