Runtime Security with Falco
Install Falco with the modern eBPF probe, write custom detection rules, and route real-time runtime security alerts to Grafana Loki and Slack.
Before you start
- ▸Linux kernel 5.8 or newer with BTF enabled (/sys/kernel/btf/vmlinux present)
- ▸Root or sudo access on the target host
- ▸A running Grafana Loki instance accessible from the host (for the Loki integration step)
- ▸A Slack incoming webhook URL if you want Slack alerts
Falco is a cloud-native runtime security tool that uses the Linux kernel's eBPF subsystem to watch every syscall your workloads make. When a container reads /etc/shadow, spawns a shell inside a running pod, or connects to an unexpected IP, Falco catches it in real time. This guide covers installing Falco with its modern eBPF probe, writing custom rules, and routing alerts to Loki and Slack so your SIEM actually has something useful to work with.
How Falco Detects Threats
Falco attaches a probe to the kernel that intercepts syscall enter/exit events. The probe feeds a ring buffer that Falco's userspace process reads, evaluates against its rule engine, and converts into structured JSON alerts. There are three probe drivers: the legacy kernel module (requires matching kernel headers), the modern eBPF probe (CO-RE, works across kernels ≥ 5.8 without header matching), and the older eBPF probe. Use the modern eBPF probe on any kernel you haven't specifically frozen.
Prerequisites and Kernel Check
Confirm your kernel supports BTF, which the modern eBPF probe requires:
uname -r
ls /sys/kernel/btf/vmlinux
If that file exists, you're ready. Kernels shipping with Ubuntu 20.04+, Fedora 31+, and Arch since mid-2020 all have BTF enabled. You also need libelf and zlib on the host — nearly always already present.
Installing Falco
Debian / Ubuntu
curl -fsSL https://falco.org/repo/falcosecurity-packages.asc \
| sudo gpg --dearmor -o /usr/share/keyrings/falco-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/falco-archive-keyring.gpg] \
https://download.falco.org/packages/deb stable main" \
| sudo tee /etc/apt/sources.list.d/falcosecurity.list
sudo apt update
sudo apt install -y falco
Fedora / RHEL / Rocky
sudo rpm --import https://falco.org/repo/falcosecurity-packages.asc
sudo tee /etc/yum.repos.d/falcosecurity.repo <<'EOF'
[falcosecurity]
name=Falco
baseurl=https://download.falco.org/packages/rpm
enabled=1
gpgcheck=1
gpgkey=https://falco.org/repo/falcosecurity-packages.asc
EOF
sudo dnf install -y falco
Arch Linux
yay -S falco-bin
Select the Modern eBPF Probe
The installer may prompt you interactively. You can also force the probe at service start by editing the systemd drop-in:
sudo mkdir -p /etc/systemd/system/falco-modern-bpf.service.d
sudo tee /etc/systemd/system/falco-modern-bpf.service.d/override.conf <<'EOF'
[Service]
Environment=FALCO_DRIVER=modern-ebpf
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now falco-modern-bpf.service
Falco ships two service units: falco-kmod.service (kernel module) and falco-modern-bpf.service. Enable only the one that matches your chosen driver.
Verifying the Installation
sudo systemctl status falco-modern-bpf.service
sudo journalctl -u falco-modern-bpf.service -n 30 --no-pager
You should see a line like Falco initialized, ready to monitor syscalls. Trigger a test event to confirm the rule engine is active:
cat /etc/shadow > /dev/null 2>&1
sudo journalctl -u falco-modern-bpf.service --since "1 minute ago" | grep shadow
Output will vary, but you should see a Read sensitive file alert with the calling process name and UID.
Understanding and Writing Rules
Falco rules live in /etc/falco/falco_rules.yaml (shipped, do not edit directly) and /etc/falco/falco_rules.local.yaml (yours). Rules have five key fields: rule, desc, condition, output, and priority.
Rule Anatomy
cat /etc/falco/falco_rules.local.yaml
Add your rules here. A rule that fires when any process other than sshd writes to /root/.ssh/authorized_keys:
sudo tee -a /etc/falco/falco_rules.local.yaml <<'EOF'
- rule: Unauthorized Write to authorized_keys
desc: A process other than sshd modified authorized_keys for root
condition: >
open_write
and fd.name = "/root/.ssh/authorized_keys"
and not proc.name = "sshd"
output: >
Suspicious authorized_keys write
(user=%user.name proc=%proc.name pid=%proc.pid
file=%fd.name container=%container.id)
priority: CRITICAL
tags: [filesystem, ssh, persistence]
EOF
Macros and Lists
Macros let you factor out shared conditions; lists group values. Both are first-class citizens in the rules file:
sudo tee -a /etc/falco/falco_rules.local.yaml <<'EOF'
- list: trusted_crypto_binaries
items: [openssl, gpg, age]
- macro: crypto_tool_running
condition: proc.name in (trusted_crypto_binaries)
- rule: Unexpected Network by Crypto Tool
desc: A trusted crypto binary made an outbound connection
condition: outbound and crypto_tool_running
output: >
Crypto binary opened network (proc=%proc.name
dst=%fd.rip:%fd.rport container=%container.id)
priority: WARNING
tags: [network, crypto]
EOF
After editing, validate syntax before reloading:
sudo falco --dry-run -c /etc/falco/falco.yaml 2>&1 | tail -5
sudo systemctl reload falco-modern-bpf.service
Structured JSON Output
Enable JSON output for machine-readable alerts — a prerequisite for Loki and Slack integrations. In /etc/falco/falco.yaml:
sudo sed -i 's/^json_output: false/json_output: true/' /etc/falco/falco.yaml
sudo sed -i 's/^json_include_output_property: false/json_include_output_property: true/' /etc/falco/falco.yaml
sudo systemctl reload falco-modern-bpf.service
Integrating with Grafana Loki
Falco can push to Loki via Falco sidekick, a companion service purpose-built for routing Falco alerts.
Install Falcosidekick
SIDEKICK_VERSION=2.28.0
curl -Lo falcosidekick.tar.gz \
"https://github.com/falcosecurity/falcosidekick/releases/download/v${SIDEKICK_VERSION}/falcosidekick_${SIDEKICK_VERSION}_linux_amd64.tar.gz"
tar xf falcosidekick.tar.gz
sudo mv falcosidekick /usr/local/bin/
Configure Falcosidekick for Loki
sudo tee /etc/falcosidekick.yaml <<'EOF'
listenaddress: "0.0.0.0"
listenport: 2801
loki:
hostport: "http://127.0.0.1:3100"
user: ""
apikey: ""
minimumpriority: "warning"
customlabels:
host: myserver
source: falco
EOF
Point Falco at Falcosidekick
In /etc/falco/falco.yaml, add a JSON HTTP output:
sudo tee -a /etc/falco/falco.yaml <<'EOF'
http_output:
enabled: true
url: "http://127.0.0.1:2801"
EOF
Run Falcosidekick as a systemd Service
sudo tee /etc/systemd/system/falcosidekick.service <<'EOF'
[Unit]
Description=Falco Sidekick
After=network.target
[Service]
Type=simple
ExecStart=/usr/local/bin/falcosidekick -c /etc/falcosidekick.yaml
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now falcosidekick.service
Routing Alerts to Slack
Add a Slack block to /etc/falcosidekick.yaml. Create an incoming webhook in your Slack workspace first, then:
sudo tee -a /etc/falcosidekick.yaml <<'EOF'
slack:
webhookurl: "https://hooks.slack.com/services/TXXXXXXXX/BXXXXXXXX/xxxxxxxxxx"
channel: "#falco-alerts"
minimumpriority: "error"
messageformat: "Falco alert on {hostname}: {rule} [{priority}] — {output}"
EOF
sudo systemctl restart falcosidekick.service
Set minimumpriority per output — you may want every WARNING in Loki for forensics but only ERROR and CRITICAL in Slack to avoid alert fatigue.
Troubleshooting
- No BTF file at boot: If
/sys/kernel/btf/vmlinuxis absent, your kernel was built withoutCONFIG_DEBUG_INFO_BTF. Switch to the kernel module driver or upgrade to a vendor LTS kernel that includes BTF. - Probe fails to load with permission error: Ensure
CAP_BPFandCAP_PERFMONare available. In locked-down environments (RHEL 9 with strict SELinux), you may need to setkernel.unprivileged_bpf_disabled=0or run Falco as root. - High CPU from the Falco process: You're likely hitting a very noisy rule. Use
sudo falco-driver-loader statsor checkfalco_events_processed_totalmetrics to find the culprit rule, then raise itsminimumpriorityor add condition exceptions withand not proc.name = "noisyapp". - Sidekick not reaching Loki: Check that Loki's
loki.auth_enabledmatches your sidekick config. If Loki requires auth, setuserandapikey(Grafana Cloud token) infalcosidekick.yaml. - Rules not reloading:
systemctl reloadsends SIGHUP; confirm Falco received it withjournalctl -u falco-modern-bpf.service --since "10 seconds ago". If rules contain YAML syntax errors, Falco will refuse the reload — always run--dry-runfirst.
Frequently asked questions
- What is the difference between the modern eBPF probe and the legacy eBPF probe?
- The modern eBPF probe uses CO-RE (Compile Once – Run Everywhere) and BTF type information embedded in the kernel, so it loads without needing kernel headers or a matching probe binary for each kernel version. The legacy eBPF probe predates CO-RE and requires header-matched compilation similar to the kernel module.
- Can Falco monitor containers and Kubernetes pods?
- Yes. Falco attaches at the kernel level, so it sees all processes regardless of containerisation. Its condition language exposes container.id, container.name, and k8s.pod.name fields so you can write rules scoped to specific workloads.
- Will Falco block a detected syscall, or only alert?
- By default Falco is detection-only — it alerts but does not block. For active response you need a companion tool like Falco Talon or a custom webhook that calls a container runtime API to kill or isolate the offending workload.
- How do I suppress noisy built-in rules without editing the shipped rules file?
- In falco_rules.local.yaml, override the rule by name and add exceptions using the exceptions field, or add a not condition macro. You can also set a higher minimumpriority per output channel in Falcosidekick so low-priority noise never reaches Slack.
- Does Falco work on ARM64 systems such as Raspberry Pi or AWS Graviton?
- Yes, Falco provides ARM64 packages and the modern eBPF probe supports ARM64 on kernels ≥ 5.8 with BTF. The kernel module driver also builds on ARM64 if you have the matching headers installed.
Related guides
Manage Secrets with Ansible Vault
Encrypt Ansible secrets with AES-256 using ansible-vault: encrypt files and inline vars, automate with password files, and isolate group-level secrets with vault IDs.
AppArmor Explained
Learn how AppArmor profiles work, how to switch between enforce and complain mode, create new profiles, and diagnose access denials on Ubuntu, Debian, and Arch.
Apply CIS Benchmarks with OpenSCAP
Use OpenSCAP and scap-security-guide to evaluate, report on, and remediate Linux systems against CIS Benchmarks — covering install, eval, and automation.
How to Audit a Linux System with auditd
Set up auditd on Linux to track file access, syscalls, and privilege use. Covers persistent rules, file watches, ausearch, and aureport across major distros.