Manage SSH Keys and the SSH Agent

SSH keys beat passwords on every axis: they are longer, unpredictable, and can be protected by a passphrase without slowing down scripted logins. This guide walks through generating modern keys, wiring up the SSH agent so you type your passphrase only once per session, forwarding credentials safely to jump hosts, and optionally anchoring keys to hardware tokens. The advice here applies to any current Linux distribution.

Choosing a Key Type: ed25519 vs RSA

Two key types dominate practical use today.

• ed25519 — Use this unless you have a specific reason not to. It produces tiny keys (68-character public key), signs and verifies fast, and has no known weaknesses. Requires OpenSSH 6.5+ (standard on any LTS released after 2016).
• RSA — Still the widest-compatibility choice. Use a minimum of 4096 bits. Necessary when connecting to legacy appliances, old cloud APIs, or hardware that predates elliptic-curve support.

Avoid DSA entirely — it is disabled by default in modern OpenSSH. Avoid ECDSA unless a specific platform requires it; ed25519 is strictly preferable for new keys.

Generating a Key Pair

ed25519 (recommended)

ssh-keygen -t ed25519 -C "[email protected]"

The -C flag adds a comment embedded in the public key — useful for identifying keys across many servers. Accept the default path (~/.ssh/id_ed25519) or supply a custom one. Always set a strong passphrase when prompted; the agent will handle re-entering it.

RSA (legacy compatibility)

ssh-keygen -t rsa -b 4096 -C "[email protected]"

File permissions matter

OpenSSH refuses to use keys with loose permissions. Set them once after generation:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

Copying the Public Key to a Remote Host

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@remote-host

This appends the public key to ~/.ssh/authorized_keys on the remote host with correct permissions. If ssh-copy-id is unavailable, do it manually:

cat ~/.ssh/id_ed25519.pub | ssh user@remote-host \
  "mkdir -p ~/.ssh && chmod 700 ~/.ssh && \
   cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

The SSH Agent: Type Your Passphrase Once

The SSH agent holds decrypted private keys in memory for the duration of a session. You unlock a key once; the agent handles all subsequent authentications silently.

Starting the agent manually

eval "$(ssh-agent -s)"

Most desktop environments (GNOME, KDE) start an agent automatically at login via systemd user units or a D-Bus service. Check whether one is already running:

echo $SSH_AUTH_SOCK

A non-empty path means an agent is active.

Adding a key to the agent

ssh-add ~/.ssh/id_ed25519

Add -t 3600 to expire the key from the agent after one hour — a sensible limit on shared or laptop systems.

ssh-add -t 3600 ~/.ssh/id_ed25519

List loaded keys

ssh-add -l

Output looks similar to: 256 SHA256:abc123... [email protected] (ED25519) — one line per loaded key. If the output is The agent has no identities, add your key with ssh-add.

Persistent Agent with keychain

keychain is a shell wrapper that re-uses a single long-running agent across logins and shells, so you type your passphrase once per reboot rather than once per terminal session. Install it first:

Debian / Ubuntu

sudo apt install keychain

Fedora / RHEL / Rocky

sudo dnf install keychain

Arch

sudo pacman -S keychain

Add the following to your ~/.bashrc (or ~/.zshrc):

eval "$(keychain --eval --quiet id_ed25519)"

On first shell open after a reboot, keychain prompts for the passphrase. Every subsequent shell sources the existing agent socket — no repeated prompts.

Agent Forwarding: Authenticate Through Jump Hosts

Agent forwarding lets a remote server use your local agent for onward SSH hops, without ever copying your private key off your machine.

Enable forwarding for specific hosts only

Edit ~/.ssh/config:

Host bastion.example.com
    ForwardAgent yes

Host *
    ForwardAgent no

Never set ForwardAgent yes for Host *. Anyone with root on a host you forward to can hijack your agent socket and authenticate as you to any server that trusts your key.

Per-connection forwarding

ssh -A [email protected]

From inside the bastion you can then ssh user@internal-host and the local agent handles authentication transparently.

ProxyJump: the modern alternative

For simple jump-host scenarios, ProxyJump is cleaner than agent forwarding and does not expose the agent socket on the intermediate host:

ssh -J [email protected] user@internal-host

Or in ~/.ssh/config:

Host internal-host
    ProxyJump [email protected]

Prefer ProxyJump over agent forwarding whenever the remote host is your final destination.

Hardware Tokens (FIDO2 / YubiKey)

OpenSSH 8.2+ supports FIDO2 hardware authenticators (YubiKey, SoloKey, etc.). The private key material never leaves the token; the token itself must be physically present for every authentication.

Generate a resident FIDO2 key

ssh-keygen -t ed25519-sk -O resident -C "yubikey@hostname"

-sk means security key. -O resident stores the key handle on the token itself, so you can retrieve the public key later with ssh-keygen -K from any machine. You will be prompted to touch the token during generation.

Non-resident (handle stored locally)

ssh-keygen -t ed25519-sk -C "yubikey@hostname"

This is slightly simpler but requires the ~/.ssh/id_ed25519_sk key handle file to be present on whatever workstation you use.

Copy the public key to remote hosts with ssh-copy-id as usual. Authentication then requires both the key handle and a physical touch of the token — two-factor by design.

Verifying Everything Works

ssh -v user@remote-host 2>&1 | grep -E 'Offering|Accepted|Authentication'

Look for a line like Server accepts key: ... and Authenticated to remote-host. The -v flag also reveals whether the agent was consulted (Offering public key) or whether password fallback occurred.

Troubleshooting

• Permission denied (publickey) — Check ~/.ssh/authorized_keys permissions on the server (must be 600). Verify the correct public key is present. Run ssh-add -l to confirm the key is in the agent.
• Agent not running / SSH_AUTH_SOCK empty — Start the agent with eval "$(ssh-agent -s)" or install keychain for persistence.
• sign_and_send_pubkey: signing failed with a hardware token — The token may not be plugged in, or udev rules for FIDO devices are missing. On Debian/Ubuntu install libpam-u2f or ensure the fido2-tools package is present. Arch users need libfido2.
• ForwardAgent not working — Confirm the server's /etc/ssh/sshd_config has AllowAgentForwarding yes (the default). Also check that SSH_AUTH_SOCK is set on the remote after connecting.
• Too many authentication failures — The client is offering too many keys. Use -o IdentitiesOnly=yes -i ~/.ssh/specific_key to force a single key.