How to Harden SSH on Linux

SSH is the primary remote-access vector on almost every Linux server. Default OpenSSH settings are functional but not hardened—they leave room for brute-force attacks, credential stuffing, and accidental root compromise. This guide walks through key-only authentication, locking out root, tightening daemon options, and restricting who can connect at all. Every change is made in /etc/ssh/sshd_config (or a drop-in under /etc/ssh/sshd_config.d/) and activated via systemd.

Before You Start

Always keep an active session open while editing SSH config. Test your changes in a second terminal before disconnecting. One typo in sshd_config can lock you out of a remote machine entirely. If you are on a cloud instance, confirm you have out-of-band console access (AWS EC2 Serial Console, DigitalOcean Recovery Console, etc.) before proceeding.

Step 1: Set Up Key-Based Authentication

Password authentication is vulnerable to brute force. SSH keys are not. Generate an Ed25519 key pair on your local workstation (not the server):

ssh-keygen -t ed25519 -C "[email protected]"

Ed25519 is preferred over RSA for new keys—it is faster, smaller, and equally secure. If you must interoperate with older systems, use -t rsa -b 4096 instead.

Copy the public key to the server:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server_ip

This appends the key to ~/.ssh/authorized_keys on the server with correct permissions. Verify you can log in with the key before continuing:

ssh -i ~/.ssh/id_ed25519 user@server_ip

Step 2: Edit sshd_config

On modern systems, drop-in files under /etc/ssh/sshd_config.d/ are preferred—they survive package upgrades without merge conflicts. Create a dedicated hardening file:

sudo nano /etc/ssh/sshd_config.d/99-hardening.conf

Add the following block and adjust to your environment:

# Disable password and keyboard-interactive auth
PasswordAuthentication no
KbdInteractiveAuthentication no
ChallengeResponseAuthentication no

# Disable root login entirely
PermitRootLogin no

# Only allow public-key auth
PubkeyAuthentication yes
AuthenticationMethods publickey

# Reduce attack surface
X11Forwarding no
AllowAgentForwarding no
AllowTcpForwarding no
PermitEmptyPasswords no

# Limit login attempts and sessions
MaxAuthTries 3
MaxSessions 5
LoginGraceTime 30

# Silence the banner that leaks OS info
PrintMotd no

# Use a non-default port (optional but reduces log noise)
# Port 2222

Important: KbdInteractiveAuthentication replaces the deprecated ChallengeResponseAuthentication directive in OpenSSH 8.7+. Include both if you have mixed server versions.

Changing the Default Port

Switching from port 22 to a high port (e.g., 2222) is not true security, but it eliminates the bulk of automated scanning noise. If you change the port, update your firewall before restarting sshd. On systems using SELinux (Fedora, RHEL, Rocky) you must also label the new port:

sudo semanage port -a -t ssh_port_t -p tcp 2222

Step 3: Restrict Access with AllowUsers / AllowGroups

An allowlist at the SSH level is one of the most effective controls. Only explicitly listed users or groups can connect, regardless of other credentials.

To restrict by user:

AllowUsers alice bob [email protected]/24

The user@host syntax restricts a user to specific source addresses. To restrict by group (often cleaner for teams), create a dedicated group:

sudo groupadd sshusers
sudo usermod -aG sshusers alice
sudo usermod -aG sshusers bob

Then add to your hardening config:

AllowGroups sshusers

Note: AllowUsers and AllowGroups are additive restrictions—if either directive is present, anyone not listed is denied. Do not set both unless you understand the interaction.

Step 4: Validate Config and Restart sshd

Always validate syntax before reloading:

sudo sshd -t

No output means no errors. Then reload without dropping existing sessions:

sudo systemctl reload sshd

On some distributions the unit is named ssh instead of sshd:

# Debian / Ubuntu
sudo systemctl reload ssh

# Fedora / RHEL / Rocky / Arch
sudo systemctl reload sshd

Step 5: Firewall Allowlisting

Pair SSH hardening with a firewall rule that restricts which source IPs can reach the SSH port at all.

UFW (Debian / Ubuntu)

# Allow SSH only from a specific IP or range
sudo ufw allow from 203.0.113.10 to any port 22 proto tcp
# If you changed the port:
sudo ufw allow from 203.0.113.10 to any port 2222 proto tcp
sudo ufw reload

firewalld (Fedora / RHEL / Rocky)

# Create a rich rule to allow only a trusted source
sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="203.0.113.10" service name="ssh" accept'
# Remove the default broad ssh rule if present
sudo firewall-cmd --permanent --remove-service=ssh
sudo firewall-cmd --reload

nftables (Arch / manual setup)

sudo nft add rule inet filter input ip saddr 203.0.113.10 tcp dport 22 accept

Replace 203.0.113.10 with your actual admin IP or CIDR range. If your IP is dynamic, consider a VPN or jump host instead of broad port exposure.

Step 6: Verify the Hardened Configuration

From a second terminal (keep your existing session open), confirm the new settings are enforced:

# Test key login works
ssh -i ~/.ssh/id_ed25519 -p 22 user@server_ip

# Confirm password auth is rejected
ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@server_ip
# Expected: "Permission denied (publickey)."

# Check active sshd config values
sudo sshd -T | grep -E 'passwordauth|permitrootlogin|allowusers|allowgroups|port'

The sshd -T command prints the full effective configuration after all drop-ins are merged—far more reliable than reading files manually.

Troubleshooting

• Locked out after reload: Use your cloud console or physical access to restore the config, then run sudo sshd -t and sudo systemctl reload sshd again.
• "Bad configuration option" on older systems: Some directives like KbdInteractiveAuthentication require OpenSSH 8.7+. Check your version with ssh -V and fall back to ChallengeResponseAuthentication no if needed.
• Drop-in files not picked up: Verify the main /etc/ssh/sshd_config includes an Include /etc/ssh/sshd_config.d/*.conf directive. Most modern distributions add this by default; older installs may not.
• SELinux blocking non-standard port: Check sudo ausearch -m avc -ts recent for denials and use the semanage command from Step 2.
• AllowGroups locking everyone out: The group must exist on the system and the user must be a member before the reload. Verify with getent group sshusers.