systemary.io

Sysadmin tutorials, Linux server hardening, monitoring & observability

How to Harden SSH on Linux: A Runbook

by Liam Foster

Most intrusions start at port 22. I've watched logs fill with brute-force attempts within hours of spinning up a new server with default SSH config. The good news: hardening SSH takes maybe 30 minutes and cuts your attack surface by 80%.

This runbook covers what I do on every Linux box before it sees traffic. Not paranoid theater—just the changes that matter.

Disable Password Authentication

Password auth is your biggest liability. Bots will find your server and hammer it. Even a "strong" password loses to persistence.

The fix: Force key-based authentication only.

Edit /etc/ssh/sshd_config:

PasswordAuthentication no
PubkeyAuthentication yes
PermitEmptyPasswords no

Then reload:

sudo systemctl reload sshd

Gotcha: Make sure you've already added your public key to ~/.ssh/authorized_keys on the target server, or you'll lock yourself out. Test the new key before reloading on production.

Change the Default Port

Port 22 is scanned by every automated tool. Moving to a non-standard port (say, 2222) won't stop a targeted attacker, but it kills 99% of the noise.

In sshd_config:

Port 2222

Reload SSH and verify you can still connect:

ssh -p 2222 user@your-server

Then update your firewall rules to allow only 2222 inbound. If you're using ufw:

sudo ufw allow 2222/tcp
sudo ufw delete allow 22/tcp

Gotcha: Document this in your runbook or you'll forget it in six months when you hand off the server.

Disable Root Login

Don't let anyone SSH directly as root. Make them use a regular user, then sudo.

PermitRootLogin no

This forces privilege escalation to be logged separately and gives you an audit trail. Plus, if someone compromises a user account, they still need the sudo password (or your sudoers config) to go further.

Limit Login Attempts

SSH has built-in rate-limiting. Use it.

MaxAuthTries 3
MaxSessions 2

Three failed attempts and the connection closes. Two concurrent sessions per user prevents resource exhaustion from a single compromised account.

Use Protocol 2 Only

If you're running anything older than RHEL 6 or Ubuntu 12.04, you might have Protocol 1 enabled. It's broken. Remove it:

Protocol 2

Modern SSH defaults to 2, but check anyway on legacy systems.

Restrict SSH Access by IP (When Possible)

If your team works from a known office or VPN, lock SSH to those IPs:

ListenAddress 0.0.0.0
AllowUsers user1@203.0.113.0 user2@203.0.113.0/24

Or use a firewall rule:

sudo ufw allow from 203.0.113.0/24 to any port 2222

Gotcha: Don't do this if you have remote staff or on-call engineers connecting from home. You'll get the 3 a.m. page about being locked out.

Enable SSH Key Rotation Reminders

Keys don't expire by default. Set a calendar reminder to rotate them every 90 days. Yes, manually. A key compromised six months ago is still valid.

Generate a new key:

ssh-keygen -t ed25519 -C "user@hostname-2024"

Add the new public key to authorized_keys, test it, then remove the old one.

Monitor Failed Logins

Hardening SSH is half the battle. The other half is knowing when someone's trying to break in.

Check logs in real-time:

sudo tail -f /var/log/auth.log | grep sshd

For production, set up log aggregation. If you're using journalctl:

journalctl -u ssh -n 50 --no-pager

Set up a simple alert if you see repeated failed attempts from the same IP. Most monitoring tools (Prometheus, Datadog, Grafana Loki) can parse auth logs and trigger alerts.

Use SSH Config on Your Local Machine

Don't type ssh -p 2222 user@your-server every time. Create ~/.ssh/config:

Host prod
  HostName your-server.com
  User deployuser
  Port 2222
  IdentityFile ~/.ssh/id_ed25519_prod
  ServerAliveInterval 60
  ServerAliveCountMax 3

Now you just type ssh prod. The ServerAlive* settings keep the connection alive through idle firewalls.

Verify Your Changes

After editing sshd_config, always validate syntax:

sudo sshd -t

If it returns nothing, you're good. If there's an error, fix it before reloading.

Then test a fresh connection from another terminal before closing your current one:

ssh -p 2222 user@your-server

If that works, reload:

sudo systemctl reload sshd

Don't restart—reload keeps existing connections alive while applying new config.

The Checklist

Before you call SSH hardened, confirm:

  • Password auth disabled
  • Public key added to authorized_keys and tested
  • Port changed from 22 to a non-standard number
  • Root login disabled
  • Max auth tries set to 3
  • Max sessions set to 2
  • sshd -t passes
  • Fresh connection test succeeds
  • Firewall rules updated
  • Changes documented in your runbook
  • Log monitoring configured

What This Actually Stops

This hardens SSH against:

  • Automated bot scans (port change alone kills most noise)
  • Dictionary and brute-force attacks (key auth + rate limiting)
  • Privilege escalation via direct root login (PermitRootLogin no)
  • Resource exhaustion (MaxSessions)

What it doesn't stop:

  • A targeted attacker with your private key (that's key management—different post)
  • SSH zero-days (update your system regularly)
  • Social engineering (that's a people problem — if you're unfamiliar with how these attacks work, the writeup on digitalwarga.id covers phishing tactics worth understanding)

Tomorrow's Action

Pull up your production server. Open sshd_config. Add the five core settings: disable password auth, change the port, disable root login, limit auth tries, and set max sessions. Validate, test, reload. Document it. Done.

If you're managing more than one server, automate this with Ansible or your config management tool. But do it once by hand first so you understand what breaks when.