How Do I Set Up SSH Securely on a Remote Server?

Quick Answer

Generate an Ed25519 key pair on your local machine with ssh-keygen -t ed25519 -C "your@email". Push the public key to the server with ssh-copy-id user@host. Then edit /etc/ssh/sshd_config on the server, set PermitRootLogin no, PasswordAuthentication no, and KbdInteractiveAuthentication no. Restart sshd with sudo systemctl restart ssh. Test the new connection in a second terminal before closing the first.

Add fail2ban to block IPs that hammer your login attempts, and consider moving SSH to a non-standard port to cut bot noise. Those four changes (keys, no root, no passwords, fail2ban) move you out of the easy target pool that automated scanners chew through every minute.

Hardening checklist at a glance

Step 1: generate a strong key pair on the client

On your local machine (Linux, Mac, or Windows with OpenSSH), open a terminal and run:

ssh-keygen -t ed25519 -C "your_email@example.com"

Ed25519 is the modern default. It is faster, shorter, and as secure as long RSA keys. Accept the default file path (~/.ssh/id_ed25519) and set a non-empty passphrase. The passphrase encrypts the private key at rest, so a stolen laptop does not become a stolen key. On a developer machine, an SSH agent (ssh-agent on Linux/Mac, or Windows OpenSSH's ssh-agent service) holds the unlocked key in memory so you only enter the passphrase once per session.

Step 2: push the public key to the server

ssh-copy-id user@your-server-ip

This appends the contents of ~/.ssh/id_ed25519.pub to ~/.ssh/authorized_keys on the server, with the correct file permissions. If the server does not have ssh-copy-id installed (some minimal distros do not), do it manually:

cat ~/.ssh/id_ed25519.pub | ssh user@host "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Test the key by opening a new terminal and running ssh user@host. You should connect without being asked for the user's password (only the key passphrase if your agent does not have it cached). Do not move on until the key works; otherwise you are about to lock yourself out.

Step 3: harden the SSH daemon

On the server, edit the SSH daemon config:

sudo nano /etc/ssh/sshd_config

Find or add these lines (uncomment them if they exist):

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
ChallengeResponseAuthentication no
PubkeyAuthentication yes
AuthenticationMethods publickey

What each line does:

• PermitRootLogin no: blocks direct SSH login as root. Log in as a normal user and use sudo for privileged commands. This alone removes a huge portion of automated attack traffic, which always targets the root account.
• PasswordAuthentication no: rejects all password attempts at the protocol level. A connection without a valid key is dropped before any password prompt appears.
• KbdInteractiveAuthentication no / ChallengeResponseAuthentication no: closes the side door that some distros leave open even when password auth is off.
• AuthenticationMethods publickey: belt and suspenders, makes public key the only acceptable method.

Save the file. Validate the config before restarting:

sudo sshd -t

If that returns no output, the syntax is valid. Restart the service:

sudo systemctl restart ssh   # Debian/Ubuntu
sudo systemctl restart sshd  # RHEL/CentOS/Rocky/Fedora

Critical: test in a second terminal before closing the first

This is the rule that saves you from a 2 AM trip to the data centre or a support ticket to the cloud provider. Open a brand-new terminal on your local machine and SSH into the server. If you connect successfully, your hardening worked. If you get rejected, the original terminal is still your administrative session, and you can fix the config and try again. Never close the original session before the new one is proven.

Step 4: change the port (optional but cheap)

Moving SSH off port 22 does not improve security on its own; a port scanner finds open ports in seconds. What it does is dramatically reduce log noise from low-effort bots, which is useful for log review and reduces resource consumption.

If you change the port, edit:

Port 2222

Then open the new port in the firewall. On Ubuntu with UFW:

sudo ufw allow 2222/tcp
sudo ufw delete allow 22/tcp

On RHEL with firewalld:

sudo firewall-cmd --permanent --add-port=2222/tcp
sudo firewall-cmd --reload

If you are using SELinux (RHEL/Rocky/Fedora), also tell it the new port:

sudo semanage port -a -t ssh_port_t -p tcp 2222

Reconnect using ssh -p 2222 user@host, confirm it works, then close the old port at the firewall.

Step 5: install fail2ban

fail2ban watches your SSH logs and bans IP addresses that fail authentication too often. Install:

sudo apt install fail2ban    # Debian/Ubuntu
sudo dnf install fail2ban    # RHEL/Rocky/Fedora

Create a local override at /etc/fail2ban/jail.local:

[sshd]
enabled = true
port = 2222
maxretry = 4
findtime = 10m
bantime = 24h

Then start and enable:

sudo systemctl enable --now fail2ban

Check status:

sudo fail2ban-client status sshd

Optional but recommended next steps

• Restrict by IP if you can. If you always log in from one or two static IPs, lock the firewall to those. Combined with key authentication, this is close to bulletproof.
• Use SSH certificates instead of authorized_keys for fleets. For more than a handful of servers, SSH CA-signed certificates scale much better than copying public keys around.
• Set LoginGraceTime 30 in sshd_config to drop slow connection attempts faster.
• Set MaxAuthTries 3 to reduce the number of attempts per connection.
• Disable unused features. If you do not use X11 forwarding, set X11Forwarding no. If you do not need TCP forwarding for tunnels, set AllowTcpForwarding no.

What about two-factor for SSH?

You can add a TOTP second factor on top of keys using libpam-google-authenticator or Duo Unix. For a single personal server, key plus passphrase is usually enough. For a server holding sensitive data or accessible to multiple admins, layering a TOTP on top of the key is worth the small inconvenience.

The takeaway

Generate an Ed25519 key pair, push the public key to the server, disable root login and password authentication in sshd_config, validate with sshd -t, and test in a second terminal before closing the first. Add fail2ban and consider a non-standard port to cut noise. Those steps take twenty minutes and move you out of the broad target pool that automated brute-force scanners hit every minute of every day.