Lesson 6 of 14 · 10 min · Ports, firewalls and remote access

SSH: secure remote access, keys and tunnels

Covers: Ubuntu 24.04 LTS, OpenSSH 9.6. Updated 13 Nov 2026. Part of the DevOps Engineer roadmap.

What you will learn

  • What SSH is and how it authenticates servers and users
  • Creating a key pair and installing the public key
  • ~/.ssh/config, jump hosts and copying files
  • Hardening sshd
  • Port forwarding (tunnels) and what they are used for

In simple words

SSH is how you sit at a computer that is far away. It opens a private, encrypted tunnel, so what you type travels safely over the internet. Instead of a password you normally use a pair of keys: a padlock you leave on the server, and the only key that opens it, which stays on your laptop.

Words to know

  • SSH: Secure Shell: the standard way to log in to a remote Linux machine.
  • Key pair: two matching files: a public key you can hand out and a private key you never share.
  • Public key: the padlock. You place it on every server you want to reach.
  • Tunnel: sending other traffic through the SSH connection, to reach something that is not open to the internet.

What SSH is

SSH (Secure Shell) gives you an encrypted remote shell, file transfer and tunnels over TCP port 22. It replaced telnet and rsh, which sent everything, including passwords, in clear text. Almost every Linux server, Git hosting, CI runner and cloud instance is managed over SSH.

ssh asha@203.0.113.10          # log in as user asha
ssh -p 2222 asha@host          # a non-standard port
ssh asha@host 'uptime'         # run one command and return

Two kinds of trust

  1. The server proves its identity to you with a host key. The first time you connect, SSH shows the key's fingerprint and asks you to confirm, then stores it in ~/.ssh/known_hosts. If the key changes later, SSH stops with a scary warning (REMOTE HOST IDENTIFICATION HAS CHANGED): either the server was rebuilt or someone is intercepting. Do not just delete the line without finding out which.
  2. You prove your identity to the server, either with a password or, much better, with a key pair.

Key pairs

A key pair has a private key (stays on your machine, never shared) and a public key (placed on servers). The server challenges you; only the holder of the private key can answer. Nothing secret crosses the network, and there is no password to guess.

ssh-keygen -t ed25519 -C "asha@laptop"

ed25519 is the modern default: short keys, fast, strong. (RSA needs at least 3072 bits if you must use it.) You are asked for a passphrase that encrypts the private key on disk; use one. Real result of generating a key (no passphrase, for the test):

-rw------- /tmp/k
-rw-r--r-- /tmp/k.pub
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINVy...
256 SHA256:zWtM1Qpj6zSoUugBUbRqfTcsf/f8zoogs+Ng25BtyAk root@dd23a794e2e4 (ED25519)

The private key is mode 600 (only you can read it), the .pub file is shareable, and the fingerprint identifies the key. Install the public key on a server:

ssh-copy-id -i ~/.ssh/id_ed25519.pub asha@203.0.113.10

which appends it to ~/.ssh/authorized_keys on the server. Cloud platforms do this for you at launch (an AWS "key pair" or the keys in your account settings).

Use an ssh-agent (ssh-add ~/.ssh/id_ed25519) so you type the passphrase once per session.

Permissions: why SSH says "Permission denied (publickey)"

SSH refuses to use files that others can modify or read:

Path Mode
~/.ssh 700
private keys (id_ed25519) 600
~/.ssh/authorized_keys 600 (or 644)
the user's home directory not writable by group or others

Wrong owner or mode on the server side is the top cause of key logins failing. On the server, journalctl -u ssh (the unit is sshd on RHEL family) tells you why: look for "Authentication refused: bad ownership or modes". On the client, ssh -v (or -vvv) shows which keys are offered.

The client config file

~/.ssh/config gives hosts short names and defaults:

Host web1
    HostName 203.0.113.10
    User asha
    IdentityFile ~/.ssh/id_ed25519

Host prod-*
    User deploy
    IdentityFile ~/.ssh/prod_key
    ServerAliveInterval 30

Host internal-db
    HostName 10.0.2.15
    User asha
    ProxyJump bastion

Then ssh web1 is enough. ServerAliveInterval keeps idle sessions from being dropped by firewalls.

Jump hosts (bastions)

Private servers have no public IP. You reach them through a hardened bastion host in the public subnet:

ssh -J asha@bastion.example.com asha@10.0.2.15

ProxyJump in the config does the same. Your private key stays on your laptop; the bastion only relays the connection. Avoid agent forwarding (-A) to untrusted hosts: root on that host could use your agent while you are connected. ProxyJump does not need it.

Copying files

scp report.pdf web1:/tmp/                  # copy to a server
scp -r web1:/var/log/app ./logs            # copy a directory from a server
rsync -avz --progress ./site/ web1:/srv/site/   # only changes, resumable; the usual choice

Hardening sshd

The server settings are in /etc/ssh/sshd_config (and drop-ins in /etc/ssh/sshd_config.d/). The ones that matter:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowUsers asha deploy
MaxAuthTries 3
  • No direct root login: log in as yourself and use sudo, so actions are traceable.
  • No passwords: internet-facing SSH is attacked constantly with password guesses. Keys cannot be guessed.
  • Limit who may log in.

Always test before you apply, and keep your current session open:

sudo sshd -t                         # syntax check; silent when fine
sudo systemctl reload ssh            # apply (service name `sshd` on RHEL family)
# in a NEW terminal: ssh in again to prove it works, then close the old one

Moving SSH to another port only reduces log noise; it is not security. Use a firewall source restriction, a VPN or a bastion, and tools like fail2ban to block repeated failures. Many teams skip public SSH entirely using cloud session managers (AWS SSM Session Manager, Google IAP).

Port forwarding (tunnels)

SSH can carry other TCP traffic through the encrypted connection.

Local forwarding (-L): make a remote service appear on your machine.

ssh -L 5433:10.0.2.20:5432 bastion        # my localhost:5433 -> (via bastion) -> 10.0.2.20:5432
psql -h 127.0.0.1 -p 5433 mydb            # now talks to the private database

Remote forwarding (-R): expose a local service on the remote machine, for example to show a development site: ssh -R 8080:localhost:3000 server.

Dynamic forwarding (-D): a SOCKS proxy through the server: ssh -D 1080 server.

Add -N (no remote command) and -f (background) for pure tunnels. Tunnels are how you reach a private database or a Kubernetes API without opening it to the internet. They are also a way around controls, so organisations govern their use.

Common mistakes

  • Leaving password logins on for an internet-facing server.
  • Committing or sharing a private key. Treat it like a password; if it leaks, remove it from every authorized_keys and generate a new one.
  • Locking yourself out after editing sshd_config. Use sshd -t and a second session.
  • Wrong modes on ~/.ssh.
  • Blindly accepting changed host keys.
  • One shared key for the whole team. Use one key per person so you can revoke individually.

Try it yourself

  1. Generate an ed25519 key with a passphrase, load it into the agent and log into a VM or container with it.
  2. Break the permissions of ~/.ssh/authorized_keys on a test server (chmod 666), watch the login fail, and find the reason in the server log. Fix it.
  3. Write a ~/.ssh/config with a bastion and a private host and connect with one command.
  4. Forward a local port to a service on a remote machine and open it from your browser or curl.

Check yourself

Why are SSH keys safer than passwords?

The private key never leaves your machine and nothing guessable crosses the network, so brute-forcing is impractical. Passwords can be guessed, reused or phished. Keys can also be revoked per person.

What does the "REMOTE HOST IDENTIFICATION HAS CHANGED" warning mean?

The server's host key differs from the one stored in known_hosts. Either the server was reinstalled or re-keyed, or someone is impersonating it. Verify out of band before removing the old entry.

What does ssh -L 5433:10.0.2.20:5432 bastion do?

It listens on local port 5433 and forwards connections through the bastion to 10.0.2.20:5432, which only the bastion can reach.

What three sshd_config settings give the biggest security gain?

PermitRootLogin no, PasswordAuthentication no (use keys), and restricting users with AllowUsers (plus network-level source restrictions).

Interview questions

How do you access a server in a private subnet?

Through a bastion host or VPN (ssh -J bastion target, or ProxyJump), or with a cloud session manager. The private server itself has no public IP.

SSH key login fails with "Permission denied (publickey)". How do you debug it?

ssh -vvv on the client to see which keys are offered, check the server's authorized_keys, ownership and modes of ~/.ssh and the home directory, and read the server's auth log or journalctl -u ssh.

What is the difference between scp and rsync?

scp copies files over SSH. rsync also uses SSH but transfers only differences, can resume and preserve attributes, so it is better for repeated or large syncs.

Summary

  • SSH provides an encrypted shell, file copy and tunnels on TCP 22; verify host keys and authenticate with key pairs.
  • Create ed25519 keys with a passphrase; keep private keys mode 600 and ~/.ssh mode 700.
  • Harden sshd (no root login, no passwords, limit users), test with sshd -t and a second session.
  • Use ProxyJump bastions and -L tunnels instead of exposing private services.

Going deeper

See how much of this you can explain

A free mock interview asks you new questions on Networking for DevOps for DevOps Engineer and scores every answer, with what to fix.