SSH Configuration Guide: Ports, Keys, and Security Hardening

The SSH Config File: Manage Every Server From One Place

Linux Fundamentals / By / October 1, 2026

If you manage more than about three machines, you are probably typing the same long ssh command over and over, with a port, a username and a key path in it. The ~/.ssh/config file removes all of that. It is the single highest-value five minutes in SSH, and most people never open it.

This guide is about that one file: what goes in it, the directives worth knowing, how to jump through a bastion, how to stop connections dropping, and how to keep one config per project without it turning into a mess.

The Complete SSH Series

Getting connected

Authentication and hardening

File transfer and tunnelling

Reference

Quick Reference

The file lives at ~/.ssh/config on the machine you connect from. It does not exist by default, so create it:

touch ~/.ssh/config
chmod 600 ~/.ssh/config

Permissions matter. SSH ignores a config file that is group or world writable, and it will not always tell you why.

Host Blocks: the Basic Idea

A config file is a list of Host blocks. Each one gives a short name to a set of connection settings:

# ~/.ssh/config

Host web
    HostName 192.168.1.50
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Host git
    HostName git.example.com
    User git
    IdentityFile ~/.ssh/id_ed25519

Host pi
    HostName 192.168.1.100
    User pi
    IdentityFile ~/.ssh/id_ed25519

Instead of ssh -p 2222 -i ~/.ssh/id_ed25519 [email protected], you type ssh web. The short name also works with scp, sftp, rsync and anything else that shells out to SSH, which is the part people underuse. scp report.csv web:/tmp/ just works.

Indentation is cosmetic. SSH reads until the next Host line.

Defaults for Every Connection

Host * matches everything. Put it at the bottom of the file, because SSH applies the first value it finds for each setting and never overrides it later:

Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    AddKeysToAgent yes
    HashKnownHosts yes

ServerAliveInterval sends a keep-alive every 60 seconds, which is what stops a session dying while you read a long log. AddKeysToAgent saves retyping a passphrase. HashKnownHosts stops your known_hosts file being a readable list of every machine you touch.

That first-value-wins rule is the single most common cause of “my config is being ignored”. A Host * block at the top silently wins over everything below it.

Wildcards and Patterns

Host patterns take wildcards, so a fleet with a naming convention needs one block rather than twenty:

Host *.internal.example.com
    User admin
    IdentityFile ~/.ssh/id_internal

Host db-*
    User postgres
    Port 2222

You can also list several patterns on one line: Host web1 web2 web3.

Jumping Through a Bastion with ProxyJump

If the machine you want is only reachable from another machine, ProxyJump handles the hop for you. No agent forwarding, no nested ssh commands:

Host bastion
    HostName bastion.example.com
    User jump
    IdentityFile ~/.ssh/id_ed25519

Host app-private
    HostName 10.0.5.20
    User deploy
    ProxyJump bastion

ssh app-private now connects through the bastion in one command, and scp follows the same path. This replaces the older ProxyCommand ssh -W %h:%p pattern you will still see in older guides; ProxyJump has been available since OpenSSH 7.3 and is clearer.

Reusing One Connection: ControlMaster

If you open several sessions to the same host, connection multiplexing makes every session after the first almost instant:

Host *
    ControlMaster auto
    ControlPath ~/.ssh/cm-%r@%h:%p
    ControlPersist 10m

One worthwhile caution: a persistent master socket outlives the session that created it. If the network drops underneath it, later connections can hang or behave strangely while the stale socket is still there. If SSH to one host starts misbehaving and everything else is fine, delete the socket in ~/.ssh/ and try again. Shorten ControlPersist, or leave multiplexing off for flaky links.

Keeping It Tidy with Include

One config file for every machine you have ever touched becomes unmanageable. Include splits it up:

# ~/.ssh/config
Include ~/.ssh/config.d/*.conf

Host *
    ServerAliveInterval 60

Then keep ~/.ssh/config.d/work.conf, home.conf, clients.conf and so on. The Include line must come before the settings you want the included files to be able to win against, because of the first-value-wins rule above.

Checking What SSH Will Actually Do

You do not have to guess which block matched. Ask:

ssh -G web

That prints the full effective configuration for the host web, every value resolved, without connecting to anything. It is the fastest way to find out why the wrong key or the wrong user is being used.

What This Page Deliberately Does Not Cover

The client config file is one piece of SSH. The rest of the series covers the others properly rather than in summary here:

Why This Matters

  • Every Linux job uses SSH — it is literally how you access servers
  • Security baseline — SSH hardening is the first thing auditors check
  • Tunnelling — accessing internal services securely is a daily operation in enterprise environments
  • Jump hosts / bastion servers — understanding SSH proxying is essential for cloud infrastructure

Next Steps

This guide is part of the Linux Fundamentals series. See the full series for more guides like this.

Before you expose any of this: the seven steps that decide whether a home lab survives contact with the internet, in the order they matter, are in How to Secure a Homelab Network in 2026.

Frequently Asked Questions

Where is the SSH config file?

~/.ssh/config on the machine you connect from. It is not created automatically. There is also a system-wide file at /etc/ssh/ssh_config which applies to every user, and a per-user file always wins over it.

How do I manage SSH connections to multiple servers?

Give each one a Host block with a short alias, then connect with ssh alias. For machines that share a naming convention, one wildcard block such as Host db-* covers the lot.

Why is my SSH config being ignored?

Three usual causes. The file is group or world writable, so SSH refuses it: chmod 600 ~/.ssh/config. A Host * block sits at the top of the file and wins, because SSH takes the first value it finds for each setting. Or the pattern does not match what you typed, which ssh -G hostname will show you.

What is ProxyJump used for?

Reaching a machine that is only accessible from another machine. ProxyJump bastion in the target’s block makes ssh target hop through the bastion in one command, and scp and rsync inherit it.

Does the SSH config file work with scp and rsync?

Yes. Anything that uses SSH as its transport reads the same file, so scp file.txt web:/tmp/ and rsync -av dir/ web:/srv/ both use the alias, the port and the key you defined.

Enjoyed this guide?

New articles on Linux, homelab, cloud, and automation every 2 days. No spam, unsubscribe anytime.

Keep Reading

Scroll to Top