First Steps to Begin Securing Your Linux Server

Although you might assume that your server is already secure, there are some simple practices you should be following to harden your server. In the event of a compromise, these practices will allow you to limit the damage that can be done by attackers.

Although this guide was written for Ubuntu Server 20.04 LTS (focal) this guide should be mostly transferable to other Linux distributions.

1. Get rid of the root account

In some scenarios, you are first given access to a Linux server via the root account. The first thing you should do is disable this account. You should be using sudo from another user account to execute privileged commands.

There are several reasons why this is beneficial, such as allowing for better auditing and accountability (since users now have to use sudo). Doing so also provides more safeguards and (in some cases) better protects you in the event of account compromises (since the user password will be required even if using private-key authentication over SSH).

First, we will create a new sudo-level user: ubuntu.

adduser ubuntu
usermod -aG sudo ubuntu
passwd --delete root

Setting an SSH key for the new user account

From our host computer, let's generate a key-pair, copy it to the server, and remove the temporary password:

Host computer

ssh-keygen -t ed25519 -f hostname-ubuntu-key.pem
ssh-copy-id -i ./hostname-ubuntu-key.pem.pub ubuntu@hostname
rm hostname-ubuntu-key.pem.pub

You can now log in to your server as the ubuntu user with our new private key:

ssh -i hostname-ubuntu-key.pem ubuntu@hostname

From now on we will be making all changes as the ubuntu user.

2. Update all packages

Ensuring your system is up-to-date is an important way of preventing vulnerabilities that allow attackers access to your server.

On Ubuntu apt is used to manage packages.

sudo apt update
sudo apt upgrade
sudo reboot

3. Tighten SSH security

Using SSH key authentication offers convenience (password-less login), while also offering better security than most passwords.

By making some changes to the SSH configuration we can disallow password authentication; empty or otherwise. This makes us rely on using the key we set up in Step 1. At the same time, we will disable root login over SSH. Since we deleted the root password earlier this is redundant, but doing so will ensure root login is never allowed should it return.

Use sudo to edit /etc/ssh/sshd_config and make sure the following are set:

• PasswordAuthentication no
• PermitEmptyPasswords no
• PermitRootLogin no
• PubkeyAuthentication yes

Lastly, we need to restart the SSH daemon:

sudo systemctl reload sshd

4. Enable the firewall

Running a firewall on your server is an additional line of defense that can help prevent illegitimate services from communicating with the outside world.

This is very simple thanks to ufw (Uncomplicated Firewall). We need to start by allowing OpenSSH (before enabling) or we will restrict ourselves from being able to access the server!

sudo ufw allow OpenSSH
sudo ufw enable

Allowing Nginx through the firewall

We can define additional custom applications to allow them through the firewall.

To create the application profile for Nginx, use sudo to create the file /etc/ufw/applications.d/nginx-service containing:

[Nginx HTTP]
title=Web Server 
description=Enable NGINX HTTP traffic
ports=80/tcp

[Nginx HTTPS] \
title=Web Server (HTTPS) \
description=Enable NGINX HTTPS traffic
ports=443/tcp

[Nginx Full]
title=Web Server (HTTP,HTTPS)
description=Enable NGINX HTTP and HTTPS traffic
ports=80,443/tcp

Then enable the application profile that you need (either Nginx HTTP, Nginx HTTPS, or Nginx Full). In most cases this will be Nginx Full:

sudo ufw allow Nginx\ Full

5. Use Docker rootless (or Podman)

By default, Docker runs as the root user. Since the Docker containers and the Docker daemon are running as root, this can be a security risk. We wouldn't want to expose ourselves to any potential vulnerabilities that could allow attackers to break out of the containers (into the host system).

Running Docker as a non-root user limits the container runtime's access to the underlying host system. This minimizes the risk of privilege escalation attacks that could potentially compromise the entire system. Rootless Docker also ensures better separation between containers, reducing the risk of one container affecting others on the same system.

Docker has wide support and availability, but Podman also offers a similarly mature and integrated solution. Podman is architected from the ground up to be daemonless and rootless. Podman also aims to be a drop-in replacement for Docker - consider using Podman for your needs instead.

See my full guide on dealing with rootless Docker here.

Final thoughts

This guide is in no way comprehensive. This guide serves as a very brief introduction to the topic. There are endless guides and resources on the topic, including The Practical Linux Hardening Guide.

Tags