How to Configure a Linux Server: Step-by-Step Guide

Configuring a Linux server from scratch is one of the most valuable skills a developer or sysadmin can have. Whether you are setting up a new dedicated server for a web application, a game server, or a personal project, this guide walks you through every step: from the first SSH connection to a production-ready, secured environment. By the end, you will have a fully configured Linux server ready to deploy workloads.

⚡ Difficulty: Beginner to Intermediate

🖥 Applies to: Ubuntu 24.04 LTS / Debian 12 / AlmaLinux 9

Key Takeaways

• You will configure a Linux server from initial SSH access to a secured, network-ready environment.
• Tools required: a terminal (or PuTTY on Windows), your server's public IP address, and root credentials provided by Kimsufi.
• Estimated time: 45 to 90 minutes depending on your Linux distribution and familiarity with the command line.
• Skill level: beginner-friendly. No prior Linux server administration experience required.
• At the end of this guide, your server will have: a non-root sudo user, SSH key authentication, a configured firewall (UFW), and up-to-date packages.

Prerequisites: What You Will Need

Before you begin, make sure you have the following ready:

• A Kimsufi dedicated server: any KS, SYS, or RISE range will work. If you have not ordered one yet, you can deploy one in minutes from the Kimsufi control panel.
• Your server's public IP address and root credentials (sent to you by email after provisioning).
• A terminal application: on macOS/Linux, use the built-in Terminal. On Windows, use PowerShell, Windows Terminal, or PuTTY.
• An SSH key pair (recommended). If you do not have one yet, Step 3 covers how to generate it.
• A chosen Linux distribution. This guide uses Ubuntu Server 24.04 LTS as the primary example, with notes for Debian 12 and AlmaLinux 9 where commands differ.

💡 Tip Ubuntu Server 24.04 LTS is the most widely documented Linux distribution for servers and the best starting point for beginners. AlmaLinux 9 is the recommended choice for enterprise-grade environments requiring RHEL compatibility.

Overview: What We Will Do

Configuring a Linux server involves a sequence of well-defined steps: connecting securely, updating the system, creating a non-root user account, hardening SSH access, configuring the firewall, and verifying that everything works. Here is the complete sequence at a glance:

• Connect to your server via SSH as root.
• Update all system packages to the latest versions.
• Create a new non-root user with sudo privileges.
• Generate and deploy SSH keys for password-free, secure login.
• Configure SSH to disable root login and password authentication.
• Set up UFW (Uncomplicated Firewall) with the correct rules.
• Configure network settings and hostname.
• Verify the configuration and test the setup.

Step 1: Connect to Your Server via SSH

The first thing to do is establish an SSH connection to your server. Kimsufi provides the root user credentials and your server's public IP address in the provisioning email.

Open your terminal and run the following command, replacing YOURSERVERIP with your actual IP address:

ssh root@YOUR_SERVER_IP

If this is your first time connecting, you will see a fingerprint prompt. Type yes to continue. You will then be asked for the root password provided in your welcome email.

✅ Expected result You are now logged in as the root user. Your terminal prompt should read: root@your-hostname:~#

On Windows using PuTTY: open PuTTY, enter your server's IP in the Host Name field, select SSH and port 22, and click Open. Enter your root credentials when prompted.

Step 2: Update System Packages

Before configuring anything, bring your system up to date. This closes known security vulnerabilities and ensures you are working with the latest stable software versions.

Ubuntu / Debian

sudo apt update && sudo apt upgrade -y

AlmaLinux / RHEL-based

sudo yum update -y

or on newer systems:

sudo dnf update -y

This process may take a few minutes depending on how many packages need updating. Once complete, reboot the server if the kernel was updated:

sudo reboot

✅ Expected result All packages are up to date. After rebooting, reconnect via SSH. You should see the updated kernel version in the login banner.

Step 3: Create a Non-Root User with Sudo Privileges

The root user has unrestricted access to everything on the system. Operating as root for daily tasks is a significant security risk: a single typo can have irreversible consequences. Best practice is to create a regular user account and grant it sudo privileges for administrative tasks.

Create a new user (replace 'youruser' with your chosen username):

adduser youruser

Follow the prompts to set a password and fill in optional user information. Then add the user to the sudo group:

Ubuntu / Debian

usermod -aG sudo youruser

AlmaLinux / RHEL-based

usermod -aG wheel youruser

Verify the user was created and has sudo access:

su - youruser

sudo whoami

✅ Expected result The command 'sudo whoami' returns 'root', confirming the user has sudo privileges without needing to log in as root.

Step 4: Set Up SSH Key Authentication

SSH key authentication is significantly more secure than password-based login. It uses a public/private key pair: the public key lives on the server, and the private key stays on your local machine. Even if someone obtains your password, they cannot log in without the private key.

Generate an SSH key pair on your local machine

Run this command in your local terminal (not on the server):

ssh-keygen -t ed25519 -C "your@email.com"

Press Enter to accept the default file location (~/.ssh/id_ed25519). Set a passphrase for an additional layer of security.

Copy the public key to your server

ssh-copy-id youruser@YOUR_SERVER_IP

If ssh-copy-id is not available (for example on Windows), manually append the content of your ~/.ssh/ided25519.pub file to ~/.ssh/authorizedkeys on the server:

mkdir -p ~/.ssh && chmod 700 ~/.ssh

nano ~/.ssh/authorized_keys

Paste your public key, save and exit

chmod 600 ~/.ssh/authorized_keys

✅ Expected result You can now connect to your server using: ssh youruser@YOURSERVERIP, with no password prompt (or only your key passphrase if you set one).

Step 5: Harden SSH Configuration

With SSH keys in place, disable password authentication and root login. This significantly reduces your server's attack surface against brute-force attacks.

Open the SSH configuration file:

sudo nano /etc/ssh/sshd_config

Locate and update (or add) the following directives:

PermitRootLogin no

PasswordAuthentication no

PubkeyAuthentication yes

AuthorizedKeysFile .ssh/authorized_keys

Save the file (Ctrl+X, Y, Enter in nano) and restart the SSH service:

sudo systemctl restart sshd

⚠️ Warning Before closing your current session, open a second terminal window and verify you can log in with your new user and SSH key. If you lock yourself out, you will need to use the Kimsufi KVM console to recover access.

Step 6: Configure the Firewall with UFW

A firewall controls which network traffic is allowed to reach your server. UFW (Uncomplicated Firewall) is the standard tool on Ubuntu/Debian for managing iptables rules with a simple interface.

Install UFW if it is not already present (it is included by default on Ubuntu):

sudo apt install ufw -y

Set the default rules: deny all incoming traffic, allow all outgoing:

sudo ufw default deny incoming

sudo ufw default allow outgoing

Allow SSH before enabling the firewall (critical: missing this will lock you out):

sudo ufw allow OpenSSH

Add rules for the services you plan to run, for example:

sudo ufw allow 80/tcp    # HTTP

sudo ufw allow 443/tcp   # HTTPS

sudo ufw allow 3306/tcp  # MySQL (only if needed remotely)

Enable UFW and check its status:

sudo ufw enable

sudo ufw status verbose

✅ Expected result UFW is active. The status output shows your rules: SSH, HTTP, and HTTPS allowed. All other inbound traffic is blocked.

For AlmaLinux / RHEL-based systems, use firewalld instead:

sudo systemctl enable --now firewalld

sudo firewall-cmd --permanent --add-service=ssh

sudo firewall-cmd --permanent --add-service=http

sudo firewall-cmd --reload

Step 7: Configure Network Settings and Hostname

Setting a meaningful hostname makes server management easier, especially when managing multiple machines. Your Kimsufi server already has a public IP address assigned, but you may want to configure a static private network address or update the hostname.

Set the hostname

sudo hostnamectl set-hostname your-server-name

Update /etc/hosts to reflect the new hostname:

sudo nano /etc/hosts

Add or update the line:

127.0.1.1 your-server-name

Check network interfaces and IP address

ip addr show

ip route show

On Kimsufi servers, the public IP address is typically assigned via DHCP and is already configured. If you need to set a static IP (common for production servers), edit the Netplan configuration file on Ubuntu:

sudo nano /etc/netplan/00-installer-config.yaml

A static IP configuration looks like this:

network:

version: 2

ethernets:

eth0:

dhcp4: no

addresses: [YOURSERVERIP/24]

gateway4: YOUR_GATEWAY

nameservers:

addresses: [8.8.8.8, 1.1.1.1]

sudo netplan apply

✅ Expected result Running 'hostnamectl' shows your new hostname. 'ip addr show' displays your server's public IP address on the primary network interface.

Verification: How to Check It Worked

Before moving on, run through this checklist to confirm your Linux server is properly configured:

• SSH access with your non-root user: ssh youruser@YOURSERVERIP should connect without prompting for a password.
• Root login disabled: ssh root@YOURSERVERIP should return 'Permission denied (publickey)'.
• Packages up to date: run sudo apt list --upgradable (Ubuntu/Debian) or sudo dnf check-update (AlmaLinux): should return an empty list.
• Firewall active: sudo ufw status should show 'Status: active' with your defined rules.
• Hostname set: hostnamectl should display your chosen hostname under 'Static hostname'.
• Sudo works: sudo whoami should return 'root'.

✅ All checks passed? Your Linux server is configured and ready for production workloads. You can now install your application stack (LAMP, LEMP, Node.js, Docker, etc.).

Troubleshooting

Here are the most common issues encountered when configuring a Linux server, and how to fix them:

Going Further: Advanced Tips

1. Install Fail2ban to block brute-force attacks

Even with SSH key authentication, automated bots will continuously attempt to connect to your server. Fail2ban monitors log files and automatically bans IP addresses that show malicious behaviour (too many failed login attempts). For a broader security checklist, see our guide on how to secure a dedicated server.

sudo apt install fail2ban -y

sudo systemctl enable --now fail2ban

The default configuration monitors SSH and bans IPs after 5 failed attempts within 10 minutes. Check its status with: sudo fail2ban-client status sshd.

2. Set up automatic security updates

Keeping your server patched is critical. On Ubuntu/Debian, the unattended-upgrades package automatically installs security updates:

sudo apt install unattended-upgrades -y

sudo dpkg-reconfigure --priority=low unattended-upgrades

3. Monitor your server with system logs

Linux provides rich logging via journald and traditional log files in /var/log/. Key logs to monitor:

• /var/log/auth.log: SSH logins, sudo usage, authentication failures (Ubuntu/Debian).
• /var/log/secure: same as above on AlmaLinux/RHEL-based systems.
• sudo journalctl -xe: real-time system events and error messages.

For a more powerful monitoring solution, consider installing Netdata or Prometheus + Grafana for real-time dashboards on your Kimsufi server. If you are ready to take the next step with your server, see our Linux dedicated server guide for more advanced configuration topics.

FAQ

How long does it take to configure a Linux server from scratch?

For a basic secure setup (steps 1 to 6 in this guide), expect 45 to 90 minutes. The time largely depends on package update times and your familiarity with the command line. Experienced sysadmins can complete the setup in under 20 minutes using automation tools like Ansible.

What is the best Linux distribution for servers?

For most use cases, Ubuntu Server LTS (Long Term Support) is the best starting point: it has the largest community, the most documentation, and 5 years of official support. For enterprise environments requiring Red Hat Enterprise Linux (RHEL) compatibility, AlmaLinux 9 or Rocky Linux are excellent free alternatives. Debian 12 is preferred when stability is the absolute priority.

What is the role of the root user?

The root user is the superuser account on a Linux system: it has unrestricted access to all files, processes, and system settings. For day-to-day server management, use a regular user account with sudo privileges rather than logging in as root directly. This limits the damage if the account is compromised or a command is run by mistake.

Can I skip setting up SSH keys and use a password instead?

Technically yes, but it is strongly discouraged. Password-based SSH authentication is vulnerable to brute-force attacks. SSH keys are mathematically secure and essentially impossible to brute-force with modern key lengths (ed25519 or RSA 4096-bit). For any server exposed to the public internet, including your Kimsufi server, SSH keys are non-negotiable.

How do I configure network settings if I do not know my gateway IP?

On Kimsufi servers, your gateway IP is typically your server's IP address with the last octet replaced by .254 (for example, if your IP is 192.168.1.100, the gateway is 192.168.1.254). You can also retrieve it before making changes with: ip route show | grep default.

What tools are needed for Linux server management?

At minimum: a terminal with SSH access, a text editor (nano for beginners, vim for advanced users), and the package manager for your distribution (apt for Ubuntu/Debian, dnf/yum for AlmaLinux). Beyond that, useful tools include htop (process monitoring), tmux (terminal multiplexer for persistent sessions), rsync (file transfers), and cron (task scheduling).

How do I configure a firewall on a Linux server?

On Ubuntu/Debian, UFW (Uncomplicated Firewall) is the standard tool: see Step 6 of this guide for the complete setup. On AlmaLinux/RHEL-based systems, firewalld is used instead. For advanced use cases requiring fine-grained rules, nftables or iptables give you full control over network traffic filtering.

Conclusion

You now have a fully configured Linux server: packages updated, a non-root sudo user in place, SSH keys set up, root login disabled, a firewall protecting your open ports, and a clean network configuration. This is the secure, production-ready foundation every Linux server should start from. The next step is to deploy your application stack on top of it, whether that is a LAMP/LEMP stack for a web server, Docker for containerised applications, or setting up automated backups to protect everything you have just built.

Deploy your Linux server today Get started with Kimsufi dedicated servers from $11.10/month. Root access, unlimited traffic, built-in DDoS protection.