Linux Essentials for Cloud — Part 2

File Systems, Users, Groups and Permissions, Package Management, System Logs and Networking.

Aug 19, 2025

The DynamoDB Book by Alex Debrie, AWS Hero(Sponsored)

If you’ve ever wanted to truly understand Amazon DynamoDB, The DynamoDB Book is one of the best resources you’ll ever read. Whether you’re just getting started or already experienced, this book breaks down DynamoDB concepts in a way that’s practical, clear, and easy to apply.

To make this even better, I’ve partnered with Alex to bring you an exclusive discount—because world-class learning should be accessible to everyone.

💡 Use code TOWARDSAWS at checkout and get 35% OFF today!

👉 Grab your copy now and start building smarter with DynamoDB.

Don’t miss your chance to level up your DynamoDB skills with the most recommended guide out there in the community.

BUY NOW (35%0FF)

In the Part 1 of Linux Essentials for Cloud, we talked about:

Linux Essentials for Cloud — Part 1

Kisan Tamang

Aug 12

Read full story

Linux Basics
Popular Linux distros
Using SSH, Terminal Shells
Basic Linux Commands
Permission and Ownerships
Process and Resource Management in Linux

This week, we are going to talk about:

File Systems and Storage

Disk management: df, du, lsblk, mount, umount
File system structure: /etc, /var, /home, /tmp, /proc, /dev
Creating and managing volumes
Persistent storage and block storage in the cloud

Users, Groups, and Permissions

Creating and managing users/groups: useradd, usermod, groupadd, passwd
Understanding /etc/passwd, /etc/group, and /etc/shadow
Sudoers and privilege escalation: sudo, visudo

Package Management

Debian-based (Ubuntu): apt, dpkg
RHEL-based (Amazon Linux, CentOS): yum, dnf, rpm
Updating and installing essential tools
Repositories and mirrors

System Logs and Troubleshooting

Viewing logs: journalctl, /var/log/, dmesg
Common logs: syslog, auth.log, cloud-init.log, messages
Log rotation and cleanup

Let’s get started!

1. File Systems and Storage

Disk Management

df - Display File System Usage:

df                        # Show disk usage for all mounted filesystems
df -h                     # Human-readable format (GB/MB)
df -T                     # Show filesystem type
df /                      # Show usage for root filesystem

du - Display Directory Usage:

du                              # Show disk usage for current directory
du -h                           # Human-readable format
du -s                           # Summary only (total size)
du -sh /var/log/                # Show total size of directory
du -ah | sort -hr | head -10    # Top 10 largest files/directories

lsblk - List Block Devices:

lsblk                            # Show all block devices in tree format
lsblk -f                         # Show filesystem information

mount and umount - Mount/Unmount Filesystems:

mount                              # Show all mounted filesystems
mount /dev/sdb1 /mnt/data          # Mount device to directory
mount -t ext4 /dev/sdb1 /mnt/data  # Mount with specific filesystem type
umount /mnt/data                   # Unmount filesystem
umount /dev/sdb1                   # Unmount by device

File System Structure

Everything in Linux is file. Understanding the Linux directory structure is most.

/etc - Configuration Files

System and application configuration files
/etc/passwd - User accounts
/etc/hosts - Hostname resolution
/etc/ssh/sshd_config - SSH daemon configuration
/etc/nginx/ - Nginx configuration
/etc/systemd/system/ - Systemd service files

/var - Variable Data

/var/log/ - Log files
/var/www/ - Web server files
/var/lib/ - Application data
/var/cache/ - Cached data
/var/spool/ - Print and mail queues

/home - User Home Directories

Personal directories for users
/home/username/ contains user-specific files
User configuration files (dotfiles)

/tmp - Temporary Files

Temporary storage, often cleared on reboot
Used by applications for temporary data
Anyone can write to /tmp

/proc - Process Information

Virtual filesystem providing process and system information
/proc/cpuinfo - CPU information
/proc/meminfo - Memory information
/proc/PID/ - Information about specific process

/dev - Device Files

Special files representing hardware devices
/dev/null - Null device (discards all writes)
/dev/zero - Provides null bytes
/dev/sda1 - First partition on first disk

Understanding Storage Types:

Block Storage:

Raw storage that appears as attached disk
Can be partitioned and formatted with any filesystem
Examples: AWS EBS, GCP Persistent Disks, Azure Managed Disks
High performance, suitable for databases and file systems

Object Storage:

Files stored as objects with metadata
Accessed via REST APIs or web interfaces
Examples: AWS S3, GCP Cloud Storage, Azure Blob Storage
Highly scalable, suitable for backups and static content

File Storage:

Network-attached storage with file system interface
Can be mounted on multiple instances simultaneously
Examples: AWS EFS, GCP Filestore, Azure Files
Suitable for shared application data

2. Users, Groups, and Permissions

Creating and Managing Users and Groups

useradd - Add New User

sudo useradd username               # Create user with defaults
sudo useradd -m username            # Create user with home directory
sudo useradd -m -s /bin/bash username  # Specify shell
sudo useradd -m -G sudo username    # Add user to sudo group
sudo useradd -c "Full Name" username   # Add description

usermod - Modify User Account

sudo usermod -aG sudo username      # Add user to sudo group
sudo usermod -s /bin/zsh username   # Change user shell
sudo usermod -l newname oldname     # Change username
sudo usermod -L username            # Lock user account
sudo usermod -U username            # Unlock user account

groupadd - Create New Group

sudo groupadd groupname             # Create new group
sudo groupadd -g 1001 groupname     # Create group with specific GID

passwd - Change Password

passwd                              # Change own password
sudo passwd username                # Change user's password
sudo passwd -l username             # Lock user's password
sudo passwd -u username             # Unlock user's password
sudo passwd -d username             # Delete user's password

Sudoers and Privilege Escalation

sudo - Execute Commands as Another User

sudo command                        # Run command as root
sudo -u username command            # Run command as specific user
sudo -i                             # Start interactive root shell
sudo -s                             # Start shell as root
sudo !!                             # Re-run last command with sudo

visudo - Edit Sudoers File:

sudo visudo                         # Safely edit /etc/sudoers

Sudoers File Examples:

# Allow user full sudo access
username ALL=(ALL:ALL) ALL

# Allow user to run specific commands without password
username ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart nginx

# Allow group members full access
%groupname ALL=(ALL:ALL) ALL

# Allow user to run commands as specific user
username ALL=(webuser) ALL

Security Best Practices:

Never edit /etc/sudoers directly, always use visudo
Use specific command paths in sudoers rules
Avoid NOPASSWD unless absolutely necessary
Regularly audit sudo usage with sudo -l
Use groups instead of individual users when possible

3. Package Management

Debian-based Systems (Ubuntu)

apt - Advanced Package Tool

apt is one of the most popular package management tool in Linux community.

sudo apt update                    # Update package index
sudo apt upgrade                   # Upgrade installed packages
sudo apt install package-name      # Install package
sudo apt remove package-name       # Remove package
sudo apt purge package-name        # Remove package and config files
sudo apt autoremove                # Remove unused packages
sudo apt search keyword            # Search for packages
sudo apt show package-name         # Show package information
sudo apt list --installed          # List installed packages

dpkg - Debian Package Manager

dpkg -l                            # List installed packages
dpkg -i package.deb                # Install .deb package
dpkg -r package-name               # Remove package
dpkg -L package-name               # List files installed by package
dpkg -S /path/to/file              # Find which package owns file

You can learn more about dpkg here: https://man7.org/linux/man-pages/man1/dpkg.1.html

RHEL-based Systems (Amazon Linux, CentOS)

yum - Yellowdog Updater Modified

sudo yum update                    # Update all packages
sudo yum install package-name      # Install package
sudo yum remove package-name       # Remove package
sudo yum search keyword            # Search for packages
sudo yum info package-name         # Show package information
sudo yum list installed            # List installed packages
sudo yum groupinstall "Development Tools"  # Install package group

dnf - Dandified YUM

sudo dnf update                    # Update all packages
sudo dnf install package-name      # Install package
sudo dnf remove package-name       # Remove package
sudo dnf search keyword            # Search for packages
sudo dnf info package-name         # Show package information
sudo dnf list installed            # List installed packages
sudo dnf groupinstall "Development Tools"  # Install package group

rpm - Red Hat Package Manager

rpm -qa                            # List all installed packages
rpm -qi package-name               # Show package information
rpm -ql package-name               # List files in package
rpm -qf /path/to/file             # Find which package owns file
sudo rpm -i package.rpm            # Install RPM package
sudo rpm -e package-name           # Remove package

Updating and Installing Essential Tools

Now let’s install some useful tools we use daily at work as a cloud builders.

Essential Tools for Cloud Engineers:

# Ubuntu/Debian
sudo apt update
sudo apt install -y curl wget git vim htop tree unzip awscli

# RHEL/CentOS/Amazon Linux
sudo yum update -y
sudo yum install -y curl wget git vim htop tree unzip awscli

Common Cloud Tools Installation:

# Install Docker (Ubuntu)
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Install kubectl
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

# Terraform
wget https://releases.hashicorp.com/terraform/1.6.0/terraform_1.6.0_linux_amd64.zip
unzip terraform_1.6.0_linux_amd64.zip

sudo mv terraform /usr/local/bin/

Repositories and Mirrors

Adding Repositories (Ubuntu):

# Add repository manually
sudo add-apt-repository "deb [arch=amd64] https://packages.docker.com/deb/ubuntu $(lsb_release -cs) stable"

# Add PPA (Personal Package Archive)
sudo add-apt-repository ppa:ondrej/php
sudo apt update

Managing Repositories (RHEL/CentOS):

# Add EPEL repository
sudo yum install epel-release

# Add custom repository
sudo yum-config-manager --add-repo https://repo.example.com/repo.repo

# List enabled repositories
yum repolist

4. System Logs and Troubleshooting

Viewing Logs

journalctl - Systemd Journal Viewer

journalctl                          # View all logs
journalctl -u nginx                 # View logs for specific service
journalctl -f                       # Follow logs in real-time
journalctl --since "2024-01-15"    # View logs since specific date
journalctl --until "2024-01-15 15:00:00"  # View logs until specific time
journalctl -p err                   # View only error level logs
journalctl -n 50                    # View last 50 entries
journalctl --disk-usage            # Check journal disk usage
journalctl --vacuum-time=2weeks    # Clean logs older than 2 weeks

/var/log/ Directory:

ls -la /var/log/                   # List all log files
tail -f /var/log/syslog            # Follow system log
tail -f /var/log/auth.log          # Follow authentication log
tail -f /var/log/nginx/error.log   # Follow Nginx error log

dmesg - Kernel Messages:

dmesg                               # View kernel messages
dmesg | grep -i error              # Filter for errors
dmesg -T                           # Show timestamps
dmesg -w                           # Follow new messages

Common Log Files

System Logs:

/var/log/syslog                     # General system messages (Ubuntu/Debian)
/var/log/messages                   # General system messages (RHEL/CentOS)
/var/log/auth.log                   # Authentication attempts
/var/log/kern.log                   # Kernel messages
/var/log/boot.log                   # Boot process messages

Application Logs:

/var/log/nginx/access.log           # Nginx access log
/var/log/nginx/error.log            # Nginx error log
/var/log/apache2/access.log         # Apache access log
/var/log/mysql/error.log            # MySQL error log
/var/log/docker.log                 # Docker daemon log

Cloud-Specific Logs:

/var/log/cloud-init.log             # Cloud-init process log
/var/log/cloud-init-output.log      # Cloud-init command output
/var/log/amazon/ssm/amazon-ssm-agent.log  # AWS SSM agent (Amazon Linux)

Log Rotation and Cleanup

logrotate Configuration:

# View logrotate configuration
cat /etc/logrotate.conf

# Check logrotate rules for specific service
cat /etc/logrotate.d/nginx

# Manually run logrotate
sudo logrotate -f /etc/logrotate.conf

# Test logrotate configuration
sudo logrotate -d /etc/logrotate.d/nginx

Example Logrotate Configuration:

/var/log/myapp/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    create 644 www-data www-data
    postrotate
        systemctl reload myapp
    endscript
}

5. Networking Basics

Network Information and Tools

ifconfig and ip - Network Interface Configuration

ifconfig                            # Show all network interfaces (older)
ip addr show                        # Show IP addresses (modern)
ip link show                        # Show network interfaces
ip route show                       # Show routing table

netstat and ss - Network Connections

netstat -tuln                       # Show listening ports
netstat -r                          # Show routing table
ss -tuln                            # Modern replacement for netstat
ss -s                               # Show network statistics

ping - Test Network Connectivity

ping google.com                     # Test connectivity to host
ping -c 4 8.8.8.8                  # Send only 4 packets
ping6 google.com                    # IPv6 ping

traceroute - Trace Network Path

traceroute google.com               # Show path to destination
traceroute -n google.com            # Show numeric IP addresses only

Testing Network Connections

telnet - Test Port Connectivity

telnet google.com 80                # Test if port 80 is open

# If connection succeeds, port is open

nc (netcat) - Network Swiss Army Knife

nc -zv google.com 80                # Test port connectivity
nc -l 8080                          # Listen on port 8080
nc -u -l 8080                       # Listen on UDP port 8080

curl - Transfer Data from Servers

curl http://example.com             # Download webpage
curl -I http://example.com          # Show only headers
curl -o file.html http://example.com # Save to file
curl -d "data=value" -X POST url    # POST request with data

wget - Download Files

wget http://example.com/file.zip    # Download file
wget -r http://example.com/         # Recursive download
wget -c http://example.com/file.zip # Continue interrupted download

Public vs Private IP

Public IP Addresses:

Accessible from the internet
Used for services that need external access
Often assigned by cloud provider's load balancer
May change when instance restarts (unless elastic/static IP)

Private IP Addresses:

Only accessible within the cloud network (VPC)
Used for internal communication between services
More secure as not directly exposed to internet
Typically remain stable throughout instance lifecycle

Cloud Networking Concepts:

VPC (Virtual Private Cloud): Isolated network environment
Subnets: Subdivisions of VPC for organizing resources
NAT Gateways: Allow private instances to access internet
Internet Gateways: Allow public instances internet access

Firewalls and Security Groups

iptables - Linux Firewall

iptables -L                         # List current rules
iptables -A INPUT -p tcp --dport 22 -j ACCEPT  # Allow SSH
iptables -A INPUT -p tcp --dport 80 -j ACCEPT  # Allow HTTP
iptables -P INPUT DROP              # Set default policy to DROP

ufw - Uncomplicated Firewall

ufw status verbose                  # Check firewall status
ufw enable                          # Enable firewall
ufw allow 22                        # Allow SSH
ufw allow 'Nginx Full'             # Allow Nginx (HTTP and HTTPS)
ufw deny 23                         # Block telnet