Programster's Blog

Tutorials focusing on Linux, programming, and open-source

Linux CLI Cheatsheet

cheatsheet cli linux-administration

Below are some short snippets of code to help you perform day-to-day tasks in Linux. If you can't find what you are looking for, make sure to look at the BASH cheatsheet.

Related Posts

Table of Contents

  1. JSON
  2. Filesystem
  3. File Manipulation
  4. Disk Management
  5. User Management
  6. Group Management
    1. Create Group
    2. Get Group's ID
    3. Get Name Of Group From ID
    4. List All Groups In System
    5. Change Group's ID
    6. Rename Group
  7. Networking
    1. Output IPs
    2. Stop Processes That Are Using Port
    3. Blink NIC
    4. Output All Interfaces
    5. Output Specific Interface
    6. Bring Interface Down
    7. Bring Interface Up
    8. Network Manager - List DNS Nameservers For Interface
    9. Get HTTP Status Code With Curl
  8. Time
    1. Get Timezone
    2. Set Timezone
    3. Get Unix Timestamp
    4. Get Date
  9. Misc
    1. Set Default Editor
    2. Hashes
    3. Copy CD To Disk
    4. Set File Timestamp
    5. Download File And Set Custom Name
    6. Get Time Taken For Command To Run
    7. CentOS - Check If Reboot Required
    8. Get Ubuntu Release Name
    9. Generate Random Passowrd
    10. Find Servers Fingerprint
    11. Regenerate OpenSSH Host Keys
    12. Change Keyboard Language
    13. Output Current Locale / Keymap
  10. References

JSON  

Allo the following examples use jq which can be installed with

sudo apt install jq -y

If you have some command or script that outputs in JSON, then it may not be in a readable format. To have it output to the terminal in a readable format, just pipe it to jq like so:

cat myJsonData.json | jq

If you then want that output to go to a file, do the following:

echo '[1,2,3,4,5,6,7,8,9,10]' | jq . > my-output-file.json

Filtering  

If you wish to filter down to a specific sub block, then you can do this like so:

cat someComplicatedJson.json | jq '.primaryBlockKeyName.subBlockKey.tertiaryBlockKey

You can chain sub-blocks with . as many times as you desire.

For more examples of using jq, refer to the following:

Filesystem  

Copy File(s) But Don't Overwrite  

Sometimes you want to copy a file to another location, but are worries about the possibility of overwriting something that already exists. If that's the case, use the -n or --no-clobber parameter.

cp -n $SOURCE $DEST

Move File(s) But Don't Overwrite  

In the same situation as above, but for moving file(s).

mv -i

or

mv --interactive

Unlike with cp -n, this will propt you if you wish to overwrite files, whereas cp -n will just immediately exit with an error.

Get Folder Sizes  

If you've ever completely filled a system to the point where you can't even install ncdu to track where all your storage is gone, you can get the size of all the folders at your current level by executing:

du -sh *

However, if you can use ncdu instead, that is a much better tool.

Move all Files of a Type To Own Folder

If you have a jumble of files with no apparent structure, perhaps from a backup, and want to move a ceratain file type (in this example PDFs) into their own directory then you can execute the following:

mkdir pdf
find . -name "*.pdf" -type f -exec /bin/mv {} ./pdf \;

Obviously if you just wanted to copy the files instead, use:

mkdir pdf
find . -name "*.pdf" -type f -exec /bin/cp {} ./pdf \;

If you forget to create the directory and it doesn't already exist then you may lose your data.

Using xargs may be better, please refer here.

List All Files Within All Subdirectories

find . -type f

If you wish to also list the directories:

find .

List All Files With Specific File Extensions

find ./ -type f \( -iname \*.jpg -o -iname \*.png \)

List all Files Over A Certain Size

The following command will recursively find individual files larger than [size] in MB. This could be useful for finding really large files to consider compressing/deleting.

find . -type f -size +[size]M

List all Files Over A Certain Age

The following command will find all the files over $NUM_DAYS in age.

find . -mtime +$NUM_DAYS

Validate Fstab

There are two possible options here:

sudo findmnt --verify --verbose

... or:

sudo mount --fake --all --verbose

The key thing here is the "fake" which causes everything to occur, except for the actual system call.

File Manipulation

Replace Text In Files (Sed)

I use variations of the sed command to manipulate config files for automated scripts

SEARCH="search text here"
REPLACE="replace text here"
FILEPATH="/path/to/file.conf"
sed -i "s;$SEARCH;$REPLACE;" $FILEPATH

If you want to replace an entire line, then just use .* as shown below.

SEARCH="bind-address.*"
REPLACE="bind-address = $PRIVATE_IP"
FILEPATH="/etc/mysql/my.cnf"
sudo sed -i "s;$SEARCH;$REPLACE;" $FILEPATH

You can use [[:space:]]\+ when you don't know how many whitespaces or tabs there are. E.g.

SEARCH="datadir[[:space:]]\+= /var/lib/mysql"
REPLACE="datadir = /media/data/mysql"
FILEPATH="/etc/mysql/my.cnf"
sudo sed -i "s;$SEARCH;$REPLACE;" $FILEPATH

To replace an entire line that contains text that you are looking for in the middle of the line, you can use the following:

SEARCH="^.*\bYour pattern or sentence goes here\b.*$"
REPLACE="This will replace the entire line"
FILEPATH="$DIR/dump.sql"
sed -i "s;$SEARCH;$REPLACE;" $FILEPATH

source

Write To File  

echo "my content here" | tee $FILEPATH

echo "my content here" > $FILEPATH

cat << EOF > [filepath]
[script content]
EOF

Write To File With Sudo Privileges  

echo "my content here" | sudo tee $FILEPATH

cat << EOF  | sudo tee $FILEPATH
[script content]
EOF

cat << EOF > [filepath]
[script content]
EOF

Append To File  

echo "my content here" >>  $FILEPATH

Append To File (With Sudo Privs)  

echo "my content here" | sudo tee -a $FILEPATH

cat << EOF  | sudo tee -a $FILEPATH
[script content]
EOF

Split File Into Chunks

Let's say you have a massive text file, perhaps a giant log file. If that text file became unmanageable, and you wanted to split it into lot of smaller files that had up to x number of lines each, then you could do the following with the split command:

INPUT_FILE="my-file.txt"
OUTPUT_FILE_PREFIX="output-"
OUTPUT_FILE_EXTENSION=".txt"
MAX_LINES_PER_FILE=10000

split \
  --verbose \
  --numeric-suffixes \
  --suffix-length=2 \
  --lines=$MAX_LINES_PER_FILE \
  --additional-suffix=$OUTPUT_FILE_EXTENSION \
  $INPUT_FILE \
  $OUTPUT_FILE_PREFIX

If you are going to need more than 99 files, then increase the --suffix-length accordingly. E.g. 3 for up to 999 files.

Binary Split

Let's say you have a giant binary file, perhaps a gzipped version of your log or SQL file. You would likely want to split this based on bytes, rather than lines like so:

INPUT_FILE="dump.sql.gz"
OUTPUT_FILE_PREFIX="dump-"
OUTPUT_FILE_EXTENSION="sql.gz"
CHUNK_SIZE="512m"

split \
  --numeric-suffixes \
  --suffix-length=3 \
  --bytes=$CHUNK_SIZE \
  --additional-suffix=$OUTPUT_FILE_EXTENSION \
  $INPUT_FILE \
  $OUTPUT_FILE_PREFIX

A common use case for this may be to split an important file into small chunks for transferring. Thus if you lose your connection whilst transferring the files, you can pretty much resume where you left off, rather than starting all over again. You will still have to replace the partially transferred file though. This should work well with something like S3 sync.

Merge Split Files

If you wish to merge files together from perhaps splitting them earlier, then you can do this through cat like so:

cat dump-* > dump.sql.gz

This works on both binary and text-based file formats.

Ordering

The example above that uses dump-* does rely on your files listing in the correct order, which may vary depending on your locale settings. If you want to specify the order, you could do the following:

cat file1 file2 file3 >> combined_file

Gzip A File

The following command will take the given file, and replace it with a gzipped one with the same name, but with a .gz extension. This is particularly useful for MySQL dumps and log files.

gzip myLogFile.txt

To then undo the operation do:

gzip --decompress myLogFile.txt.gz

Alternatively, the --uncompress flag is a synonym that will do the same thing:

gzip --uncompress myLogFile.txt.gz

This will replace the .gz file with the decompressed file. If you want to keep the file, simply add the --keep flag.

Disk Management  

Get Utilization  

df -h

Or you could install pydf for a simpler interface.

Secure Erase Disk  

The best way to erase a disk is to use secure erase. For an SSD, this will reach areas that the other commands won't, and return it to peak performance.

Alternatively, you can use shred

shred -n 1 -vz /dev/sdb

... or use scrub

scrub -p dod /dev/sdb

... or you could use dd.

Check SATA Connection of Drives  

dmesg | grep -i  ahci | grep -i --color Gbps

Check the SATA Connection Of Specific Drive  

smartctl -a /dev/sda | grep "^SATA"

You will need to install the smartctl package if you haven't already.

Get Block Size  

sudo blockdev --getbsz /dev/sd[x]

The output will be in bytes, so typically this will be 4096 to represent a 4k block size.

List Block Devices

sudo lsblk

Example output:

NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0   15G  0 disk 
└─sda1   8:1    0   15G  0 part /
sdb      8:16   0 1000G  0 disk /mnt/data
sdc      8:32   0 1000G  0 disk

List Disk Labels

Using lsblk, we can get all of the details of the block devices, including their labels like so:

sudo lsblk -o name,mountpoint,label,size,uuid

Output When Disk Last Checked

The following command works on a disk or disk-partition basis to find out when it was last checked for errors:

DISK=/dev/sda1
sudo tune2fs -l $DISK | grep checked

Be sure to change the path for the DISK variable.

User Management  

Add User With Adduser  

The simplest way to add users is to use the command below which will ask you a series of questions and set up their home directory.

sudo adduser [username]

Add User With Useradd  

If you are wanting to add a user as part of a script, or just don't want to set up a home directory, you can use

sudo useradd [username]

In that format, it will create a user without a home directory or even a password, use the options -m to automatically create a home directory, and -p to specify the encrypted password. For example, the commands below will create a user called test with a random password:

USER="test"
PASSWORD=`openssl rand -base64 16`
PASSWORD=${PASSWORD::-2}
ENCRYPTED_PASSWORD=$(openssl passwd -1 ${PASSWORD})
sudo useradd -m -p $ENCRYPTED_PASSWORD $USER
echo "Please log in with password: $PASSWORD"

Delete User  

sudo deluser [user]

A full-blown command in which we specify the user ID, group ID, whether to create a home, and the shell to use would be:

sudo useradd \
  --no-create-home  \
  --uid 1019  \
  --gid 100  \
  --shell /usr/sbin/nologin  \
   myUserName

Get User's ID  

id --user $USERNAME

If you just want your own user ID, you can run:

id --user

Change User's ID  

The following command would change the programster user to user ID 1026

sudo usermod -u 1026 programster

You would need to run this command as another user, when the specified user is not logged in.

Get Users Primary Group ID  

id --group $USER

Change Users Primary Group ID  

You can also do the following to set a user's primary group:

usermod --gid primary_group_name programster

The user's primary group determines the default group ID on all files/folders the user creates, thus all files the user now creates will have group ownership set to that by default.

Add User To Group  

sudo adduser $USERNAME $GROUPNAME

The adduser command may not always be available. In such a scenario, you may wish to use:

sudo usermod \
  --append $USER_NAME \
  --groups $GROUP_NAME

All other tutorials I see mention doing usermod -a -G $GROUP_NAME $USER_NAME, but I have tested that this way round works, and I feel it is more memorable and intutitive.

Append User To Multiple Groups  

If you want to append a user to multiple groups in one go, then you can do the following:

sudo usermod \
  --append $USER_NAME \
  --groups $GROUP1,$GROUP2,$GROUP3

Set User's Groups  

If you want to specify the list of groups a user should be part of, which would append/remove the user from groups as appropriate to ensure they match the list, then you would do the following:

sudo usermod \
  $USER_NAME \
  --groups $GROUP1,$GROUP2,$GROUP3

Remove User From Group  

sudo deluser $USERNAME $GORUPNAME

List Groups User Belongs To

groups $USER_TO_CHECK

Group Management  

Create Group  

sudo groupadd myGroupName

If you wish to specify the ID of the group, you would use:

sudo groupadd --gid 1003 myGroupName

Get Group's ID  

The following will output the ID of the specified group:

cut -d: -f3 < <(getent group $GROUP_NAME)

Get Name Of Group From ID  

Use the following to convert a group ID, to the name of the group

getent group $GROUP_ID | cut -d: -f1

List All Groups In System  

The following command will list all the groups on the system.

cut -d: -f1 /etc/group | sort

Change Group's ID  

groupmod --gid $DESIRED_GROUP_ID $GROUP_TO_CHANGE

For example, the following command would change the "admins" group to 1026.

sudo groupmod --gid 1026 admins

If the name of the group being changed, is the primary group of the logged in users ID, then you would need to run this command as another user.

Rename Group  

groupmod --new-name $NEW_GROUP_NAME $CURRENT_GROUP_NAME

e.g.

groupmod --new-name apacheUser www-data

Networking

Output IPs

hostname -I

If you want each IP on a newline, use the command below:

hostname -I | xargs | tr [:space:] '\n'

Stop Processes That Are Using Port  

The following command will kill all processes that are using the port 1234.

lsof -ti:1234 | xargs kill -9

If you have multiple NICs in a computer and want to know which one is which, you can make it blink. The command below will make eth0 blink for 30 seconds.

ethtool -p eth0 30

Output All Interfaces  

Networkctl

sudo networkctl status --all

IP Addr

ip addr

Output Specific Interface  

If you just want details about a single interface, you can do either of the following:

Networkctl

INTERFACE="enp2s0"
sudo networkctl status $INTERFACE

IP Addr

INTERFACE="enp2s0"
ip addr show $INTERFACE

Bring Interface Down

DEVICE_NAME="eno1"
sudo ip link set dev $DEVICE_NAME down

The command will immediately return, even before it has fully taken effect.

Bring Interface Up

DEVICE_NAME="eno1"
sudo ip link set dev $DEVICE_NAME up

The command will immediately return, even before it has fully taken effect.

Network Manager - List DNS Nameservers For Interface

If you are using the network manager that comes bundled with distros like Ubuntu Desktop, then you may wish to list the DNS namservers your machine is set to use from all the different sources (DHCP etc).

DEVICE_NAME="wlp0s20f3"
nmcli device show $DEVICE_NAME | grep DNS

Get HTTP Status Code With Curl

If you need to find out what the HTTP status code returned from a URL is, you can do this using curl like so:

curl -s -o /dev/null -w "%{http_code}\n" https://some.website.com/

The %{http_code}\n is the important part, and the \n is just to neatly print out the code on a separate line in the terminal. If using in a script, you probably want to take that out.

Time

Get Timezone

date +'%:z %Z' # +01:00 BST

Set Timezone

sudo dpkg-reconfigure tzdata

Get Unix Timestamp

date +%s

Get Date

date +%d-%m-%Y # 29-12-2012

Misc  

Set Default Editor

There are multiple ways one can do this. For a tempory solution for your current session, you can do:

export EDITOR=vim

To permanently make the change, you can run:

sudo update-alternatives --config editor

... or:

select-editor

Hashes  

md5sum $FILEPATH | awk '{print $1}'
sha1sum $FILEPATH | awk '{print $1}'
sha256sum $FILEPATH | awk '{print $1}'

Copy CD To Disk  

If you have a CD-Rom drive (remember those things?), and want to create an ISO version on your computer, you can do it like so:

dd if=/dev/cdrom of=filename.iso

Set File Timestamp  

We all know that you can use touch $FILENAME to update a file to say it was edited right now. However, if you want to manually set a time that is in the past, you can use:

touch -d '1 June 2018 11:02' $FILENAME

Here is a little php snippet for programmatically setting time of files:

<?php

$filename = "test-php-file.txt"; // specify name for file to create/edit timestmap of
$timestamp = time() - (60*60); // set a unix timestamp here.
$dateString = date("d F Y H:i:s", $timestamp);
$cmd = "touch -d '{$dateString}' {$filename}";
shell_exec($cmd);

Download File And Set Custom Name  

The following command will use wget to download a file, but set your own custom name for the file when it is downloaded.

wget --output-document="my-custom-filename.zip" http://my.domain.com/path-to-file

Get Time Taken For Command To Run  

Sometimes it's useful to know how long a command takes. All you have to do is wrap it like so:

CMD="YOUR COMMAND GOES HERE"
utime="$( TIMEFORMAT='%lU';time ( $CMD ) 2>&1 1>/dev/null )"
echo "$utime"

CentOS - Check If Reboot Required  

#!/bin/bash
LAST_KERNEL=$(rpm -q --last kernel | perl -pe 's/^kernel-(\S+).*/$1/' | head -1)
CURRENT_KERNEL=$(uname -r)

test $LAST_KERNEL = $CURRENT_KERNEL || echo REBOOT

Get Ubuntu Release Name  

lsb_release -cs

This should output something like:

xenial

Generate Random Password  

This is my favourite option from How-To-Geek: 10 Ways to Generate a Random Password from the Command Line

openssl rand -base64 32

If you need it to not contain special characters, you could use:

NUM_CHARACTERS=24
head /dev/urandom | tr -dc A-Za-z0-9 | head -c$NUM_CHARACTERS

... or if hexadecimal is ok (all lowercase):

NUM_CHARACTERS=24
openssl rand -hex $NUM_CHARACTERS

Find Server's Fingerprint  

If you swap change a server around on an IP or domain name, you may see the following message:

ECDSA host key for dns.programster.org has changed and you have requested strict checking.
Host key verification failed.

If you wish to manually check what your server's fingerprint is, you can run the following command:

ssh-keygen -l -f /etc/ssh/ssh_host_ecdsa_key.pub

The -l tells ssh-keygen to "show fingerprint of specified public key file." and the -f is for specifying the filepath to the file we wish to work on.

Regenerate OpenSSH Host Keys

You may wish to regenerate your server's SSH identity if you are creating it from cloning, and receive messages like these:

This host key is known by the following other names/addresses:
    ~/.ssh/known_hosts:797: [hashed name]
    ~/.ssh/known_hosts:800: [hashed name]
    ~/.ssh/known_hosts:801: [hashed name]

To regenerate your server's SSH host key/identity run the following (on the server):

sudo rm /etc/ssh/ssh_host_* \
  && sudo dpkg-reconfigure openssh-server \
  && sudo systemctl restart ssh

Any clients that connected in the past will need to be updated to use the new public key. Otherwise they wills see the warning WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!

Change Keyboard Language

KEYMAP="gb"
sudo localectl set-keymap $KEYMAP

To list possible keymaps, use the command: sudo localectl list-keymaps

Output Current Locale / Keymap

sudo localectl

References

Last updated: 21st November 2023
First published: 16th August 2018
Programster's Blog Ⓒ 2023