Linux Tricks Cheatsheet

Password Generators

# pwgen
$ brew install pwgen            # MacOS
$ sudo dnf install pwgen        # Fedora
$ pwgen 12
sei9AhPiokai jaezooThahn4 yahghooCh5ha uquiCh7soog0 Pahthe0fe5Ku owaht4xooPhu
eet2eijeeT6E Chie8oz1Enee Piemu7pi1uqu TheebohNg8se eil2AhNeiF2s WueGh8guoxie

# apg
$ sudo dnf install apg          # Fedora
$ apg -n 3 -m 12 -M SNCL        # pronounceable
tit8OdcigEp~
Plaf@dryRec6
JavJu%Wudek6
$ apg -a 1 -n 3 -m 12 -M SNCL   # random
,+c"e3P[Me%H
ZlL0qw_8L)Ff
1x=2nizg}x!X

# gpg
$ brew install gpg              # MacOS
$ sudo dnf install gpg          # Fedora
$ gpg --gen-random --armor 1 14
spdhuOS2il2JjhOq2ZU=

tree Directory Viewer

dnf Useful Commands

Fedora package installer DNF

$ sudo dnf install httpd                            # install httpd
$ sudo dnf install httpd-manual -y                  # assume yes
$ sudo dnf dnf check-update                         # check for available updates
$ sudo dnf update                                   # update installed packages
$ sudo dnf install unbound-1.4.20-28.el7.x86_64.rpm # install local package
$ sudo dnf remove httpd                             # remove package
$ sudo dnf reinstall httpd -y                       # reinstall package
$ sudo dnf search php                               # search for a package
$ sudo dnf provides /etc/httpd/conf/httpd.conf      # which package provides the file
$ sudo dnf provides httpd                           # which package provides 'http'
$ sudo dnf info httpd                               # package info
$ sudo dnf repoquery --list httpd                   # list the files installed by package
$ sudo dnf history                                  # installation history
$ sudo dnf history info 13                          # what did install 13 do
$ sudo dnf history undo 13 -y                       # undo install 13
$ sudo dnf history redo 13 -y                       # redo install 13
$ sudo dnf list installed                           # list installed packages
$ sudo dnf grouplist                                # list which groups are available, installed, not-installed.
$ sudo dnf groupinfo "System Tools"                 # what is installed by this group

Terminal Pagers

Stolen from the Fedora Magazine: 5 cool terminal pages post.

$ more --help                   # trusty original with limited features
$ more <file>                   #
$ more <file1> <file2> <file3>  # ':n' next file, ':p' previous file

$ less --help                   # many features
$ less <file>                   #
$ less <file1> <file2> <file3>  # ':n' next file, ':p' previous file, ':e' new file

$ most --help                   # good for 'wide' files
$ most <file>                   # screens: 'ctl-x 2' split, 'ctl-x 1' close , 'ctl-x o' switch
$ most <file1> <file2> <file3>  # split-screen and ':n' next file, ':p' previous file

$ pspg --help                   # table friendly pager
$ cat t.csv
a;b;c;d;e
1;2;3;4;5
$ cat t.csv | pspg --csv

mysql> pager pspg;              # replace less or more as pager
$ export PAGER=pspg; mycli ...  # MySQL CLI example
$ export PAGER=pspg; pgcli ...  # PostgreSQL CLI example

Cat File Tricks

$ cat -n <filename>                      # adds line number prefix
$ cat -e <filename>                      # shows crlf ending (Unix, DOS, MacOS)
$ cat -n <filename> | head -5            # (beginning) first 5 lines
$ cat -n <filename> | tail -5            # (ending) last 5 lines
$ cat -n <filename> | tail -10 | head -5 # (middle) first 5 of last 10 lines

Grep File Tricks

$ cat flintstones.yaml
---
family: flintstones
members:
  - Name: Fred
    Age: 35
    Gender: male
  - Name: Wilma
    Age: 25
    Gender: female
  - Name: Pebbles
    Age: 1
    Gender: female
  - Name: Dino
    Age: 5
    Gender: male

$ grep Fred flintstones.yaml
  - Name: Fred

$ grep Name flintstones.yaml
  - Name: Fred
  - Name: Wilma
  - Name: Pebbles
  - Name: Dino

$ grep "Name|Age" flintstones.yaml    # no output
$ grep -E "Name|Age" flintstones.yaml # Extended (a.k.a egrep)
  - Name: Fred
    Age: 35
  - Name: Wilma
    Age: 25
  - Name: Pebbles
    Age: 1
  - Name: Dino
    Age: 5

$ grep Age flintstones.yaml -A 1     # one line After match
    Age: 35
    Gender: male
--
    Age: 25
    Gender: female
--
    Age: 1
    Gender: female
--
    Age: 5
    Gender: male

$ grep Age flintstones.yaml -B 1     # one line Before match
  - Name: Fred
    Age: 35
--
  - Name: Wilma
    Age: 25
--
  - Name: Pebbles
    Age: 1
--
  - Name: Dino
    Age: 5

$ grep Age flintstones.yaml -C 1     # one line Context (before/after) match
  - Name: Fred
    Age: 35
    Gender: male
  - Name: Wilma
    Age: 25
    Gender: female
  - Name: Pebbles
    Age: 1
    Gender: female
  - Name: Dino
    Age: 5
    Gender: male

JSON, YAML File Filtering

  • jq is a lightweight command-line JSON processor, similar to sed.

  • yq is a Python command-line (jq wrapper) YAML/XML processor.

# Installation
$ sudo dnf install jq      # Fedora
$ brew install jq          # MacOS
$ pip install yq           # Python
$ winget install jqlang.jq # Windows

# Command Line examples
$ echo '{"fruit":{"name":"apple","color":"green","price":1.20}}' | jq '.' # pretty-print
{
  "fruit": {
    "name": "apple",
    "color": "green",
    "price": 1.2
  }
}

# Get International Space Station Current Location
$ curl http://api.open-notify.org/iss-now.json | jq '.' # pretty-print HTTP response
{
  "message": "success",
  "iss_position": {
    "longitude": "103.2534",
    "latitude": "-44.3309"
  },
  "timestamp": 1719322950
}
# Installation
# Linux
$ VERSION=v4.43.1
$ BINARY=yq_linux_amd64
$ sudo wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq
$ sudo chmod +x /usr/bin/yq

$ brew install yq                  # MacOS
$ winget install --id MikeFarah.yq # Windows

# Command Line examples
$ echo '{"fruit":{"name":"apple","color":"green","price":1.20}}' | yq '.'
{"fruit": {"name": "apple", "color": "green", "price": 1.20}}

# Get International Space Station Current Location
$ curl http://api.open-notify.org/iss-now.json | yq '.' # pretty-print HTTP GET response
{"message": "success", "iss_position": {"longitude": "103.9546", "latitude": "-44.0234"}, "timestamp": 1719322960}

XML, HTML File Filtering

# Installation
$ sudo dnf install xq                               # Fedora
$ brew install xq                                   # MacOS
$ curl -sSL https://bit.ly/install-xq | sudo bash   # Linux, installs into /usr/local/bin

# Command Line example
$ curl -s https://www.w3schools.com/xml/note.xml | xq
<?xml version="1.0" encoding="UTF-8"?>
<note>
  <to>Tove</to>
  <from>Jani</from>
  <heading>Reminder</heading>
  <body>Don't forget me this weekend!</body>
</note>

Repology

Repology shows you in which repositories a given project is packaged, which version is the latest and which needs updating, who maintains the package, and other related information.

HTTP Header Checking

$ curl -I 127.0.0.1:8080
HTTP/1.1 200 OK
Server: nginx/1.27.0
Date: Sat, 01 Jun 2024 15:14:01 GMT
Content-Type: text/html
Content-Length: 4253
Last-Modified: Sat, 01 Jun 2024 14:14:45 GMT
Connection: keep-alive
ETag: "665b2cd5-109d"
Accept-Ranges: bytes

$ wget -S --spider 127.0.0.1:8080
Spider mode enabled. Check if remote file exists.
--2024-06-01 17:13:56--  http://127.0.0.1:8080/
Connecting to 127.0.0.1:8080... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Server: nginx/1.27.0
  Date: Sat, 01 Jun 2024 15:13:56 GMT
  Content-Type: text/html
  Content-Length: 4253
  Last-Modified: Sat, 01 Jun 2024 14:14:45 GMT
  Connection: keep-alive
  ETag: "665b2cd5-109d"
  Accept-Ranges: bytes
Length: 4253 (4.2K) [text/html]
Remote file exists and could contain further links,
but recursion is disabled -- not retrieving.

Email Checking

Shameless copy of the LinkedIn post by Jan Schaumann

$ sudo dnf install bind-utils                            # Install dig, if necessary
$ dig +short MX yahoo.com                                # DNS MX records
$ dig +short TXT yahoo.com | grep spf                    # domain spoofing check
$ dig +short TXT selector._domainkey.yahoo.com           # DKIM email authentication method
$ dig +short TXT _dmarc.yahoo.com                        # DMARC (spf and/or DKIM)
$ dig +short TXT _mta-sts.yahoo.com                      # MTA-STS (is TLS enforced)
$ curl https://mta-sts.yahoo.com/.well-known/mta-sts.txt # MTA-STS (is TLS enforced)
$ dig +short TXT _smtp._tls.yahoo.com                    # SMTP TLS Reporting
$ dig +short TLSA _port._tcp.yahoo.com                   # DANE check (no results?)
$ dig +short TXT default._bimi.yahoo.com                 # BIMI check (no results?)

To help understand these commands

Gnome Desktop Custom Launcher

Using PyCharm Community Edition as an example, download the PyCharm Community Edition and unpack the tar.gz file into $HOME/Applications

Create the com.jetbrains.pycharm.community.desktop file, modify it as necessary, and then copy it to $HOME/.local/share/applications

$ cat com.jetbrains.pycharm.community.desktop
[Desktop Entry]
Encoding=UTF-8
Name=PyCharm
Exec=/home/<user>/Applications//bin/pycharm.sh
Icon=/home/<user>/Applications/pycharm-community/bin/pycharm.png
Type=Application
Version=2022.2.2
Terminal=false
Categories=Development;

$ cp ./com.jetbrains.pycharm.community.desktop $HOME/.local/share/applications

Base 64 Encode/Decode

$ echo -n "EncodeMe-in-Base64" | base64
RW5jb2RlTWUtaW4tQmFzZTY0

$ echo -n "RW5jb2RlTWUtaW4tQmFzZTY0" | base64 -d
EncodeMe-in-Base64

Using Python

>>> import base64
>>> _ascii = "EncodeMe-in-Base64".encode("ascii")
>>> _b64bytes = base64.b64encode(_ascii)
>>> print(_b64bytes.decode("ascii"))
RW5jb2RlTWUtaW4tQmFzZTY0

>>> import base64
>>> _ascii = "RW5jb2RlTWUtaW4tQmFzZTY0".encode("ascii")
>>> _b64bytes = base64.b64decode(_ascii)
>>> print(_b64bytes.decode("ascii"))
EncodeMe-in-Base64

WSL2 on Windows

Read the prerequisites in, Install Linux on Windows with WSL

Installation can now be done via the Microsoft Store

First enable Windows optional features to run WSL, so the sequence is as follows.

1. Windows -> Settings -> Optional Features -> More Windows Features
    - [x] Virtual Machine Platform
    - [x] Windows Subsystem for Linux
2. Reboot
3. Install WSL from Microsoft Store
4. Reboot
5. Install Ubuntu (20.04.6 LTS) from Microsoft Store

Update Ubuntu

$ man apt-get
$ sudo apt-get update  # sync the package index files
$ sudo apt-get upgrade # install the newest versions
$ sudo reboot

$ man apt
$ sudo apt update      # sync the package index files
$ sudo apt upgrade     # install the newest versions
$ sudo reboot

$ apt --help

Linux Network Tools

Command

Description

ping, ping6

Send ICMP ECHO_REQUEST to network hosts

hping3

TCP/IP equivalent of ping

curl, wget, HTTPie

Access URL meta-data or content

tc

Show / manipulate traffic control settings

dig, nslookup, host, whois

DNS lookup utilities

ssh, scp sftp

Secure client connection and copy

telnet, ftp,

Insecure client connection and copy

rsync

Sophisticated remote/local file-copying

tcpdump, wireshark, tshark

Dump and analyze network traffic

ngrep

Network grep

ifconfig, route, ethtool, ip

Show/manipulate ip routing, devices, and tunnels

iw, iwconfig

Configure a wireless network interface

nmap, zenmap

Network exploration tool and security/port scanner

p0f

Identify remote systems passively

openvpn, wireguard stunnel

Secure VPN tunnels

nc, socat

Arbitrary TCP and UDP connections and listeners

netstat, ss, lsof, fuser

Troubleshoot connections, processes, file usage

iptables, ip6tables, nftables

Firewall, TCP/IP packet filtering and NAT

arp, arptables,

Manipulate the system ARP cache

traceroute, mtr, tcptraceroute

Troubleshoot connections, processes, file usage

iptraf, nethogs, iftop, ntop

Troubleshoot network bandwidth issues

ab, nload, iperf

Benchmarking tools

ipcalc

Simple manipulation of IP addresses

nsenter

Enter container process’s namespace

Python HTTP modules

HTTP serve files in CWD, python -m http.server

Brendan Gregg’s Homepage

G’Day. I use this site to share and bookmark various things, mostly my work with computers. While I currently work on large scale cloud computing performance at Intel (previously Netflix), this site reflects my own opinions and work from over the years. I have a personal blog, and I’m also on twitter.

This page lists everything: Documentation, Videos, Software, Misc. For a short selection of most popular content, see my Overview page.

Managing .rc files

By default, rcm uses ~/.dotfiles for storing all the dotfiles it manages.

A managed dotfile is actually stored inside ~/.dotfiles, and a symlinked.

For example, if ~/.bashrc is tracked by rcm, a long listing would look like this.

$ ls -l ~/.bashrc
lrwxrwxrwx. 1 link link 27 Dec 16 05:19 .bashrc -> /home/geoff/.dotfiles/bashrc

rcm consists of 4 commands:

  • mkrc – convert a file into a dotfile managed by rcm

  • lsrc – list files managed by rcm

  • rcup – synchronize dotfiles managed by rcm

  • rcdn – remove all the symlinks managed by rcm

Fedora 36 Live CD install

Of course backup everything you want to keep because you are going to reformat the HDD or SSD!

The live installation is process is well documented and robust so simply follow:

Next add the RPM Fusion repositories, by installing and configuring them as described in RPMFusion Configuration

Finally consult Fedora Quick Docs especially the Adding and managing software section.

Some of the perennial audio and video playback issues are still there, so follow these instructions.

$ sudo dnf install gstreamer1-plugins-{bad-\*,good-\*,base} gstreamer1-plugin-openh264 gstreamer1-libav --exclude=gstreamer1-plugins-bad-free-devel
$ sudo dnf install lame\* --exclude=lame-devel
$ sudo dnf group upgrade --with-optional Multimedia