Javad's mail server guide

Unix notes

Follow these steps to add a new disk into the system. The idea is to correctly label the new disk based on its physical location in the rack. This would help you to easily locate the disk in the future.

  1. When installing the new disks, write down the serial number printed on the disks, and the physical location of the disks in the rack (rack number, shelf number, disk slot).
  2. On Linux, use lsblk to get a list of all disks. On FreeBSD, use camcontrol devlist. Then use below commands to fetch the S/N of every disk:
    3. 	# diskinfo -v {diskname}        # FreeBSD
	# lsblk -o +MODEL,SERIAL      # Linux
  3. From step 1, you have a list of phyisical locations and serial numbers. From step 2, you have a list of device nodes and serial numbers. Both lists have serial numbers, so you can match these up to get a list of device nodes and physical locations.
  4. Now you can label the disks based on their physical location. Use a consistent labeling scheme and type for all disks. Some label types have limitation in their number of characters, so you may want to shorten the serial number. If you want to do so, strip out the first few characters, as the last characters are the most unique ones. My hard disk's SN is WD-WXF1AC079835, it is installed on rack number 3, on shelf 2, and on slot 7. So its label would be r3s2d7AC079835.
  5. Use this label everywhere, if possible, disable other labeling schems, so that the disk can be accessed only and only through this lable.
  6. Now if a disk goes faulty, you can easily locate the disk based on its label, which itself is based on the physical location of the disk. When replacing the disk, write down its SN again, so that you could create a proper label for it.

Before you dispose of a disk, you need to wipe out all of its data. There is an ATA `secure erase` command which can do it for you. But this command is password-protected on the disk level. This is to reduce the chance of accidental invokation of the command. Therefore you need to set a password on the drive before invoking the secure erase command.

On Linux: 

$ sudo hdparm --user-master u --security-set-pass {password} {disk node}
$ sudo hdparm --user-master u --security-erase {password} {disk node}
On FreeBSD: 
# camcontrol security {disk} -U user -s {password} -e {password}
If the disk doesn't support this command, you can use dd(1), but use a large block size: 
# dd if=/dev/zero of=/dev/disk bs=128k

There are several label types for disk partitions. `filesystem labels` are stored inside the filesystem. If the filesystem is erased, the label is gone. You set filesystem labels with filesystem utilities like mkfs(8) and e2label(8). In fstab, use LABEL= to use filesystem labels. 

# mkfs -t ext4 -L {label_name} {disk node}

Another type of label for disk partitions is `GPT label`, which resides in the partition itself, not on the filesystem. In fstab, use PARTLABEL= to use GPT labels. On Linux, you can set this type of label on a partition using the gdisk(8) utility. 

# gdisk {disk node}
Use the c command to set a label, and then use the w command to save your changes.

To find out what package a particular file belongs to:

On Redhat: 

$ rpm -qf /path/to/file
Debian: 
	$ dpkg-query -S /path/to/file
FreeBSD: 
$ pkg which /path/to/file

Handy ps command which gives more technical details than ps aux: 

$ ps lax

Add ww to have unlimited column width, which is very handy if you want to examine long command-line arguments

Advanced shell redirections

>&		To redirect both STDOUT and STDERR
2>		To redirect STDERR only
1>&2		To redirect STDOUT to STDERR

To read kernel logs (on any Unix system):

$ dmesg

On Linux: 

journalctl -k
journalctl --dmesg

Always use vipw and vigr to edit /etc/passwd and /etc/group files. Do not edit these files directly, specifically on FreeBSD. These commands will open the files in the editor specified by the EDITOR environmental variable.

On Linux, ss(8) is the new netstat. Examples:

To display all sockets: 

ss -a

Add -n to avoid name resolution.

To display listening sockets only: 

ss -l

You can add -t or -u to display TCP and UDP sockets, and -4 or -6 to display IPv4 or IPv6 sockets

To include the process info in the output: 

ss -p

See ss(8) for more information

Memorize the shell's for loop, it will come in handy. Some commands can't handle multiple arguments, and can operate on just a single file. unzip(1) is an example. You can work around this using the for loop.

for VAR in ITEM_LIST; do cmd $VAR; done

To unzip many files at once:

for zfile in *.zip; do unzip $zfile; done

Both Linux and FreeBSD have monolithic kernels and additional modules can be added and removed dynamically. In Linux, kernel modules are stored in /lib/modules/{version}, and in FreeBSD, in /boot/kernel.

To load a module into the kernel:

# modprobe {module_name}               # Linux
# kldload {module_name}                # FreeBSD

To unload a module:

# modprobe -r {module_name}           # Linux
# kldunload {module_name}             # FreeBSD

To list all currently loaded modules:

# lsmod                               # Linux
# kldstat                             # FreeBSD

If you want to load a module automatically at boot time, you must edit /etc/modprobe.conf on Linux, and /boot/loader.conf on FreeBSD. On Linux, you can dynamically generate an /etc/modprobe.conf file that corresponds to all your currently installed modules by running modprobe -c.

Unless statically compiled, a binary file most likely uses some shared libraries (.so files), which are its run-time dependencies. You can find out which libraries a binary file is linked to with the following command:

	$ ldd {binary-file-path}

Example on /bin/ls: 

% ldd /bin/ls
/bin/ls:
	libutil.so.9 => /lib/libutil.so.9 (0x80024c000)
	libncursesw.so.9 => /lib/libncursesw.so.9 (0x800264000)
	libc.so.7 => /lib/libc.so.7 (0x8002d8000)

Before unmounting a filesystem, make sure that no process is using any file inside the mountpoint: 

% fuser -c {mountpoint}

The above command will print the PID of every process that’s using a file or directory on that filesystem. The nature of activity is reported by a series of letters, of which the most imporant are: 

c		current working directory
x		executes a program from there

To perform a reverse DNS lookup:

% dig -x {addr}     # from BIND package
% drill -x {addr}    # from NSD package
where {addr} is an IP address.

On your public, Internet-facing servers, disable ICMP redirects. ICMP redirects can maliciously reroute traffic and tamper with your routing tables.

# sysctl net.ipv4.conf.default.accept_redirects=0         # Linux
# sysctl net.inet.icmp.drop_redirect=1                    # FreeBSD

Disable broadcast pings on your servers; such packets have been used in denial-of-service attacks. 

# sysctl net.ipv4.icmp_echo_ignore_broadcasts=1         # Linux
# sysctl net.inet.icmp.bmcastecho=0                     # FreeBSD

Encrypt your swap space. Passwords, private keys and other important data are stored in memory and if swapped out into the persistant storage (disks), they can be recovered.

Use column(1) to better stylize the output of commands.

% cat /etc/passwd | column -t -s:

In above command, -s specifies the delimiter character, which is : for /etc/passwd and -t creates a table out of the input.

You can easily create an .iso image from a CD-ROM by running:

# cat /dev/cdrom > cdrom.iso

You really don't need dos2unix to convert text-files with Microsoft Windows line-endings into Unix format:

% tr -d \\r {dosfile} {newfile}

To run the last command again, just type

!!

in your shell and hit the Enter key (works in tcsh, zsh, bash).

You don't need to re-type a long command if you know its first few characters.

% !find

will run the last command starting with the literal string find.

On systems that use UEFI, use efibootmgr(8) to manipulate UEFI boot manager. To print a summary of boot config:

# efibootmgr -v

To boot a Linux system into the single-user mode, reboot the system, then highlight your desired kernel at GRUB splash screen, and press e. Then append systemd.unit=rescue.target to the end of the line, like:

	linux16 /vmlinuz-3.10.0-229.el7.x86_64 root=/dev/mapper/rhel_rhel-root
	ro crashkernel=auto rd.lvm.lv=rhel_rhel/swap rd.lvm.lv=rhel_rhel/root
	rhgb quiet LANG=en_US.UTF-8 systemd.unit=rescue.target

Type <Control-X> to contiune the boot process.

The root filesystem is mounted as read-only, you may need to remount it as rw:

# mount -o rw,remount /                # Linux
# mount -o rw /                        # FreeBSD

You can trace a process and see what system calls it invokes. This is extremely useful for debugging a process.

On Linux:

$ strace ls -l			# runs ls -l and tells you what system calls it invokes
$ strace -p {pid}		# to trace an already running process

On FreeBSD:

# sysctl security.bsd.unprivileged_proc_debug=1
% truss ls -l

Note that strace may interrupt the system calls invoked by the process, so the if the process doesn't restart interrupted system calls, it may not behave correctly, keep this in mind.

cron(8) uses SIGHUP to re-read the crontab files.

rpm(8) command cheat-sheet:

To list all installed packages:

$ rpm -qa

To find out which packages depend on a specific package:

$ rpm -q --whatrequires {package}

If multiple .rpm files are listed on the command-line, rpm(8) sorts them by dependency before installation.

To list the dependencies of an .rpm file or an installed package:

$ rpm -qpR {.rpm-file}
$ rpm -qR {package}

Other useful operations

rpm -ivh {rpm-file}                to install a package from an .rpm file
rpm -Uvh {rpm-file}                to upgrade a package from an .rpm file
rpm -ev {package}                  to remove a package
rpm -qf {/path/to/file}            which package owns the file?
rpm -qi {package}                  display info about {package}
rpm -qc {pacakge-name}             shows the list of config files
rpm -qdf {/path/to/file}           to query docs of a file (ex binary)
rpm -qa --last                     displays the recently installed packages
rpm -qpR {rpm-file}                list dependencies before installation
rpm -U --test {rpm-file}           test-only, do not install
dpkg(1) cheat-cheet: 
dpkg -i {.deb package}            installs a package or upgrades it if it's already installed
dpkg -R {directory-name}          installs all packages recursively from a directory
dpkg -r {package}                 removes the package, keeps the config files
dpkg -P {package}                 removes everything, including config files
dpkg -l                           lists all installed packages
dpkg -L {package}                 lists installed files by a packages
dpkg -p {package}                 displays info about a package
dpkg -S {/path/to/file}           finds what package owns the file
dpkg -c {.Deb package}            lists files inside the .deb
dpkg -x {.deb} {dir}              extracts .deb into {dir}
dpkg --configure {package}        reconfigures a package

If you are filtering the output generated by tail -f with the grep command, add the --line-buffered flag to see each matching line as soon as it becomes available. Works with FreeBSD's grep as well.

$ tail -f /var/log/messages | grep --line-buffered {pattern}

To read a file line by line in sh:

while IFS= read -r LINE; do
	echo $LINE
done < {filename}

wtmp (sometimes wtmpx) contains a record of users' logins and logouts as well as entries that record when the system was rebooted or shut down. the file is binary. Use the last(1) command to decode the information.

lastlog contains information similar to that in wtmp, but it records only the time of last login for each user.

Many network interfaces can calculate TCP checksums. Normally this is a great thing because it save CPU from having to compute the checksums. But on VMs where virtual network interfaces have no real chipset for this calculation, you may want to offload TCP checksumming from your network interfaces. If you are planning to capture network traffic with Wireshark or TCPDump, it's better to off-load checksumming. In Linux, this is done through ethtool and in FreeBSD, it is done through ifconfig.

Always run tcpdump with the -n option, because tcpdump may miss some packets while it is waiting for a slow DNS query.

Among other things, the file /etc/nsswitch.conf specifies how name resolution should be performed. It specifies the order in which various name databases (like DNS, /etc/hosts) should be queried. For example, the following line specifies that for any given doman name, the system should look /etc/hosts first, and only if it doesn't contain the answer, DNS must be queried. 
hosts: files dns
powered by FreeBSD logo powered by vi logo Valid HTML 4.0!


This page was edited on Sat Jun 11 04:02:03 EEST 2022