Linux

Discussing Linux commands and apps that aren't directly related to self hosting but are generally useful.

Linux Troubleshooting

How to deal with specific problems I encountered in Linux.

Linux Troubleshooting

How to fix Read-Only EXT4 drive

An external ext4 drive mounts as read-only in Linux usually because the kernel detected file system corruption or an improper unmount and switched it to read-only mode to prevent data loss. It can also be caused by hardware failure or root-only root directory permissions. [1, 2, 3, 4]
How to Fix and Troubleshoot
1. Identify the Drive Partition
Find your external drive's device name (e.g., /dev/sdb1 or /dev/sdc1):
Run: lsblk or df -h
2. Unmount and Repair the File System
You must unmount the drive before running a file system check (fsck):
Unmount the drive: sudo umount /dev/sdX1 (replace sdX1 with your actual partition)
Repair errors: sudo e2fsck -f /dev/sdX1
Follow any prompts to fix or clear blocks. [1, 2, 3, 4]
3. Remount as Read-Write
Once the repair finishes cleanly, mount it back normally: [1]
Mount command: sudo mount -o remount,rw /dev/sdX1 or unplug and plug it back in. [1, 2]
4. Fix Ownership Issues (If Not Corrupted)
If fsck finds no errors but you still cannot write, the root folder of the drive likely belongs to the root user: [1]
Mount the drive manually to a folder or navigate to its mount point.
Change ownership to your current user: sudo chown -R $USER:$USER /path/to/mount/point

Useful Commands

"sudo pacman -Sy"
"sudo pacman -S git fakeroot binutils"
"cd ~/Downloads"
"git clone https://aur.archlinux.org/realvnc-vnc-viewer.git"
"cd realvnc-vnc-viewer"
"sudo pacman -U realvnc-vnc-viewer-*-x86_64.pkg.tar.zst"

"su" - elevate prompt so you don't need sudo
"pwd" - show current directory
"sudo fdisk -l" - ?
"mhwd -li" - show ??
"pamac install gvfs-smb" - install samba

"find . -type d -empty -delete" - delete empty subfolders

"rsync -rvh /path/to/dir1 /path/to/destination" - basic copy command with rsync that copies one folder and all sub-folders into a destination

"rsync -avh /path/to/dir1/ /path/to/dir2/ /path/to/destination/" - copy the contents of two folders into one destination with rsync
Example:  "rsync -avh /a/Media/ /b/Media/ /c/Media/" - results in /c/Media containing the contents of the two source Media folders
"rsync -avh /a/Media /b/Media /c/" - another way to get the same results as the previous one
"rsync -avh /a/Media /b/Media /c/Media" - Results in /a/Media and /b/Media being merged into /c/Media/Media

-a (archive): Preserves symbolic links, permissions, ownership, and timestamps.  Identical to -dR --preserve=all, preserving recursive structure, symlinks, and attributes.
     --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
     --no-OPTION             turn off an implied OPTION (e.g. --no-D)
-H (Hard Links): Preserves hard links in the transfer (not included in -a).
-A and -X: Preserve ACLs (Access Control Lists) and Extended Attributes, respectively.
-v (Verbose): Increases the information provided during the transfer, listing each file as it is processed.
-q, --quiet                 suppress non-error messages
     --no-motd               suppress daemon-mode MOTD (see caveat)
-c (Checksum): Forces rsync to compare files based on checksums rather than the default "quick check" of file size and modification time.
-r, --recursive             recurse into directories
-R, --relative              use relative path names
     --no-implied-dirs       don't send implied dirs with --relative
-P: Combines --partial (keeps partially transferred files if the sync is interrupted) and --progress (shows a progress bar during the transfer).
-u, --update                skip files that are newer (modified time) on the receiver
     --inplace               update destination files in-place
     --append                append data onto shorter files
     --append-verify         --append w/old data in file checksum
-z (Compress): Compresses data during the transfer to reduce network usage, which is ideal for slow connections.
-h (Human-Readable): Displays file sizes and transfer speeds in a more readable format (e.g., KB, MB, GB) instead of raw bytes.
-n or --dry-run: Performs a trial run without making any actual changes. This is critical for testing your command before execution

If you're moving large files and you want the option to resume a cancelled or failed rsync operation from the exact point that rsync stopped, you need to use the --append or --append-verify switch on the next attempt.

Another example:  rsync -achvuXPn --remove-source-files Media/ /mnt/omv/media/

In this example, it does it's best to preserve the original files, uses a checksum to verify integrity of existing files, human readable verbose logs, skips newer (modified time) files, preserves extended attributes, allows resuming, deletes source after copy and is a test run.

--delete: Deletes files at the destination that no longer exist at the source. Use this to create an exact mirror, but always use --dry-run first to avoid accidental data loss.
-e <command>: Specifies the remote shell to use, most commonly ssh. For example: rsync -avz -e ssh source/ user@remote:/dest/.
--exclude='PATTERN': Excludes files or directories matching a specific pattern from the transfer.
--ignore-existing (Optional but Recommended): Only copies files that do not exist on the destination, ignoring existing files, which saves time.
Trailing / (Crucial): Ensure you include a trailing slash on the source path (e.g., /source/) to copy the contents of the folder, not the folder itself.

References:  
https://linuxize.com/post/how-to-use-rsync-for-local-and-remote-data-transfer-and-synchronization/
https://linux.die.net/man/1/rsync

"cp" - alternate copy command

Never use "cp -r" if you need to maintain soft links (symlinks) as links; -r copies the actual file the link points to. Use -a instead.

The importance of sudo (rsync)

Adding sudo prevents "permission denied" errors if the blockage is caused by a lack of read or write privileges on files and folders. However, how you apply sudo depends on whether you are running rsync locally or over a remote SSH connection. [1, 2, 3]
Local Transfers
Put sudo at the very front of your command: sudo rsync -av /source/ /destination/.
This runs the entire process with root permissions, bypassing local file restrictions. [1, 2, 3]
Remote Transfers (via SSH)
Typing sudo rsync locally only elevates privileges on your local machine. If the permission error happens on the remote server, you must tell rsync to invoke sudo on the remote side using the --rsync-path flag: [1, 2, 3, 4]
Command: rsync -av -e ssh --rsync-path="sudo rsync" /local/path/ user@remote:/remote/path/
Important: The remote user account must be configured for passwordless sudo (NOPASSWD) for rsync in the remote /etc/sudoers file. Otherwise, rsync will hang or fail when it tries to prompt you interactively for a password.

Docker Commands

docker stats --format "table {{.ID}}\t{{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.MemPerc}}\t{{.NetIO}}\t{{.BlockIO}}\t{{.PIDs}}" --no-stream

docker stats --format "table {{.ID}}\t{{.Name}}\t{{.CPUPerc}}\t{{.MemPerc}}\t{{.NetIO}}\t{{.BlockIO}}" --no-stream