File management

GNU coreutils

https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands

https://wiki.archlinux.org/title/Core_utilities

test – check file types and compare values
uniq – report or omit repeated lines
dirname – strip last component from file name
basename – strip directory and suffix from filenames
pathchk – check whether file names are valid or portable
dd – convert and copy a file
readlink – print resolved symbolic links or canonical file names

ls – list directory contents

-A, –almost-all	do not list implied . and ..
-h, –human-readable	with -l and -s, print sizes like 1K 234M 2G etc.
-k, –kibibytes	default to 1024-byte blocks for disk usage; used only with -s and per directory totals
-l	use a long listing format
-n, –numeric-uid-gid	like -l, but list numeric user and group IDs
-r, –reverse	reverse order while sorting
-R, –recursive	list subdirectories recursively
-S	sort by file size, largest first
-t	sort by time, newest first; see –time
-d, –directory	list directories themselves, not their contents

# ls -rlS

List only directories:

ls -l -d */

cp – copy files and directories

`-l, --link`	hard link files instead of copying
`-R, -r, --recursive`	copy directories recursively

Merge two folders using cp:

cp -rl source/folder destination
rm -r source/folder

Note that the source folder shouldn’t be in use, otherwise there’s a chance that new files in the source will be deleted without having been linked…

mkdir – make directories

`-m, --mode=MODE`	set file mode (as in chmod), not a=rwx – umask
`-p, --parents`	no error if existing, make parent directories as needed

Create a new folder and sets its permissions/mode

mkdir -pm755 /etc/apt/keyrings

move all files except some

mkdir archived
mv !(archived) archived/

Create several folders at once

mkdir /docs/folder{1..5}

du – estimate file space usage

`-h, --human-readable`	print sizes in human readable format (e.g., 1K 234M 2G)
`-x, --one-file-system`	skip directories on different file systems
`-d, --max-depth=N`	print the total for a directory (or file, with –all) only if it is N or fewer levels below the command line argument; –max-depth=0 is the same as –summarize
`--apparent-size`	print apparent sizes, rather than disk usage; although the apparent size is usually smaller, it may be larger due to holes in (‘sparse’) files, internal fragmentation, indirect blocks, and the like

du -hxd 1 Documents/

Something important to note on compressed filesystems, du might show a different size for two identical datasets while ls won’t. To avoid this problem, one should use –apparent-size.

du --apparent-size -hxd 1 Documents/

Coupled with sort:

du --apparent-size -hxd1 Documents/archived | sort -hr
du --apparent-size -hxd1 Documents/archived | sort --human-numeric-sort --reverse

cut – remove sections from each line of files

btrfs subvolume list -a /mnt/storage | cut -d' ' -f9

sort – sort lines of text files

btrfs subvolume list -a /mnt/storage | cut -d' ' -f9 | sort

realpath – print the resolved path

leo@computer:~$ realpath todo
/home/leo/todo

me@computer:~$ realpath ../../bin/cd
/usr/bin/cd

tee – read from standard input and write to standard output and files

tar – an archiving utility

`-x, --extract, --get`	Extract files from an archive. Arguments are optional. When given, they specify names of the archive members to be extracted.
`-v, --verbose`	Verbosely list files processed.
`-f, --file=ARCHIVE`	Use archive file or device ARCHIVE.
`-z, --gzip, --gunzip, --ungzip`	Filter the archive through gzip(1).
`-C, --directory=DIR`	Change to DIR before performing any operations. This option is order-sensitive, i.e. it affects all options that follow.

Untar shit

tar -xvf DKU_16.4.0.tar
tar -zxvf lnx_drv_celerity8_2220f1.tgz

tail – output the last part of files

head – output the first part of files

head file.txt               # first 10 lines
tail file.txt               # last 10 lines
head -n 20 file.txt         # first 20 lines
tail -n 20 file.txt         # last 20 lines
head -20 file.txt           # first 20 lines
tail -20 file.txt           # last 20 lines
head -n -5 file.txt         # all lines except the 5 last
tail -n +5 file.txt         # all lines except the 4 first, starts at line 5

Others utilities

watch – execute a program periodically, showing output fullscreen

`-d, --differences [permanent]`	Highlight the differences between successive updates.
`-n, --interval seconds`	Specify update interval.

To watch the contents of a directory change, you could use:

watch -d ls -l-d

Watch open ports variation:

watch -d -n 0.5 ss -tulpn

xargs – build and execute command lines from standard input

Switches	Function
`-0, --null`	Input items are terminated by a null character instead of by whitespace, and the quotes and backslash are not special (every character is taken literally). Disables the end of file string, which is treated like any other argument. Useful when input items might contain white space, quote marks, or backslashes. The GNU find -print0 option produces input suitable for this mode.
`-I replace-str`	Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character. Implies -x and -L 1.
`-P max-procs --max-procs=max-procs`	Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xargs will run as many processes as possible at a time.

Removes all BTRFS subvolumes in one go (there’s no recursive switch…)

btrfs subvolume list . | grep timeshift-btrfs | cut -d ' ' -f 9 | xargs -I % btrfs subvolume delete /mnt/KINGSTON_SA1000M8/%

Use several processes

find /path -name '*.foo' | xargs -P 24 -I '{}' /cpu/bound/process '{}' -o '{}'.out

https://en.wikipedia.org/wiki/Xargs

lsof – list open files

lsof +r2 | grep '/some/dir'

IT Hero's adventures