Monitoring

Utilities

top – display Linux processes

htop – interactive process viewer

atop – Advanced System & Process Monitor

iostat – Report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions

-d Display the device utilization report
-h Make the Device Utilization Report easier to read by a human. –human is enabled implicitly with this option

Shows disk activity every 2 seconds (measured since the last 2 seconds)

iostat -m -d 2

iotop – simple top-like I/O monitor

Use the left and right arrows to change the sorting,

r to reverse the sorting order,
o to toggle the –only option,
p to toggle the –processes option,
a to toggle the –accumulated option,
q to quit or
i to change the priority of a thread or a process’ thread(s).

Any other key will force a refresh.

iftop – display bandwidth usage on an interface by host

-i interface	Listen to packets on interface
-F net/mask	Specifies an IPv4 network for traffic analysis. If specified, iftop will only include packets flowing in to or out of the given network, and packet direction is determined relative to the network boundary, rather than to the interface.

Shows the rate at which data has been sent and received over the preceding 2, 10 and 40 second intervals

iftop -i eth0
iftop -F 10.0.0.0/255.0.0.0

bwm-ng – Bandwidth Monitor NG (Next Generation), a live bandwidth monitor for network and disk io

`-i, --input method`	`selects which method to use. It can be one of the above (see INPUT METHODS) if support for it was compiled in.`
`-u, --unit value`	`selects which unit to show. It can be one of bytes, bits, packets or errors.`
`-I, --interfaces list`	`show only interfaces which are in this comma separated list (whitelist). If the list is prefixed by a '%' its meaning is negated and interfaces in this list are hidden from output (blacklist). (Example: %eth0,tun0)`
`-o, --output method`	`selects which method to use for output. It can be one of the above (see OUTPUT METHODS) if support for it was compiled in. (curses,curses2,plain, csv, html)`

bwm-ng --input disk --output curses

bwm-ng -i disk -I sda,dm-0,dm-1,dm-2,dm-3,dm-4,dm-5,dm-6,dm-7

bwm-ng -i disk -I nvme0n1,nvme1n1,sda,sdb,sdd,sde,sdf -o curses2

  bwm-ng v0.6.1 (probing every 0.500s), press 'h' for help
  input: disk IO type: rate
  |         iface                   Rx                   Tx                Total
  ==============================================================================
              sda:          31.94 KB/s        12263.47 KB/s        12295.41 KB/s
              sdc:      113692.62 KB/s            0.00 KB/s       113692.62 KB/s
              sdd:           0.00 KB/s            0.00 KB/s            0.00 KB/s
             sdd1:           0.00 KB/s            0.00 KB/s            0.00 KB/s
             sdd5:           0.00 KB/s            0.00 KB/s            0.00 KB/s
              sdb:           0.00 KB/s            0.00 KB/s            0.00 KB/s
             dm-0:      113972.06 KB/s        73580.84 KB/s       187552.91 KB/s
  ------------------------------------------------------------------------------
            total:      227696.61 KB/s        85844.31 KB/s       313540.94 KB/s

ps – report a snapshot of the current processes

List processes and also shows their PGID. The second command kills an entire PGID (all processes belonging to this group). Note the – before 5112 which means not the PID 5112 but the PGID 5112.

ps x -o  "%p %r %y %x %c "
kill -TERM -- -5112

ps fax

# ps -C ssh
    PID TTY          TIME CMD
  20307 pts/0    00:00:03 ssh
  21510 pts/1    00:00:00 ssh

ax	causes ps to list all processes with a terminal (tty), or to list all processes when used together with the x option
f	ASCII art process hierarchy (forest)
w	Wide output. Use this option twice for unlimited width.
-C cmdlist	Select by command name. This selects the processes whose executable name is given in cmdlist.

Kill all processes of a command

ps -axo pid:1,cmd | grep anki | cut -d' ' -f1 | xargs kill -9

lsof – list open files

List open files listening to something (needs elevated credentials/sudo):

lsof -i -P -n | grep LISTEN

ss – another utility to investigate sockets

ss -ltup

watch – execute a program periodically, showing output fullscreen

-d, –differences[=permanent]	Highlight the differences between successive updates. If the optional permanent argument is specified then watch will show all changes since the first iteration.
-n, –interval seconds	Specify update interval. The command will not allow quicker than 0.1 second interval, in which the smaller values are converted.

watch ssh leo@openmediavault.fritz.box systemctl list-jobs

Packages

AuditD

The audit package contains the user space utilities for storing and searching the audit records generated by the audit subsystem in the Linux 2.6 kernel.

To know what is writing/accessing a folder:

auditctl -w /some/dir/ -p war -k whatsgoingon

Then follow the results in: /var/log/audit/audit.log

Watch a file:

auditctl -w /path/to/that/file -p wa

Then watch for entries to be written to /var/log/audit/audit.log

To list the current audit rules:

auditctl -l