waldek
/
linux_introduction
1
0
Fork 0
Code Issues 1 Pull Requests Projects Releases Wiki Activity

adds processes from course doc

This commit is contained in:
waldek 2022-08-22 14:33:56 +02:00
parent b5b308e84c
commit b2c5e93fe1
1 changed files with 410 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

410
advanced/learning_processes.md Normal file
View File

@ -0,0 +1,410 @@
# Processes and management in Linux
## Foreground and background
By default a process is launched in the **foreground** of a terminal.
We can observe this behavior by executing a simple `ls -la` command in our home.
It writes it's result to **STDOUT** and gives us back a terminal when the command completes.
```
➜ ~ ls -la
total 184
drwxr-xr-x 8 waldek waldek 4096 Jul 5 08:15 .
drwxr-xr-x 6 root root 4096 Jun 3 12:41 ..
-rw------- 1 waldek waldek 291 Mar 4 13:10 .bash_history
-rw-r--r-- 1 waldek waldek 220 Mar 4 13:05 .bash_logout
-rw-r--r-- 1 waldek waldek 3526 Mar 4 13:05 .bashrc
drwx------ 4 waldek waldek 4096 May 13 15:14 .config
drwx------ 2 waldek waldek 4096 Jul 4 19:05 .elinks
drwx------ 3 waldek waldek 4096 Mar 4 13:06 .gnupg
drwxr-xr-x 12 waldek waldek 4096 May 3 20:06 .oh-my-zsh
drwxr-x--- 2 waldek waldek 4096 Jul 4 09:25 ovpns
-rw-r--r-- 1 waldek waldek 807 Mar 4 13:05 .profile
-rw------- 1 waldek waldek 0 Mar 18 22:32 .python_history
-rw-r--r-- 1 waldek waldek 10 Mar 4 13:08 .shell.pre-oh-my-zsh
drwxr-xr-x 2 waldek waldek 4096 Jul 1 12:37 .ssh
-rw------- 1 waldek waldek 15035 Jul 1 12:37 .viminfo
-rw-r--r-- 1 waldek waldek 277 Mar 25 12:26 .wget-hsts
-rw-r--r-- 1 waldek waldek 49005 Jun 29 11:42 .zcompdump-vps-42975ad1-5.7.1
-rw------- 1 waldek waldek 60990 Jul 5 08:15 .zsh_history
-rw-r--r-- 1 waldek waldek 3689 Mar 4 13:08 .zshrc
➜ ~
```
This is probably very obvious behaviour by now but now consider the following command `sleep 10`.
This command just **sleeps** for 10 seconds and returns our prompt after.
We use `sleep` to simulate a long running process such as a heavy calculation, think password cracking, or a server of some sort.
We can use **bash syntax** or **signals** to manipulate running processes.
## Jobs
In a new shell execute the `jobs` command.
It will probably return nothing because you don't have any jobs running.
So how can we create jobs?
As mentioned before, we can do it with **bash syntax** or **signals**.
Let's do it with syntax first.
### Bash syntax
If we add a `&` at the end of a command `bash` will send it to the background.
Execute `sleep 10 &` and observe the output.
```
➜ ~ sleep 10 &
[1] 996
➜ ~
```
The sleep command is executed, and running in the background.
We immediately gain control of our terminal again to perform more tasks but after 10 seconds we get the following output indicating our job is done.
```
➜ ~ sleep 10 &
[1] 996
➜ ~
[1] + 996 done sleep 10
➜ ~
```
We can have multiple jobs running at the same time and can inspect them with the `jobs` command.
Try the following in a shell `sleep 5 & sleep 10 & sleep 20 & sleep 30 & sleep 50 &`.
You gain immediate control of the terminal but a list of *background tasks* is displayed first.
```
➜ ~ sleep 5 & sleep 10 & sleep 20 & sleep 30 & sleep 50 &
[1] 1057
[2] 1058
[3] 1059
[4] 1060
[5] 1061
➜ ~ jobs
[1] running sleep 5
[2] running sleep 10
[3] running sleep 20
[4] - running sleep 30
[5] + running sleep 50
➜ ~
[1] 1057 done sleep 5
➜ ~
[2] 1058 done sleep 10
➜ ~
[3] 1059 done sleep 20
➜ ~
[4] - 1060 done sleep 30
➜ ~
[5] + 1061 done sleep 50
➜ ~ jobs
➜ ~
```
Indeed, that's a lot of numbers on your screen.
The numbers between `[]` are the **job ID** numbers and the four digit ones are the **process ID** numbers, or **PID**.
When using the `jobs` command you can sue the job ID to reference a particular job.
For example, run `sleep 30 & sleep 60 & sleep 90 &` and observe the output.
Next run the `jobs` command and not the more verbose output.
All three jobs are **running** and will terminate one by one.
We can bring back a process to the foreground, so we can interact with it from **STDIN**, by running the `fg` command.
If we only have one process running it will bring back this single process but you can choose which one to bring to the foreground by specifying the job ID as such `fg %2` or `fg %3`.
**Can you tell me what the `+` and `-` mean in the jobs list?**
Now, how can we gain control of our terminal again?
Observe the following output:
```
➜ ~ sleep 30 & sleep 60 & sleep 90 &
[1] 13207
[2] 13208
[3] 13209
➜ ~ fg %3
[3] - 13209 running sleep 90
^Z
[3] + 13209 suspended sleep 90
➜ ~ jobs
[1] running sleep 30
[2] - running sleep 60
[3] + suspended sleep 90
➜ ~
```
First we create three jobs that are sent to the background.
Next we bring job ID number 3 back to the foreground.
We send the **suspend** signal to this job by pressing CTRL-Z.
Note the output from `jobs` which now notes two running jobs and one suspended.
This brings us to **signals**.
### Signals
We use signals all the time without realizing it.
The most common signal we have used is the **SIGINT** that we send when pressing **CTRL-C** on a running process.
A second one most of you know by know is CTRL-Z to suspend a running job.
To see all key combination and their signals we can run the `stty -a` command.
```
speed 38400 baud; rows 30; columns 122; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>; swtch = <undef>; start = ^Q;
stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V; discard = ^O; min = 1; time = 0;
-parenb -parodd -cmspar cs8 -hupcl -cstopb cread -clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon -ixoff -iuclc -ixany -imaxbel iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke -flusho -extproc
```
We can also send signals with the `kill` command.
Contrary to `jobs`, `kill` uses the **PID** numbers to reference running processes.
The PID of a process is shown when you launch it, or you can inspect the PID of all your jobs by executing `jobs -l`.
To demonstrate how to send signals I advise you to run a few long running sleep commands as follows: `sleep 32234 & sleep 324234 & sleep 72552 & sleep 453445 & sleep 96986996 &`
You can now send signals to these processes with the following syntax `kill -$signal_to_send $PID` where `$signal_to_send` is the signal and `$PID` is the process ID.
For example:
```
➜ ~ sleep 32234 & sleep 324234 & sleep 72552 & sleep 453445 & sleep 96986996 &
[1] 13477
[2] 13478
[3] 13479
[4] 13480
[5] 13481
➜ ~ jobs
[1] running sleep 32234
[2] running sleep 324234
[3] running sleep 72552
[4] - running sleep 453445
[5] + running sleep 96986996
➜ ~ jobs -l
[1] 13477 running sleep 32234
[2] 13478 running sleep 324234
[3] 13479 running sleep 72552
[4] - 13480 running sleep 453445
[5] + 13481 running sleep 96986996
➜ ~ kill -STOP 13479
[3] + 13479 suspended (signal) sleep 72552
➜ ~ jobs -l
[1] 13477 running sleep 32234
[2] 13478 running sleep 324234
[3] + 13479 suspended (signal) sleep 72552
[4] 13480 running sleep 453445
[5] - 13481 running sleep 96986996
➜ ~
```
Analyse the output above step by step to make sense of it.
All of this might seem to complicated but there are some handy features of the shell to help us.
First, to get a list of available signals just type `kill -l` and it will output them to STDOUT.
Secondly, `kill` does **autocomplete** on both **signals** and on the **PID**.
Thirdly, you can specify **multiple PID's** to the `kill` command.
You can use `htop` as well to send signals!
Have a try at this with the same long list of sleep command and not the behavior of the processes.
By stopping and continuing a process you can probably explain me what the `S` column means now no?
## Disown
Up until now all of the commands and examples should work in both `bash` and `zsh`.
To test the following command I advise you to take a `bash` shell because it's [posix](https://en.wikipedia.org/wiki/POSIX) compliant.
When a process starts it's always the **child** of a **parent** process.
You can investigate who is a process's parent with `htop` in the *tree* mode.
An other handy tool is `ps` which reports a snapshot of the current processes.
Let's give `ps` a go.
If you run `ps` in a new shell you should get output similar to codeblock below which shows all running jobs in the current shell.
```
➜ ~ ps
PID TTY TIME CMD
13510 pts/0 00:00:00 zsh
14154 pts/0 00:00:00 ps
➜ ~
```
If I add a few background jobs the output becomes as follows:
```
➜ ~ sleep 32234 & sleep 324234 & sleep 72552 & sleep 453445 & sleep 96986996 &
[1] 14164
[2] 14165
[3] 14166
[4] 14167
[5] 14168
➜ ~ ps
PID TTY TIME CMD
13510 pts/0 00:00:00 zsh
14164 pts/0 00:00:00 sleep
14165 pts/0 00:00:00 sleep
14166 pts/0 00:00:00 sleep
14167 pts/0 00:00:00 sleep
14168 pts/0 00:00:00 sleep
14171 pts/0 00:00:00 ps
➜ ~
```
The information above is already quite interesting but we can add or remove columns to the output by using the `o` argument as follows.
Note that each process has a **unique** PID but they all share the same PPID (parent process ID).
Or do they?
Why does the first line, in my case `zsh` have a different PPID?
```
➜ ~ ps o pid,ppid,cmd
PID PPID CMD
13510 13509 -zsh
14164 13510 sleep 32234
14165 13510 sleep 324234
14166 13510 sleep 72552
14167 13510 sleep 453445
14168 13510 sleep 96986996
14199 13510 ps o pid,ppid,cmd
➜ ~
```
The list of available columns can be found in the `man ps` pages in the **STANDARD FORMAT SPECIFIERS** section (around line 500).
We can specify a specific process with the `-p $PID` argument.
```
➜ ~ ps o pid,ppid,cmd
PID PPID CMD
14466 14465 -zsh
14640 14466 tmux
14643 14642 -zsh
14681 14643 ps o pid,ppid,cmd
➜ ~ ps o pid,ppid,cmd -p 14643
PID PPID CMD
14643 14642 -zsh
➜ ~
```
Now in this shell I can start a few specific background jobs, simulated with `sleep`.
```
➜ ~ sleep 1111 & sleep 2222 & sleep 3333 &
[1] 14697
[2] 14698
[3] 14699
➜ ~ ps o pid,ppid,cmd
PID PPID CMD
14466 14465 -zsh
14640 14466 tmux
14643 14642 -zsh
14697 14643 sleep 1111
14698 14643 sleep 2222
14699 14643 sleep 3333
14702 14643 ps o pid,ppid,cmd
➜ ~
```
If I now `disown` a specific job ID, or all with the `-a` flag the processes will not be dependent on the parent's existance.
A quick `ps o pid,ppid,cmd` will still show the PPID as parent *but* when you close the parent shell and inspect the specific PID of the disowned process you'll see it's now owned by a *different* parent.
I know it sounds complicated but I urge you to test this all out in a few shells.
The practice will explain it a lot better than some codeblocks.
```
➜ ~ ps o pid,ppid,cmd -p 14698
PID PPID CMD
14698 1 sleep 2222
➜ ~
```
Now why is the process only changing parent once the original parent terminates?
I'm asking you to look for an answer online but the solution can be found the realm of *signals*, especially the *hang up* [signal](https://en.wikipedia.org/wiki/SIGHUP).
## Nohup
TODO
## Zombie processes
Yes, there are such things as zombie processes.
Learning how to create them is a bit out of our scope but I highly advise you to read up a bit on [what](https://en.wikipedia.org/wiki/Zombie_process) they are and [how](https://www.howtogeek.com/701971/how-to-kill-zombie-processes-on-linux/) to deal with them.
## Process priorities
Life is all about setting priorities and while Linux is very good at managing it's CPU time all by itself, sometimes we know better.
We've seen the priorities before in `htop` in the `NI` column but we can view them as well via `ps o nice`.
A more detailed command would be `ps o nice,pid,ppid,args` which for my laptop returns the following:
```
➜ ~ git:(master) ✗ ps o nice,pid,args
NI PID COMMAND
0 2220 zsh
0 2283 -zsh
0 2323 /bin/sh /usr/bin/startx
0 2345 xinit /etc/X11/xinit/xinitrc -- /etc/X11/xinit/xserverrc :0 vt1 -keeptty -auth /tmp/serverauth.8jVsAiU2KQ
0 2346 /usr/lib/xorg/Xorg -nolisten tcp :0 vt1 -keeptty -auth /tmp/serverauth.8jVsAiU2KQ
0 2354 x-window-manager -a --restart /run/user/1000/i3/restart-state.2354
0 5848 zsh
0 8365 zsh
0 9036 zsh
0 9065 newsboat
0 10478 ssh waldek@86thumbs.net
0 13113 vim learning_processes.md
0 13860 ps o nice,pid,args
0 28084 zsh
➜ ~ git:(master) ✗
```
All my processes are neutral on a scale from *nice* to *not-very-nice*.
You can tell because they are at `0`.
The **nice** scale goes from `-20` being not-at-all-nice to `20` being super friendly towards other processes.
The nicer a process the less aggressive it will be when demanding CPU time.
### Nice
Depending on your system a new process will get a specific nice value.
On my Debian laptop by default processes get `5` as nice value.
We can inspect this as follows where the `ping` command is the new process:
```
➜ ~ git:(master) ✗ ping 8.8.8.8 > /dev/null &
[1] 15428
➜ ~ git:(master) ✗ ps o nice,pid,args -p 15428
NI PID COMMAND
5 15428 ping 8.8.8.8
➜ ~ git:(master) ✗
```
Let's be nice to start with and set the process to be not aggressive at all.
You can launch a command with a specific nice value by prepending `nice -n 15` before the command.
The value you set will be **added** to the default value as seen below (but tops out at 19 and -19).
```
➜ ~ git:(master) ✗ nice -n 15 ping 8.8.8.8 > /dev/null &
[1] 15632
➜ ~ git:(master) ✗ ps o nice,pid,args -p 15632
NI PID COMMAND
19 15632 ping 8.8.8.8
➜ ~ git:(master) ✗
```
Now what about *aggressive* processes?
I would like you to try and set a very *not-nice* value for a `ping` or `sleep` process?
You can probably guess but it won't work.
Why do you think this is?
### Renice
Nice values are not that practical if we need to set them before we start a process no?
That's where the `renice` program comes into play.
It allows us to change the nice value of a running process with a very simple syntax.
I would advise you to use `sudo` when changing the nice values because otherwise you'll constantly run into either `operation not permitted` or `permission denied` errors.
```
➜ ~ git:(master) ✗ ping 8.8.8.8 > /dev/null &
[1] 16877
➜ ~ git:(master) ✗ ps o nice,pid,args -p 16877
NI PID COMMAND
5 16877 ping 8.8.8.8
➜ ~ git:(master) ✗ sudo renice -n 20 -p 16877
16877 (process ID) old priority 5, new priority 19
➜ ~ git:(master) ✗
```
## /proc virtual filesystem
TODO
## Exercises
To help you understand what happens to running and stopped processes I made a few python scripts you can download below.
Run them either with `python3 $SCRIPT_NAME` or `./$SCRIPT_NAME`.
* [simple timer](./assets/processes_ex_01.py)
* [timer with random keyboard prompt](./assets/processes_ex_02.py)
* [custom callback function for SIGALRM](./assets/processes_ex_03.py)