waldek
/
linux_introduction
1
0
Fork 0
Code Issues 1 Pull Requests Projects Releases Wiki Activity
linux_introduction/advanced/learning_bash_scripting.md

1358 lines
45 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# bash
A `bash` script is a sequence of command that are executed one by one.
Most of the time we just execute one command and wait for the result to then make a decision and execute an other command.
We can however create a sequence of commands on the command line.
```
(baseline-) ➜ ~ echo hello world
hello world
(baseline-) ➜ ~ date
Wed 16 Mar 2022 07:06:02 PM CET
(baseline-) ➜ ~ cal
March 2022
Su Mo Tu We Th Fr Sa
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31
(baseline-) ➜ ~ echo hello world date cal
hello world date cal
(baseline-) ➜ ~ echo hello world; date; cal
hello world
Wed 16 Mar 2022 07:06:19 PM CET
March 2022
Su Mo Tu We Th Fr Sa
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31
```
A very visually similar result, but completely different operation, can be obtained by replacing the `;` with `&` or `&&`.
`&` will launch a new process and send it to the background, `&&` will **evaluate** the return status of your process and only continue **if** the status was `0`, or in other words successful.
```
(baseline-) ➜ ~ echo hello world & date & cal
[1] 3075524
hello world
[2] 3075525
[1] - 3075524 done echo hello world
Wed 16 Mar 2022 07:06:33 PM CET
[2] + 3075525 done date
March 2022
Su Mo Tu We Th Fr Sa
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31
(baseline-) ➜ ~ echo hello world && date && cal
hello world
Wed 16 Mar 2022 07:06:40 PM CET
March 2022
Su Mo Tu We Th Fr Sa
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31
(baseline-) ➜ ~
```
This should make you think we can create quite complicated logical flows in `bash` and you're right!
But first things first, let's create our first script!
```
(baseline-) ➜ bash file test.sh
test.sh: Bourne-Again shell script, ASCII text executable
(baseline-) ➜ bash ls -l test.sh
-rwxr-xr-x 1 waldek waldek 39 Mar 16 19:13 test.sh
(baseline-) ➜ bash cat test.sh
#!/bin/bash
echo hello world
cal
date
(baseline-) ➜ bash ./test.sh
hello world
March 2022
Su Mo Tu We Th Fr Sa
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31
Wed 16 Mar 2022 07:27:22 PM CET
(baseline-) ➜ bash bash test.sh
hello world
March 2022
Su Mo Tu We Th Fr Sa
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31
Wed 16 Mar 2022 07:27:25 PM CET
(baseline-) ➜ bash
```
If you observe the output above you can conclude multiple things.
* the `test.sh` file is a simple text file
* it has executable permissions for the owner, group and others
* we can execute the sequence of commands in the script in two ways
* `./test.sh`
* `bash test.sh`
* the content is a series of commands but with a *weird* first line
That first line is called a [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) and is a way of explaining which interpreter understands the lines that follow.
If you venture out into the python universe you'll encounter the same norm but with a different path to an interpretor, often `/bin/python3` or `/usr/bin/env python3`.
The later is a sort of shortcut that points to the local python installation, even if it's not in a standard location.
A shebang is not necessary for a script to function but is **highly** advised.
The `env` program is actually very interesting!
Let's try it by itself.
```
➜ ~ env
PAGER=less
LANGUAGE=en_US:en
GNOME_TERMINAL_SCREEN=/org/gnome/Terminal/screen/022a5a78_fe95_445c_9b33_f6dcb35cfd27
LANG=en_US.UTF-8
DISPLAY=:0
SWAYSOCK=/run/user/1000/sway-ipc.1000.1565.sock
WAYLAND_DISPLAY=wayland-0
AUTOSWITCH_VERSION=3.4.0
HUSHLOGIN=FALSE
USER=waldek
OLDPWD=/home/waldek/bin/bash
HOME=/home/waldek
MOZ_ENABLE_WAYLAND=1
VIRTUAL_ENV=/home/waldek/.virtualenvs/baseline-
DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-acMe3wyyoo,guid=ad1233c6039551be866f78ae61f911b2
XDG_VTNR=1
XDG_SEAT=seat0
I3SOCK=/run/user/1000/sway-ipc.1000.1565.sock
LESS=-R
_=/usr/bin/env
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
AUTOSWITCH_DEFAULTENV=baseline
VTE_VERSION=6203
LSCOLORS=Gxfxcxdxbxegedabagacad
ZSH=/home/waldek/.oh-my-zsh
AUTOSWITCH_FILE=.venv
MAIL=/var/mail/waldek
LOGNAME=waldek
PS1=(baseline-) %(?:%{%}➜ :%{%}➜ ) %{$fg[cyan]%}%c%{$reset_color%} $(git_prompt_info)
GDK_BACKEND=wayland
PATH=/home/waldek/.virtualenvs/baseline-/bin:/home/waldek/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/waldek/.local/bin:/home/waldek/bin/python/:/home/waldek/.local/bin:/home/waldek/bin/python/
XDG_RUNTIME_DIR=/run/user/1000
QT_QPA_PLATFORM=wayland
XDG_SESSION_ID=2
XDG_SESSION_TYPE=wayland
FPATH=/home/waldek/.oh-my-zsh/functions:/home/waldek/.oh-my-zsh/completions:/home/waldek/.oh-my-zsh/cache/completions:/home/waldek/.oh-my-zsh/custom/plugins/autoswitch_virtualenv:/home/waldek/.oh-my-zsh/plugins/git:/home/waldek/.oh-my-zsh/functions:/home/waldek/.oh-my-zsh/completions:/home/waldek/.oh-my-zsh/cache/completions:/usr/local/share/zsh/site-functions:/usr/share/zsh/vendor-functions:/usr/share/zsh/vendor-completions:/usr/share/zsh/functions/Calendar:/usr/share/zsh/functions/Chpwd:/usr/share/zsh/functions/Completion:/usr/share/zsh/functions/Completion/AIX:/usr/share/zsh/functions/Completion/BSD:/usr/share/zsh/functions/Completion/Base:/usr/share/zsh/functions/Completion/Cygwin:/usr/share/zsh/functions/Completion/Darwin:/usr/share/zsh/functions/Completion/Debian:/usr/share/zsh/functions/Completion/Linux:/usr/share/zsh/functions/Completion/Mandriva:/usr/share/zsh/functions/Completion/Redhat:/usr/share/zsh/functions/Completion/Solaris:/usr/share/zsh/functions/Completion/Unix:/usr/share/zsh/functions/Completion/X:/usr/share/zsh/functions/Completion/Zsh:/usr/share/zsh/functions/Completion/openSUSE:/usr/share/zsh/functions/Exceptions:/usr/share/zsh/functions/MIME:/usr/share/zsh/functions/Math:/usr/share/zsh/functions/Misc:/usr/share/zsh/functions/Newuser:/usr/share/zsh/functions/Prompts:/usr/share/zsh/functions/TCP:/usr/share/zsh/functions/VCS_Info:/usr/share/zsh/functions/VCS_Info/Backends:/usr/share/zsh/functions/Zftp:/usr/share/zsh/functions/Zle:/usr/share/zsh:/usr/share/zsh
SHELL=/usr/bin/zsh
AUTOSWITCH_SILENT=1
EDITOR=vim
MOTD_SHOWN=pam
SHLVL=2
GNOME_TERMINAL_SERVICE=:1.10
COLORTERM=truecolor
XCURSOR_SIZE=24
TERM=xterm-256color
PWD=/home/waldek
XDG_SESSION_CLASS=user
➜ ~
```
What is all of this?
Well, the are the current **environment variables** defined in your shell.
You'll probably recognize some of them.
Let's play around with what we have defined.
```
➜ ~ echo SHELL
SHELL
➜ ~ echo $SHELL
/usr/bin/zsh
➜ ~ echo $USER
waldek
➜ ~ echo $HOME
/home/waldek
➜ ~ echo $
zsh: do you wish to see all 223 possibilities (56 lines)?
```
Yes, tab complete works!
Your variables will be slightly different but that's not an issue at all.
```
➜ ~ echo $
! EUID MAILPATH SWAYSOCK
# fg manpath TERM
* FG MANPATH termcap
- fg_bold module_path terminfo
? fg_no_bold MODULE_PATH TIMEFMT
@ fignore modules TMPPREFIX
_ FIGNORE MOTD_SHOWN TRY_BLOCK_ERROR
$ fpath MOZ_ENABLE_WAYLAND TRY_BLOCK_INTERRUPT
0 FPATH nameddirs TTY
aliases funcfiletrace NULLCMD TTYIDLE
ARGC FUNCNEST OLDPWD UID
argv funcsourcetrace OPTARG USER
AUTOSWITCH_DEFAULTENV funcstack OPTIND userdirs
AUTOSWITCH_FILE functions options usergroups
AUTOSWITCH_SILENT functions_source OSTYPE USERNAME
AUTOSWITCH_VERSION functrace PAGER VENDOR
bg FX parameters VIRTUAL_ENV
BG galiases patchars VTE_VERSION
bg_bold GDK_BACKEND _patcomps watch
bg_no_bold GID path WATCH
bold_color GNOME_TERMINAL_SCREEN PATH WATCHFMT
builtins GNOME_TERMINAL_SERVICE pipestatus WAYLAND_DISPLAY
cdpath histchars plugins widgets
CDPATH HISTCHARS _postpatcomps WORDCHARS
color HISTCMD PPID XCURSOR_SIZE
COLORTERM HISTFILE precmd_functions XDG_RUNTIME_DIR
colour history preexec_functions XDG_SEAT
COLUMNS historywords prompt XDG_SESSION_CLASS
commands HISTSIZE PROMPT XDG_SESSION_ID
_compautos HOME PROMPT2 XDG_SESSION_TYPE
_comp_dumpfile HOST PROMPT3 XDG_VTNR
COMPLETION_WAITING_DOTS HUSHLOGIN PROMPT4 zle_bracketed_paste
_comp_options I3SOCK PS1 ZLS_COLORS
comppostfuncs jobdirs PS2 ZSH
compprefuncs jobstates PS3 ZSH_ARGZERO
_comps jobtexts PS4 ZSH_CACHE_DIR
_comp_setup key psvar ZSH_COMPDUMP
CPUTYPE KEYBOARD_HACK PSVAR ZSH_CUSTOM
d keymaps PWD zsh_eval_context
DBUS_SESSION_BUS_ADDRESS KEYTIMEOUT QT_QPA_PLATFORM ZSH_EVAL_CONTEXT
debian_missing_features LANG RANDOM ZSH_NAME
dirstack langinfo READNULLCMD ZSH_PATCHLEVEL
dis_aliases LANGUAGE reset_color zsh_scheduled_events
dis_builtins _lastcomp reswords ZSH_SUBSHELL
dis_functions LESS saliases ZSH_THEME
dis_functions_source LINENO __savecursor ZSH_THEME_GIT_PROMPT_CLEAN
dis_galiases LINES SAVEHIST ZSH_THEME_GIT_PROMPT_DIRTY
dis_patchars LISTMAX SCREEN_NO ZSH_THEME_GIT_PROMPT_PREFIX
DISPLAY LOGCHECK __searching ZSH_THEME_GIT_PROMPT_SUFFIX
dis_reswords LOGNAME _services ZSH_THEME_RUBY_PROMPT_PREFIX
dis_saliases LS_COLORS SHELL ZSH_THEME_RUBY_PROMPT_SUFFIX
EDITOR LSCOLORS SHLVL ZSH_THEME_RVM_PROMPT_OPTIONS
EGID MACHTYPE SHORT_HOST ZSH_THEME_TERM_TAB_TITLE_IDLE
EPOCHREALTIME MAIL signals ZSH_THEME_TERM_TITLE_IDLE
EPOCHSECONDS MAILCHECK SPROMPT ZSH_VERSION
epochtime mailpath status
➜ ~ echo $
```
We can create our own variables as follows.
Notice how an undefined variable does not throw an error.
This is very typical for shell scripting, python on the other hand would crash over an undefined variable.
```
waldek@helloworld:~$ echo $USER
waldek
waldek@helloworld:~$ echo $name
waldek@helloworld:~$ name="wouter gordts"
waldek@helloworld:~$ echo $name
wouter gordts
waldek@helloworld:~$
```
If you open up a new shell this `$name` variable will not be defined because variables are local to each instance of `bash` that is running.
This can be observed as follows.
We can *export* variables to children with the `export` keyword.
```
waldek@helloworld:~$ name="wouter gordts"
waldek@helloworld:~$ echo $name
wouter gordts
waldek@helloworld:~$ bash
waldek@helloworld:~$ echo $name
waldek@helloworld:~$ exit
exit
waldek@helloworld:~$ export name
waldek@helloworld:~$ bash
waldek@helloworld:~$ echo $name
wouter gordts
waldek@helloworld:~$ exit
exit
waldek@helloworld:~$
```
# Using variables to store the output of command
Bash only really knows *characters*, both for sending and receiving.
We can temporarily store the output of a command using variables.
The syntax is a bit tricky at first but quickly becomes quite natural.
We can try this out on the command line.
Next we'll write a small script to leverage the power of variables and pipes.
```
waldek@metal:~$ grep $USER /etc/passwd
waldek:x:1000:1000:waldek,,,:/home/local/waldek:/bin/zsh
waldek@metal:~$ my_name=$(grep $USER /etc/passwd)
waldek@metal:~$ echo $my_name
waldek:x:1000:1000:waldek,,,:/home/local/waldek:/bin/zsh
waldek@metal:~$
```
A little bit more complicated.
```
waldek@metal:~$ count=$(grep "/home" /etc/passwd | wc -l)
waldek@metal:~$ msg="there are $count users on this machine"
waldek@metal:~$ echo $msg
there are 4 users on this machine
waldek@metal:~$
```
Now a small script.
```bash
#!/bin/bash
FULLNAME="wouter gordts"
CITY="Brussels"
echo "this script was written by $FULLNAME in $CITY"
IP=$(ip a | grep -v "127.0.0.1" | grep -o -E "[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\/[[:digit:]]{1,3}")
TIME=$(date +%X)
DAY=$(date +%A)
YEAR=$(date +%Y)
echo "this computer has $IP as IP address"
echo "it is $TIME and we are a $DAY in $YEAR"
```
Which if we run it gives us the following output.
```
waldek@metal:~$ bash test.sh
this script was written by wouter gordts in Brussels
this computer has 10.1.12.53/24 as IP address
it is 02:17:02 PM and we are a Tuesday in 2022
waldek@metal:~$
```
# Coding challenge - Output system stats
Write a program that prints information about your computer such as:
* the hostname
* the FQDN
* number of cpus
* type of cpu
* amount of RAM
Write a program that prints information about a given user such as:
* name
* UID
* their default shell
* groups they are a member of
* number of files in their home directory
* amount of disk space they use
# Getting input into the script
## With `read`
Observe the output of the following *program*.
It's not really complicated but it will demonstrate we can do arithmetic in bash scripts as well!
```
waldek@metal:~$ bash test.sh
In which year where you born?
1986
your are probably around 36...
waldek@metal:~$
```
The output above was generated with the following code.
The **two** things to notice are the `read year` line and the `$(( $this_year - $year ))`.
The former offers the possibility to **prompt** the user for input, the latter performs a mathematical calculation with two numbers.
```bash
#!/bin/bash
echo "In which year where you born?"
read year
this_year=$(date +%Y)
echo "your are probably around $(( $this_year - $year ))..."
waldek@metal:~$
```
# Coding challenge - Secret input
Can you create me a secret password prompt?
Something like this.
```
waldek@metal:~$ bash test.sh
what is your secret password?
hmmm, I don't know how to compare helloworld to supersecret
waldek@metal:~$
```
We haven't learned how to *evaluate* values but if you're eager and quick you can try to figure it out yourself.
If not, no worries, we'll get to *conditional logic* soon enough.
<details>
<summary>Spoiler warning!</summary>
```bash
#!/bin/bash
my_pass="supersecret"
read -s -p "what is your secret password? " pass
echo
echo "hmmm, I don't know how to compare $pass to $my_pass"
```
</details>
### `read` multiple variables
`read` can be used to unpack multiple values right on the spot!
By *unpacking* I mean that each value is separated by **white space**.
A demonstration can be seen below, plus the actual code right after.
```
waldek@debian:~$ bash test.sh
first and last name please: wouter gordts
hello mr gordts...
or may I call you wouter?
waldek@debian:~$
```
```
#!/bin/bash
read -p "first and last name please: " first last
echo "hello mr $last..."
echo "or may I call you $first?"
```
#### Exercise - unpack values
It is worth discovering what happens when you supply too many or to little values.
Please try this out!
# Coding challenge - Birthday **day**
Write me `bash` program that asks for your date of birth and prints the day of the week that was.
Like the output below.
```
waldek@server:~$ bash birthday.sh
what is your birthday (day month year)? 07 10 1986
you where born on a Tuesday
```
<details>
<summary>Spoiler warning!</summary>
```bash
#!/bin/bash
read -p "what is your date of birth? (day month year) " day month year
day_of_the_week=$(date +%A -d "$month/$day/$year")
echo "you where born on a $day_of_the_week"
```
</details>
## With command line arguments
We can create a similar behaviour but with command line arguments.
By doing so we don't have to answer any questions the script poses at runtime.
If we create a script that will run for a long time, it doesn't require any interaction mid way!
All necessary information is supplied by at execution time.
```
waldek@metal:~$ bash test.sh 1986
your are probably around 36...
waldek@metal:~$
```
```bash
#!/bin/bash
year=$1
this_year=$(date +%Y)
echo "your are probably around $(( $this_year - $year ))..."
```
The magic behind is the `$1` variable.
This variable represents the *first* argument on the command line.
Knowing this, what would `$4` mean?
Indeed, the *fourth* argument...
# Coding Challenge - output the exact output below
```
waldek@metal:~$ bash test.sh hello world 1986 35 foo bar linux rulez...
hello waldek, my name is test.sh
you supplied 8 arguments on the command line...
here are all of them on one line: hello world 1986 35 foo bar linux rulez...
waldek@metal:~$
```
<details>
<summary>Spoiler warning!</summary>
```bash
#!/bin/bash
echo "hello $USER, my name is $0"
echo "you supplied $# arguments on the command line..."
echo "here are all of them on one line: $@"
```
</details>
# More math!
## The `let` keyword
TODO
## The `expr` keyword
TODO
## Double parenthesis
We've already seen the basic syntax before but here are some more examples.
```
waldek@metal:~$ a=11
waldek@metal:~$ b=202
waldek@metal:~$ echo $(( $a + $b ))
213
waldek@metal:~$ echo $(( $a - $b ))
-191
waldek@metal:~$ echo $(( $a * $b ))
2222
waldek@metal:~$ echo $(( $a / $b ))
0
waldek@metal:~$ echo $(( $a % $b ))
11
```
Incrementing variables can also be done with the double parenthesis syntax.
We can't use the `$` to reference the variable though.
This is a *classic* example of bash's finicky behaviour.
```
waldek@metal:~$ echo $(( b++ ))
202
waldek@metal:~$ echo $(( b++ ))
203
waldek@metal:~$ echo $(( b++ ))
204
waldek@metal:~$ echo $(( b++ ))
205
waldek@metal:~$ echo $(( b++ ))
206
waldek@metal:~$
```
## Variable length
As bash only *knows* characters it has a built in feature to determine a variable's length.
You can print the length of a variable, or use it to calculate something, with the following syntax.
```
waldek@metal:~$ test="hello world! bash is pretty sweet..."
waldek@metal:~$ echo ${#test}
36
waldek@metal:~$ echo $(( ${#test} + 1986 ))
2022
waldek@metal:~$
```
[Ryan's tutorials](https://ryanstutorials.net/bash-scripting-tutorial/bash-arithmitic.php)
# If Statements - How to make decisions within your Bash script.
The small *password* checker we made before could use some [conditional logic](https://en.wikipedia.org/wiki/Conditional_(computer_programming)).
We can easily implement this in `bash` with the following syntax.
```bash
#!/bin/bash
my_pass="supersecret"
read -s -p "what is your secret password? " pass
echo
if [ $pass == $my_pass ]; then
echo "access granted!"
fi
```
The `[ $pass == $my_pass ]` is the actual *evaluation* and will **always** evaluate to either `true` or `false`.
If the statement is `true`, the following code get's executed.
If **not**, currently nothing happens.
We can introduce a second keyword `else` to handle this.
```bash
#!/bin/bash
my_pass="supersecret"
read -s -p "what is your secret password? " pass
echo
if [ $pass == $my_pass ]; then
echo "access granted!"
else
echo "access denied..."
fi
```
The script above gives us the following behaviour.
```
waldek@helloworld:~$ bash test.sh
what is your secret password?
access denied...
waldek@helloworld:~$ bash test.sh
what is your secret password?
access granted!
waldek@helloworld:~$
```
We can add a bit more complexity to our possible branches with the `elif` keyword.
This keyword allows us to construct a second and third branch of execution.
Consider the sentence below.
> If you are younger than 27 you are still young so if you're older than 27 you're considered old, but if you are 27 on the dot your life might be at risk!
This sentence can be converted to a conditional logic block as follows.
```bash
#!/bin/bash
read -p "how old are you? " age
echo
if [ "$age" -lt "27" ]; then
echo "you are so young! enjoy it"
elif [ "$age" -gt "27" ]; then
echo "you're sooo old!"
elif [ "$age" -eq "27" ]; then
echo "your life might be at risk..."
else
echo "I'm not sure I understand you."
fi
```
There is a little problem here though!
We can input anything we want, not only numbers, and this creates some error messages.
Bash is a bit special, compared to a language like `python3`, because it doesn't *crash* on an error.
It just *keeps going*.
```
waldek@helloworld:~$ bash test.sh
how old are you? helloworld
test.sh: line 6: [: helloworld: integer expression expected
test.sh: line 8: [: helloworld: integer expression expected
test.sh: line 10: [: helloworld: integer expression expected
I'm not sure I understand you.
waldek@helloworld:~$
```
We can check if the input is *really* a number and redirect the error to `/dev/null`.
If the number is not a real number we can't continue so we `exit` the script.
```bash
#!/bin/bash
read -p "how old are you? " age
echo
if ! [ "$age" -eq "$age" ] 2> /dev/null
then
echo "Sorry integers only"
exit 1
fi
if [ "$age" -lt "27" ]; then
echo "you are so young! enjoy it"
elif [ "$age" -gt "27" ]; then
echo "you're sooo old!"
elif [ "$age" -eq "27" ]; then
echo "your life might be at risk..."
fi
```
## How does it work behind the scenes?
## exit status
Every command you execute on the command line has an **exit code**.
You can [read up](https://en.wikipedia.org/wiki/Exit_status) a bit on what they are but the most important things to know are:
* the code is always a **number**
* on our systems it's a [uint8](https://en.wikipedia.org/wiki/Integer_(computer_science)#Common_integral_data_types) which means a value between **0** and **255**
* the [**convention**](https://tldp.org/LDP/abs/html/exitcodes.html) is that `0` means **success**, everything else is an **error**
In your bash shell, the variable `$?` always references the *last* exit status code.
We can discover it's behaviour as follows.
```
waldek@debian:~$ ls does_exist
does_exist
waldek@debian:~$ echo $?
0
waldek@debian:~$ ls does_not_exist
ls: cannot access 'does_not_exist': No such file or directory
waldek@debian:~$ echo $?
2
waldek@debian:~$ echo $?
0
waldek@debian:~$
```
The **first** `echo $?` prints the exit code of `ls does_exist`.
The **second** prints the exit code of the *failed* command `ls does_not_exist`.
The **third** prints the exit code of the `echo $?` that failed!
## `test`
The presence of exit codes means we can **evaluate** their value and make **decisions** based on the outcome.
The main workhorse for this is a builtin called `test`.
```
waldek@debian:~$ whatis test
test (1) - check file types and compare values
waldek@debian:~$
```
I **highly** recommend you take some time to read the `man test`.
Because the convention of exit codes is *no news, good news* there are two tiny programs that just serve to output `true` and `false`, where true is `0` and false is `1`.
A little demonstration.
```
waldek@debian:~$ which true
/usr/bin/true
waldek@debian:~$ which false
/usr/bin/false
waldek@debian:~$ whatis true
true (1) - do nothing, successfully
waldek@debian:~$ whatis false
false (1) - do nothing, unsuccessfully
waldek@debian:~$ true
waldek@debian:~$ echo $?
0
waldek@debian:~$ false
waldek@debian:~$ echo $?
1
waldek@debian:~$ test true == true
waldek@debian:~$ echo $?
0
waldek@debian:~$ test true == false
waldek@debian:~$ echo $?
1
waldek@debian:~$
```
Again, I highly advise you to read the `man test`.
If you did this then the following will make a lot of sense.
```
waldek@debian:~$ test -a does_exist
waldek@debian:~$ echo $?
0
waldek@debian:~$ test -d does_exist
waldek@debian:~$ echo $?
1
waldek@debian:~$ test -a does_not_exist
waldek@debian:~$ echo $?
1
waldek@debian:~$ test ! -a does_not_exist; echo $?
0
waldek@debian:~$
```
The table below is taken from the bash reference manual you can find [here](https://www.gnu.org/software/bash/manual/bash.html#Bash-Conditional-Expressions).
| <div style="width:300px">command</div> | status |
| --- | --- |
| -a file | True if file exists. |
| -b file | True if file exists and is a block special file. |
| -c file | True if file exists and is a character special file. |
| -d file | True if file exists and is a directory. |
| -e file | True if file exists. |
| -f file | True if file exists and is a regular file. |
| -g file | True if file exists and its set-group-id bit is set. |
| -h file | True if file exists and is a symbolic link. |
| -k file | True if file exists and its "sticky" bit is set. |
| -p file | True if file exists and is a named pipe (FIFO). |
| -r file | True if file exists and is readable. |
| -s file | True if file exists and has a size greater than zero. |
| -t fd | True if file descriptor fd is open and refers to a terminal. |
| -u file | True if file exists and its set-user-id bit is set. |
| -w file | True if file exists and is writable. |
| -x file | True if file exists and is executable. |
| -G file | True if file exists and is owned by the effective group id. |
| -L file | True if file exists and is a symbolic link. |
| -N file | True if file exists and has been modified since it was last read. |
| -O file | True if file exists and is owned by the effective user id. |
| -S file | True if file exists and is a socket. |
| file1 -ef file2 | True if file1 and file2 refer to the same device and inode numbers. |
| file1 -nt file2 | True if file1 is newer (according to modification date) than file2, or if file1 exists and file2 does not. |
| file1 -ot file2 | True if file1 is older than file2, or if file2 exists and file1 does not. |
| -o optname | True if the shell option optname is enabled. The list of options appears in the description of the -o option to the set builtin (see The Set Builtin). |
| -v varname | True if the shell variable varname is set (has been assigned a value). |
| -R varname | True if the shell variable varname is set and is a name reference. |
| -z string | True if the length of string is zero. |
| -n string string | True if the length of string is non-zero. |
| string1 == string2 | True if the strings are equal. When used with the [[ command, this performs pattern matching as described above (see Conditional Constructs). ‘=’ should be used with the test command for POSIX conformance. |
| string1 = string2 | True if the strings are equal. When used with the [[ command, this performs pattern matching as described above (see Conditional Constructs). ‘=’ should be used with the test command for POSIX conformance. |
| string1 != string2 | True if the strings are not equal. |
| string1 < string2 | True if string1 sorts before string2 lexicographically. |
| string1 > string2 | True if string1 sorts after string2 lexicographically. |
## Nested `if` statements
It's worth pointing out we can *nest* `if` statements *inside* other `if` statements.
There is no real theoretical limit to how *deep* we can go, but it's advised to keep the limit to two or three levels.
```bash
#!/bin/bash
num="40"
if [ "$num" -lt "300" ]; then
echo "$num is a small number"
if [ "$(( $num % 2))" -eq 0 ]; then
echo "and it is even"
else
echo "$num is not even"
fi
fi
```
## A *modern* version of `test`
I'll be the first to admit that the syntax of `bash` can be confusing and is rarely reader friendly.
A nice, but brief, explication of the nuances of single and double brackets can be found in [this](https://unix.stackexchange.com/questions/306111/what-is-the-difference-between-the-bash-operators-vs-vs-vs) stack overflow post.
The double bracket command are called [compound commands](https://www.gnu.org/software/bash/manual/html_node/Compound-Commands.html).
### `[[ ]]`
```
waldek@debian:~$ [[ 3 = [[:digit:]] ]] ; echo $?
0
waldek@debian:~$ [ 3 = [[:digit:]] ] ; echo $?
1
```
TODO
### `(( ))`
TODO
### `&&` and `||`
```
waldek@debian:~$ test true == true && echo "yes sir!" || echo "nope..."
yes sir!
waldek@debian:~$ test true == false && echo "yes sir!" || echo "nope..."
nope...
waldek@debian:~$
```
[Ryan's tutorials](https://ryanstutorials.net/bash-scripting-tutorial/bash-if-statements.php)
# Coding challenge - File information
Write a script that takes one argument which should be a valid file path.
The program should print out what type of file this is, and if it is readable, print the first and last 5 lines.
If the file does not exist, an error message should be shown.
Something along these lines.
```
waldek@helloworld:~$ bash test.sh .bashrc
.bashrc exists, I'll dig a little deeper
it is indeed a file
and I can read it!
here are the first 5 lines
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
# If not running interactively, don't do anything
and here are the last 5 lines
elif [ -f /etc/bash_completion ]; then
. /etc/bash_completion
fi
fi
. "$HOME/.cargo/env"
waldek@helloworld:~$ bash test.sh .not_a_file
that's not a file!
waldek@helloworld:~$
```
<details>
<summary>Spoiler warning!</summary>
```bash
#!/bin/bash
filepath=$1
if [ -e "$filepath" ]; then
echo "$filepath exists, I'll dig a little deeper"
if [ -f "$filepath" ]; then
echo "it is indeed a file"
if [ -r "$filepath" ]; then
echo "and I can read it!"
echo "here are the first 5 lines"
head -n 5 $filepath
echo "and here are the last 5 lines"
tail -n 5 $filepath
fi
fi
else
echo "that's not a file!"
fi
```
</details>
# Coding challenge - pipe or argument?
Can you code me a script that depending on how it is called, with argument or via pipe, prints a different message?
Along these lines...
```
waldek@debian:~$ bash test.sh
No input was found on stdin, skipping!
No input given!
waldek@debian:~$ bash test.sh shopping.list
No input was found on stdin, skipping!
Filename specified: shopping.list
Doing things now..
waldek@debian:~$ cat shopping.list | bash test.sh
Data was piped to this script!
waldek@debian:~$
```
<details>
<summary>Spoiler warning!</summary>
**It might take you some time but we've seen all the necessary building blocks!**
<details>
<summary>Spoiler warning!</summary>
```bash
#!/bin/bash
if [ -p /dev/stdin ]; then
echo "Data was piped to this script!"
else
echo "No input was found on stdin, skipping!"
if [ -f "$1" ]; then
echo "Filename specified: ${1}"
echo "Doing things now.."
else
echo "No input given!"
fi
fi
```
</details>
</details>
# Loops - A variety of ways to perform repetitive tasks.
Up until now all our scripts run from top to bottom, executing each line as they are read.
Most, if not all, programming languages offer some form of looping to **repeat** lines of code either based on a **condition** or for a predefined number of *items*.
## `while` loop
```bash
#!/bin/bash
counter=0
while [[ counter -lt 10 ]]; do
echo $counter
(( counter++ ))
done
```
### Read from a file with a `while` loop
Below you can see I have two files, one is my *shopping list*, the other my script.
The script loops over each item I have to buy and prints me a verbose message.
```
waldek@debian:~$ cat shopping.list
6 eggs
1 bread
2 milk
4 pasta
waldek@debian:~$ bash test.sh shopping.list
you need to buy: 6 eggs
you need to buy: 1 bread
you need to buy: 2 milk
you need to buy: 4 pasta
waldek@debian:~$
```
One way to do this is with a `while` loop.
The syntax is a follows.
```bash
#!/bin/bash
while read line; do
echo "you need to buy: $line"
done < $1
```
The syntax is not what I would call *pretty* but it is quite effective.
The **file** `$1` is redirected **into** the `while` loop which performs a `read`.
**When** there are no more lines in the file, the condition **fails** and the loop exits.
### Read from a pipe with a `while` loop
We can use a *very* similar construct to read data from a pipe.
The utilisation is as follows.
```
waldek@debian:~$ cat shopping.list | ./test.sh
you need to buy: 6 eggs
you need to buy: 1 bread
you need to buy: 2 milk
you need to buy: 4 pasta
waldek@debian:~$
```
And the code that does this as such.
```bash
#!/bin/bash
while IFS= read line; do
echo "you need to buy: $line"
done
```
What on earth is this `IFS=`?
I'm glad you [asked](https://unix.stackexchange.com/questions/184863/what-is-the-meaning-of-ifs-n-in-bash-scripting)!
## `for` loop
Ah, my favorite loop!
While the `while` loop runs as long as a condition is `true`, the `for` loop **iterates** over *items*.
The simplest way to visualize it would be as follows.
For each *friend* in my list of friends I'll say hello.
```
waldek@debian:~$ ./test.sh
these are my friends: adam camille alice bob steve
hello adam!
hello camille!
hello alice!
hello bob!
hello steve!
waldek@debian:~$
```
The syntax to achieve this goes as follows.
```bash
#!/bin/bash
friends="adam camille alice bob steve"
echo "these are my friends: $friends"
for f in $friends; do
echo "hello $f!"
done
```
### Counter loops with `for`
We can do a counter loop as well via a `for` loop.
An easy way to do this is by using the `seq` program.
Remember to read the `man seq` for more information.
```bash
#!/bin/bash
for number in $(seq 0 10); do
echo "number is $number"
done
```
### Finicky behaviour
`bash` is not the prettiest language and the difference between single and double quotes can be a bit confusing.
```bash
#!/bin/bash
for animal in dog cat horse 'racing horse' whale; do
echo "$animal says: I'm an animal..."
done
```
Which gives the following output.
```
waldek@debian:~$ ./test.sh
dog says: I'm an animal...
cat says: I'm an animal...
horse says: I'm an animal...
racing horse says: I'm an animal...
whale says: I'm an animal...
waldek@debian:~$
```
## `break` and `continue`
When we consider the `secret.sh` password checker we made, we have a problem.
The program always exits, either with a success code `0` or with an error of `1`.
In order to *break* a loop conditionally we need a new keyword, `break`.
I rewrote the same script but with a more logical flow of operation.
```bash
#!/bin/bash
secret="test123"
tries=3
while true; do
read -s -p "your password please: " password
if [[ $password == $secret ]]; then
echo "access granted!"
break
else
(( tries-- ))
if [[ $tries -eq 0 ]]; then
echo "access denied!"
exit 1
fi
echo "wrong password, you have $tries left..."
fi
done
echo "we're in!"
echo "the code keeps on flowing..."
```
`continue` is very similar to `break`.
It *breaks* the current iteration and **continues** to the next cycle.
Consider the example below.
```bash
#!/bin/bash
counter=0
value=0
while [[ $counter -lt 100 ]]; do
if (( $counter % 2 )); then
echo "$counter is even"
random=$(( $RANDOM % 20 ))
echo "adding $random to the counter"
counter=$(( $random + $counter ))
echo "and I'll loop again"
continue
fi
echo "incrementing $counter by just one..."
(( counter++ ))
done
```
# Functions - Reuse code to make life easier.
## defining a function
## function arguments
## global vs local variable
## return values
## the `command` builtin
[Ryan's tutorials](https://ryanstutorials.net/bash-scripting-tutorial/bash-functions.php)
# Coding challenge - pipe or argument plus action!
Can you expand the previous coding challenge, where you perform a conditional logic on the input of the script and actually *use* the incoming data?
For example, print a shopping list or make a sum of all items you need to purchase.
# Coding challenge - Rename files
Rename all files from [this](../assets/simple_sort_01.tar.gz) file with an prefix or postfix.
Can you give the files that start with an **uppercase** letter a different pre or postfix?
Can you move the files with lowercase into a different folder?
If you did it with a prefix, now try with a postfix.
# Coding challenge - Rename jpeg files
Download [these](../assets/jpeg_ext.tar) files, which are all `jpeg` files.
The extensions however are not really uniform.
Find the most common extension type and rename all files with that extension.
# Coding challenge - Remove duplicate files
Find all duplicate files from [this](../assets/shasum.tar) and move them to a different directory.
How certain are you that they are duplicates?
# Coding challenge - File tree
Download [this](../assets/tree.tar) file and use the content of the directory to write me a program that prints the content as a tree structure.
You can be *creative* but it's interesting to learn how to perform these action by hand.
As an extra challenge I would like you to add different behaviour to the script, depending on the input.
For example, if the input file is a `tar.gz` file, the program will automatically uncompress and then show the tree.
[hint](https://www.shellscript.sh/eg/directories/)
# Coding challenge - Compare and move
Compare [these](../assets/) files from `one` directory with `second` directory and if they exist in the second directory, copy them to a `third` directory.
<details>
<summary>Spoiler warning!</summary>
```bash
dir1=directory1
dir2=directory2
dir3=directory3
v=v # change to v= to make it quieter
shopt -s globstar
mkdir -p$v "$dir3"
for f1 in "$dir1"/*.dll; do
for f2 in "$dir2"/**/"${f1##*/}"; do
if [[ -f "$f2" ]]; then
cp ${v:+-v} "$f1" "$dir3/"
break
fi
done
done
```
</details>
# Coding challenge - Guess the number
Can you make me a small guessing game like the one below?
```
waldek@debian:~$ ./test.sh
I have a number in mind between 0 and 100
your quess: 10
my number is smaller...
your quess: 5
my number is bigger...
your quess: helloworld
I don't understand you...
your quess: 6
disco! I had 6 in mind...
waldek@debian:~$
```
<details>
<summary>Spoiler warning!</summary>
```bash
#!/bin/bash
difficulty=100
computernumber=$(( $RANDOM % ( $difficulty + 1)))
echo "I have a number in mind between 0 and $difficulty"
while true; do
read -p "your quess: " usernumber
if [ $usernumber -lt $computernumber ] 2> /dev/null; then
echo "my number is bigger..."
elif [ $usernumber -gt $computernumber ] 2> /dev/null; then
echo "my number is smaller..."
elif [ $usernumber -eq $computernumber ] 2> /dev/null; then
echo "disco! I had $computernumber in mind..."
break
else
echo "I don't understand you..."
fi
done
```
</details>
# Coding challenge - Student reports
With the [following](../assets/scores.csv) file write me a program that prints:
* the total average score for each student
* individual scores for each student
* list of passed student or list of failed students
# User Interface - Make your scripts user friendly.
* `case` for command line arguments
* `select` for small menu's
* [dialog tutorial](https://www.linuxjournal.com/article/2807)
* [better dialog tutorial](https://linuxcommand.org/lc3_adv_dialog.php)
* [Ryan's tutorials](https://ryanstutorials.net/bash-scripting-tutorial/bash-user-interfaces.php)
# Coding challenge - Address book
* one file db or folder db
* case insensitive search
* edit record when found
* format cell phone number to standard representation
TODO
# Arrays in `bash`
TODO
## Python
[This](https://gitea.86thumbs.net/waldek/python_course_doc) repository has a twenty day course to learn python written by me.
The main file you need to follow is [this](https://gitea.86thumbs.net/waldek/python_course_doc/src/branch/master/learning_python3.md) one.
Some practical exercises can be found [here](https://gitea.86thumbs.net/waldek/linux_course_doc/src/branch/master/modules/qualifying/exercise_python.md) together with the needed [source files](https://gitea.86thumbs.net/waldek/linux_course_doc/src/branch/master/modules/qualifying/assets/files.tar.gz).
## Vim as an IDE
I made a tutorial on the essentials of [vim customization](https://gitea.86thumbs.net/waldek/linux_course_doc/src/branch/master/modules/qualifying/learning_vim_configuration.md).
My real world configuration can be found at [this](https://gitea.86thumbs.net/waldek/vimrc) repository.
Reference in New Issue View Git Blame Copy Permalink