From fe76dac62a9b6a3b74ce26081261f39d9b168c24 Mon Sep 17 00:00:00 2001 From: waldek Date: Wed, 15 Jun 2022 17:19:33 +0200 Subject: [PATCH] first documentation of shell modding --- advanced/learning_shells.md | 490 +++++++++++++++++++++++++++++++++++- 1 file changed, 487 insertions(+), 3 deletions(-) diff --git a/advanced/learning_shells.md b/advanced/learning_shells.md index ef5c416..d06a48f 100644 --- a/advanced/learning_shells.md +++ b/advanced/learning_shells.md @@ -1,14 +1,498 @@ # `bash` login -TODO +What happens when we log into a server or when we open up a terminal with `bash` running in it? +As always the manual can tell us quite a bit. +Quite often you'll find a list of configuration files used by a program near the end of the manual. +Sometimes not, your millage may vary but here is the files section of the `bash` manual. + +``` +FILES + /bin/bash + The bash executable + /etc/profile + The systemwide initialization file, executed for login shells + /etc/bash.bashrc + The systemwide per-interactive-shell startup file + /etc/bash.bash.logout + The systemwide login shell cleanup file, executed when a login shell exits + ~/.bash_profile + The personal initialization file, executed for login shells + ~/.bashrc + The individual per-interactive-shell startup file + ~/.bash_logout + The individual login shell cleanup file, executed when a login shell exits + ~/.inputrc + Individual readline initialization file + +``` + +I created a minimal container to play around a bit. +This is what I have in my home directory. + +``` +waldek@tester:~$ ls -la +total 24 +drwxr-xr-x 2 waldek waldek 4096 Jun 15 15:49 . +drwxr-xr-x 3 root root 4096 Jun 15 15:47 .. +-rw------- 1 waldek waldek 39 Jun 15 15:50 .bash_history +-rw-r--r-- 1 waldek waldek 220 Jun 15 15:47 .bash_logout +-rw-r--r-- 1 waldek waldek 3526 Jun 15 15:47 .bashrc +-rw-r--r-- 1 waldek waldek 807 Jun 15 15:47 .profile +waldek@tester:~$ +``` + +Let's have a look at them one by one. +First the `.bash_history`. +This contains a history of all the commands I ran on this computer! +Quite handy for when we forget how to do something but we know we've done it before. +You can use `grep` to search the file but there is also a shortcut you can use, `ctrl-r`. + +``` +waldek@tester:~$ cat .bash_history +vim .profile +su +ls -la +exit +su +logout +waldek@tester:~$ +``` + +Next up is `.bash_logout`. +It's purpose is written in the comment on the first line! +This file gets executed when a login shell exits. +Is there a difference between a *shell* and a *login shell*? +I'll let you ponder this a bit and we'll come back to this later. + +``` +waldek@tester:~$ cat .bash_logout +# ~/.bash_logout: executed by bash(1) when login shell exits. + +# when leaving the console clear the screen to increase privacy + +if [ "$SHLVL" = 1 ]; then + [ -x /usr/bin/clear_console ] && /usr/bin/clear_console -q +fi +waldek@tester:~$ +``` + +Now the `.bashrc` file. +I know it's a pretty long file so I'll just show the `head`. + +``` +waldek@tester:~$ head .bashrc +# ~/.bashrc: executed by bash(1) for non-login shells. +# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc) +# for examples + +# If not running interactively, don't do anything +case $- in + *i*) ;; + *) return;; +esac + +waldek@tester:~$ +``` + +Aha! +The comment on the first line mentions *non-login shells* so there must be a difference! +This is a very important configuration file and I urge you to read it, especially the comments. +What kind of content is in this file? + +Last but not least `.profile`. + +``` +waldek@tester:~$ cat .profile +# ~/.profile: executed by the command interpreter for login shells. +# This file is not read by bash(1), if ~/.bash_profile or ~/.bash_login +# exists. +# see /usr/share/doc/bash/examples/startup-files for examples. +# the files are located in the bash-doc package. + +# the default umask is set in /etc/profile; for setting the umask +# for ssh logins, install and configure the libpam-umask package. +#umask 022 + +# if running bash +if [ -n "$BASH_VERSION" ]; then + # include .bashrc if it exists + if [ -f "$HOME/.bashrc" ]; then + . "$HOME/.bashrc" + fi +fi + +# set PATH so it includes user's private bin if it exists +if [ -d "$HOME/bin" ] ; then + PATH="$HOME/bin:$PATH" +fi + +# set PATH so it includes user's private bin if it exists +if [ -d "$HOME/.local/bin" ] ; then + PATH="$HOME/.local/bin:$PATH" +fi +waldek@tester:~$ +``` ## which files are sources when -TODO +So, we have different files that are sourced at different times. +Let's try to play around a bit. +I added some comments to them so we get a clear idea which are sourced when. + +``` +waldek@tester:~$ tail -n 1 .bashrc .profile .bash_logout +==> .bashrc <== +echo I am .bachrc + +==> .profile <== +echo I am .profile + +==> .bash_logout <== +echo I am .bash_logout +waldek@tester:~$ +``` + +When I log in on a tty I get the following output. +This is an actual interactive login so we source both `.bashrc` and `.profile`. +You might ask yourself where the *other* output comes from. +That's up to you to find out! + +``` +Debian GNU/Linux 11 tester pts/1 + +tester login: waldek +Password: +Linux tester 5.10.0-15-amd64 #1 SMP Debian 5.10.120-1 (2022-06-09) x86_64 + +The programs included with the Debian GNU/Linux system are free software; +the exact distribution terms for each program are described in the +individual files in /usr/share/doc/*/copyright. + +Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent +permitted by applicable law. +Last login: Wed Jun 15 16:03:17 CEST 2022 on pts/1 +I am .bachrc +I am .profile +waldek@tester:~$ +``` + +What if we `su` to ourselves? + +``` +waldek@tester:~$ su waldek +Password: +I am .bachrc +waldek@tester:~$ +``` + +The `.profile` is not sourced! +We can however change the behaviour of `su` with arguments. +A quick look at the `man su` tells us that. + +``` + -, -l, --login + Start the shell as a login shell with an environment similar to a real login: + + o clears all the environment variables except TERM and variables specified by --whitelist-environment + + o initializes the environment variables HOME, SHELL, USER, LOGNAME, and PATH + + o changes to the target user's home directory + + o sets argv[0] of the shell to '-' in order to make the shell a login shell + +``` + +So we can do the following. + +``` +waldek@tester:~$ su - waldek +Password: +I am .bachrc +I am .profile +waldek@tester:~$ +``` + +If we `exit` this session we get the following. + +``` +waldek@tester:~$ exit +logout +I am .bash_logout +waldek@tester:~$ +``` + +Ah, there is out `.bash_logout` message! +And if we exit the tty session we get this. + +``` +waldek@tester:~$ exit +logout +I am .bash_logout + +Debian GNU/Linux 11 tester pts/1 + +tester login: +``` + +When we launch a bash in a bash we get the following. +But when we launch a script, none of our files get sourced! + +``` +waldek@tester:~$ bash +I am .bachrc +waldek@tester:~$ exit +exit +waldek@tester:~$ bash script.sh +I'm a script +waldek@tester:~$ +``` + +What about `ssh` connections? +I'm running the server on port 2222 because it's a contained with host networking. +It's a bit too early to go into too much detail but by then end of the course you'll fully understand what's happening! + +``` +waldek@tester:~$ ssh localhost -p 2222 +waldek@localhost's password: +Linux tester 5.10.0-15-amd64 #1 SMP Debian 5.10.120-1 (2022-06-09) x86_64 + +The programs included with the Debian GNU/Linux system are free software; +the exact distribution terms for each program are described in the +individual files in /usr/share/doc/*/copyright. + +Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent +permitted by applicable law. +Last login: Wed Jun 15 16:29:26 2022 from ::1 +I am .bachrc +I am .profile +waldek@tester:~$ exit +logout +I am .bash_logout +Connection to localhost closed. +waldek@tester:~$ +``` + +What if we run a command over `ssh`? +Then none of these files are sourced! + +``` +waldek@tester:~$ ssh localhost -p 2222 ls -la +waldek@localhost's password: +total 36 +drwxr-xr-x 3 waldek waldek 4096 Jun 15 16:29 . +drwxr-xr-x 3 root root 4096 Jun 15 15:47 .. +-rw------- 1 waldek waldek 649 Jun 15 16:27 .bash_history +-rw-r--r-- 1 waldek waldek 243 Jun 15 16:19 .bash_logout +-rw-r--r-- 1 waldek waldek 3545 Jun 15 16:20 .bashrc +-rw-r--r-- 1 waldek waldek 826 Jun 15 16:17 .profile +drwx------ 2 waldek waldek 4096 Jun 15 16:29 .ssh +-rw------- 1 waldek waldek 3141 Jun 15 16:20 .viminfo +-rw-r--r-- 1 waldek waldek 34 Jun 15 16:18 script.sh +waldek@tester:~$ +``` + +# A clean slate + +I deleted all the content of our three files. + +``` +waldek@tester:~$ cat .bashrc .bash_logout .profile + + + +waldek@tester:~$ +``` + +We don't see any differences but our shell is *less* powerful. +For example, autocomplete does not work anymore on things like `systemctl`. +We now have a `bash` shell that has only sourced actual content from the following files. + +* `/etc/profile` +* `/etc/bash.bashrc` +* `/etc/bash.bash.logout` if it exists + +As these are system files we can only edit them as `root`. +I went a head and added a comment `#` in front of every line. +When I now log in on a tty I get the following. + +``` +Debian GNU/Linux 11 tester pts/1 + +tester login: waldek +Password: +Linux tester 5.10.0-15-amd64 #1 SMP Debian 5.10.120-1 (2022-06-09) x86_64 + +The programs included with the Debian GNU/Linux system are free software; +the exact distribution terms for each program are described in the +individual files in /usr/share/doc/*/copyright. + +Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent +permitted by applicable law. +Last login: Wed Jun 15 16:44:05 CEST 2022 on pts/1 +-bash-5.1$ +``` + +This is a fully barebones `bash` shell, no bells and whistles at all. +You can see our prompt has changed to a super simple one, no more username nor hostname nor path. +Tab complete works, but only for programs and paths, not for arguments. # Prompt customization -TODO +Form this basic session we can learn quite a bit. +The most obvious change is our prompt. +The fact that it changed means we can modify it too! +Let's discover how. + +From the manual we can conclude that `$PSX` are the variables that hold the prompt. + +``` + PS0 The value of this parameter is expanded (see PROMPTING below) and displayed by interactive shells after reading a command and before + the command is executed. + PS1 The value of this parameter is expanded (see PROMPTING below) and used as the primary prompt string. The default value is ``\s-\v\$ + ''. + PS2 The value of this parameter is expanded as with PS1 and used as the secondary prompt string. The default is ``> ''. + PS3 The value of this parameter is used as the prompt for the select command (see SHELL GRAMMAR above). + PS4 The value of this parameter is expanded as with PS1 and the value is printed before each command bash displays during an execution + trace. The first character of the expanded value of PS4 is replicated multiple times, as necessary, to indicate multiple levels of + indirection. The default is ``+ ''. +``` + +``` +-bash-5.1$ echo $PS +$PS1 $PS2 $PS4 +-bash-5.1$ echo $PS1 +\s-\v\$ +-bash-5.1$ +``` + +Your prompt is defined by the `$PS1` variable. +By changing this we can change it's behaviour. + +``` +-bash-5.1$ PS1="helloworld" +helloworldls +script.sh +helloworldcd / +helloworldls +bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var +helloworld +``` + +This is *far* from useful but it's just to prove a point. +Let's try to get our username back. +Again, by reading the manual we can see all variables we can use in a prompt. +I include the **PROMPTING** section for completeness. + +``` +PROMPTING + When executing interactively, bash displays the primary prompt PS1 when it is ready to read a command, and the secondary prompt PS2 when it + needs more input to complete a command. Bash displays PS0 after it reads a command but before executing it. Bash displays PS4 as described + above before tracing each command when the -x option is enabled. Bash allows these prompt strings to be customized by inserting a number of + backslash-escaped special characters that are decoded as follows: + \a an ASCII bell character (07) + \d the date in "Weekday Month Date" format (e.g., "Tue May 26") + \D{format} + the format is passed to strftime(3) and the result is inserted into the prompt string; an empty format results in a locale- + specific time representation. The braces are required + \e an ASCII escape character (033) +\h the hostname up to the first `.' + \H the hostname + \j the number of jobs currently managed by the shell + \l the basename of the shell's terminal device name + \n newline + \r carriage return + \s the name of the shell, the basename of $0 (the portion following the final slash) + \t the current time in 24-hour HH:MM:SS format + \T the current time in 12-hour HH:MM:SS format + \@ the current time in 12-hour am/pm format + \A the current time in 24-hour HH:MM format + \u the username of the current user + \v the version of bash (e.g., 2.00) + \V the release of bash, version + patch level (e.g., 2.00.0) + \w the current working directory, with $HOME abbreviated with a tilde (uses the value of the PROMPT_DIRTRIM variable) + \W the basename of the current working directory, with $HOME abbreviated with a tilde + \! the history number of this command + \# the command number of this command + \$ if the effective UID is 0, a #, otherwise a $ + \nnn the character corresponding to the octal number nnn + \\ a backslash + \[ begin a sequence of non-printing characters, which could be used to embed a terminal control sequence into the prompt + \] end a sequence of non-printing characters + + The command number and the history number are usually different: the history number of a command is its position in the history list, which + may include commands restored from the history file (see HISTORY below), while the command number is the position in the sequence of com‐ + mands executed during the current shell session. After the string is decoded, it is expanded via parameter expansion, command substitution + or contain characters special to word expansion. +``` + +So, to get our username back we need the `\u` variable. + +``` +helloworldPS1="\u" +waldekls +bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var +waldek +``` + +That *glue* between our username and command is rather annoying, let's modify that a bit. + +``` +waldekPS1="\u -> " +waldek -> ls +bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var +waldek -> +``` + +Ah, much better! +Now, what time is it? + +``` +waldek -> PS1="\u \t -> " +waldek 17:02:53 -> ls +bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var +waldek 17:02:56 -> +``` + +And where am I? + +``` +waldek@tester 17:04:10 -> PS1="\u@\H \t -> " +waldek@tester 17:04:19 -> ls +bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var +waldek@tester 17:04:20 -> +``` + +And in which directory? + +``` +waldek@tester 17:04:20 -> PS1="\u@\H:\w \t -> " +waldek@tester:/ 17:05:06 -> cd /var/log/ +waldek@tester:/var/log 17:05:13 -> +``` + +What about the *other* `$PS` variables? +Well, if we want to know the exact time when we launch a command we can modify the `$PS0` variable. + +``` +waldek@tester:/var/log -> PS0="it is \t \n" +waldek@tester:/var/log -> ls +it is 17:06:48 +alternatives.log auth.log btmp dpkg.log journal messages runit user.log +apt bootstrap.log daemon.log faillog lastlog private syslog wtmp +waldek@tester:/var/log -> cd +it is 17:06:58 +waldek@tester:~ -> +``` + +## Decoding the *base* prompt + +On a modern install your `$PS1` will probably be as one of the two below. +They are actually the *same*, just one has **colors** in it, the other one not! +Customizing colors of a prompt is a bit of a nightmare but well worth the exercise! + +* `PS1='\u@\h:\w\$ '` +* `PS1='\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '']]]]'` # Alternative shells