linux_introduction/advanced/learning_shells.md

18 KiB
Raw Blame History

bash login

What happens when we log into a server or when we open up a terminal with bash running in it? As always the manual can tell us quite a bit. Quite often you'll find a list of configuration files used by a program near the end of the manual. Sometimes not, your millage may vary but here is the files section of the bash manual. If you can't find a list of files the program you want to investigate searches for, there are other ways of digging deep into what a program is doing but that is for a later date.

FILES
       /bin/bash
              The bash executable
       /etc/profile
              The systemwide initialization file, executed for login shells
       /etc/bash.bashrc
              The systemwide per-interactive-shell startup file
       /etc/bash.bash.logout
              The systemwide login shell cleanup file, executed when a login shell exits
       ~/.bash_profile
              The personal initialization file, executed for login shells
       ~/.bashrc
              The individual per-interactive-shell startup file
       ~/.bash_logout
              The individual login shell cleanup file, executed when a login shell exits
       ~/.inputrc
              Individual readline initialization file

I created a minimal container to play around a bit. This is what I have in my home directory.

waldek@tester:~$ ls -la
total 24
drwxr-xr-x 2 waldek waldek 4096 Jun 15 15:49 .
drwxr-xr-x 3 root   root   4096 Jun 15 15:47 ..
-rw------- 1 waldek waldek   39 Jun 15 15:50 .bash_history
-rw-r--r-- 1 waldek waldek  220 Jun 15 15:47 .bash_logout
-rw-r--r-- 1 waldek waldek 3526 Jun 15 15:47 .bashrc
-rw-r--r-- 1 waldek waldek  807 Jun 15 15:47 .profile
waldek@tester:~$ 

Let's have a look at them one by one. First the .bash_history. This contains a history of all the commands I ran on this computer! The arrow keys navigate his file when you search through your history. Quite handy for when we forget how to do something but we know we've done it before. You can use grep to search the file but there is also a shortcut you can use, ctrl-r.

waldek@tester:~$ cat .bash_history 
vim .profile 
su
ls -la
exit
su
logout
waldek@tester:~$ 

Next up is .bash_logout. It's purpose is written in the comment on the first line! This file gets executed when a login shell exits. Is there a difference between a shell and a login shell? I'll let you ponder this a bit and we'll come back to this later.

waldek@tester:~$ cat .bash_logout 
# ~/.bash_logout: executed by bash(1) when login shell exits.

# when leaving the console clear the screen to increase privacy

if [ "$SHLVL" = 1  ]; then
    [ -x /usr/bin/clear_console  ] && /usr/bin/clear_console -q
fi
waldek@tester:~$ 

Now the .bashrc file. I know it's a pretty long file so I'll just show the head.

waldek@tester:~$ head .bashrc
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples

# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *) return;;
esac

waldek@tester:~$ 

Aha! The comment on the first line mentions non-login shells so there must be a difference! This is a very important configuration file and I urge you to read it, especially the comments. Notice how the length of your history file is set in here? What kind of content is in this file?

Last but not least .profile.

waldek@tester:~$ cat .profile 
# ~/.profile: executed by the command interpreter for login shells.
# This file is not read by bash(1), if ~/.bash_profile or ~/.bash_login
# exists.
# see /usr/share/doc/bash/examples/startup-files for examples.
# the files are located in the bash-doc package.

# the default umask is set in /etc/profile; for setting the umask
# for ssh logins, install and configure the libpam-umask package.
#umask 022

# if running bash
if [ -n "$BASH_VERSION"  ]; then
    # include .bashrc if it exists
    if [ -f "$HOME/.bashrc"  ]; then
	. "$HOME/.bashrc"
    fi
fi

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/bin"  ] ; then
    PATH="$HOME/bin:$PATH"
fi

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/.local/bin"  ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi
waldek@tester:~$ 

which files are sources when

So, we have different files that are sourced at different times. Let's try to play around a bit. I added some comments to them so we get a clear idea which are sourced when.

waldek@tester:~$ tail -n 1 .bashrc .profile .bash_logout 
==> .bashrc <==
echo I am .bachrc

==> .profile <==
echo I am .profile

==> .bash_logout <==
echo I am .bash_logout
waldek@tester:~$ 

When I log in on a tty I get the following output. This is an actual interactive login so we source both .bashrc and .profile. You might ask yourself where the other output comes from. That's up to you to find out!

Debian GNU/Linux 11 tester pts/1

tester login: waldek
Password: 
Linux tester 5.10.0-15-amd64 #1 SMP Debian 5.10.120-1 (2022-06-09) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Jun 15 16:03:17 CEST 2022 on pts/1
I am .bachrc
I am .profile
waldek@tester:~$ 

What if we su to ourselves?

waldek@tester:~$ su waldek
Password: 
I am .bachrc
waldek@tester:~$ 

The .profile is not sourced! We can however change the behaviour of su with arguments. A quick look at the man su tells us that.

       -, -l, --login
              Start the shell as a login shell with an environment similar to a real login:

                 o      clears all the environment variables except TERM and variables specified by --whitelist-environment

                 o      initializes the environment variables HOME, SHELL, USER, LOGNAME, and PATH

                 o      changes to the target user's home directory

                 o      sets argv[0] of the shell to '-' in order to make the shell a login shell

So we can do the following.

waldek@tester:~$ su - waldek
Password: 
I am .bachrc
I am .profile
waldek@tester:~$ 

If we exit this session we get the following.

waldek@tester:~$ exit
logout
I am .bash_logout
waldek@tester:~$ 

Ah, there is out .bash_logout message! And if we exit the tty session we get this.

waldek@tester:~$ exit
logout
I am .bash_logout

Debian GNU/Linux 11 tester pts/1

tester login: 

When we launch a bash in a bash we get the following. But when we launch a script, none of our files get sourced!

waldek@tester:~$ bash
I am .bachrc
waldek@tester:~$ exit
exit
waldek@tester:~$ bash script.sh 
I'm a script
waldek@tester:~$ 

What about ssh connections? I'm running the server on port 2222 because it's a container with host networking. It's a bit too early to go into too much detail but by then end of the course you'll fully understand what's happening!

waldek@tester:~$ ssh localhost -p 2222
waldek@localhost's password: 
Linux tester 5.10.0-15-amd64 #1 SMP Debian 5.10.120-1 (2022-06-09) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Jun 15 16:29:26 2022 from ::1
I am .bachrc
I am .profile
waldek@tester:~$ exit
logout
I am .bash_logout
Connection to localhost closed.
waldek@tester:~$ 

What if we run a command over ssh? Then none of these files are sourced!

waldek@tester:~$ ssh localhost -p 2222 ls -la
waldek@localhost's password: 
total 36
drwxr-xr-x 3 waldek waldek 4096 Jun 15 16:29 .
drwxr-xr-x 3 root   root   4096 Jun 15 15:47 ..
-rw------- 1 waldek waldek  649 Jun 15 16:27 .bash_history
-rw-r--r-- 1 waldek waldek  243 Jun 15 16:19 .bash_logout
-rw-r--r-- 1 waldek waldek 3545 Jun 15 16:20 .bashrc
-rw-r--r-- 1 waldek waldek  826 Jun 15 16:17 .profile
drwx------ 2 waldek waldek 4096 Jun 15 16:29 .ssh
-rw------- 1 waldek waldek 3141 Jun 15 16:20 .viminfo
-rw-r--r-- 1 waldek waldek   34 Jun 15 16:18 script.sh
waldek@tester:~$ 

A clean slate

I deleted all the content of our three files.

waldek@tester:~$ cat .bashrc .bash_logout .profile 



waldek@tester:~$ 

We don't see any big differences but our shell is less powerful. For example, autocomplete does not work anymore on things like systemctl or apt. We now have a bash shell that has only sourced actual content from the following files.

  • /etc/profile
  • /etc/bash.bashrc
  • /etc/bash.bash.logout if it exists

As these are system files we can only edit them as root or via sudo. I went a head and added a comment # in front of every line. When I now log in on a tty I get the following.

Debian GNU/Linux 11 tester pts/1

tester login: waldek
Password: 
Linux tester 5.10.0-15-amd64 #1 SMP Debian 5.10.120-1 (2022-06-09) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Jun 15 16:44:05 CEST 2022 on pts/1
-bash-5.1$ 

This is a fully barebones bash shell, no bells and whistles at all. You can see our prompt has changed to a super simple one, no more username nor hostname nor path. Tab complete works, but only for programs and paths, not for arguments.

Prompt customization

Form this basic session we can learn quite a bit. The most obvious change is our prompt. The fact that it changed means we can modify it too! Let's discover how.

From the manual we can conclude that $PSX are the variables that hold the prompt.

       PS0    The value of this parameter is expanded (see PROMPTING below) and displayed by interactive shells after reading a command and  before
              the command is executed.
       PS1    The  value of this parameter is expanded (see PROMPTING below) and used as the primary prompt string.  The default value is ``\s-\v\$
              ''.
       PS2    The value of this parameter is expanded as with PS1 and used as the secondary prompt string.  The default is ``> ''.
       PS3    The value of this parameter is used as the prompt for the select command (see SHELL GRAMMAR above).
       PS4    The value of this parameter is expanded as with PS1 and the value is printed before each command bash displays  during  an  execution
              trace.   The  first character of the expanded value of PS4 is replicated multiple times, as necessary, to indicate multiple levels of
              indirection.  The default is ``+ ''.
-bash-5.1$ echo $PS
$PS1  $PS2  $PS4  
-bash-5.1$ echo $PS1
\s-\v\$
-bash-5.1$ 

Your prompt is defined by the $PS1 variable. By setting this we can change it's behaviour.

-bash-5.1$ PS1="helloworld"
helloworldls
script.sh
helloworldcd /
helloworldls
bin  boot  dev	etc  home  lib	lib32  lib64  libx32  media  mnt  opt  proc  root  run	sbin  srv  sys	tmp  usr  var
helloworld

This is far from useful but it's just to prove a point. Let's try to get our username back. Again, by reading the manual we can see all variables we can use in a prompt. I include the PROMPTING section for completeness.

PROMPTING
       When  executing interactively, bash displays the primary prompt PS1 when it is ready to read a command, and the secondary prompt PS2 when it
       needs more input to complete a command.  Bash displays PS0 after it reads a command but before executing it.  Bash displays PS4 as described
       above before tracing each command when the -x option is enabled.  Bash allows these prompt strings to be customized by inserting a number of
       backslash-escaped special characters that are decoded as follows:
              \a     an ASCII bell character (07)
              \d     the date in "Weekday Month Date" format (e.g., "Tue May 26")
              \D{format}
                     the format is passed to strftime(3) and the result is inserted into the prompt string; an empty format results  in  a  locale-
                     specific time representation.  The braces are required
              \e     an ASCII escape character (033)
\h     the hostname up to the first `.'
              \H     the hostname
              \j     the number of jobs currently managed by the shell
              \l     the basename of the shell's terminal device name
              \n     newline
              \r     carriage return
              \s     the name of the shell, the basename of $0 (the portion following the final slash)
              \t     the current time in 24-hour HH:MM:SS format
              \T     the current time in 12-hour HH:MM:SS format
              \@     the current time in 12-hour am/pm format
              \A     the current time in 24-hour HH:MM format
              \u     the username of the current user
              \v     the version of bash (e.g., 2.00)
              \V     the release of bash, version + patch level (e.g., 2.00.0)
              \w     the current working directory, with $HOME abbreviated with a tilde (uses the value of the PROMPT_DIRTRIM variable)
              \W     the basename of the current working directory, with $HOME abbreviated with a tilde
              \!     the history number of this command
              \#     the command number of this command
              \$     if the effective UID is 0, a #, otherwise a $
              \nnn   the character corresponding to the octal number nnn
              \\     a backslash
              \[     begin a sequence of non-printing characters, which could be used to embed a terminal control sequence into the prompt
              \]     end a sequence of non-printing characters

       The  command number and the history number are usually different: the history number of a command is its position in the history list, which
       may include commands restored from the history file (see HISTORY below), while the command number is the position in the  sequence  of  com
       mands executed during the current shell session.  After the string is decoded, it is expanded via parameter expansion, command substitution
       or contain characters special to word expansion.

So, to get our username back we need the \u variable.

helloworldPS1="\u"
waldekls
bin  boot  dev	etc  home  lib	lib32  lib64  libx32  media  mnt  opt  proc  root  run	sbin  srv  sys	tmp  usr  var
waldek

That glue between our username and command is rather annoying, let's modify that a bit.

waldekPS1="\u -> "
waldek -> ls
bin  boot  dev	etc  home  lib	lib32  lib64  libx32  media  mnt  opt  proc  root  run	sbin  srv  sys	tmp  usr  var
waldek -> 

Ah, much better! Now, what time is it?

waldek -> PS1="\u \t -> "
waldek 17:02:53 -> ls
bin  boot  dev	etc  home  lib	lib32  lib64  libx32  media  mnt  opt  proc  root  run	sbin  srv  sys	tmp  usr  var
waldek 17:02:56 -> 

And where am I?

waldek@tester 17:04:10 -> PS1="\u@\H \t -> "
waldek@tester 17:04:19 -> ls
bin  boot  dev	etc  home  lib	lib32  lib64  libx32  media  mnt  opt  proc  root  run	sbin  srv  sys	tmp  usr  var
waldek@tester 17:04:20 -> 

And in which directory?

waldek@tester 17:04:20 -> PS1="\u@\H:\w \t -> "
waldek@tester:/ 17:05:06 -> cd /var/log/
waldek@tester:/var/log 17:05:13 -> 

What about the other $PS variables? Well, if we want to know the exact time when we launch a command we can modify the $PS0 variable.

waldek@tester:/var/log -> PS0="it is \t \n"
waldek@tester:/var/log -> ls
it is 17:06:48 
alternatives.log  auth.log	 btmp	     dpkg.log  journal	messages  runit   user.log
apt		  bootstrap.log  daemon.log  faillog   lastlog	private   syslog  wtmp
waldek@tester:/var/log -> cd
it is 17:06:58 
waldek@tester:~ -> 

Saving our changes

When you exit your shell, and log back in, you'll notice the prompt is back to normal. If we want to save the one we have we need to place it in one of the files that get sourced when our session starts. I would recommend you make your changes to the .bashrc file and have a .profile that sources the .bashrc if we run an interactive bash shell. You can find some interesting notes here. So let's get started!

waldek@tester:~ -> tail -n +1 .profile .bashrc # a nice trick to show file content with header ;)
==> .profile <==
if [ "$BASH"  ]; then
    . ~/.bashrc
fi

==> .bashrc <==
PS1="\u@\H:\w -> "
echo "sourced"
waldek@tester:~ -> 

I'm writing this documentation on a laptop so it would be nice to see my battery status on the command line. By now you probably guessed it, but the sourced files are scripts so we can use every trick we've learned during our bash scripting classes. I'll make a function that prints the status of the battery and will include this function in the prompt.

function get_battery_status () {
	status=$(cat /sys/class/power_supply/BAT0/status)
	echo -e $status # -e for no newline

}

PS1="\u@\H:\w [\$(get_battery_status)] -> " # need to escape the $ sign or use single quotes!
#PS1='\u@\H:\w [$(get_battery_status)] -> ' # this would work as well.. ah, bash...

echo "sourced"

Decoding the base prompt

On a modern install your $PS1 will probably be as one of the two below. They are actually the same, just one has colors in it, the other one not! Customizing colors of a prompt is a bit of a nightmare but well worth the exercise!

  • PS1='\u@\h:\w\$ '
  • PS1='\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '']]]]'

Alternative shells

TODO

zsh

TODO

fish

TODO

xonsh

TODO

Shell completion

TODO

Frameworks

TODO

oh my zsh

TODO

oh my bash

TODO

xxh

TODO