1751 lines
80 KiB
Markdown
1751 lines
80 KiB
Markdown
# Introduction to the command line
|
||
|
||
We'll be using a few new *words* to reference the command line such as *shell*, *bash* and *terminal* through out the course.
|
||
They all pretty much mean the same thing but with some small, and not so important, differences between them.
|
||
Essentially a command line is a **textual interface** for humans to operate a computer.
|
||
What is very important to understand is that textual commands and graphical actions operate on the **same** computer.
|
||
For example, if you create a file via the command line, it will show up in you file explorer and vice versa.
|
||
The graphical and textual interfaces are just different *representations* of the same machine.
|
||
|
||
Now open up a terminal and you'll see the following.
|
||
|
||
```
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
This is what we call a [prompt](https://en.wikipedia.org/wiki/Command-line_interface#Command_prompt).
|
||
It's not much but it's our *window* into the computer.
|
||
As with most things in life you can question it's *who, where, what and when*.
|
||
This information is actually **embedded** in the prompt.
|
||
Let's break it down.
|
||
|
||
* `waldek` is **who** I am on this computer
|
||
* `hellodebian` is the **what**, as in what computer I'm operating on
|
||
* `~` is **where** I am located on this computer
|
||
|
||
What about the *when* then?
|
||
Let's type in `date` and see what happens.
|
||
|
||
```
|
||
waldek@hellodebian:~$ date
|
||
Fri 18 Feb 2022 03:46:59 PM CET
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
There we see our **when**!
|
||
But this miniscule operation illustrates us the **fundamental** operation of a command line!
|
||
|
||
1. we have a prompt where we can run a program
|
||
2. the program runs and outputs it's information on the terminal
|
||
3. once the program **finishes** we can run an other program
|
||
|
||
I'm deliberately saying program here but is `date` *really* a program?
|
||
It's a bit basic no?
|
||
Well, it is a program and most *commands* you'll type into your terminal are actually programs.
|
||
We can illustrate this as follows.
|
||
|
||
```
|
||
waldek@hellodebian:~$ vlc
|
||
VLC media player 3.0.16 Vetinari (revision 3.0.13-8-g41878ff4f2)
|
||
[0000559c46ee95b0] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
|
||
[0000559c46f89790] main playlist: playlist is empty
|
||
|
||
```
|
||
|
||
Vlc is now running and the terminal is *blocked* meaning we can't run other programs or commands in it.
|
||
If you try to run the `date` command again, or `ls` or `htop`, it won't work!
|
||
Try it out if you don't believe me.
|
||
But what happens if you **close** vlc?
|
||
The commands you typed get executed!
|
||
This is an illustration of the **sequential** nature of a command line.
|
||
|
||
Now, I don't think we have sufficiently *proven* that `date` is a full blown program so let's dig a bit deeper.
|
||
|
||
```
|
||
waldek@hellodebian:~$ which vlc
|
||
/usr/bin/vlc
|
||
waldek@hellodebian:~$ which date
|
||
/usr/bin/date
|
||
waldek@hellodebian:~$ which which
|
||
/usr/bin/which
|
||
```
|
||
|
||
There is quite bit to *unpack* in the example above.
|
||
First, what on earth is `which`?
|
||
Well, it's *also* a program and it's sole purpose in life is to tell you **where** a program is located on your system.
|
||
Because `which` by itself does not make a lot of sense it needs what we call an **argument**.
|
||
Here the argument is the name of the program we want to know it's location of.
|
||
The existence of arguments is the second big thing we discovered here.
|
||
The third new thing we can observe here is what we call **paths**, meaning locations on the system.
|
||
For example, `vlc` is a *binary* program located in a **folder** called `bin` which is located in a folder called `usr` which is at the *root* of your system.
|
||
If this sounds complicated, don't worry, we'll go into detail a bit later.
|
||
|
||
Now that we know **where** some of our programs are located, let's find out **what** they are.
|
||
The methodology is the same as with `which` but we'll use an other program called `file` who's purpose in life is to tell more about the **content** of a certain file.
|
||
Logically, `file` needs an argument and this argument is the **path** to the file you want to inspect.
|
||
|
||
```
|
||
waldek@hellodebian:~$ file /usr/bin/vlc
|
||
/usr/bin/vlc: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=51c40f8234213415771b3a344cab25a140543f8a, for GNU/Linux 3.2.0, stripped
|
||
waldek@hellodebian:~$ file /usr/bin/date
|
||
/usr/bin/date: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=b740f054aaef6a85aff024858c914c7eae70a6a5, for GNU/Linux 3.2.0, stripped
|
||
waldek@hellodebian:~$ file /usr/bin/which
|
||
/usr/bin/which: POSIX shell script, ASCII text executable
|
||
waldek@hellodebian:~$ file $(which which) # this is some command line kung fu...
|
||
/usr/bin/which: POSIX shell script, ASCII text executable
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
Here we learn that both `vlc` and `date` are **executables**, compiled for an x86-64 system.
|
||
I would say they are both created *equally* no?
|
||
Both are actual programs.
|
||
But what about `which`?
|
||
It's also an executable but not compiled, it's a *POSIX shell script*.
|
||
So not all programs *are* created equally?
|
||
|
||
## Compiled vs interpreted
|
||
|
||
Executing a file, or program, means you take this file and tell the computer it needs to execute the actions that are stored in the file.
|
||
Compiled programs contain actual [instructions](https://en.wikipedia.org/wiki/Instruction_set_architecture#Instructions) the computer understands out of the box.
|
||
This means that programs that are compiled are always compiled for a specify architecture which in our case is x86-64.
|
||
On a Raspberry Pi this would be armhf or arm64.
|
||
|
||
Interpreted programs are not compiled to machine code but when we run them each line is passed to an appropriate **interpreter** and executed **line by line**.
|
||
The first line of a script is often the path to the interpreter that *understands* the code that will follow.
|
||
Popular interpreted languages are `bash`, `sh`, `python`, `php`, ...
|
||
|
||
We can take it *one* more step forward and peak into the *content* of the files.
|
||
A nice little program to do this is called `head` who's purpose in life is to show the first few lines of a file.
|
||
First for `vlc`.
|
||
|
||
```
|
||
waldek@hellodebian:~$ head /usr/bin/vlc
|
||
ELF>@<40>1@8
|
||
HHh,h<h<<3C><><EFBFBD>,<2C><<3C>< <20><><EFBFBD>DDP<44>td###<23><>Q<EFBFBD>tdR<64>tdh,h<h<<3C><>/lib64/ld-linux-x86-64.so.2GNUQ<EFBFBD><EFBFBD>4!4w4L<34>%<25>@T?<3F>GNU)<29>)*<2A>e<EFBFBD>m9<6D>y<EFBFBD><79><EFBFBD><EFBFBD> <20><>)F <20>, X<>`
|
||
Pco<EFBFBD><EFBFBD><EFBFBD><EFBFBD>v<EFBFBD><EFBFBD>ZD<EFBFBD>
|
||
<20><>/<2F><><EFBFBD>h<EFBFBD>"<22> _ITM_deregisterTMCloneTable__gmon_start___ITM_registerTMCloneTablelibvlc_set_app_idlibvlc_newlibvlc_set_user_agentlibvlc_get_changesetlibvlc_get_versionlibvlc_set_exit_handlerlibvlc_add_intflibvlc_playlist_playlibvlc_releasesigwaitflockfilepthread_killpthread_sigmaskpthread_mutex_lockpthread_mutex_unlockfunlockfilesigactiondlsymdlerrorfflushsignal__stack_chk_failabortisattysigemptysetrand_r__fprintf_chksigaddsetmemcpystderrsigdelsetalarmfwritegeteuid__vfprintf_chk__cxa_finalizepthread_self__libc_start_main__register_atforklibvlc.so.5libpthread.so.0libdl.so.2libc.so.6GLIBC_2<EFBFBD>ri5GLIB<EFBFBD>ti.14GL<EFBFBD>u<EFBFBD>i2.4Glh<EFBFBD>p<EFBFBD>@<40>?<3F>?GLIBC_2.3.G u<>i lW u<>i lb<6C><62>xii
|
||
<EFBFBD>?
|
||
<20>?<3F>?) @*<2A>><3E>><3E>><3E>><3E>><3E>><3E><>> <09>>
|
||
??? ?(?0?8?@?H?P?X?`?<3F>h??x?<3F>?<3F>?<3F>? <20>?!<21>?"<22>?#<23>?$<24>?%<25>?&<26>?'<27>?(H<>H<EFBFBD><48>/H<><48>t<EFBFBD><74>H<EFBFBD><48><EFBFBD>5<EFBFBD>.<2E>%<25>.@<40>%<25>.h<><68><EFBFBD><EFBFBD><EFBFBD><EFBFBD>%z.h<><68><EFBFBD><EFBFBD><EFBFBD><EFBFBD>%r.h<><68><EFBFBD><EFBFBD><EFBFBD><EFBFBD>%j.h<><68><EFBFBD><EFBFBD><EFBFBD>%b.h<><68><EFBFBD><EFBFBD><EFBFBD>%Z.h<><68><EFBFBD><EFBFBD><EFBFBD>%R.h<><68><EFBFBD><EFBFBD><EFBFBD>%J.h<>p<EFBFBD><70><EFBFBD><EFBFBD>%B.<2E>`<60><><EFBFBD><EFBFBD>%:.h <09>P<EFBFBD><50><EFBFBD><EFBFBD>%2.h
|
||
<EFBFBD>@<40><><EFBFBD><EFBFBD>%*.h
|
||
<20>0<EFBFBD><30><EFBFBD><EFBFBD>%".h
|
||
<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>%.h<><68><EFBFBD><EFBFBD><EFBFBD>% <20> <20><><EFBFBD><EFBFBD>%<25>.h
|
||
.h<><68><EFBFBD><EFBFBD><EFBFBD><EFBFBD>%.h<><68><EFBFBD><EFBFBD><EFBFBD><EFBFBD>%<25>-h<><68><EFBFBD><EFBFBD><EFBFBD><EFBFBD>%<25>-h<><68><EFBFBD><EFBFBD><EFBFBD><EFBFBD>%<25>-h<><68><EFBFBD><EFBFBD><EFBFBD>%<25>-h<><68><EFBFBD><EFBFBD><EFBFBD>%<25>-h<><68><EFBFBD><EFBFBD><EFBFBD>%<25>-h<><68><EFBFBD><EFBFBD><EFBFBD>%<25>-h<>p<EFBFBD><70><EFBFBD><EFBFBD>%<25>-h<>`<60><><EFBFBD><EFBFBD>%<25>-h<>P<EFBFBD><50><EFBFBD><EFBFBD>%<25>-h<><68>@<40><><EFBFBD><EFBFBD>%<25>-h0<68><30><EFBFBD><EFBFBD>%<25>-h<> <20><><EFBFBD><EFBFBD>%<25>-h<><68><EFBFBD><EFBFBD><EFBFBD>%<25>-h<><68><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>L<EFBFBD><4C><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>1<EFBFBD>L<EFBFBD><4C>H<EFBFBD><48><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>k<EFBFBD><6B><EFBFBD><EFBFBD>S<EFBFBD>[Hc<48>H<EFBFBD>G<EFBFBD><47><EFBFBD><EFBFBD>L<EFBFBD><4C><EFBFBD>:<3A><><EFBFBD><EFBFBD>L<EFBFBD><4C><EFBFBD>-<2D><><EFBFBD><EFBFBD>L<EFBFBD><4C><EFBFBD> <20><><EFBFBD><EFBFBD>
|
||
L<><4C>
|
||
H<><48>H<EFBFBD>5<EFBFBD>
|
||
fHn<48>H<EFBFBD>BfHn<48>H<EFBFBD><48> H<><48><EFBFBD>fl<66>I<EFBFBD>H)<29>L<EFBFBD>t$L<><4C>I<EFBFBD><49><EFBFBD>H<EFBFBD><48>L<EFBFBD><4C><EFBFBD>I<EFBFBD>~<7E>o<EFBFBD><6F><EFBFBD>Lc<4C>K<EFBFBD><4B><EFBFBD>L<EFBFBD><4C><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>I<EFBFBD><49>H<EFBFBD><48><EFBFBD><EFBFBD>H<EFBFBD><48>H<EFBFBD><48><EFBFBD><EFBFBD><EFBFBD><EFBFBD>H<EFBFBD>5<EFBFBD><35>S<EFBFBD><53><EFBFBD>H<EFBFBD>
|
||
H<EFBFBD>/
|
||
L<><4C>H<EFBFBD>5
|
||
<20>v<EFBFBD><76><EFBFBD>H<EFBFBD>
|
||
H<>5
|
||
L<><4C><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>H<EFBFBD>5!
|
||
L<><4C><EFBFBD>!<21><><EFBFBD>H<EFBFBD>5
|
||
|
||
L<EFBFBD><EFBFBD><EFBFBD>'<27><><EFBFBD>H<EFBFBD><48><EFBFBD><EFBFBD><EFBFBD><EFBFBD>L<EFBFBD><4C><EFBFBD>H<EFBFBD><48><EFBFBD>L<EFBFBD><4C><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>L<EFBFBD><4C><EFBFBD>S<EFBFBD><53><EFBFBD><EFBFBD>L<EFBFBD><4C><EFBFBD>F<EFBFBD><46><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>H<EFBFBD>5<EFBFBD><35><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>1<EFBFBD>L<EFBFBD><4C><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>L<EFBFBD><4C><EFBFBD>5<EFBFBD><35><EFBFBD><EFBFBD>8<EFBFBD><38><EFBFBD><EFBFBD><EFBFBD>I<EFBFBD><49><EFBFBD>&<26><><EFBFBD>H<EFBFBD>=<3D>*M<><4D>H<EFBFBD><48><EFBFBD>@<40><><EFBFBD>u
|
||
H<><48><EFBFBD>1<EFBFBD><31><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>n<EFBFBD><6E><EFBFBD>A<EFBFBD>H<EFBFBD>E<EFBFBD>dH+%(u?H<>e<EFBFBD>D<EFBFBD><44>[A\A]A^]<5D>I<EFBFBD>M<EFBFBD>H<EFBFBD>=\*H<><48>
|
||
CH<EFBFBD>=L<><4C><EFBFBD><EFBFBD><EFBFBD>)<29>DH<44>=<3D>)H<><48>)H9<48>tH<74><48>)H<><48>t<EFBFBD>f.<2E>1<EFBFBD><31><EFBFBD><EFBFBD><EFBFBD><EFBFBD>H<EFBFBD>=<3D>)H<>5<EFBFBD>)H)<29>H<EFBFBD><48>H<EFBFBD><48>?H<><48>H<EFBFBD>H<EFBFBD><48>tH<74>U)H<><48><EFBFBD><EFBFBD>fD<66><44><EFBFBD>=q)u/UH<55>=6)H<><48>t
|
||
H<>=:)<29><><EFBFBD><EFBFBD><EFBFBD>h<EFBFBD><68><EFBFBD><EFBFBD>I)]<5D><><EFBFBD><EFBFBD><EFBFBD>{<7B><><EFBFBD>f.<2E><>1<EFBFBD><31><EFBFBD>$<24><><EFBFBD>@H<>?<3F><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>)<29><>AUI<55><49>ATI<54><49>UH<55><48>SH<53><48>H<EFBFBD><48><EFBFBD>L<EFBFBD>D$@L<>L$H<><48>t7)D$P)L$`)T$p)<29>$<24>)<29>$<24>)<29>$<24>)<29>$<24>)<29>$<24>dH<64>%(H<>D$1<>H<EFBFBD>={(H<>T$ H<><48>$<24>$ <20>D$0H<30>DH<44>T$<24><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>
|
||
<20><>v1H<31>=B(<28><><EFBFBD><EFBFBD><EFBFBD>H<EFBFBD>D$dH+%(uH<75><48><EFBFBD>[]A\A]<5D><><EFBFBD><EFBFBD>H<EFBFBD>=(I<><49>1<EFBFBD><31>
|
||
H<EFBFBD><EFBFBD>H<EFBFBD>=7 H<>5H<35>=?'<27><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>e<EFBFBD><65><EFBFBD><EFBFBD>#<23><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>f<EFBFBD>ATI<54><49><EFBFBD>f<EFBFBD><66><EFBFBD>H<EFBFBD>=<3D>'L<><4C>H<EFBFBD>I<EFBFBD><49><EFBFBD>1<EFBFBD><31><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>H<EFBFBD>i<EFBFBD><69><EFBFBD>1<EFBFBD>1<EFBFBD><31>.(<28>f<EFBFBD><66>=(UH<55><48>u#H<>5H<35><48><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>H<EFBFBD><48>t5H<35><48>]<5D><>DI<44><49>1<EFBFBD>H<EFBFBD>
|
||
<EFBFBD>H<EFBFBD>H<EFBFBD>5<EFBFBD>H<EFBFBD>=7&<26><>&<26><><EFBFBD><EFBFBD>X1<58>Z]A\A]<5D>H<EFBFBD><48><EFBFBD><EFBFBD><EFBFBD>ff.<2E><><EFBFBD>='UH<55><48>u#H<>5<EFBFBD>H<EFBFBD><48><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>H<EFBFBD><48>t5H<35><48>]<5D><>DI<44><49>1<EFBFBD>H<EFBFBD><48>RH<52>
|
||
!"<22><><EFBFBD><EFBFBD>H<EFBFBD>H<EFBFBD><48>DESKTOP_STARTUP_ID--no-ignore-config--media-library--dbusvlcorg.VideoLAN.VLCVLC/3.0.16VLC media playerglobalhotkeys,noneVLC is not supposed to be run as root. Sorry.
|
||
If you need to use real-time priorities and/or privileged TCP ports
|
||
you can use %s-wrapper (make sure it is Set-UID root and
|
||
```
|
||
|
||
And now of `which`.
|
||
|
||
```
|
||
waldek@hellodebian:~$ head /usr/bin/which
|
||
#! /bin/sh
|
||
set -ef
|
||
|
||
if test -n "$KSH_VERSION"; then
|
||
puts() {
|
||
print -r -- "$*"
|
||
}
|
||
else
|
||
puts() {
|
||
printf '%s\n' "$*"
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
Notice how the `vlc` file is mostly **not human readable** but we can still make out some keywords?
|
||
The `which` file however is perfectly readable!
|
||
The first line of the `which` file is `#! /bin/sh` which is the path to the interpreter.
|
||
The `#!` is called a [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)).
|
||
We can dig into this to learn more about this mysterious `sh`.
|
||
|
||
```
|
||
waldek@hellodebian:~$ file /bin/sh
|
||
/bin/sh: symbolic link to dash
|
||
waldek@hellodebian:~$ which dash
|
||
/usr/bin/dash
|
||
waldek@hellodebian:~$ file /usr/bin/dash
|
||
/usr/bin/dash: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=cb6911fd56559717336c938bef1ce479b0a85b35, for GNU/Linux 3.2.0, stripped
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
Surprise surprise!
|
||
The *interpreter* is a compiled program!
|
||
When you think about it it does make sense because at the end of the day the computer *only* understands machine instructions.
|
||
So when you execute a script each line of code is converted to machine instructions on the spot.
|
||
This makes scripts easier to write but slower at execution.
|
||
The above is a *generalization* so take it with a grain of salt please!
|
||
|
||
## Note on file extensions
|
||
|
||
Linux does not rely as much on file extensions as windows does.
|
||
We could even say it often does not care about them.
|
||
Due to the decentralized nature of Linux development, some programs *do* care about them while others don't.
|
||
It's a bit of hit and miss but I advise you to read [this](https://askubuntu.com/questions/803434/do-file-extensions-have-any-purpose-in-linux) post on AskUbuntu for a more detailed explanation.
|
||
|
||
# More about paths
|
||
|
||
The prompt in our terminal tells us who we are, on which machine, and **where** we are located.
|
||
This where is symbolized with the `~` character.
|
||
But where is this where?
|
||
|
||
```
|
||
waldek@hellodebian:~$ pwd
|
||
/home/waldek
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
The `pwd` program **p**rints our **w**orking **d**irectory, meaning where we are located on our system.
|
||
A user's home directory is symbolized with this tilde character.
|
||
We can move around our system with the `cd` command which is an abbreviation of **c**hange **d**irectory.
|
||
`cd` by itself seems to to *nothing* but this is not the case.
|
||
It's a *shortcut* to go to your user's home directory.
|
||
Let's try this out.
|
||
|
||
```
|
||
waldek@hellodebian:~$ cd
|
||
waldek@hellodebian:~$ cd /usr/bin
|
||
waldek@hellodebian:/usr/bin$ cd
|
||
waldek@hellodebian:~$ cd -
|
||
/usr/bin
|
||
waldek@hellodebian:/usr/bin$ cd
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
The first line appears to do nothing because we are already in our home directory.
|
||
Next we move to the directory which contains the programs we discovered before and we see our prompt change.
|
||
It now shows *where* we are located!
|
||
The next line takes us back to our home and from there we can move *back to where we were before* with the `cd -` command.
|
||
This last command is also a handy *shortcut*!
|
||
|
||
Moving around is nice but once we arrive at our destination it would be practical to be able to list the files in that directory no?
|
||
This can be done with the `ls` program.
|
||
For example.
|
||
|
||
```
|
||
waldek@hellodebian:~$ cd /usr/bin/
|
||
waldek@hellodebian:/usr/bin$ ls
|
||
'[' getconf nmtui-hostname systemd-escape
|
||
7z getent nohup systemd-hwdb
|
||
7za getfacl notify-send systemd-id128
|
||
7zr getkeycodes nproc systemd-inhibit
|
||
aa-enabled getopt nroff systemd-machine-id-setup
|
||
aa-exec gettext nsenter systemd-mount
|
||
aconnect gettext.sh nslookup systemd-notify
|
||
add-apt-repository gio nstat systemd-path
|
||
addpart gio-querymodules nsupdate systemd-resolve
|
||
addr2line gjs ntfs-3g systemd-run
|
||
alsabat gjs-console ntfs-3g.probe systemd-socket-activate
|
||
alsaloop gkbd-keyboard-display ntfscat systemd-stdio-bridge
|
||
alsamixer glib-compile-schemas ntfscluster systemd-sysusers
|
||
alsatplg gmake ntfscmp systemd-tmpfiles
|
||
alsaucm gnome-calculator ntfsdecrypt systemd-tty-ask-password-agent
|
||
|
||
# ------------------------------------
|
||
# --- cropped the list for clarity ---
|
||
# ------------------------------------
|
||
|
||
gdk-pixbuf-csource nisdomainname system-config-printer-applet zforce
|
||
gdk-pixbuf-pixdata nl systemctl zgrep
|
||
gdk-pixbuf-thumbnailer nm systemd zipdetails
|
||
gdm-control nm-applet systemd-analyze zipgrep
|
||
gdmflexiserver nmcli systemd-ask-password zipinfo
|
||
gdm-screenshot nm-connection-editor systemd-cat zless
|
||
gedit nm-online systemd-cgls zmore
|
||
gencat nmtui systemd-cgtop znew
|
||
geqn nmtui-connect systemd-delta
|
||
GET nmtui-edit systemd-detect-virt
|
||
waldek@hellodebian:/usr/bin$
|
||
```
|
||
|
||
That's a **lot** of programs!
|
||
Can you find `cd` in there?
|
||
Spoiler alert, you wont...
|
||
Maybe it's located somewhere else?
|
||
Let's have a look.
|
||
|
||
```
|
||
waldek@hellodebian:/usr/bin$ which cd
|
||
waldek@hellodebian:/usr/bin$
|
||
```
|
||
|
||
`cd` is not a program but a builtin command.
|
||
From a practical point of view there is no difference but I do like to mention it for completeness.
|
||
Later down the line it will help you to contextualize the differences between shells a bit better.
|
||
But built into *what*?
|
||
Below you can see a snippet of a part of the `bash-builtin` manual.
|
||
For now just accept the specific shell we're using is called `bash`.
|
||
|
||
```
|
||
BASH-BUILTINS(7) Miscellaneous Information Manual BASH-BUILTINS(7)
|
||
|
||
NAME
|
||
bash-builtins - bash built-in commands, see bash(1)
|
||
|
||
SYNOPSIS
|
||
bash defines the following built-in commands: :, ., [, alias, bg, bind, break, builtin, case, cd, command, compgen, complete, continue, de‐
|
||
clare, dirs, disown, echo, enable, eval, exec, exit, export, fc, fg, getopts, hash, help, history, if, jobs, kill, let, local, logout, popd,
|
||
printf, pushd, pwd, read, readonly, return, set, shift, shopt, source, suspend, test, times, trap, type, typeset, ulimit, umask, unalias,
|
||
unset, until, wait, while.
|
||
```
|
||
|
||
## Absolute and relative paths
|
||
|
||
> All roads lead to Rome.
|
||
|
||
There are multiple ways to go to the same location on your system.
|
||
You can always go either in an **absolute** way, or in *multiple* **relative** ways.
|
||
The root of your system is `/` and to list what we can find there we can use `ls`.
|
||
|
||
```
|
||
waldek@hellodebian:~$ cd /
|
||
waldek@hellodebian:/$ ls
|
||
bin dev home initrd.img.old lib32 libx32 media opt root sbin sys usr vmlinuz
|
||
boot etc initrd.img lib lib64 lost+found mnt proc run srv tmp var vmlinuz.old
|
||
waldek@hellodebian:/$
|
||
```
|
||
|
||
From here we can go back home in multiple ways.
|
||
|
||
```
|
||
waldek@hellodebian:/$ cd
|
||
waldek@hellodebian:~$ cd -
|
||
/
|
||
waldek@hellodebian:/$ cd home/waldek/
|
||
waldek@hellodebian:~$ cd -
|
||
/
|
||
waldek@hellodebian:/$ cd /home/waldek/
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
The first one is the handy shortcut we learned, and we go back to the root of our system with the `cd -` shortcut.
|
||
The following two manipulations look very similar but there is a subtle difference.
|
||
The first one `home/waldek` is a **relative** path, and the second one `/home/waldek` is an **absolute** one.
|
||
Relative paths depend on where you are located, absolute ones always point to the same location.
|
||
An example.
|
||
|
||
```
|
||
waldek@hellodebian:~$ cd /usr/bin/
|
||
waldek@hellodebian:/usr/bin$ cd /home/waldek/
|
||
waldek@hellodebian:~$ cd -
|
||
/usr/bin
|
||
waldek@hellodebian:/usr/bin$ cd home/waldek
|
||
-bash: cd: home/waldek: No such file or directory
|
||
waldek@hellodebian:/usr/bin$
|
||
```
|
||
|
||
The second command fails because from where I'm standing, there is no folder called `home/waldek`!
|
||
I can however still go to my home in a relative way but I need to *go back* a few directories first.
|
||
|
||
```
|
||
waldek@hellodebian:/usr/bin$ cd ../../home/waldek/
|
||
waldek@hellodebian:~$ pwd
|
||
/home/waldek
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
The `..` means go **back** one directory so in our example we go back **two** directories, which brings us to the `/` of our system and from there we go **up** to `home` and then `waldek`.
|
||
|
||
## `root` and `/` and `/root` are not the same thing
|
||
|
||
The word *root* has multiple meanings and can be a bit confusing at the start.
|
||
Every Linux machine will have a `root` user **and** a `/` of the system.
|
||
The `root` user is an account essential to the proper working of a Linux system, whereas `/` is a location, the *root* of the filesystem.
|
||
The `/root` is also a location but is the **home** of the user `root`, located at the `/` of the filesystem.
|
||
If this sounds confusing read the above paragraph slowly until you see the differences.
|
||
I do want to note that the name of the user `root` *can* change but it's ID will always be `0`.
|
||
We'll go into detail about users and ID's later down the line.
|
||
|
||
# Exercise
|
||
|
||
Explore your system a bit using `cd` and `ls`.
|
||
You'll probably encounter some weird messages along the way.
|
||
Keep note of them so we can discuss it together.
|
||
|
||
# A pit stop to review what we've learned so far
|
||
|
||
Below is a list of programs and command we've seen so far.
|
||
We've seen that some take arguments, some don't.
|
||
This list will grow as we go so I advise you to take some notes of your own.
|
||
|
||
|command|description|
|
||
|---|---|
|
||
|systemctl|manipulate running services|
|
||
|apt|the Debian package manager|
|
||
|htop|a command line task manager we installed|
|
||
|vlc|a video player we installed|
|
||
|date|display the time of day|
|
||
|which|show the path to an executable|
|
||
|file|print more information about a file's content|
|
||
|head|show the first lines of a file|
|
||
|pwd|print working directory|
|
||
|cd|change directory|
|
||
|ls|list content of a directory|
|
||
|
||
# Arguments and flags
|
||
|
||
The `ls` program is one of the most used commands.
|
||
It's like a very basic *file explorer* in a graphical environment and just like it's graphical counter part you can use it to display it's data in different ways.
|
||
Think of the *details* versus *thumbnail* views in your favorite explorer.
|
||
But how do you do this?
|
||
We can **add flags or options** to the program to modify it's behavior.
|
||
Have a look at the output below.
|
||
|
||
```
|
||
waldek@hellodebian:~$ ls
|
||
Desktop Documents Downloads Music Pictures Public Templates Videos
|
||
waldek@hellodebian:~$ ls -l
|
||
total 32
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Desktop
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Documents
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Downloads
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Music
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Pictures
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Public
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Templates
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Videos
|
||
waldek@hellodebian:~$ ls -a
|
||
. .bashrc Desktop .gnupg .profile Templates .vboxclient-seamless.pid
|
||
.. .cache .dmrc .local Public .vboxclient-clipboard.pid Videos
|
||
.bash_history .config Documents Music .python_history .vboxclient-display-svga-x11.pid .Xauthority
|
||
.bash_logout .dbus Downloads Pictures .ssh .vboxclient-draganddrop.pid .xsession-errors
|
||
```
|
||
|
||
The `-l` and `-a` additions are what we call *flags* or *options*.
|
||
Aren't they just extra *arguments*?
|
||
There is no real rule to the naming of this concept but [this](https://unix.stackexchange.com/questions/285575/whats-the-difference-between-a-flag-an-option-and-an-argument) is a rather interesting take.
|
||
|
||
We can combine arguments if we want, and some programs even allow *concatenation* of arguments.
|
||
Have a look below to understand what I mean by that.
|
||
|
||
```
|
||
waldek@hellodebian:~$ ls -a -l
|
||
total 112
|
||
drwxr-xr-x 16 waldek waldek 4096 Feb 21 18:12 .
|
||
drwxr-xr-x 3 root root 4096 Feb 18 12:49 ..
|
||
-rw------- 1 waldek waldek 2772 Feb 22 10:45 .bash_history
|
||
-rw-r--r-- 1 waldek waldek 220 Feb 18 12:49 .bash_logout
|
||
-rw-r--r-- 1 waldek waldek 3526 Feb 18 12:49 .bashrc
|
||
drwx------ 16 waldek waldek 4096 Feb 18 14:24 .cache
|
||
drwx------ 20 waldek waldek 4096 Feb 18 14:23 .config
|
||
drwx------ 3 waldek waldek 4096 Feb 18 12:52 .dbus
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Desktop
|
||
-rw-r--r-- 1 waldek waldek 35 Feb 18 13:33 .dmrc
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Documents
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Downloads
|
||
drwx------ 2 waldek waldek 4096 Feb 18 15:07 .gnupg
|
||
drwxr-xr-x 3 waldek waldek 4096 Feb 18 12:50 .local
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Music
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Pictures
|
||
-rw-r--r-- 1 waldek waldek 807 Feb 18 12:49 .profile
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Public
|
||
-rw------- 1 waldek waldek 11 Feb 21 18:12 .python_history
|
||
drwx------ 2 waldek waldek 4096 Feb 21 12:45 .ssh
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Templates
|
||
-rw-r----- 1 waldek waldek 5 Feb 18 15:06 .vboxclient-clipboard.pid
|
||
-rw-r----- 1 waldek waldek 5 Feb 18 15:06 .vboxclient-display-svga-x11.pid
|
||
-rw-r----- 1 waldek waldek 5 Feb 18 15:06 .vboxclient-draganddrop.pid
|
||
-rw-r----- 1 waldek waldek 5 Feb 18 15:06 .vboxclient-seamless.pid
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Videos
|
||
-rw------- 1 waldek waldek 56 Feb 18 13:33 .Xauthority
|
||
-rw------- 1 waldek waldek 2701 Feb 18 13:33 .xsession-errors
|
||
waldek@hellodebian:~$ ls -la
|
||
total 112
|
||
drwxr-xr-x 16 waldek waldek 4096 Feb 21 18:12 .
|
||
drwxr-xr-x 3 root root 4096 Feb 18 12:49 ..
|
||
-rw------- 1 waldek waldek 2772 Feb 22 10:45 .bash_history
|
||
-rw-r--r-- 1 waldek waldek 220 Feb 18 12:49 .bash_logout
|
||
-rw-r--r-- 1 waldek waldek 3526 Feb 18 12:49 .bashrc
|
||
drwx------ 16 waldek waldek 4096 Feb 18 14:24 .cache
|
||
drwx------ 20 waldek waldek 4096 Feb 18 14:23 .config
|
||
drwx------ 3 waldek waldek 4096 Feb 18 12:52 .dbus
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Desktop
|
||
-rw-r--r-- 1 waldek waldek 35 Feb 18 13:33 .dmrc
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Documents
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Downloads
|
||
drwx------ 2 waldek waldek 4096 Feb 18 15:07 .gnupg
|
||
drwxr-xr-x 3 waldek waldek 4096 Feb 18 12:50 .local
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Music
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Pictures
|
||
-rw-r--r-- 1 waldek waldek 807 Feb 18 12:49 .profile
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Public
|
||
-rw------- 1 waldek waldek 11 Feb 21 18:12 .python_history
|
||
drwx------ 2 waldek waldek 4096 Feb 21 12:45 .ssh
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Templates
|
||
-rw-r----- 1 waldek waldek 5 Feb 18 15:06 .vboxclient-clipboard.pid
|
||
-rw-r----- 1 waldek waldek 5 Feb 18 15:06 .vboxclient-display-svga-x11.pid
|
||
-rw-r----- 1 waldek waldek 5 Feb 18 15:06 .vboxclient-draganddrop.pid
|
||
-rw-r----- 1 waldek waldek 5 Feb 18 15:06 .vboxclient-seamless.pid
|
||
drwxr-xr-x 2 waldek waldek 4096 Feb 18 12:50 Videos
|
||
-rw------- 1 waldek waldek 56 Feb 18 13:33 .Xauthority
|
||
-rw------- 1 waldek waldek 2701 Feb 18 13:33 .xsession-errors
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
# Getting help
|
||
|
||
## Options
|
||
|
||
OK, so we have *a lot* of programs we can use on the command line, plus we can change how they work by adding obscure characters.
|
||
How on earth can we *discover* what a program can and can't do?
|
||
Introducing the **most important flag**: `--help`.
|
||
|
||
Most programs come with builtin help to explain you how to use the program.
|
||
Often, but not always, this help can be displayed on the terminal by adding the `--help` flag.
|
||
Let's investigate this.
|
||
|
||
```
|
||
waldek@hellodebian:~$ ls --help
|
||
Usage: ls [OPTION]... [FILE]...
|
||
List information about the FILEs (the current directory by default).
|
||
Sort entries alphabetically if none of -cftuvSUX nor --sort is specified.
|
||
|
||
Mandatory arguments to long options are mandatory for short options too.
|
||
-a, --all do not ignore entries starting with .
|
||
-A, --almost-all do not list implied . and ..
|
||
--author with -l, print the author of each file
|
||
-b, --escape print C-style escapes for nongraphic characters
|
||
--block-size=SIZE with -l, scale sizes by SIZE when printing them;
|
||
e.g., '--block-size=M'; see SIZE format below
|
||
-B, --ignore-backups do not list implied entries ending with ~
|
||
-c with -lt: sort by, and show, ctime (time of last
|
||
modification of file status information);
|
||
with -l: show ctime and sort by name;
|
||
otherwise: sort by ctime, newest first
|
||
-C list entries by columns
|
||
--color[=WHEN] colorize the output; WHEN can be 'always' (default
|
||
if omitted), 'auto', or 'never'; more info below
|
||
-d, --directory list directories themselves, not their contents
|
||
-D, --dired generate output designed for Emacs' dired mode
|
||
-f do not sort, enable -aU, disable -ls --color
|
||
-F, --classify append indicator (one of */=>@|) to entries
|
||
--file-type likewise, except do not append '*'
|
||
--format=WORD across -x, commas -m, horizontal -x, long -l,
|
||
single-column -1, verbose -l, vertical -C
|
||
--full-time like -l --time-style=full-iso
|
||
-g like -l, but do not list owner
|
||
--group-directories-first
|
||
group directories before files;
|
||
can be augmented with a --sort option, but any
|
||
use of --sort=none (-U) disables grouping
|
||
-G, --no-group in a long listing, don't print group names
|
||
-h, --human-readable with -l and -s, print sizes like 1K 234M 2G etc.
|
||
--si likewise, but use powers of 1000 not 1024
|
||
-H, --dereference-command-line
|
||
follow symbolic links listed on the command line
|
||
--dereference-command-line-symlink-to-dir
|
||
follow each command line symbolic link
|
||
that points to a directory
|
||
--hide=PATTERN do not list implied entries matching shell PATTERN
|
||
(overridden by -a or -A)
|
||
--hyperlink[=WHEN] hyperlink file names; WHEN can be 'always'
|
||
(default if omitted), 'auto', or 'never'
|
||
--indicator-style=WORD append indicator with style WORD to entry names:
|
||
none (default), slash (-p),
|
||
file-type (--file-type), classify (-F)
|
||
-i, --inode print the index number of each file
|
||
-I, --ignore=PATTERN do not list implied entries matching shell PATTERN
|
||
-k, --kibibytes default to 1024-byte blocks for disk usage;
|
||
used only with -s and per directory totals
|
||
-l use a long listing format
|
||
-L, --dereference when showing file information for a symbolic
|
||
link, show information for the file the link
|
||
references rather than for the link itself
|
||
-m fill width with a comma separated list of entries
|
||
-n, --numeric-uid-gid like -l, but list numeric user and group IDs
|
||
-N, --literal print entry names without quoting
|
||
-o like -l, but do not list group information
|
||
-p, --indicator-style=slash
|
||
append / indicator to directories
|
||
-q, --hide-control-chars print ? instead of nongraphic characters
|
||
--show-control-chars show nongraphic characters as-is (the default,
|
||
unless program is 'ls' and output is a terminal)
|
||
-Q, --quote-name enclose entry names in double quotes
|
||
--quoting-style=WORD use quoting style WORD for entry names:
|
||
literal, locale, shell, shell-always,
|
||
shell-escape, shell-escape-always, c, escape
|
||
(overrides QUOTING_STYLE environment variable)
|
||
-r, --reverse reverse order while sorting
|
||
-R, --recursive list subdirectories recursively
|
||
-s, --size print the allocated size of each file, in blocks
|
||
-S sort by file size, largest first
|
||
--sort=WORD sort by WORD instead of name: none (-U), size (-S),
|
||
time (-t), version (-v), extension (-X)
|
||
--time=WORD change the default of using modification times;
|
||
access time (-u): atime, access, use;
|
||
change time (-c): ctime, status;
|
||
birth time: birth, creation;
|
||
with -l, WORD determines which time to show;
|
||
with --sort=time, sort by WORD (newest first)
|
||
--time-style=TIME_STYLE time/date format with -l; see TIME_STYLE below
|
||
-t sort by time, newest first; see --time
|
||
-T, --tabsize=COLS assume tab stops at each COLS instead of 8
|
||
-u with -lt: sort by, and show, access time;
|
||
with -l: show access time and sort by name;
|
||
otherwise: sort by access time, newest first
|
||
-U do not sort; list entries in directory order
|
||
-v natural sort of (version) numbers within text
|
||
-w, --width=COLS set output width to COLS. 0 means no limit
|
||
-x list entries by lines instead of by columns
|
||
-X sort alphabetically by entry extension
|
||
-Z, --context print any security context of each file
|
||
-1 list one file per line. Avoid '\n' with -q or -b
|
||
--help display this help and exit
|
||
--version output version information and exit
|
||
|
||
The SIZE argument is an integer and optional unit (example: 10K is 10*1024).
|
||
Units are K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,... (powers of 1000).
|
||
Binary prefixes can be used, too: KiB=K, MiB=M, and so on.
|
||
|
||
The TIME_STYLE argument can be full-iso, long-iso, iso, locale, or +FORMAT.
|
||
FORMAT is interpreted like in date(1). If FORMAT is FORMAT1<newline>FORMAT2,
|
||
then FORMAT1 applies to non-recent files and FORMAT2 to recent files.
|
||
TIME_STYLE prefixed with 'posix-' takes effect only outside the POSIX locale.
|
||
Also the TIME_STYLE environment variable sets the default style to use.
|
||
|
||
Using color to distinguish file types is disabled both by default and
|
||
with --color=never. With --color=auto, ls emits color codes only when
|
||
standard output is connected to a terminal. The LS_COLORS environment
|
||
variable can change the settings. Use the dircolors command to set it.
|
||
|
||
Exit status:
|
||
0 if OK,
|
||
1 if minor problems (e.g., cannot access subdirectory),
|
||
2 if serious trouble (e.g., cannot access command-line argument).
|
||
|
||
GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
|
||
Full documentation <https://www.gnu.org/software/coreutils/ls>
|
||
or available locally via: info '(coreutils) ls invocation'
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
The output above teaches us that the options `-la` change the output so it becomes:
|
||
|
||
* `-a, --all do not ignore entries starting with .`
|
||
* `-l use a long listing format`
|
||
|
||
To *not ignore entries starting with .* we can use two different flags, `-a` or `--all`.
|
||
The former is called short arguments and the latter long ones.
|
||
The long ones are more *verbose* so it's often easier to understand what a line *does* just by reading it.
|
||
You can read up on the [history of arguments](https://blog.liw.fi/posts/2022/05/07/unix-cli/).
|
||
Note that `ls` only has one way of showing the help by adding `--help` whereas the help for `htop` can be shown with both `-h` or `--help`.
|
||
This inconsistency is sadly a byproduct of the decentralized nature of Linux.
|
||
To add to this mess some programs use *single dash long form* (`-help`) but those programs are rather rare.
|
||
|
||
## Manuals
|
||
|
||
The `--help` flag is tremendously useful as it gives you an overview of how you can modify a programs behaviour, kind of like settings, but it doesn't always explain you what a program is *for*.
|
||
The is where the manuals come into play.
|
||
I see the `--help` flag as a *cheat sheet* to quickly discover options and the man pages as a reference.
|
||
We can read the manual by calling the `man` program with an argument.
|
||
This argument is the *name* of the program you want to read the manual for.
|
||
Same idea as with `which`.
|
||
|
||
```
|
||
waldek@hellodebian:~$ man ls
|
||
```
|
||
|
||
You're now in a program called `less`, which is a [pager](https://en.wikipedia.org/wiki/Terminal_pager).
|
||
You can scroll the manual with the arrows of `vi` based navigation (which we'll get into later).
|
||
If you hit `h` you'll see the builtin documentation of `less` and press `q` to quit (twice if you're in the help section).
|
||
This brings you back to your prompt.
|
||
It's worth reading the [wikipedia entry](https://en.wikipedia.org/wiki/Man_page) of the `man` command to understand it's history and operation.
|
||
|
||
The output below illustrates the *tree* of a terminal that's reading a manual.
|
||
In a `bash` shell you invoke `man ls` which in turn runs the `pager` program to display the content.
|
||
If you're intrigued by this `pstree` program go and read the manual!
|
||
|
||
```
|
||
waldek@hellodebian:~$ pstree 10845 -a
|
||
bash
|
||
└─man ls
|
||
└─pager
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
Not every command has a manual.
|
||
For example, `cd` does not have one.
|
||
This is kind of logical because `cd` is a builtin.
|
||
To learn more about builtin commands you can use the `help` program.
|
||
On it's own it gives you all the builtin command you can invoke and with an argument it outputs the help about that specific command.
|
||
|
||
```
|
||
waldek@hellodebian:~$ help
|
||
GNU bash, version 5.1.4(1)-release (x86_64-pc-linux-gnu)
|
||
These shell commands are defined internally. Type `help' to see this list.
|
||
Type `help name' to find out more about the function `name'.
|
||
Use `info bash' to find out more about the shell in general.
|
||
Use `man -k' or `info' to find out more about commands not in this list.
|
||
|
||
A star (*) next to a name means that the command is disabled.
|
||
|
||
job_spec [&] history [-c] [-d offset] [n] or history -anrw [filename] or history -ps>
|
||
(( expression )) if COMMANDS; then COMMANDS; [ elif COMMANDS; then COMMANDS; ]... [ else>
|
||
. filename [arguments] jobs [-lnprs] [jobspec ...] or jobs -x command [args]
|
||
: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [>
|
||
[ arg... ] let arg [arg ...]
|
||
[[ expression ]] local [option] name[=value] ...
|
||
alias [-p] [name[=value] ... ] logout [n]
|
||
bg [job_spec ...] mapfile [-d delim] [-n count] [-O origin] [-s count] [-t] [-u fd] [-C c>
|
||
bind [-lpsvPSVX] [-m keymap] [-f filename] [-q name] [-u name] [-r keyse> popd [-n] [+N | -N]
|
||
break [n] printf [-v var] format [arguments]
|
||
builtin [shell-builtin [arg ...]] pushd [-n] [+N | -N | dir]
|
||
caller [expr] pwd [-LP]
|
||
case WORD in [PATTERN [| PATTERN]...) COMMANDS ;;]... esac read [-ers] [-a array] [-d delim] [-i text] [-n nchars] [-N nchars] [-p>
|
||
cd [-L|[-P [-e]] [-@]] [dir] readarray [-d delim] [-n count] [-O origin] [-s count] [-t] [-u fd] [-C>
|
||
command [-pVv] command [arg ...] readonly [-aAf] [name[=value] ...] or readonly -p
|
||
compgen [-abcdefgjksuv] [-o option] [-A action] [-G globpat] [-W wordlis> return [n]
|
||
complete [-abcdefgjksuv] [-pr] [-DEI] [-o option] [-A action] [-G globpa> select NAME [in WORDS ... ;] do COMMANDS; done
|
||
compopt [-o|+o option] [-DEI] [name ...] set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
|
||
continue [n] shift [n]
|
||
coproc [NAME] command [redirections] shopt [-pqsu] [-o] [optname ...]
|
||
declare [-aAfFgiIlnrtux] [-p] [name[=value] ...] source filename [arguments]
|
||
dirs [-clpv] [+N] [-N] suspend [-f]
|
||
disown [-h] [-ar] [jobspec ... | pid ...] test [expr]
|
||
echo [-neE] [arg ...] time [-p] pipeline
|
||
enable [-a] [-dnps] [-f filename] [name ...] times
|
||
eval [arg ...] trap [-lp] [[arg] signal_spec ...]
|
||
exec [-cl] [-a name] [command [argument ...]] [redirection ...] true
|
||
exit [n] type [-afptP] name [name ...]
|
||
export [-fn] [name[=value] ...] or export -p typeset [-aAfFgiIlnrtux] [-p] name[=value] ...
|
||
false ulimit [-SHabcdefiklmnpqrstuvxPT] [limit]
|
||
fc [-e ename] [-lnr] [first] [last] or fc -s [pat=rep] [command] umask [-p] [-S] [mode]
|
||
fg [job_spec] unalias [-a] name [name ...]
|
||
for NAME [in WORDS ... ] ; do COMMANDS; done unset [-f] [-v] [-n] [name ...]
|
||
for (( exp1; exp2; exp3 )); do COMMANDS; done until COMMANDS; do COMMANDS; done
|
||
function name { COMMANDS ; } or name () { COMMANDS ; } variables - Names and meanings of some shell variables
|
||
getopts optstring name [arg ...] wait [-fn] [-p var] [id ...]
|
||
hash [-lr] [-p pathname] [-dt] [name ...] while COMMANDS; do COMMANDS; done
|
||
help [-dms] [pattern ...] { COMMANDS ; }
|
||
waldek@hellodebian:~$ help cd
|
||
cd: cd [-L|[-P [-e]] [-@]] [dir]
|
||
Change the shell working directory.
|
||
|
||
Change the current directory to DIR. The default DIR is the value of the
|
||
HOME shell variable.
|
||
|
||
The variable CDPATH defines the search path for the directory containing
|
||
DIR. Alternative directory names in CDPATH are separated by a colon (:).
|
||
A null directory name is the same as the current directory. If DIR begins
|
||
with a slash (/), then CDPATH is not used.
|
||
|
||
If the directory is not found, and the shell option `cdable_vars' is set,
|
||
the word is assumed to be a variable name. If that variable has a value,
|
||
its value is used for DIR.
|
||
|
||
Options:
|
||
-L force symbolic links to be followed: resolve symbolic
|
||
links in DIR after processing instances of `..'
|
||
-P use the physical directory structure without following
|
||
symbolic links: resolve symbolic links in DIR before
|
||
processing instances of `..'
|
||
-e if the -P option is supplied, and the current working
|
||
directory cannot be determined successfully, exit with
|
||
a non-zero status
|
||
-@ on systems that support it, present a file with extended
|
||
attributes as a directory containing the file attributes
|
||
|
||
The default is to follow symbolic links, as if `-L' were specified.
|
||
`..' is processed by removing the immediately previous pathname component
|
||
back to a slash or the beginning of DIR.
|
||
|
||
Exit Status:
|
||
Returns 0 if the directory is changed, and if $PWD is set successfully when
|
||
-P is used; non-zero otherwise.
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
Last but not least there are a few extra programs to give you quick and condensed information about a program, `whatis` and `apropos`.
|
||
|
||
```
|
||
waldek@hellodebian:~$ whatis bash
|
||
bash (1) - GNU Bourne-Again SHell
|
||
waldek@hellodebian:~$ apropos bash
|
||
bash (1) - GNU Bourne-Again SHell
|
||
bash-builtins (7) - bash built-in commands, see bash(1)
|
||
bashbug (1) - report a bug in bash
|
||
builtins (7) - bash built-in commands, see bash(1)
|
||
dh_bash-completion (1) - install bash completions for package
|
||
rbash (1) - restricted bash, see bash(1)
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
## Exercise
|
||
|
||
**Read** some manual pages on the commands we've seen until now.
|
||
**Apply** some options you read about in the manual to experiment with said programs.
|
||
|
||
# Creating, modifying and deleting
|
||
|
||
We've seen how to navigate our filesystem and how to list files and directories at certain locations.
|
||
Now let's learn how to add, modify and remove files and directories.
|
||
|
||
## Directories
|
||
|
||
The main tool to create directories is `mkdir`.
|
||
It's usage is pretty simple, it will create a directory at the location you specify as argument.
|
||
This location can be absolute or relative.
|
||
By default it will **not** create parent directories if needed but we can force it to do so with the `-p` flag.
|
||
|
||
```
|
||
waldek@hellodebian:~$ ls
|
||
Desktop Documents Downloads Music Pictures Public Templates Videos
|
||
waldek@hellodebian:~$ mkdir website
|
||
waldek@hellodebian:~$ ls
|
||
Desktop Documents Downloads Music Pictures Public Templates Videos website
|
||
waldek@hellodebian:~$ ls website/
|
||
waldek@hellodebian:~$ mkdir website/page_one/assets
|
||
mkdir: cannot create directory ‘website/page_one/assets’: No such file or directory
|
||
waldek@hellodebian:~$ mkdir -p website/page_one/assets
|
||
waldek@hellodebian:~$ ls -R website/
|
||
website/:
|
||
page_one
|
||
|
||
website/page_one:
|
||
assets
|
||
|
||
website/page_one/assets:
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
The counterpart to `mkdir` is `rmdir` which can be used to remove **empty** directories.
|
||
As it can only remove empty directories it's not used that much but it's worth knowing it exists.
|
||
|
||
```
|
||
waldek@hellodebian:~$ rmdir website/
|
||
rmdir: failed to remove 'website/': Directory not empty
|
||
waldek@hellodebian:~$ rmdir website/page_one/
|
||
rmdir: failed to remove 'website/page_one/': Directory not empty
|
||
waldek@hellodebian:~$ rmdir website/page_one/assets/
|
||
waldek@hellodebian:~$ rmdir website/page_one/
|
||
waldek@hellodebian:~$ rmdir website/
|
||
waldek@hellodebian:~$ ls
|
||
Desktop Documents Downloads Music Pictures Public Templates Videos
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
To remove directories that have subdirectories or files in them we need to use an other program, `rm` but with a specific flag `-r`.
|
||
It works as such.
|
||
|
||
```
|
||
waldek@hellodebian:~$ mkdir -p website/page_one/assets
|
||
waldek@hellodebian:~$ rm website/
|
||
rm: cannot remove 'website/': Is a directory
|
||
waldek@hellodebian:~$ rm -r website/
|
||
waldek@hellodebian:~$ ls
|
||
Desktop Documents Downloads Music Pictures Public Templates Videos
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
**There is no trash or recycle bin on the command line! Removed is removed so think before you hit enter.**
|
||
|
||
## Files
|
||
|
||
We can create empty files with the `touch` command.
|
||
It's worth pointing out that using `touch` to create files is not it's main purpose.
|
||
Reading the manual the fact that it creates files is kind of an extra feature.
|
||
Non the less, it's widely used to create files.
|
||
|
||
|
||
```
|
||
TOUCH(1) User Commands TOUCH(1)
|
||
|
||
NAME
|
||
touch - change file timestamps
|
||
|
||
SYNOPSIS
|
||
touch [OPTION]... FILE...
|
||
|
||
DESCRIPTION
|
||
Update the access and modification times of each FILE to the current time.
|
||
|
||
A FILE argument that does not exist is created empty, unless -c or -h is supplied.
|
||
|
||
A FILE argument string of - is handled specially and causes touch to change the times of the file associated with standard output.
|
||
|
||
Mandatory arguments to long options are mandatory for short options too.
|
||
|
||
```
|
||
|
||
In practice it works as follows.
|
||
We can create files with `touch` and delete them with `rm`.
|
||
For deleting files we don't need any extra flags as opposed to directories.
|
||
|
||
```
|
||
waldek@hellodebian:~$ mkdir -p website/page_one/assets
|
||
waldek@hellodebian:~$ ls website/
|
||
page_one
|
||
waldek@hellodebian:~$ touch website/index.html
|
||
waldek@hellodebian:~$ ls website/
|
||
index.html page_one
|
||
waldek@hellodebian:~$ file website/index.html
|
||
website/index.html: empty
|
||
waldek@hellodebian:~$ rm website/index.html
|
||
waldek@hellodebian:~$ ls website/
|
||
page_one
|
||
waldek@hellodebian:~$ file website/index.html
|
||
website/index.html: cannot open `website/index.html' (No such file or directory)
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
What if you created the file in the wrong place?
|
||
With `mv` we can move files from one location to an other.
|
||
When you think about it, renaming is actually the *same* as moving so there is no specific program to rename files or folders.
|
||
|
||
```
|
||
waldek@hellodebian:~$ touch index.html
|
||
waldek@hellodebian:~$ ls
|
||
Desktop Documents Downloads index.html Music Pictures Public Templates Videos website
|
||
waldek@hellodebian:~$ mv index.html website/indexxxx.html
|
||
waldek@hellodebian:~$ ls
|
||
Desktop Documents Downloads Music Pictures Public Templates Videos website
|
||
waldek@hellodebian:~$ ls website/
|
||
indexxxx.html page_one
|
||
waldek@hellodebian:~$ mv website/indexxxx.html website/index.html
|
||
waldek@hellodebian:~$ ls website/
|
||
index.html page_one
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
We can also move entire directories from one location to another.
|
||
|
||
```
|
||
waldek@hellodebian:~$ ls
|
||
Desktop Documents Downloads Music Pictures Public Templates Videos website
|
||
waldek@hellodebian:~$ mv website/ Documents/
|
||
waldek@hellodebian:~$ ls
|
||
Desktop Documents Downloads Music Pictures Public Templates Videos
|
||
waldek@hellodebian:~$ ls -R Documents/
|
||
Documents/:
|
||
website
|
||
|
||
Documents/website:
|
||
index.html page_one
|
||
|
||
Documents/website/page_one:
|
||
assets
|
||
|
||
Documents/website/page_one/assets:
|
||
waldek@hellodebian:~$
|
||
```
|
||
|
||
Copying is done with `cp` and behaves very similar to `mv`.
|
||
For example.
|
||
|
||
```
|
||
waldek@hellodebian:~/Documents$ ls
|
||
website
|
||
waldek@hellodebian:~/Documents$ mv website/ website_1
|
||
waldek@hellodebian:~/Documents$ ls
|
||
website_1
|
||
waldek@hellodebian:~/Documents$ cp website_1/ website_2
|
||
cp: -r not specified; omitting directory 'website_1/'
|
||
waldek@hellodebian:~/Documents$ cp -r website_1/ website_2
|
||
waldek@hellodebian:~/Documents$ ls
|
||
website_1 website_2
|
||
waldek@hellodebian:~/Documents$ ls -R
|
||
.:
|
||
website_1 website_2
|
||
|
||
./website_1:
|
||
index.html page_one
|
||
|
||
./website_1/page_one:
|
||
assets
|
||
|
||
./website_1/page_one/assets:
|
||
|
||
./website_2:
|
||
index.html page_one
|
||
|
||
./website_2/page_one:
|
||
assets
|
||
|
||
./website_2/page_one/assets:
|
||
waldek@hellodebian:~/Documents$
|
||
```
|
||
|
||
## Links
|
||
|
||
TODO
|
||
|
||
|
||
# Editors
|
||
|
||
## `nano`
|
||
|
||
The *easiest* to use command line editor that comes installed with most Linux distributions is called `nano`.
|
||
Below is a screenshot of an empty text file, opened up in `nano`.
|
||
|
||
|
||
|
||
In this window we can write anything we want.
|
||
I added two lines of text, but how do I *save and quit* this program?
|
||
At the bottom of the screen there is a *menu*.
|
||
`^X` is the *button* to exit `nano` but how do we invoke it?
|
||
Well, by pressing **CRTL-x**.
|
||
By logical consequence to search the text file we can use **CTRL-w**.
|
||
When you try to exit, dialog will pop up to ask you to save.
|
||
If you answer **Y**, you'll be prompted for a filename.
|
||
Here you can just hit **ENTER**.
|
||
|
||
|
||
|
||
|
||
|
||
### Exercise
|
||
|
||
Create some text files with `nano`.
|
||
Move them around with `mv`, delete them with `rm` and copy them with `cp`.
|
||
Can you edit the `/etc/passwd` file?
|
||
|
||
## `vi` and `vim`
|
||
|
||
On of the most [popular](https://www.journaldev.com/41292/top-best-text-editors-linux) text editors is by far `vim`.
|
||
It is an *improvement* on an older editor called `vi`, vim(proved).
|
||
Personally it is my favorite editor, or even program, on Linux and I can highly advise you to become somewhat comfortable with it.
|
||
The learning curve is quite *steep* but so very much worth it!
|
||
The reason *why* it is a bit complicated at first is because `vim` is a [modal](https://www.wikidata.org/wiki/Q80592893) text editor.
|
||
The paragraph below is taken from the [wikipedia](https://en.wikipedia.org/wiki/Vi) page of `vi`.
|
||
|
||
> vi is a modal editor: it operates in either insert mode (where typed text becomes part of the document) or command mode (where keystrokes are interpreted as commands that control the edit session). For example, typing i while in command mode switches the editor to insert mode, but typing i again at this point places an "i" character in the document. From insert mode, pressing ESC switches the editor back to command mode. A perceived advantage of vi's separation of text entry and command modes is that both text editing and command operations can be performed without requiring the removal of the user's hands from the home row. As non-modal editors usually have to reserve all keys with letters and symbols for the printing of characters, any special commands for actions other than adding text to the buffer must be assigned to keys that do not produce characters, such as function keys, or combinations of modifier keys such as Ctrl, and Alt with regular keys. Vi has the property that most ordinary keys are connected to some kind of command for positioning, altering text, searching and so forth, either singly or in key combinations. Many commands can be touch typed without the use of Ctrl or Alt. Other types of editors generally require the user to move their hands from the home row when touch typing:
|
||
|
||
### Exercise
|
||
|
||
In order to use `vim` we need to install it.
|
||
Most Linux distributions come with some form of `vi` or `vim` installed.
|
||
On a modern Debian system this is:
|
||
|
||
```
|
||
waldek@debian:~$ vi
|
||
vi view vim.tiny
|
||
waldek@debian:~$ vi
|
||
```
|
||
|
||
`vim.tiny` is a version of `vim` compiled with only the basic features.
|
||
We can see what this means by adding the `--version` argument.
|
||
|
||
```
|
||
waldek@debian:~$ vim.tiny --version
|
||
VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Oct 01 2021 01:51:08)
|
||
Included patches: 1-2434
|
||
Extra patches: 8.2.3402, 8.2.3403, 8.2.3409, 8.2.3428
|
||
Modified by team+vim@tracker.debian.org
|
||
Compiled by team+vim@tracker.debian.org
|
||
Small version without GUI. Features included (+) or not (-):
|
||
+acl -farsi +mouse_sgr -tag_old_static
|
||
-arabic -file_in_path -mouse_sysmouse -tag_any_white
|
||
+autocmd -find_in_path -mouse_urxvt -tcl
|
||
-autochdir -float +mouse_xterm -termguicolors
|
||
-autoservername -folding +multi_byte -terminal
|
||
-balloon_eval -footer -multi_lang +terminfo
|
||
-balloon_eval_term +fork() -mzscheme +termresponse
|
||
-browse -gettext -netbeans_intg -textobjects
|
||
+builtin_terms -hangul_input +num64 -textprop
|
||
-byte_offset +iconv -packages -timers
|
||
-channel +insert_expand -path_extra -title
|
||
-cindent -ipv6 -perl -toolbar
|
||
-clientserver -job -persistent_undo +user_commands
|
||
-clipboard +jumplist -popupwin -vartabs
|
||
+cmdline_compl -keymap -printer +vertsplit
|
||
+cmdline_hist -lambda -profile +virtualedit
|
||
-cmdline_info -langmap -python +visual
|
||
+comments -libcall -python3 +visualextra
|
||
-conceal -linebreak -quickfix -viminfo
|
||
-cryptv -lispindent -reltime +vreplace
|
||
-cscope +listcmds -rightleft +wildignore
|
||
+cursorbind +localmap -ruby -wildmenu
|
||
-cursorshape -lua +scrollbind +windows
|
||
-dialog -menu -signs +writebackup
|
||
-diff -mksession -smartindent -X11
|
||
-digraphs +modify_fname -sound +xfontset
|
||
-dnd +mouse -spell -xim
|
||
-ebcdic -mouseshape -startuptime -xpm
|
||
-emacs_tags -mouse_dec -statusline -xsmp
|
||
-eval -mouse_gpm -sun_workshop -xterm_clipboard
|
||
+ex_extra -mouse_jsbterm -syntax -xterm_save
|
||
-extra_search -mouse_netterm +tag_binary
|
||
system vimrc file: "$VIM/vimrc"
|
||
user vimrc file: "$HOME/.vimrc"
|
||
2nd user vimrc file: "~/.vim/vimrc"
|
||
user exrc file: "$HOME/.exrc"
|
||
defaults file: "$VIMRUNTIME/defaults.vim"
|
||
fall-back for $VIM: "/usr/share/vim"
|
||
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -Wdate-time -g -O2 -ffile-prefix-map=/build/vim-DtwDbo/vim-8.2.2434=. -fstack-protector-strong -Wformat -Werror=format-security -DTINY_VIMRC -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
|
||
Linking: gcc -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim -lSM -lICE -lXpm -lXt -lX11 -lXdmcp -lSM -lICE -lm -ltinfo -lselinux -lacl -lattr -ldl
|
||
waldek@debian:~$
|
||
```
|
||
|
||
We can install a more complete version which is called `vim-nox`.
|
||
|
||
```
|
||
waldek@debian:~$ apt search vim-nox
|
||
Sorting... Done
|
||
Full Text Search... Done
|
||
vim-nox/stable 2:8.2.2434-3+deb11u1 amd64
|
||
Vi IMproved - enhanced vi editor - with scripting languages support
|
||
|
||
vim-tiny/stable,now 2:8.2.2434-3+deb11u1 amd64 [installed]
|
||
Vi IMproved - enhanced vi editor - compact version
|
||
|
||
waldek@debian:~$
|
||
```
|
||
|
||
I highly advise you to install the more complete version.
|
||
Once done, have a look at the *new* programs installed with tab complete.
|
||
|
||
```
|
||
waldek@debian:~$ vim
|
||
vim vimdiff vim.nox vim.tiny vimtutor
|
||
waldek@debian:~$ which vim
|
||
/usr/bin/vim
|
||
waldek@debian:~$ ls -l /usr/bin/vim
|
||
lrwxrwxrwx 1 root root 21 May 13 11:20 /usr/bin/vim -> /etc/alternatives/vim
|
||
```
|
||
|
||
The only way to learn `vim` is to *use* it.
|
||
Luckily `vim-nox` comes with a dedicated *program* to learn it called `vimtutor`.
|
||
It's actually a text file that you open up in `vim` where you just need to **read** and **execute** what you've read.
|
||
Enjoy!
|
||
|
||
Once you have completed the `vimtutor` I urge you to have a look at one of the *many* cheat sheets [online](https://vim.rtorr.com/).
|
||
|
||
# Opening or peaking into files
|
||
|
||
We can use `vim` or `nano` to open text files and write to them, but there are a few programs worth knowing that can *just* show the content of a text file.
|
||
You'll use these all the time!
|
||
Before we've seen that `head` shows the first few lines of a file.
|
||
To see the *last lines* of a file we use `tail`.
|
||
Get it, *head and tail*?
|
||
The linux world is full of little inside jokes like this.
|
||
Anyway, the two most used programs to **read** file content are `cat` and `less`.
|
||
|
||
`cat` prints the content of a file straight to the terminal, where as `less` is what we call a [pager](https://en.wikipedia.org/wiki/Terminal_pager).
|
||
If you need to search or read a long file you'll find that a pager is more practical.
|
||
Reading the `man less` page we see the following.
|
||
|
||
```
|
||
LESS(1) General Commands Manual LESS(1)
|
||
|
||
NAME
|
||
less - opposite of more
|
||
```
|
||
|
||
Again, the joke here is that `less` can actually do *more* than `more`.
|
||
|
||
# Exercise
|
||
|
||
Open up some files with `cat` and `less`.
|
||
If you need some inspiration, try a few files in the `/etc` directory.
|
||
|
||
# Finding stuff
|
||
|
||
## Finding files
|
||
|
||
### The easy way
|
||
|
||
On most distributions `locate` will not be installed by default but I advise you to install it, especially in the beginning of your linux career.
|
||
The syntax is super simple and it fetches results from a database instead of the hard drive so it's quite quick and doesn't have permission problems.
|
||
To install and use it, you should proceed as follows.
|
||
|
||
```
|
||
waldek@debian:~$ sudo apt install locate
|
||
Reading package lists... Done
|
||
Building dependency tree... Done
|
||
Reading state information... Done
|
||
locate is already the newest version (4.8.0-1).
|
||
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
|
||
waldek@debian:~$ locate bin/zip
|
||
/usr/bin/zipdetails
|
||
waldek@debian:~$ locate zip | grep bin # this is a weird thing...
|
||
/usr/bin/bunzip2
|
||
/usr/bin/bzip2
|
||
/usr/bin/bzip2recover
|
||
/usr/bin/gunzip
|
||
/usr/bin/gzip
|
||
/usr/bin/streamzip
|
||
/usr/bin/zipdetails
|
||
/usr/lib/klibc/bin/gunzip
|
||
/usr/lib/klibc/bin/gzip
|
||
waldek@debian:~$
|
||
```
|
||
|
||
You'll probably be a bit confused by the `|` part of the commands above.
|
||
Don't worry, give it a few hours paragraphs and it will become fully transparent!
|
||
|
||
Next, there is a very high probability that the commands above don't immediately work for you.
|
||
This is because `locate` functions with a database which needs to be updated from time to time.
|
||
This is handled automatically by a [cron](https://en.wikipedia.org/wiki/Cron) job.
|
||
We'll go into detail on how to schedule jobs ourselves but for now just remember you can trigger a refresh of the database yourself with the following command.
|
||
|
||
```
|
||
waldek@debian:~$ locate nano.html
|
||
waldek@debian:~$ sudo updatedb
|
||
waldek@debian:~$ locate nano.html
|
||
/usr/share/doc/nano/nano.html
|
||
waldek@debian:~$
|
||
```
|
||
|
||
### The hard way
|
||
|
||
A program you'll find on most Linux distributions to search for files is `find`.
|
||
It performs a real time search a crossed the entire file system and prints out the path of the file found.
|
||
A simple example can be seen below.
|
||
|
||
```
|
||
waldek@debian:~$ find /usr -iname "nano.html"
|
||
/usr/share/doc/nano/nano.html
|
||
waldek@debian:~$
|
||
```
|
||
|
||
The real power lies in it's various command line arguments.
|
||
We can for example search for all files which are bigger than 100MB, owned by a specific user or of a certain permission.
|
||
All of these conditions can be combined to improve the specificity of your query.
|
||
There are a few exercises below so I won't give too many spoiler.
|
||
My best advise is to read `man find` and experiment.
|
||
|
||
# Exercise
|
||
|
||
Inside a clean minimal installation can you please:
|
||
|
||
* find all files that are 3286 bytes
|
||
* find all files not owned by `root` **and** `$USER` (yourself)
|
||
* find all symbolic links
|
||
* find all files that contain *ssh* in the filename **and** are smaller that 10 bytes
|
||
|
||
For the eager ones:
|
||
|
||
* find all files in `/usr` that contain the word *hippo*
|
||
* find all files in `/usr` that contain the word *hippop* or *hiphop*
|
||
* find all Belgian email addresses in `/usr`
|
||
|
||
# Searching inside files
|
||
|
||
`locate` and `find` are used to search **for** files.
|
||
To search for content **inside** files we can use an other very powerful tool called `grep`.
|
||
It is often combined with [regular expressions](https://www.gnu.org/software/grep/manual/html_node/Regular-Expressions.html), or regex, for search for patterns.
|
||
The syntax of regex's is a bit challenging but, like vim, it is very much worth the effort.
|
||
Even more, you can use regex's *inside* vim to search and replace text!
|
||
This being said, you can use `grep` without regular expressions as well and this is what we'll do first.
|
||
|
||
Most distributions will come with some dictionaries installed.
|
||
We can find the American one with the `locate` command.
|
||
The *third* one in line is the one we need, but have a look at the other files if you're intrigued.
|
||
|
||
```
|
||
waldek@debian:~$ locate american
|
||
/usr/lib/ispell/american.aff
|
||
/usr/lib/ispell/american.hash
|
||
/usr/share/dict/american-english
|
||
/usr/share/doc/iamerican
|
||
/usr/share/doc/wamerican
|
||
/usr/share/doc/wamerican/NEWS.Debian.gz
|
||
/usr/share/doc/wamerican/README.Debian
|
||
/usr/share/doc/wamerican/changelog.Debian.gz
|
||
/usr/share/doc/wamerican/copyright
|
||
/usr/share/doc/wamerican/wamerican.scowl-word-lists-used
|
||
/usr/share/ispell/american.med+.mwl.gz
|
||
/usr/share/ispell/american.mwl.gz
|
||
/usr/share/man/man5/american-english.5.gz
|
||
/var/lib/dictionaries-common/ispell/iamerican
|
||
/var/lib/dictionaries-common/wordlist/wamerican
|
||
/var/lib/dpkg/info/iamerican.config
|
||
/var/lib/dpkg/info/iamerican.list
|
||
/var/lib/dpkg/info/iamerican.md5sums
|
||
/var/lib/dpkg/info/iamerican.postinst
|
||
/var/lib/dpkg/info/iamerican.postrm
|
||
/var/lib/dpkg/info/iamerican.preinst
|
||
/var/lib/dpkg/info/iamerican.templates
|
||
/var/lib/dpkg/info/wamerican.config
|
||
/var/lib/dpkg/info/wamerican.list
|
||
/var/lib/dpkg/info/wamerican.md5sums
|
||
/var/lib/dpkg/info/wamerican.postinst
|
||
/var/lib/dpkg/info/wamerican.postrm
|
||
/var/lib/dpkg/info/wamerican.templates
|
||
/var/lib/ispell/american.compat
|
||
/var/lib/ispell/american.hash
|
||
/var/lib/ispell/american.remove
|
||
```
|
||
|
||
We can use `grep` to find all occurrences of the word *hippop* with the following syntax.
|
||
|
||
```
|
||
waldek@debian:~$ grep "hippop" /usr/share/dict/american-english
|
||
hippopotami
|
||
hippopotamus
|
||
hippopotamus's
|
||
hippopotamuses
|
||
waldek@debian:~$
|
||
```
|
||
|
||
We can search for multiple patterns with the `|` character **but** this character has a special meaning in the shell so we need to escape it with a `\` **or** we can use *extended* `grep` with either `egrep` or `grep -E`.
|
||
The exact *meaning* of `|` will come clear in a bit, but in this context it means **or**.
|
||
|
||
```
|
||
waldek@debian:~$ grep "hippo\|tamu" /usr/share/dict/american-english
|
||
Metamucil
|
||
Metamucil's
|
||
hippo
|
||
hippo's
|
||
hippopotami
|
||
hippopotamus
|
||
hippopotamus's
|
||
hippopotamuses
|
||
hippos
|
||
whippoorwill
|
||
whippoorwill's
|
||
whippoorwills
|
||
waldek@debian:~$ grep -E "hippo|tamu" /usr/share/dict/american-english
|
||
Metamucil
|
||
Metamucil's
|
||
hippo
|
||
hippo's
|
||
hippopotami
|
||
hippopotamus
|
||
hippopotamus's
|
||
hippopotamuses
|
||
hippos
|
||
whippoorwill
|
||
whippoorwill's
|
||
whippoorwills
|
||
waldek@debian:~$
|
||
```
|
||
|
||
Adding the `--color` argument to `grep` will make the matched patterns jump out with a color, depending on the color scheme of your terminal.
|
||
|
||
## Wildcards and regular expressions
|
||
|
||
Both are quite related and for simplicities sake you can view regular expressions as wildcards on steroids.
|
||
A more detailed explaination can be found [here](https://unix.stackexchange.com/questions/57957/how-do-regular-expressions-differ-from-wildcards-used-to-filter-files).
|
||
|
||
### Wildcards
|
||
|
||
The syntax for wildcards is rater simple and also goes by the name of [globbing](https://en.wikipedia.org/wiki/Globbing) of [filename expansion](https://www.gnu.org/software/bash/manual/html_node/Filename-Expansion.html).
|
||
The most important rules to remember are the following.
|
||
|
||
| character | meaning |
|
||
| --- | --- |
|
||
| ? | one character, one times |
|
||
| * | any character, any number of times |
|
||
| [a-z] | matches all lowercase letters |
|
||
| [0-9] | matches all numbers |
|
||
| [ac9] | matches a *or* c *or* 9 |
|
||
|
||
# Exercise
|
||
|
||
You can download and extract [this](../assets/a_lot_of_empty_files_1.tar.gz) to try and match some patterns with wildcards.
|
||
For example.
|
||
|
||
```
|
||
waldek@helloworld:~$ ls *bo?ma*
|
||
aboymadozscmguidzrfh.txt bopmamzrcoqjmkjmjdow.txt obotmaorizmnrnycrjld.txt
|
||
waldek@helloworld:~$ ls cat*
|
||
catdmkskxbpgbijcsrbw.txt catlauqronedqtgyjrlj.txt catlocxdfvuqucyhrtti.txt catxvlgrwtvhgnohwqsc.txt
|
||
waldek@helloworld:~$ ls *boom*
|
||
mmnjgibpbvqkpboomgpu.txt phxbprfdyzboomqedgwu.txt yqrjsjdizlwjenlboomk.txt
|
||
waldek@helloworld:~$ ls *bo?ma*
|
||
aboymadozscmguidzrfh.txt bopmamzrcoqjmkjmjdow.txt obotmaorizmnrnycrjld.txt
|
||
waldek@helloworld:~$
|
||
```
|
||
|
||
I understand that the examples don't make any practical sense but it's a quick and easy way to learn some basic wildcards
|
||
You can download a second [file](../assets/a_lot_of_empty_files_2.tar.gz) that contains *a lot* of files with lowercase letters and numbers.
|
||
There you can try and match some patterns to make your list smaller and smaller.
|
||
For example.
|
||
|
||
```
|
||
waldek@helloworld:~$ ls [6-9][0-3][ab]?x*
|
||
63aaxc38fp7t5celiiqj 70b2xt9iu9my6amgdeyu 93ayxw6c4tf1du3y4pgu 93bsxleuz0t8gogga301
|
||
70abx7v1dzhun41fywdz 92bvx59br7goqqbo4h5o 93azx9zfm5mpf9ukm412
|
||
waldek@helloworld:~$
|
||
```
|
||
|
||
# Exercise
|
||
|
||
I hid a lot of words in [this](../assets/find_words_in_here.txt) file.
|
||
How many can you find?
|
||
What do they all have in common?
|
||
Can you count the occurences?
|
||
|
||
* [exercise from Linux long](https://gitea.86thumbs.net/waldek/linux_course_doc/src/branch/master/modules/qualifying/learning_regex.md)
|
||
|
||
### Regular expressions
|
||
|
||
TODO
|
||
|
||
# Pipes and redirects
|
||
|
||
TODO basic overview
|
||
|
||
## Redirecting
|
||
|
||
|
||
|
||
| name | stream ID | description |
|
||
| --- | --- | --- |
|
||
| STDIN | 0 | standard input, by default the keyboard |
|
||
| STDOUT | 1 | standard output, by default your terminal |
|
||
| STDERR | 2 | standard error, by default your terminal |
|
||
|
||
We can however **redirect** the input and output's!
|
||
This is done with the `>` character.
|
||
An example...
|
||
|
||
```
|
||
waldek@debian:~$ ls -l
|
||
total 0
|
||
waldek@debian:~$ date
|
||
Tue 10 May 2022 12:23:13 PM CEST
|
||
waldek@debian:~$ date > date_now.txt
|
||
waldek@debian:~$ ls -l
|
||
total 4
|
||
-rw-r--r-- 1 waldek waldek 33 May 10 12:23 date_now.txt
|
||
waldek@debian:~$ cat date_now.txt
|
||
Tue 10 May 2022 12:23:25 PM CEST
|
||
waldek@debian:~$
|
||
```
|
||
|
||
|
||
|
||
It's very important to realize that the file you redirect too will be overwritten!
|
||
So if you run the above `date > date_now.txt` again, the file will show a more recent date.
|
||
We can also [append](https://en.wikipedia.org/wiki/Append) to an existing file by ising the `>>`.
|
||
This redirection will add the date to the end of the file if the file exists, otherwise it will just create one.
|
||
|
||
```
|
||
waldek@debian:~$ date >> date_now.txt
|
||
waldek@debian:~$ date >> date_now.txt
|
||
waldek@debian:~$ date >> date_now.txt
|
||
waldek@debian:~$ cat date_now.txt
|
||
Tue 10 May 2022 12:23:25 PM CEST
|
||
Tue 10 May 2022 12:27:23 PM CEST
|
||
Tue 10 May 2022 12:27:24 PM CEST
|
||
Tue 10 May 2022 12:27:25 PM CEST
|
||
waldek@debian:~$
|
||
```
|
||
|
||
We can also choose which of the two outputs, STDOUT or STDERR, we want to redirect.
|
||
This is done by adding the **stream ID** to the redirection.
|
||
Observe the following output where I search for my username in each file located at `/etc`.
|
||
|
||
```
|
||
waldek@debian:~$ grep -R waldek /etc/
|
||
/etc/group-:cdrom:x:24:waldek
|
||
/etc/group-:floppy:x:25:waldek
|
||
/etc/group-:sudo:x:27:waldek
|
||
/etc/group-:audio:x:29:waldek
|
||
/etc/group-:dip:x:30:waldek
|
||
/etc/group-:video:x:44:waldek
|
||
/etc/group-:plugdev:x:46:waldek
|
||
/etc/group-:netdev:x:108:waldek
|
||
/etc/group-:waldek:x:1000:
|
||
grep: /etc/sv/ssh/log/supervise: No such file or directory
|
||
grep: /etc/sv/ssh/supervise: No such file or directory
|
||
/etc/passwd-:waldek:x:1000:1000:waldek,,,:/home/waldek:/bin/bash
|
||
grep: /etc/gshadow: Permission denied
|
||
/etc/subuid:waldek:100000:65536
|
||
grep: /etc/security/opasswd: Permission denied
|
||
grep: /etc/gshadow-: Permission denied
|
||
grep: /etc/sudoers.d/README: Permission denied
|
||
grep: /etc/.pwd.lock: Permission denied
|
||
grep: /etc/runit/runsvdir/default/ssh/log/supervise: No such file or directory
|
||
grep: /etc/runit/runsvdir/default/ssh/supervise: No such file or directory
|
||
grep: /etc/ssh/ssh_host_ecdsa_key: Permission denied
|
||
grep: /etc/ssh/ssh_host_ed25519_key: Permission denied
|
||
grep: /etc/ssh/ssh_host_rsa_key: Permission denied
|
||
grep: /etc/shadow-: Permission denied
|
||
grep: /etc/ssl/private: Permission denied
|
||
/etc/group:cdrom:x:24:waldek
|
||
/etc/group:floppy:x:25:waldek
|
||
/etc/group:sudo:x:27:waldek
|
||
/etc/group:audio:x:29:waldek
|
||
/etc/group:dip:x:30:waldek
|
||
/etc/group:video:x:44:waldek
|
||
/etc/group:plugdev:x:46:waldek
|
||
/etc/group:netdev:x:108:waldek
|
||
/etc/group:waldek:x:1000:
|
||
grep: /etc/shadow: Permission denied
|
||
grep: /etc/sudoers: Permission denied
|
||
/etc/subgid:waldek:100000:65536
|
||
/etc/passwd:waldek:x:1000:1000:waldek,,,:/home/waldek:/bin/bash
|
||
waldek@debian:~$
|
||
```
|
||
|
||
We can see quite a few `Permission denied` lines which are errors.
|
||
Because STDERR is connected to our terminal by default, we also see them printed on the screen.
|
||
Let's play around with the above command a bit.
|
||
First a simple redirection.
|
||
|
||
```
|
||
waldek@debian:~$ grep -R waldek /etc/ > files_with_my_name
|
||
grep: /etc/sv/ssh/log/supervise: No such file or directory
|
||
grep: /etc/sv/ssh/supervise: No such file or directory
|
||
grep: /etc/gshadow: Permission denied
|
||
grep: /etc/security/opasswd: Permission denied
|
||
grep: /etc/gshadow-: Permission denied
|
||
grep: /etc/sudoers.d/README: Permission denied
|
||
grep: /etc/.pwd.lock: Permission denied
|
||
grep: /etc/runit/runsvdir/default/ssh/log/supervise: No such file or directory
|
||
grep: /etc/runit/runsvdir/default/ssh/supervise: No such file or directory
|
||
grep: /etc/ssh/ssh_host_ecdsa_key: Permission denied
|
||
grep: /etc/ssh/ssh_host_ed25519_key: Permission denied
|
||
grep: /etc/ssh/ssh_host_rsa_key: Permission denied
|
||
grep: /etc/shadow-: Permission denied
|
||
grep: /etc/ssl/private: Permission denied
|
||
grep: /etc/shadow: Permission denied
|
||
grep: /etc/sudoers: Permission denied
|
||
```
|
||
|
||
We only see the errors now!
|
||
Where did our actual output go?
|
||
|
||
```
|
||
waldek@debian:~$ ls -l
|
||
total 4
|
||
-rw-r--r-- 1 waldek waldek 722 May 10 12:34 files_with_my_name
|
||
waldek@debian:~$ cat files_with_my_name
|
||
/etc/group-:cdrom:x:24:waldek
|
||
/etc/group-:floppy:x:25:waldek
|
||
/etc/group-:sudo:x:27:waldek
|
||
/etc/group-:audio:x:29:waldek
|
||
/etc/group-:dip:x:30:waldek
|
||
/etc/group-:video:x:44:waldek
|
||
/etc/group-:plugdev:x:46:waldek
|
||
/etc/group-:netdev:x:108:waldek
|
||
/etc/group-:waldek:x:1000:
|
||
/etc/passwd-:waldek:x:1000:1000:waldek,,,:/home/waldek:/bin/bash
|
||
/etc/subuid:waldek:100000:65536
|
||
/etc/group:cdrom:x:24:waldek
|
||
/etc/group:floppy:x:25:waldek
|
||
/etc/group:sudo:x:27:waldek
|
||
/etc/group:audio:x:29:waldek
|
||
/etc/group:dip:x:30:waldek
|
||
/etc/group:video:x:44:waldek
|
||
/etc/group:plugdev:x:46:waldek
|
||
/etc/group:netdev:x:108:waldek
|
||
/etc/group:waldek:x:1000:
|
||
/etc/subgid:waldek:100000:65536
|
||
/etc/passwd:waldek:x:1000:1000:waldek,,,:/home/waldek:/bin/bash
|
||
waldek@debian:~$
|
||
```
|
||
|
||
If we want to do the *reverse* we can redirect the STDERR by using `2>`.
|
||
An example...
|
||
|
||
```
|
||
waldek@debian:~$ grep -R waldek /etc/ 2> files_with_my_name
|
||
/etc/group-:cdrom:x:24:waldek
|
||
/etc/group-:floppy:x:25:waldek
|
||
/etc/group-:sudo:x:27:waldek
|
||
/etc/group-:audio:x:29:waldek
|
||
/etc/group-:dip:x:30:waldek
|
||
/etc/group-:video:x:44:waldek
|
||
/etc/group-:plugdev:x:46:waldek
|
||
/etc/group-:netdev:x:108:waldek
|
||
/etc/group-:waldek:x:1000:
|
||
/etc/passwd-:waldek:x:1000:1000:waldek,,,:/home/waldek:/bin/bash
|
||
/etc/subuid:waldek:100000:65536
|
||
/etc/group:cdrom:x:24:waldek
|
||
/etc/group:floppy:x:25:waldek
|
||
/etc/group:sudo:x:27:waldek
|
||
/etc/group:audio:x:29:waldek
|
||
/etc/group:dip:x:30:waldek
|
||
/etc/group:video:x:44:waldek
|
||
/etc/group:plugdev:x:46:waldek
|
||
/etc/group:netdev:x:108:waldek
|
||
/etc/group:waldek:x:1000:
|
||
/etc/subgid:waldek:100000:65536
|
||
/etc/passwd:waldek:x:1000:1000:waldek,,,:/home/waldek:/bin/bash
|
||
waldek@debian:~$
|
||
```
|
||
|
||
And the file content.
|
||
|
||
```
|
||
waldek@debian:~$ cat files_with_my_name
|
||
grep: /etc/sv/ssh/log/supervise: No such file or directory
|
||
grep: /etc/sv/ssh/supervise: No such file or directory
|
||
grep: /etc/gshadow: Permission denied
|
||
grep: /etc/security/opasswd: Permission denied
|
||
grep: /etc/gshadow-: Permission denied
|
||
grep: /etc/sudoers.d/README: Permission denied
|
||
grep: /etc/.pwd.lock: Permission denied
|
||
grep: /etc/runit/runsvdir/default/ssh/log/supervise: No such file or directory
|
||
grep: /etc/runit/runsvdir/default/ssh/supervise: No such file or directory
|
||
grep: /etc/ssh/ssh_host_ecdsa_key: Permission denied
|
||
grep: /etc/ssh/ssh_host_ed25519_key: Permission denied
|
||
grep: /etc/ssh/ssh_host_rsa_key: Permission denied
|
||
grep: /etc/shadow-: Permission denied
|
||
grep: /etc/ssl/private: Permission denied
|
||
grep: /etc/shadow: Permission denied
|
||
grep: /etc/sudoers: Permission denied
|
||
waldek@debian:~$
|
||
```
|
||
|
||
Now the errors are not *that* interesting too us so we can send this output to a *black hole* or *trash* location.
|
||
This is done by sending STDERR to `/dev/null`.
|
||
You'll see this redirection in a lot of places!
|
||
Try it out when you're searching for files with `find` and set the start of the search at the root of your system `/`.
|
||
|
||
```
|
||
waldek@debian:~$ grep -R waldek /etc/ 2> /dev/null
|
||
/etc/group-:cdrom:x:24:waldek
|
||
/etc/group-:floppy:x:25:waldek
|
||
/etc/group-:sudo:x:27:waldek
|
||
/etc/group-:audio:x:29:waldek
|
||
/etc/group-:dip:x:30:waldek
|
||
/etc/group-:video:x:44:waldek
|
||
/etc/group-:plugdev:x:46:waldek
|
||
/etc/group-:netdev:x:108:waldek
|
||
/etc/group-:waldek:x:1000:
|
||
/etc/passwd-:waldek:x:1000:1000:waldek,,,:/home/waldek:/bin/bash
|
||
/etc/subuid:waldek:100000:65536
|
||
/etc/group:cdrom:x:24:waldek
|
||
/etc/group:floppy:x:25:waldek
|
||
/etc/group:sudo:x:27:waldek
|
||
/etc/group:audio:x:29:waldek
|
||
/etc/group:dip:x:30:waldek
|
||
/etc/group:video:x:44:waldek
|
||
/etc/group:plugdev:x:46:waldek
|
||
/etc/group:netdev:x:108:waldek
|
||
/etc/group:waldek:x:1000:
|
||
/etc/subgid:waldek:100000:65536
|
||
/etc/passwd:waldek:x:1000:1000:waldek,,,:/home/waldek:/bin/bash
|
||
waldek@debian:~$ cat /dev/null
|
||
waldek@debian:~$
|
||
```
|
||
|
||
If you want to redirect **both** outputs to a file you can use the `&>` syntax.
|
||
The file will then contain **everything** that would be printed out on the terminal.
|
||
|
||
```
|
||
waldek@debian:~$ grep -R waldek /etc/ &> files_with_my_name
|
||
waldek@debian:~$
|
||
```
|
||
|
||
TODO - input redirection
|
||
|
||
## Piping
|
||
|
||
A *very* powerful fundamental concept of the command line is are **pipes** which are created with the `|` character.
|
||
Their purpose is to send the STDOUT of one process to the STDIN of a following.
|
||
They might seem similar to redirection but they are fundamentally different.
|
||
A redirect sends **to** or **from** a **file** while a pipe sends to and from a **process**.
|
||
|
||
|
||
|
||
As always, an example should clarify it better.
|
||
|
||
```
|
||
waldek@debian:~$ ls -l /etc/* | grep "d.conf"
|
||
-rw-r--r-- 1 root root 6169 Feb 27 2021 /etc/sudo_logsrvd.conf
|
||
-rw-r--r-- 1 root root 3289 Mar 13 2021 sshd_config
|
||
drwxr-xr-x 2 root root 4096 Mar 13 2021 sshd_config.d
|
||
-rw-r--r-- 1 root root 1052 Mar 20 20:55 journald.conf
|
||
-rw-r--r-- 1 root root 1145 Mar 20 20:55 logind.conf
|
||
-rw-r--r-- 1 root root 609 Feb 2 2021 networkd.conf
|
||
-rw-r--r-- 1 root root 943 Mar 20 20:55 resolved.conf
|
||
-rw-r--r-- 1 root root 677 Mar 20 20:55 timesyncd.conf
|
||
waldek@debian:~$
|
||
```
|
||
|
||
In the example above we send the output of `ls -l /etc/*` to `grep "d.conf"`.
|
||
We go from *a lot* of output to *less*.
|
||
Pipes are a very powerful tool to do text manipulations!
|
||
|
||
# Text manipulation
|
||
|
||
Below is a list of some of the essential tools to perform text manipulation.
|
||
I highly recommend reading the man pages for each of these programs.
|
||
Some of them have a myriad of options, others are more limited but they all have their purpose.
|
||
This might a good time to read up a bit on the fundamental [philosophy](https://en.wikipedia.org/wiki/Unix_philosophy) of Linux and Unix, especial the idea to create small programs that do one thing well.
|
||
|
||
|
||
| command | description |
|
||
| --- | --- |
|
||
| cut | remove sections from each line of files |
|
||
| sort | sort lines of text files |
|
||
| uniq | report or omit repeated lines |
|
||
| grep | print lines that match patterns |
|
||
| tr | translate or delete characters |
|
||
| sed | stream editor for filtering and transforming text |
|
||
| nl | number lines of files |
|
||
| wc | print newline, word, and byte counts for each file |
|
||
| seq | print a sequence of numbers |
|
||
| diff | compare files |
|
||
| cat | concatenate files and print on the standard output |
|
||
| tac | concatenate and print files in reverse |
|
||
| comm | compare two sorted files line by line |
|
||
| shuf | generate random permutations |
|
||
| paste | merge lines of file |
|
||
| join | join lines of two files on a common field |
|
||
| patch | apply a diff file to an original |
|
||
| aspell | interactive spell checker |
|
||
|
||
Below are some examples to illustrate the power of pipes and text manipulation programs.
|
||
You can copy/paste the command in your terminal and remove each pipe, one by one, to discover it's effect.
|
||
Most of these pipes are not actually *useful* but I hope they illustrate the flexibility of the concept.
|
||
|
||
1. sort by size
|
||
```
|
||
waldek@debian:~$ ls -la | sort --key 5 -g -r | grep -v -E "total|root" | nl
|
||
1 drwxr-xr-x 2 waldek waldek 4096 May 10 15:10 .
|
||
2 -rw-r--r-- 1 waldek waldek 3526 May 9 11:59 .bashrc
|
||
3 -rw-r--r-- 1 waldek waldek 807 May 9 11:59 .profile
|
||
4 -rw-r--r-- 1 waldek waldek 722 May 10 12:45 files_with_my_name
|
||
5 -rw-r--r-- 1 waldek waldek 220 May 9 11:59 .bash_logout
|
||
6 -rw-r--r-- 1 waldek waldek 74 May 9 16:06 .selected_editor
|
||
7 -rw------- 1 waldek waldek 49 May 10 15:10 .lesshst
|
||
waldek@debian:~$
|
||
```
|
||
1. number of users with `bash` as login shell
|
||
```
|
||
waldek@debian:~$ cat /etc/passwd | grep bash | cut -d ":" -f 1 | wc -l
|
||
2
|
||
waldek@debian:~$
|
||
```
|
||
1. files in `/etc` that contain my username
|
||
```
|
||
waldek@debian:~$ grep -R $USER /etc 2> /dev/null | cut -d ":" -f 1 | sort | uniq
|
||
/etc/group
|
||
/etc/group-
|
||
/etc/passwd
|
||
/etc/passwd-
|
||
/etc/subgid
|
||
/etc/subuid
|
||
waldek@debian:~$
|
||
```
|
||
1. files in `/etc` that contain the world *password* but listed by number of occurrences and only the three most populated ones are shown
|
||
```
|
||
waldek@debian:~$ grep -R password /etc 2> /dev/null | cut -d ":" -f 1 | sort | uniq -c | sort -g | tail -3
|
||
7 /etc/pam.d/common-password
|
||
9 /etc/debconf.conf
|
||
12 /etc/login.defs
|
||
waldek@debian:~$
|
||
```
|
||
|
||
# Ideas
|
||
|
||
* find unique phone numbers
|
||
* find valid mobile phone numbers [based on](https://en.wikipedia.org/wiki/Telephone_numbers_in_Belgium#Mobile_numbers)
|
||
* find valid IP addresses
|
||
* find all ugly filenames and replace with better names (replace with \_)
|
||
*
|