diff --git a/essential/introduction_to_the_commandline.md b/essential/introduction_to_the_commandline.md index 53ff2f2..388c48d 100644 --- a/essential/introduction_to_the_commandline.md +++ b/essential/introduction_to_the_commandline.md @@ -1235,6 +1235,9 @@ Adding the `--color` argument to `grep` will make the matched patterns jump out Both are quite related and for simplicities sake you can view regular expressions as wildcards on steroids. A more detailed explaination can be found [here](https://unix.stackexchange.com/questions/57957/how-do-regular-expressions-differ-from-wildcards-used-to-filter-files). + +### Wildcards + The syntax for wildcards is rater simple and also goes by the name of [globbing](https://en.wikipedia.org/wiki/Globbing) of [filename expansion](https://www.gnu.org/software/bash/manual/html_node/Filename-Expansion.html). The most important rules to remember are the following. @@ -1284,6 +1287,9 @@ Can you count the occurences? * [exercise from Linux long](https://gitea.86thumbs.net/waldek/linux_course_doc/src/branch/master/modules/qualifying/learning_regex.md) +### Regular expressions + +TODO # Pipes and redirects @@ -1567,6 +1573,12 @@ Pipes are a very powerful tool to do text manipulations! # Text manipulation +Below is a list of some of the essential tools to perform text manipulation. +I highly recommend reading the man pages for each of these programs. +Some of them have a myriad of options, others are more limited but they all have their purpose. +This might a good time to read up a bit on the fundamental [philosophy](https://en.wikipedia.org/wiki/Unix_philosophy) of Linux and Unix, especial the idea to create small programs that do one thing well. + + | command | description | | --- | --- | | cut | remove sections from each line of files | @@ -1583,5 +1595,57 @@ Pipes are a very powerful tool to do text manipulations! | tac | concatenate and print files in reverse | | comm | compare two sorted files line by line | | shuf | generate random permutations | +| paste | merge lines of file | +| join | join lines of two files on a common field | +| patch | apply a diff file to an original | +| aspell | interactive spell checker | +Below are some examples to illustrate the power of pipes and text manipulation programs. +You can copy/paste the command in your terminal and remove each pipe, one by one, to discover it's effect. +Most of these pipes are not actually *useful* but I hope they illustrate the flexibility of the concept. +1. sort by size + ``` + waldek@debian:~$ ls -la | sort --key 5 -g -r | grep -v -E "total|root" | nl + 1 drwxr-xr-x 2 waldek waldek 4096 May 10 15:10 . + 2 -rw-r--r-- 1 waldek waldek 3526 May 9 11:59 .bashrc + 3 -rw-r--r-- 1 waldek waldek 807 May 9 11:59 .profile + 4 -rw-r--r-- 1 waldek waldek 722 May 10 12:45 files_with_my_name + 5 -rw-r--r-- 1 waldek waldek 220 May 9 11:59 .bash_logout + 6 -rw-r--r-- 1 waldek waldek 74 May 9 16:06 .selected_editor + 7 -rw------- 1 waldek waldek 49 May 10 15:10 .lesshst + waldek@debian:~$ + ``` +1. number of users with `bash` as login shell + ``` + waldek@debian:~$ cat /etc/passwd | grep bash | cut -d ":" -f 1 | wc -l + 2 + waldek@debian:~$ + ``` +1. files in `/etc` that contain my username + ``` + waldek@debian:~$ grep -R $USER /etc 2> /dev/null | cut -d ":" -f 1 | sort | uniq + /etc/group + /etc/group- + /etc/passwd + /etc/passwd- + /etc/subgid + /etc/subuid + waldek@debian:~$ + ``` +1. files in `/etc` that contain the world *password* but listed by number of occurrences and only the three most populated ones are shown + ``` + waldek@debian:~$ grep -R password /etc 2> /dev/null | cut -d ":" -f 1 | sort | uniq -c | sort -g | tail -3 + 7 /etc/pam.d/common-password + 9 /etc/debconf.conf + 12 /etc/login.defs + waldek@debian:~$ + ``` + +# Ideas + +* find unique phone numbers +* find valid mobile phone numbers [based on](https://en.wikipedia.org/wiki/Telephone_numbers_in_Belgium#Mobile_numbers) +* find valid IP addresses +* find all ugly filenames and replace with better names (replace with \_) +*