From fe46f6a2a2fb258f26780ef79227cb707119b851 Mon Sep 17 00:00:00 2001
From: waldek <waldek@mailbox.org>
Date: Tue, 20 Apr 2021 07:25:00 +0200
Subject: [PATCH] adds solution for ftp excersise

---
 modules/resources/solution_raspberrypi.md | 271 ++++++++++++++++++++++
 1 file changed, 271 insertions(+)
 create mode 100644 modules/resources/solution_raspberrypi.md

diff --git a/modules/resources/solution_raspberrypi.md b/modules/resources/solution_raspberrypi.md
new file mode 100644
index 0000000..f239889
--- /dev/null
+++ b/modules/resources/solution_raspberrypi.md
@@ -0,0 +1,271 @@
+# The FTP challenge
+
+## Getting the file
+
+The first step to solving this exersise is to download the CSV file to your Raspberry.
+For those wondering what on earth a CSV file is I invite you to a detailed [read](https://en.wikipedia.org/wiki/Comma-separated_values) but to make a long story short it stands for *comma separated values* and is one of the most basic ways to structure data.
+You can use Libreoffice calc to open it and you'll quickly understand how it works.
+
+Now, to get the file onto our Raspberry PI we need to download it from the webserver.
+One way would be to use the `wget` program to do so.
+We know *where* the server is by it's IP address, plus we also know the *filename*.
+Putting these two together we can construct the following line.
+
+```bash
+wget 172.30.6.96/accounts.csv
+```
+
+This will download the file to the directory we're in and will save it as `accounts.csv`.
+You can change the output filename it you want, just have a look at the `wget` options via `wget --help` or our trusty `man wget`.
+
+## Extracting the data we need
+
+A quick `cat accounts.csv` gives us the following output:
+
+```bash
+EMAIL,LASTNAME,FIRSTNAME,MATRIX,GITEA,TEAM
+1h.lust.hugo@gmail.com,Lust,Hugo,@hugo_lust:86thumbs.net,https://gitea.86thumbs.net/Hugo,red
+ticus@kraland.net,krstev,vladimir,@vl4dd:86thumbs.net,https://gitea.86thumbs.net/vl4dd,blue
+adamd@outlook.be,Adam,David,@adamd73:matrix.org,https://gitea.86thumbs.net/adamd,red
+nicohawai@gmail.com,Perez,Nicolas,@hawai:86thumbs.net,https://gitea.86thumbs.net/Hawai,blue
+nicolas.wattripont@gmail.com,Wattripont,Nicolas,@wawa142:86thumbs.net,https://gitea.86thumbs.net/wawa142,red
+laurentdelvigne@hotmail.com,Delvigne,Laurent,@ldelvigne:86thumbs.net,https://gitea.86thumbs.net/ldelvigne,blue
+sselcukaslan@gmail.com,Aslan,Selçuk,@slck:86thumbs.net,https://gitea.86thumbs.net/selcuk,blue
+Sarah24886@hotmail.com,Rmiki,Sarah,@sarahrm95:matrix.org,https://gitea.86thumbs.net/sarahrm95,blue
+knoppixs@hotmail.com,Abbamoulay,Abdellah,@knoppixs:86thumbs.net,https://gitea.86thumbs.net/Abdellah,red
+JonathanDechief@hotmail.com,Dechief,Jonathan,@elewene:matrix.org,https://gitea.86thumbs.net/Elewene,red
+51207@etu.he2b.be,,Aliou,@aliou:86thumbs.net,https://gitea.86thumbs.net/aliou,blue
+```
+
+### The $USERNAME
+
+To extract the `$USERNAME` we're interested in the fourth collumn which has the MATRIX login handle.
+We can extract only this collumn by using `cut` with `,` as a delimiter.
+
+```bash
+cat accounts.csv | cut -d "," -f 4
+```
+
+This leaves us with only the MATRIX login handles which is a good start but there is still a bit too much information .
+We need to drop the first line, which is the *header* of the CSV file, plus crop between the `@` and the `:`.
+These two operations can be done is multiple ways but I suggest this modification.
+It is not the most elegant solution but it uses only tools you have used this far.
+The `tail` command drops the *header* line, and the two `cut` commands trim the username to just what we need.
+
+```bash
+cat accounts.csv | cut -d "," -f 4 | tail -n +2 | cut -d ":" -f 1 | cut -d "@" -f 2
+```
+
+Done!
+We have all the usernames we need.
+We can save this to a file by redirecting the STDOUT to a file with the following command.
+
+```bash
+cat accounts.csv | cut -d "," -f 4 | tail -n +2 | cut -d ":" -f 1 | cut -d "@" -f 2 > usernames.list
+```
+
+### The $PASSWORD
+
+To extract the password we need to combine two field from the CSV file.
+A *really* good command line program to accieve this is `awk`.
+Don't forget the man pages!
+
+```bash
+cat accounts.csv | awk -F "," '{print $2 $3}' | tail -n +2
+```
+
+If you feel like making the password complexer, you can try to add in extra data into the `awk` command, or even append random numbers to the end.
+
+```bash
+cat accounts.csv | awk -F "," '{print $2 "_helloworld_" $3}' | tail -n +2
+```
+
+We can now save these passwords to a file in the same way as before.
+
+### The $GROUP
+
+This is an *easy* one because it requires no real modification of the field.
+
+```bash
+cat accounts.csv | tail -n +2 | cut -d "," -f 6 > groups.list
+```
+
+## Using this information to create accounts
+
+We now have command that extracts the information we need, plus three separate files that contain all the information.
+This is a good moment to try and write a very simple script.
+I'll first try to do it without a loop but you'll quickly see it's a *lot* easier with a loop in there.
+Remember that `$1` represents the first argument on the command line so that when calling our script with `./script.sh 3` we'll get the username, password and group for the third user.
+A combination of `head | tail` is a [classic](https://stackoverflow.com/questions/6022384/bash-tool-to-get-nth-line-from-a-file) way of selection only one file from a file.
+Last but not least, don't forget to add execution permissions to this script with `chmod`.
+
+```bash
+#!/bin/bash
+
+LINE=$1
+
+head -$1 usernames.list | tail -1
+head -$1 passwords.list | tail -1
+head -$1 groups.list | tail -1
+```
+
+We can then *copy/paste* this information to create the following line.
+We'll be prompted to paste in the proper information.
+
+```bash
+sudo adduser $USERNAME
+```
+
+Needless to say this is a labour intensive operation that we can automate with a mini script.
+
+### Putting it together as a script
+
+Brace yourselves a bit but I promise it's worth it!
+The only thing we have not seen is how to save the output of a command into a variable.
+This can be done with the `$(...)` syntax.
+I know it looks a bit cryptic but an example speaks more than words.
+
+```bash
+NOW=$(date)
+echo $NOW
+```
+
+With this in mind, the following code should make sense.
+We're doing the exact same thing but saving the output of each command into a variable.
+At the last line we *use* the variables to create a message.
+
+```bash
+#!/bin/bash
+
+LINE=$1
+
+USERNAME=$(head -$1 usernames.list | tail -1)
+PASSWORD=$(head -$1 passwords.list | tail -1)
+GROUP=$(head -$1 groups.list | tail -1)
+
+echo "user: $USERNAME password: $PASSWORD group: $GROUP"
+```
+
+This just output's all information onto one line, but why not *use* this information to actually create the accounts?
+A counterpart to the `adduser` program you're used to using, there is also `useradd` which is better suited for scripting purposes.
+By default `useradd` is very *barebones* and does not create a home directory for the user but a quick look at the `man useradd` pages tells us we can use the `-m` flag to do so. 
+This tells us the the command `useradd $USERNAME -m` will create a user for us with his/her own home directory.
+A [google search](https://linux.die.net/man/8/chpasswd) pointed me to `chpasswd` to set passwords from within a script.
+The line to set the password will be `echo $USERNAME:$PASSWORD | chpasswd`.
+This gives us the following script.
+
+```bash
+#!/bin/bash
+
+LINE=$1
+
+USERNAME=$(head -$1 usernames.list | tail -1)
+PASSWORD=$(head -$1 passwords.list | tail -1)
+GROUP=$(head -$1 groups.list | tail -1)
+
+echo "adding user: $USERNAME"
+useradd $USERNAME -m
+
+echo "setting password: $PASSWORD for $USERNAME"
+echo $USERNAME:$PASSWORD | chpasswd
+```
+
+You probably noticed I did not add the users to their *red/blue* groups.
+We can add them on the `useradd` line with the `-G` flag but it would fail if the group does not exist yet.
+The `groupadd` command will add a group to the system and if we add the `-f` flag to it will do so without complaining if the group already exists.
+This way we can just execute that line each time without worrying wether the group exists or not.
+
+```bash 
+#!/bin/bash
+
+LINE=$1
+
+USERNAME=$(head -$1 usernames.list | tail -1)
+PASSWORD=$(head -$1 passwords.list | tail -1)
+GROUP=$(head -$1 groups.list | tail -1)
+
+echo "making sure $GROUP exists..."
+groupadd -f $GROUP
+
+echo "adding user: $USERNAME"
+useradd $USERNAME -m -G $GROUP
+
+echo "setting password: $PASSWORD for $USERNAME"
+echo $USERNAME:$PASSWORD | chpasswd
+```
+
+Those who switched to the newly created user noticed that the shell is *very* basic one.
+You can find out which shell these new accounts use by looking at the `/etc/passwd` file.
+There are multiple ways to sort this problem but a look at the `man useradd` pages tells us we can use the `-s` flag to set the shell we want for the user.
+We probably want to use `/bin/bash` for this option!
+
+```bash 
+#!/bin/bash
+
+LINE=$1
+
+USERNAME=$(head -$1 usernames.list | tail -1)
+PASSWORD=$(head -$1 passwords.list | tail -1)
+GROUP=$(head -$1 groups.list | tail -1)
+
+echo "making sure $GROUP exists..."
+groupadd -f $GROUP
+
+echo "adding user: $USERNAME"
+useradd $USERNAME -m -G $GROUP -s "/bin/bash"
+
+echo "setting password: $PASSWORD for $USERNAME"
+echo $USERNAME:$PASSWORD | chpasswd
+
+echo "adding $USERNAME to $GROUP"
+```
+
+This is getting pretty close to perfect!
+We can now run through all of the lines of the file one by one and automatically create the proper user, password and group combinations.
+To know how many accounts we have to create we can use `wc -l accounts.csv` and then just run the script, incrementing the number each time.
+
+```bash
+sudo ./script.sh 1
+sudo ./script.sh 2
+sudo ./script.sh 3
+echo "etc..."
+```
+
+### Taking it further as an extra challenge
+
+If we want to automate the entire thing we'll need a loop to *loop through* every line in the `accounts.csv` file.
+Bash loops are for a further class but I'll leave you with this quick example for those who feel like messing around.
+Remember the *oneliners* we constructed at the beginning to extract the relevant information from the line!
+Don't worry if this looks to complicated at the moment, we'll do this exersise again when we're looking into [bash scripting](https://ryanstutorials.net/bash-scripting-tutorial/).
+
+```bash
+#!/bin/bash
+
+FILE=$1
+LINES=$(cat $1)
+
+for LINE in $LINES;
+do
+	echo $LINE
+done
+```
+
+## Setting up the fileserver
+
+### FTP
+
+A quick [google](https://likegeeks.com/ftp-server-linux/) tells us `vsftpd` is a popular *FileTransferProtocol* server.
+As expected you can find it in the main Debian repositories.
+You know how to install it by now!
+
+```bash 
+sudo apt search vsftpd
+```
+
+To know *where* to configure the server we can look at the `man vsftpd` pages.
+Scroll all the way to the bottom to see which files it uses to configure itself.
+The configuration file itself is very *verbose* and should explain itself.
+
+### SFTP
+
+TODO