waldek
/
linux_course_doc
1
0
Fork 0
Code Issues 2 Pull Requests Projects Releases Wiki Activity

fixes spelling mistakes

This commit is contained in:
waldek 2021-04-20 22:15:08 +02:00
parent dc9a70ab95
commit c35934b33b
1 changed files with 59 additions and 49 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

108
modules/resources/solution_raspberrypi.md
View File

@ -2,13 +2,14 @@
## Getting the file ## Getting the file
The first step to solving this exersise is to download the CSV file to your Raspberry. The first step to solving this exercise is to download the CSV file to your Raspberry.
For those wondering what on earth a CSV file is I invite you to a detailed [read](https://en.wikipedia.org/wiki/Comma-separated_values) but to make a long story short it stands for *comma separated values* and is one of the most basic ways to structure data. For those wondering what on earth a CSV file is I invite you to a detailed [read](https://en.wikipedia.org/wiki/Comma-separated_values) but to make a long story short it stands for *comma separated values* and is one of the most basic ways to structure data.
You can use Libreoffice calc to open it and you'll quickly understand how it works. You can use Libreoffice calc to open it and you'll quickly understand how it works.
Now, to get the file onto our Raspberry PI we need to download it from the webserver. Now, to get the file onto our Raspberry PI we need to download it from the webserver.
One way would be to use the `wget` program to do so. One way would be to use the `wget` program to do so.
We know *where* the server is by it's IP address, plus we also know the *filename*. Can you think of some alternative ways?
We know *where* the server is because we know it's IP address, plus we also know the *filename*.
Putting these two together we can construct the following line. Putting these two together we can construct the following line.
```bash ```bash
@ -37,10 +38,13 @@ JonathanDechief@hotmail.com,Dechief,Jonathan,@elewene:matrix.org,https://gitea.8
51207@etu.he2b.be,,Aliou,@aliou:86thumbs.net,https://gitea.86thumbs.net/aliou,blue 51207@etu.he2b.be,,Aliou,@aliou:86thumbs.net,https://gitea.86thumbs.net/aliou,blue
``` ```
The last line is a rather interesting one that illustrates how CSV files work.
Notice that *Aliou* does not have a LASTNAME so it's just two consecutive `,` to mark the empty field.
### The $USERNAME ### The $USERNAME
To extract the `$USERNAME` we're interested in the fourth collumn which has the MATRIX login handle. To extract the `$USERNAME` we're interested in we need the fourth column which has the MATRIX login handle.
We can extract only this collumn by using `cut` with `,` as a delimiter. We can extract only this column by using `cut` program with `,` as a delimiter.
```bash ```bash
cat accounts.csv | cut -d "," -f 4 cat accounts.csv | cut -d "," -f 4
@ -48,17 +52,18 @@ cat accounts.csv | cut -d "," -f 4
This leaves us with only the MATRIX login handles which is a good start but there is still a bit too much information . This leaves us with only the MATRIX login handles which is a good start but there is still a bit too much information .
We need to drop the first line, which is the *header* of the CSV file, plus crop between the `@` and the `:`. We need to drop the first line, which is the *header* of the CSV file, plus crop between the `@` and the `:`.
These two operations can be done is multiple ways but I suggest this modification. These two operations can be done in multiple ways but I suggest these additional pipes.
It is not the most elegant solution but it uses only tools you have used this far. It is not the most elegant solution but it uses only tools you have used so far.
The `tail` command drops the *header* line, and the two `cut` commands trim the username to just what we need. The `tail` command drops the *header* line, and the two `cut` commands trim the username to just what we need.
Can you think of a better way to do this?
Remember `tr` from your [bandit](https://www.overthewire.org) days?
```bash ```bash
cat accounts.csv | cut -d "," -f 4 | tail -n +2 | cut -d ":" -f 1 | cut -d "@" -f 2 cat accounts.csv | cut -d "," -f 4 | tail -n +2 | cut -d ":" -f 1 | cut -d "@" -f 2
``` ```
Done! Done!
We have all the usernames we need. Now we have all the usernames we need and we can save this to a file by redirecting the STDOUT to a file with the following command.
We can save this to a file by redirecting the STDOUT to a file with the following command.
```bash ```bash
cat accounts.csv | cut -d "," -f 4 | tail -n +2 | cut -d ":" -f 1 | cut -d "@" -f 2 > usernames.list cat accounts.csv | cut -d "," -f 4 | tail -n +2 | cut -d ":" -f 1 | cut -d "@" -f 2 > usernames.list
@ -67,7 +72,8 @@ cat accounts.csv | cut -d "," -f 4 | tail -n +2 | cut -d ":" -f 1 | cut -d "@" -
### The $PASSWORD ### The $PASSWORD
To extract the password we need to combine two field from the CSV file. To extract the password we need to combine two field from the CSV file.
A *really* good command line program to accieve this is `awk`. A *really* good command line program to achieve this is `awk`.
We haven't used it but [this](https://linuxhandbook.com/awk-command-tutorial/) is a good tutorial.
Don't forget the man pages! Don't forget the man pages!
```bash ```bash
@ -75,12 +81,13 @@ cat accounts.csv | awk -F "," '{print $2 $3}' | tail -n +2
``` ```
If you feel like making the password complexer, you can try to add in extra data into the `awk` command, or even append random numbers to the end. If you feel like making the password complexer, you can try to add in extra data into the `awk` command, or even append random numbers to the end.
How would you do this?
```bash ```bash
cat accounts.csv | awk -F "," '{print $2 "_helloworld_" $3}' | tail -n +2 cat accounts.csv | awk -F "," '{print $2 "_helloworld_" $3}' | tail -n +2
``` ```
We can now save these passwords to a file in the same way as before. We can now save these passwords to a file the same way we did before.
### The $GROUP ### The $GROUP
@ -92,11 +99,11 @@ cat accounts.csv | tail -n +2 | cut -d "," -f 6 > groups.list
## Using this information to create accounts ## Using this information to create accounts
We now have command that extracts the information we need, plus three separate files that contain all the information. We now have commands that extract the information we need, plus three separate files that contain all the information as lists.
This is a good moment to try and write a very simple script. This is a good moment to introduce you to writing a very simple script.
I'll first try to do it without a loop but you'll quickly see it's a *lot* easier with a loop in there. I'll do it without a loop but you'll quickly understand it's a *lot* easier and more functional with a loop in there.
Remember that `$1` represents the first argument on the command line so that when calling our script with `./script.sh 3` we'll get the username, password and group for the third user. Remember that `$1` represents the first argument on the command line so that when calling our script with `./script.sh 3` we'll get the username, password and group for the third user.
A combination of `head | tail` is a [classic](https://stackoverflow.com/questions/6022384/bash-tool-to-get-nth-line-from-a-file) way of selection only one file from a file. A combination of `head | tail` is a [classic](https://stackoverflow.com/questions/6022384/bash-tool-to-get-nth-line-from-a-file) way of selection only one specific line from a file.
Last but not least, don't forget to add execution permissions to this script with `chmod`. Last but not least, don't forget to add execution permissions to this script with `chmod`.
```bash ```bash
@ -109,14 +116,14 @@ head -$1 passwords.list | tail -1
head -$1 groups.list | tail -1 head -$1 groups.list | tail -1
``` ```
We can then *copy/paste* this information to create the following line. Calling our script `./script.sh 1` outputs the necessary information we can *copy/paste* to compliment the following command.
We'll be prompted to paste in the proper information. We'll be prompted to paste in the proper information.
```bash ```bash
sudo adduser $USERNAME sudo adduser $USERNAME
``` ```
Needless to say this is a labour intensive operation that we can automate with a mini script. Needless to say this is a labour intensive operation that we can automate adding some extra commands to our script.
### Putting it together as a script ### Putting it together as a script
@ -132,7 +139,7 @@ echo $NOW
With this in mind, the following code should make sense. With this in mind, the following code should make sense.
We're doing the exact same thing but saving the output of each command into a variable. We're doing the exact same thing but saving the output of each command into a variable.
At the last line we *use* the variables to create a message. At the last line we *use* the variables to create a message we display on our STDOUT.
```bash ```bash
#!/bin/bash #!/bin/bash
@ -147,11 +154,11 @@ echo "user: $USERNAME password: $PASSWORD group: $GROUP"
``` ```
This just output's all information onto one line, but why not *use* this information to actually create the accounts? This just output's all information onto one line, but why not *use* this information to actually create the accounts?
A counterpart to the `adduser` program you're used to using, there is also `useradd` which is better suited for scripting purposes. A counterpart to the `adduser` program you're used to using, there is `useradd` which is better suited for scripting purposes.
By default `useradd` is very *barebones* and does not create a home directory for the user but a quick look at the `man useradd` pages tells us we can use the `-m` flag to do so. By default `useradd` is very *barebones* and does not create a home directory for the user but a quick look at the `man useradd` pages tells us we can use the `-m` flag to do so.
This tells us the the command `useradd $USERNAME -m` will create a user for us with his/her own home directory. This tells us the command `useradd $USERNAME -m` will create a user for us with his/her own home directory.
A [google search](https://linux.die.net/man/8/chpasswd) pointed me to `chpasswd` to set passwords from within a script. A [google search](https://linux.die.net/man/8/chpasswd) pointed me to `chpasswd` to set passwords from within a script.
The line to set the password will be `echo $USERNAME:$PASSWORD | chpasswd`. The syntax, which I found on [stackoverflow](https://unix.stackexchange.com/questions/197448/change-password-programmatically) used to set the password will be `echo $USERNAME:$PASSWORD | chpasswd`.
This gives us the following script. This gives us the following script.
```bash ```bash
@ -171,9 +178,10 @@ echo $USERNAME:$PASSWORD | chpasswd
``` ```
You probably noticed I did not add the users to their *red/blue* groups. You probably noticed I did not add the users to their *red/blue* groups.
We can add them on the `useradd` line with the `-G` flag but it would fail if the group does not exist yet. We can add them to the `useradd` line by using the `-G` flag but it would fail if the group does not exist yet.
The `groupadd` command will add a group to the system and if we add the `-f` flag to it will do so without complaining if the group already exists. The `groupadd` command will add a group to the system and if we add the `-f` flag to it will do so without complaining if the group already exists.
This way we can just execute that line each time without worrying wether the group exists or not. This way we can just execute that line each time without worrying whether the group exists or not.
Nice!
```bash ```bash
#!/bin/bash #!/bin/bash
@ -194,9 +202,9 @@ echo "setting password: $PASSWORD for $USERNAME"
echo $USERNAME:$PASSWORD | chpasswd echo $USERNAME:$PASSWORD | chpasswd
``` ```
Those who switched to the newly created user noticed that the shell is *very* basic one. Those who switched to the newly created user to check whether they actually *work* probably noticed that the shell is *very* basic one.
You can find out which shell these new accounts use by looking at the `/etc/passwd` file. You can find out which shell these new accounts use by looking at the `/etc/passwd` file.
There are multiple ways to sort this problem but a look at the `man useradd` pages tells us we can use the `-s` flag to set the shell we want for the user. There are multiple ways to sort this problem but a look at the `man useradd` pages tells us we can use the `-s` flag to set the shell we want for the user we're creating.
We probably want to use `/bin/bash` for this option! We probably want to use `/bin/bash` for this option!
```bash ```bash
@ -221,7 +229,7 @@ echo "adding $USERNAME to $GROUP"
``` ```
This is getting pretty close to perfect! This is getting pretty close to perfect!
We can now run through all of the lines of the file one by one and automatically create the proper user, password and group combinations. We can now run through all of the lines of the file, one by one, and automatically create the proper user, password and group combinations.
To know how many accounts we have to create we can use `wc -l accounts.csv` and then just run the script, incrementing the number each time. To know how many accounts we have to create we can use `wc -l accounts.csv` and then just run the script, incrementing the number each time.
```bash ```bash
@ -233,10 +241,10 @@ echo "etc..."
### Taking it further as an extra challenge ### Taking it further as an extra challenge
If we want to automate the entire thing we'll need a loop to *loop through* every line in the `accounts.csv` file. If we want to automate the entire thing we'll need a loop to *loop through* every line of the `accounts.csv` file.
Bash loops are for a further class but I'll leave you with this quick example for those who feel like messing around. Bash loops are for a future class but I'll leave you with this quick example for those who feel like messing around.
Remember the *oneliners* we constructed at the beginning to extract the relevant information from the line! Remember the *oneliners* we constructed at the beginning to extract the relevant information from the line!
Don't worry if this looks to complicated at the moment, we'll do this exersise again when we're looking into [bash scripting](https://ryanstutorials.net/bash-scripting-tutorial/). Don't worry if this looks to complicated at the moment, we'll do this exercise again when we're looking into [bash scripting](https://ryanstutorials.net/bash-scripting-tutorial/).
```bash ```bash
#!/bin/bash #!/bin/bash
@ -252,13 +260,13 @@ done
## Setting up the fileserver ## Setting up the fileserver
The concept of a fileserver is about as old as the the world wide web, email, newsgroups etc. The concept of a fileserver is about as old as the world wide web, email, newsgroups etc.
It's main purpose in life is to offer a place to store and retrieve large files. It's main purpose in life is to offer a place to store and retrieve large files.
The Debian repositories where you download all your packages from is contactable over the FTP protocol. The Debian repositories where you download all your packages from is contactable over the FTP protocol.
There are multiple implementations available, some plain, some secure, some less secure. There are multiple implementations available, some plain, some secure, some less secure.
We'll have a look at the two most common solutions to host and transfer files. We'll have a look at the two most common solutions to host and transfer files.
To use the FTP protocol we can install any FTP tool we find. To use the FTP protocol as a client we can install any FTP client we can [find](https://www.slant.co/topics/12056/~ftp-clients-for-linux).
A good candidate is `filezilla` which you can find in the Debian repositories, or online if you want to use it on Windows. A good candidate is `filezilla` which you can find in the Debian repositories, or online if you want to use it on Windows.
### FTP ### FTP
@ -274,6 +282,7 @@ You know how to install it by now!
To know *where* to configure the server we can look at the `man vsftpd` pages. To know *where* to configure the server we can look at the `man vsftpd` pages.
Scroll all the way to the bottom to see which files it uses to configure itself. Scroll all the way to the bottom to see which files it uses to configure itself.
The configuration file itself is very *verbose* and should explain itself. The configuration file itself is very *verbose* and should explain itself.
Before changing the configuration do have a quick read of *Taking changes to configuration files into account*.
### SFTP ### SFTP
@ -281,26 +290,27 @@ Even without installing any additional server, we can offer our users a way to d
When you install `openssh-server`, and pay attention to the installation dialog, you'll see it will also install an additional package called `openssh-sftp-server` When you install `openssh-server`, and pay attention to the installation dialog, you'll see it will also install an additional package called `openssh-sftp-server`
This in itself is enough to have an SFTP server up and running. This in itself is enough to have an SFTP server up and running.
Done! Done!
Try it out on one of your virtual machines or the Raspberry PI.
It does raise the question of security. It does raise the question of security.
When installing the `vsftpd` package your users can *only* upload and download files but when you install an ssh server they can also get shell access! When installing the `vsftpd` package your users can *only* upload and download files but when you install an ssh server they can also get shell access!
This might not be the desired behaviour for all your users. This might not be the desired behaviour for all your users.
Luckily we can change the server configuration to modify it to our requirements. Luckily we can change the system, or server, configuration to modify it to our requirements.
How will we find *which* configuration file we need to change *where*?
`man sshd` should do the trick.
Navigate to to end of the manual and you'll see a huge list of files the server uses to configure itself.
Why `sshd` and not `ssh` you ask?
Well, `ssh` is the **client** you use to connect to the `sshd` **server** and thet are to different *programs*.
Remember that `ssh` was installed on your Linux machine even before installing `openssh-server`!
I can think of two ways to limit the shell access for specific users. I can think of two ways to limit the shell access for specific users.
The first one should ring a bell. The first one should ring a bell.
As we define our default shell in the `/etc/passwd` file to be `/bin/bash`, we can also change this for specific users to `/usr/sbin/nologin`.
As we define our default shell, found in the `/etc/passwd` file, to be `/bin/bash`, we can also change this for specific users to `/usr/sbin/nologin`.
This will block those users from getting an actual shell when sshing into the machine but SFTP should still work. This will block those users from getting an actual shell when sshing into the machine but SFTP should still work.
There are multiple downsides to this solution, some of which we'll investigate further later one, but the main one I see from a beginner point of view is that it requires a customisation for each account we want to block. There are multiple downsides to this solution, some of which we'll investigate further later one, but the main one I see from a beginner point of view is that it requires a customisation for each account we want to block.
As all users we want to block are part of a group, either *red* or *blue* there should be a way to restrict access on a group based level no? As all users we want to block are part of a group, either *red* or *blue* there should be a way to restrict access on a group based level no?
This will be done by modifying the ssh server.
How will we find *which* configuration files we need to change and *where* can be find those files?
`man sshd` should do the trick.
Navigate to to end of the manual and you'll see a huge list of files the server uses to configure itself.
Why `sshd` and not `ssh` you may ask?
Well, `ssh` is the **client** you use to connect to the `sshd` **server** and they are two different *programs*.
Remember that `ssh` was installed on your Linux machine even before installing `openssh-server`!
It's the last lines of the configuration file that point out some interesting features. It's the last lines of the configuration file that point out some interesting features.
```bash ```bash
@ -315,7 +325,7 @@ Subsystem sftp /usr/lib/openssh/sftp-server
# ForceCommand cvs server # ForceCommand cvs server
``` ```
Remember that the `#` are comments. Remember that the `#` are comments and not taken into account when the server configures itself.
These settings point out that we can add different rules for different users, and probably groups as well. These settings point out that we can add different rules for different users, and probably groups as well.
So, if you trust all people in the *blue* group, you can add a rule to match all users in the *red* group and restrict them to only sftp login and no shell access. So, if you trust all people in the *blue* group, you can add a rule to match all users in the *red* group and restrict them to only sftp login and no shell access.
With this in mind, what would adding the following to the configuration file do? With this in mind, what would adding the following to the configuration file do?
@ -326,22 +336,22 @@ Match Group blue
``` ```
But the users can still walk around the entire file system, which is not always a good idea. But the users can still walk around the entire file system, which is not always a good idea.
Using the keywords **sftp**, **chroot**, **group** in [google](https://googlethatforyou.com?q=sftp%20chroot%20group) lists us the [arch](https://wiki.archlinux.org/index.php/SFTP_chroot) wiki which is *always* a solid place to inform ourselves. Using the keywords **sftp**, **chroot**, **group** in [google](https://googlethatforyou.com?q=sftp%20chroot%20group) we find a link to the [arch](https://wiki.archlinux.org/index.php/SFTP_chroot) wiki which is *always* a solid place to inform ourselves.
Try to have get this set up as an additional exersise! You can have a go this as an additional exercise!
### Taking changes to configuration files into account ### Taking changes to configuration files into account
When make these changes to the configuration files you probably noticed they are not applied immidiatly. When make these changes to the configuration files you probably noticed they are not applied immediately.
So when are they taken into account then? So when are they taken into account then?
Most servers we'll install onto our various machines run as [daemons](https://en.wikipedia.org/wiki/Daemon_(computing)) in the background. Most servers we'll install onto our various machines run as [daemons](https://en.wikipedia.org/wiki/Daemon_(computing)) in the background.
They listen on for an incoming client connection on a port, do whatever the client requests of them and go back to sleep when there are no tasks for them to perform. They listen for an incoming client connection on a port, do whatever the client requests of them and go back to sleep when there are no tasks for them to perform.
They might have internal tasks scheduled but we'll set this aside for the moment. They might have internal tasks scheduled but we'll set this aside for the moment.
As with most things Linux, configuration is done by a text file and *most* daemons will read their corresponding configuration files upon startup. As with most things Linux, configuration is done by a text file and *most* daemons will read their corresponding configuration files upon startup.
Think of the .bashrc file, which is read every time a new shell is started. Think of the .bashrc file, which is read every time a new shell is started.
Once they know how to behave, from reading their configuration file, they will act accordingly. Once they know how to behave, from reading their configuration file, they will act accordingly.
This raises the question, how do you *restart* a running server? This raises the question, how do you *restart* a running server?
The system responsible for launching most daemons on our Debian installions, and also our Raspberry PI's, is called **systemd**. The system responsible for launching most daemons on our Debian installations, and also our Raspberry PI's, is called **systemd**.
The thee most used commands you need to know are as follows. The thee most used commands you need to know are as follows.
How do to *start* a service, how to *stop* one and how to *check* if one is running properly. How do to *start* a service, how to *stop* one and how to *check* if one is running properly.
For system daemons, such as `sshd` or most webservers, you need administrator powers to interact with them. For system daemons, such as `sshd` or most webservers, you need administrator powers to interact with them.
@ -354,10 +364,10 @@ sudo systemctl status sshd.service
These three commands will go a long way but there are other handy ones you can try out. These three commands will go a long way but there are other handy ones you can try out.
I advise you to use *tab-completion* as much as possible as it will help you to construct the proper commands. I advise you to use *tab-completion* as much as possible as it will help you to construct the proper commands.
Especially at the beginning of your Linux journey because it takes some time to rememeber specific service names. Especially at the beginning of your Linux journey because it takes some time to remember specific service names.
Have a look at all the base commands of `systemctl` and you'll notice quite a few interesting things. Have a look at all the base commands of `systemctl` and you'll notice quite a few interesting things.
For example, it can be used to reboot or shutdown your computer as well! For example, it can be used to reboot or shutdown your computer as well!
For those who want to digg deeper into systemd itself I advise you to have a browse in it's configuration directory. For those who want to dig deeper into systemd itself I advise you to have a browse in it's configuration directory.
You'll find it at `/etc/systemd` and the manpages `man systemd` are also very helpfull, but a bit verbose. You'll find it at `/etc/systemd` and the manpages `man systemd` are also very helpful, but a bit verbose.
We will have a deeper look at the internals of systemd later throughout the course. We will have a deeper look at the internals of systemd later throughout the course.