linux_course_doc/modules/resources/solution_postproduction.md

# Post production SFTP solution

## Mapping out the users

From the information given in the text I would create the following users and groups:

We could write a script that takes a CSV file as input but I decided to go the old school way and just write a full script.
First we add the groups we think we'll need.

```bash
#!/bin/bash

groupadd production
groupadd planning
groupadd script
groupadd technical
groupadd videoeditors
groupadd audioengineers
groupadd sftpjailed
```

Most of these are self explanatory, but the `production`, `technical` and `sftpjailed` require a bit more explantation.
They are *overarching* groups, meaning they group other groups.
From a Linux standpoint they are no different from the normal groups, but we'll use the to group together the technical department and all users that don't need ssh.

Next we'll add the users and we'll set their passwords to a *test* password.
In order for them to share a `$HOME` directory we'll need to create one as well.


```bash
#!/bin/bash

# adding the groups
groupadd production
groupadd planning
groupadd script
groupadd technical
groupadd videoeditors
groupadd audioengineers
groupadd sftpjailed

# creating the shared home directory
mkdir -p /home/postproduction

# adding the users
useradd marie
echo "marie:test" | chpasswd
usermod -d /home/postproduction marie

useradd hugo
echo "hugo:test" | chpasswd
usermod -d /home/postproduction hugo

useradd victor
echo "victor:test" | chpasswd
usermod -d /home/postproduction victor

useradd camille
echo "camille:test" | chpasswd
usermod -d /home/postproduction camille

useradd dave
echo "dave:test" | chpasswd
usermod -d /home/postproduction dave

useradd sarah
echo "sarah:test" | chpasswd
usermod -d /home/postproduction sarah

useradd ester
echo "ester:test" | chpasswd
usermod -d /home/postproduction ester

useradd adam
echo "adam:test" | chpasswd
usermod -d /home/postproduction adam

useradd eefje
echo "eefje:test" | chpasswd
usermod -d /home/postproduction eefje

useradd alex
echo "alex:test" | chpasswd
usermod -d /home/postproduction alex
```

The quick ones among you probably see there is quite a lot of room for error here when typing the same names over and over again.
We can optimise this with some easy variables.

```bash
#!/bin/bash

# the shared home for all users
sharedhome="/home/postproduction"
mkdir -p $sharedhome

# adding the groups
groupadd production
groupadd planning
groupadd script
groupadd technical
groupadd videoeditors
groupadd audioengineers
groupadd sftpjailed

# adding the users
username="marie"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username

username="victor"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username

username="camille"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username

username="dave"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username

username="sarah"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username

username="ester"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username

username="adam"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username

username="eefje"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username

username="alex"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
```

Those who already did programming before see this is an ideal situation to create our own [functions](https://ryanstutorials.net/bash-scripting-tutorial/bash-functions.php).
Try to optimise the script if you can with your own function!

Next we need to add all the users to their groups.

```bash
#!/bin/bash

# the shared home for all users
sharedhome="/home/postproduction"
mkdir -p $sharedhome

# adding the groups
groupadd production
groupadd planning
groupadd script
groupadd technical
groupadd videoeditors
groupadd audioengineers
groupadd sftpjailed

# adding the users
username="marie"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
usermod -a -G production $username
usermod -a -G planning $username
usermod -a -G script $username
usermod -a -G technical $username
usermod -a -G videoeditors $username
usermod -a -G audioengineers $username
usermod -a -G sftpjailed $username

username="hugo"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
usermod -a -G production $username
usermod -a -G planning $username
usermod -a -G script $username
usermod -a -G sftpjailed $username

username="victor"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
usermod -a -G production $username
usermod -a -G planning $username
usermod -a -G sftpjailed $username

username="camille"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
usermod -a -G production $username
usermod -a -G planning $username
usermod -a -G sftpjailed $username

username="dave"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
usermod -a -G videoeditors $username
usermod -a -G technical $username
usermod -a -G sftpjailed $username

username="sarah"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
usermod -a -G videoeditors $username
usermod -a -G technical $username
usermod -a -G sftpjailed $username

username="ester"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
usermod -a -G videoeditors $username
usermod -a -G technical $username
usermod -a -G sftpjailed $username

username="adam"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
usermod -a -G audioengineers $username
usermod -a -G technical $username
usermod -a -G sftpjailed $username

username="eefje"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
usermod -a -G audioengineers $username
usermod -a -G technical $username
usermod -a -G sftpjailed $username

username="alex"
useradd $username
echo "$username:test" | chpasswd
usermod -d $sharedhome $username
usermod -a -G production $username
usermod -a -G planning $username
usermod -a -G script $username
usermod -a -G technical $username
usermod -a -G videoeditors $username
usermod -a -G audioengineers $username
```

When testing this out you'll quickly discover the need for a second script, one that removes all the users and groups from your system.
I would advise something along these lines.

```bash
#!/bin/bash

sharedhome="/home/postproduction"

# removing the groups
groupdel production
groupdel planning
groupdel script
groupdel technical
groupdel videoeditors
groupdel audioengineers
groupdel sftpjailed

# removing the users
userdel marie
userdel hugo
userdel victor
userdel camille
userdel dave
userdel sarah
userdel ester
userdel adam
userdel eefje
userdel alex

# removing the groups created for the users
groupdel marie
groupdel hugo
groupdel victor
groupdel camille
groupdel dave
groupdel sarah
groupdel ester
groupdel adam
groupdel eefje
groupdel alex

# removing the home
rm -r $sharedhome
```

Both scripts together now give us a basic tool set to add and remove the users we need.
They work quite nicely but are not *verbose* at all!
To add debug messages with `echo` all around will become very prone to errors unless we start adding `for` loops and `functions`.
Let's start with a for loop in the remove script.

```bash 
#!/bin/bash

sharedhome="/home/postproduction"

groups="production planning script technical videoeditors audioengineers sftpjailed"
users="marie hugo victor camille dave sarah ester adam eefje alex"

# removing the groups
for group in $groups
do
	echo "removing group $group"
	groupdel $group
done

# removing the users and their primary groups
for user in $users
do
	echo "removing user $user"
	userdel $user
	echo "removing group $user"
	groupdel $user
done

# removing the home
rm -r $sharedhome
```

So much cleaner no?
This is also a good place to start controlling our `STDERR` output.
Can you think of a clean way to deal with the error messages?
Remember what `$?` does?
And you remember what `2> /dev/null` does?
A combination of these two concepts can tame the error message output quite nicely.
Have a test yourself!

The script to create the users will be a bit more complicated as there are a lot more variables.
A function would be ideal, but we'll try to deal with the problem without first!

```bash
#!/bin/bash

sharedhome="/home/postproduction"

groups="production planning script technical videoeditors audioengineers sftpjailed"
users="marie hugo victor camille dave sarah ester adam eefje alex"

marie="production planning script technical videoeditors audioengineers sftpjailed"
hugo="production planning script sftpjailed"
victor="production planning sftpjailed"
camille="production script sftpjailed"
dave="technical videoeditors sftpjailed"
sarah="technical videoeditors sftpjailed"
ester="technical videoeditors sftpjailed"
adam="technical audioengineers sftpjailed"
eefje="technical audioengineers sftpjailed"
alex="production planning script technical videoeditors audioengineers"

# adding the home
mkdir -p $sharedhome

# adding the groups
for group in $groups
do
	echo "adding group $group"
	groupadd $group
done

# adding the users
for user in $users
do
	echo "adding user $user"
	useradd $user
	echo "setting the password for $user"
	echo "$user:test" | chpasswd
done

# adding the users to their groups
for user in $users
do
	echo "adding groups for $user"
	usergroups=$(eval echo \$$user)
	for group in $usergroups 
	do
		echo "adding $user to $group"
		usermod -a -G $group $user
	done
done
```

Two things worth pointing out in the above script are:

1. the loop in a loop
2. the `eval` statement

The loop in a loop might look intimidating at first sight but it's not super exotic.
For each user is the `$users` list we will look at each group they need to belong to.
For each of those groups we add them with `usermod`.

The `eval` is a bit trickier to explain and I would consider it a bit of a *hack* to make it all work.
For each user in the `$users` list we also have a variable with their name.
This variable contains the groups they need to belong to.
The `eval` statement will *evaluate* the expression following and `\$` will interpretate the $ sign *literally*.
All of this serves to make a call to `$camille` and get `"production script sftpjailed"` in return so we can iterate over the groups.

There are *cleaner* ways of doing this and I would advise you to look into bash [arrays](https://www.gnu.org/software/bash/manual/html_node/Arrays.html).
If you decide to try this out, you *might* run into compatibility issues when testing in `zsh` but from inside a `bash` script you should be fine.

That's it, we now have two fully functional scripts to add and remove our users from the system.
Next up is the directory structure.

## Mapping out the files and folders

I'll be following my suggestion from the challenge outline.
First I think we should create a folder for the film project.
Then we'll go over each department and their files.
It might look a bit daunting but it's a lot of the same command.

* create a directory
* change the ownership
* add the files
* set the permissions

```bash
#!/bin/bash

sharedhome="/home/postproduction/"
movie="awesome_movie_project"

# we'll make a folder to house all the movie data
mkdir -p $sharedhome$movie

# we cd into that directory to make our life easier
cd $sharedhome$movie
echo "we'll create all files here: $(pwd)"

# planning
touch planning.ics
chown marie:planning planning.ics
chmod 664 planning.ics

# scenario
touch scenario.md
chown marie:script scenario.md
chmod 664 scenario.md

# audiofiles
mkdir audiofiles
chown marie:audioengineers audiofiles
chmod 2770 audiofiles
mkdir -p audiofiles/day{01..14}

touch audiofiles/day{01..14}/recording_{00..99}.wav
chown marie:audioengineers -R audiofiles
chmod 2770 audiofiles
chmod 2770 audiofiles/day{01..14}
chmod 0660 audiofiles/day{01..14}/*.wav

# videofiles
mkdir videofiles
chown marie:videoeditors videofiles
chmod 2770 videofiles
mkdir -p videofiles/day{01..14}

touch videofiles/day{01..14}/clip_{00..99}.mp4
chown marie:videoeditors -R videofiles
chmod 2770 videofiles
chmod 2770 videofiles/day{01..14}
chmod 0660 videofiles/day{01..14}/*.mp4

# renders
mkdir -p renders
touch renders/final_render.{wav,mp4}
chmod 664 renders/final_render.{wav,mp4}
chown marie:technical renders
chown marie:audioengineers renders/final_render.wav
chown marie:videoeditors renders/final_render.mp4
```

You can now log in with `filezilla` as any user of the film team and check if their permissions are correct.
Also try to log in with `ssh` and see if you can?
You should not be able to but we'll get to that in a minute.
I urge you to try it out and see think a bit about how you would improve it.
I can think of a few things.

* limit the group `sftpjailed` to only sftp and not ssh
* limit them to the project home directory so they can't walk around (and get lost) the entire file system
* when creating new files the entire owner and permission structure will become one gigantic mess!

We'll tackle all three problems one at a time.

### Limiting the group to sftp

This an `sshd` configuration setting so we'll need to edit the configuration file.
You should know *where* you can find it and if not *how* you can find out where is is located.
At the bottom of the file you'll see a *hint* towards adding rules from specific users and groups.
What would adding the following do you think?

```bash
Match Group sftpjailed
	ForceCommand internal-sftp
```

Remember how to *apply* these changes to the `sshd` server?
Go ahead and restart it now.
Can you still log in over sftp?
What about ssh?

### Limiting the users to walk around the server

We can force users or groups to see a different *root* as lowest point of the directory tree.
With *root* I mean `/` and **not** the user `root`!

```bash
Match Group sftpjailed
	ForceCommand internal-sftp
	ChrootDirectory /home/postproduction
```

Relaunch the server and fire up a new connection.
Can you still walk around all over the server?

### Limiting the total mess new files will make

This a more tricky problem and it can be tackled in multiple ways.
Selçuk and Hugo solved it by using [acl](https://www.redhat.com/sysadmin/linux-access-control-lists) and I'll let them do a presentation on how to do it but I'm going old school with [setuid and setgid](https://en.wikipedia.org/wiki/Setuid) and [umasks](https://en.wikipedia.org/wiki/Umask).
I'll let you be the judge on which way is the easiest/best.

#### The last and final permissions

Those who did not just *copy/pasted* but read the file creation script must have noticed I set the permissions with four numbers instead of three!
The first number allows us to set the setuid and setgid values.
On directories it forces the ownership of newly created files and directories inside this directory to either the `user` or the `group` who owns said directory.
Sounds confusing?
Test it out on some throwaway directory you create in `/tmp` and you'll quickly see how it behaves.
On files it forces the file or script to be run as the owner or group.
This can be handy but also quite dangerous, especially on scripts.
Luckily Debian does [not allow](https://unix.stackexchange.com/questions/364/allow-setuid-on-shell-scripts) it's usage on scripts.
A bit more information for you reading pleasure can be found [here](https://www.techrepublic.com/blog/it-security/understand-the-setuid-and-setgid-permissions-to-improve-security/).

#### Permissions for new files

I invite you to create a throwaway directory somewhere in your own home directory.
Go into it and create a file.
What are the *standard* permissions this new files has?
Now ask yourself, where does this rule come from?

Enter the `umask`.
No, literally, try and see what happens when you enter the command `umask`.
These numbers are a *mask* that gets subtracted from fully open permissions.
For files they get subtracted form `666` and for folders from `777`.
Why would they need to be different for files and folders?

Knowing this, the umask you have is `022` and this get's subtracted from `666`, it makes sense the permissions for your newly created file are `644` no?
Try to change your umask and see how it behaves.

So with all of this in mind, how can we change the default umask for a user?
The solution is twofold.
For shell sessions such as `bash` or `zsh` we would do this in the `.profile` or `.zprofile` files but as most of our movie team users don't have shell access this won't be enough.
I'll leave you with [this](https://googlethatforyou.com?q=linux%20sftp%20set%20umask%20) to figure it out!