This is an old revision of the document!
- RAID
- Jonathan Haack
- Haack's Networking
- netcmnd@jonathanhaack.com
This tutorial is based on me wanting to set up a server solution for self-hosters that can survive a remote reboot, but also encrypt the data on the server that matters most. For that reason, I chose RAID1 array, and used luks and pam_mount and once that array and install were created - for short, I call this the RAID tutorial, since that was the primary non-negotiable. Here is what I did using an ncurses Debian stretch installer. I recommend not using a netinst.iso but the full DVD.iso instead and also, do NOT use a network mirror, and do not use any extra packages. Once the system is built, you can install the packages and rebuild the sources.list as needed. If you add all of those options during installation, older hardware in particular, will take up to a day to install since it will also be simultaneously syncing the array. At any rate, once you get that all ready, proceed:
Hardware Required
- Matching 4TB drives (some prefer identical models, others prefer identical size and speed but different models)
- Host Machine (4GB RAM minimum, 16GB ideal)
Using the ncruses installer, with manual partitioning, select each drive heading, continue by pressing enter when each of those are selected, and for each one it will prompt you to issue the drive a new GPT table. After that, I partition as follows:
/dev/sda1 - BIOS /dev/sda2 - 128GB (use as RAID) /dev/sda3 - 16GB (use as RAID) /dev/sda4 - 3.9TB (use as RAID)
The other drive was set up identically.
/dev/sdg1 - BIOS /dev/sdg2 - 128GB (use as RAID) /dev/sdg3 - 16GB (use as RAID) /dev/sdg4 - 3.9TB (use as RAID)
From there, I use the configure software RAID option and create new md devices for sda2/sdg2, sda3/sdg3, and sda4/sdg4. Do not create a mirror between the BIOS partitions - do not worry, if you attempted to, it will not allow you to utilize the md partition in that manner in the following screen, but it takes a lot of time to back up and reformat everything and build the array properly, so do not do this from the beginning. Once this is done, you will have
md0 - 128GB - format it as xfs, mount point "/" md1 - 16GB - format this as a swap partition md2 - 3.9TB - format it as xfs, use as "/home"
Once that is done (still in the ncurses/text-based installer), you can continue with the installation and the options as you desire. However, as I mentioned earlier, I do not add any additional software or utitlize the mirror during install in order to speed up the install process. Later, I will add all the packages I need and adjust sources.list, etc., but I do this … after the syncing completes. However … since I intend to encrypt the md2 partition within the RAID1 array, when I say that in this case, I mean one should first encrypt the md2 partition, and then after that reboot, and once all is working, then you wait and let everything sync before doing any package work. Remember, if you already have your system up and need to check its RAID snync status, issue this command:
cat /proc/mdstat
But again … ignore the status until you complete the step below. To get your partition encrypted with luks, you must still do the following to ensure that swap and /home are encrypted. Open a tty1 (ctrl-f1 or -fx) when the system boots in and log in as root at the tty1 shell. Once logged in, perform the following on md2 (the “/home” partition on the RAID mirror). I adapted the encryption instructions that follow from my friend Jason Schaefer's blog:
apt-get install cryptsetup libpam-mount rsync su - root rsync -av /home /backup umount /home/ cryptsetup luksFormat /dev/md2 cryptsetup luksOpen /dev/md2 home mkfs.xfs -L home /dev/mapper/home mount /dev/mapper/home /home/ rsync -av /backup/home/ /home nano /etc/fstab
Just add “#” before the lines for “/home” since that is now getting changed. Now, let's configure the pam_mount set up, which is what we are changing that to:
cp /etc/security/pam_mount.conf.xml /root/ nano /etc/security/pam_mount.conf.xml
The last command above opens a text editor nano with the pam_mount configuration file. Once this file is opened, locate the “<!– Volume definitions –>” section, and immediately underneat, enter a configuration similar to the one I use:
<volume user="user" fstype="crypt" path="/dev/disk/by-uuid/entertheuuidofmd2partition" mountpoint="/home" options="noatime,exec,fsck,nodev,nosuid"/>
If you forgot how to locate the UUID or ID, here's some different ways:
ls -lah /dev/disk/by-uuid/ ls -lah /dev/disk/by-id/ blkid
Anyway, once this is done, the crypt for home is set up. Now, let's encyrpt swap. Again, we will not encrypt the file system root because pam_mount will allow an easy remote reboot if we only encrypt the home directory. Additionally, the web server root will also be located in /home/server/ in this tutorial, but that set up and configuration is beyond this tutorial's scope. I am in the process of adding the topic of virtual hosts outside of the /var/www configuration to my tutorial, entitled “apachesurvival.” Anyways, for swap, do the following:
swapoff -a cryptdisks_start md1_crypt nano /etc/crypttab
Enter something like this in the crypttab file that just opened:
md1_crypt /dev/disk/by-id/md1byidcodejustdolslahondevdiskbyid /dev/urandom cipher=aes-xts-plain64,size=256,swap
Now, make sure to comment out the /etc/fstab entry for swap and replace with something like this:
/dev/mapper/md1_crypt none swap sw 0 0
Once that configuration is entered, do the following to bring the encrypted swap back up:
swapon -av swapon -sv
When you reboot, you may find that swap stopped syncing. If so, remember you can always check the syncing status of the RAID1 array or restart the syncing as follows:
cat /proc/mdstat [check] mdadm --readwrite /dev/mdx [restart]
If a drive fails, you need to deactivate it, activate the replacement and then allow enough time for syncing. That tutorial is next and will go below. It centers around mdadm commands to deactivate the old drive, and with sfdisk to create an identical partition table on the new device. More later on that …
The following are miscellaneous commands and sites that I am using while I add to this tutorial.
The sfdisk command is not complete … the proc shows kernel report on mirroring / syncing status …
sfdisk -d | sfdisk cat /proc/mdstat
Instructions for debian install:
https://blog.sleeplessbeastie.eu/2013/10/04/how-to-configure-software-raid1-during-installation-process/ https://www.howtoforge.com/replacing_hard_disks_in_a_raid1_array https://linuxgazette.net/140/pfeiffer.html http://jasonwryan.com/blog/2012/02/11/lvm/
— oemb1905 2018/11/23 21:20
