Differences

This shows you the differences between two versions of the page.

--- computing:proxmux [2023/12/23 22:09] – oemb1905
+++ computing:proxmux [2023/12/24 11:26] (current) – removed oemb1905
@@ Line 1: / Line 1: @@
--------------------------------------------
-  * **proxmux**
-  * **Jonathan Haack**
-  * **Haack's Networking**
-  * **webmaster@haacksnetworking.org**
--------------------------------------------
-//proxmux//
--------------------------------------------
-Been testing proxmux on home back up server. The server is my old production SuperMicro, which is now used for offsite backups in the home office. I have two 6TB drives in a zfs mirror for vm spinning (no spare), and the other 6x 6TBs are in 3 two-way mirrors for actual production backups. I am using the Debian underbelly with the 3 two-way mirrors to run my normal rsnapshot version control scripts, etc., while I use one pool for testing proxmux functionality. So far everything is working fine, and it's fun to use. Until I am offering advanced business / small enterprise support though, there's not really a need for the tools. But/and, that's not the purpose - the purpose is testing for now for a later date. The first thing I tested was how to bring in an existing virtual machine. To do that, create a machine in the GUI with no OS, and a trivial small HD size. Make sure the other resources match what you need for the VM. Then, run these commands on the command line within the proxmux host:
-  qm importdisk 500 hub.jonathanhaack.com.qcow2 <dataset>
-  qm set 500 --scsi0 vms:vm-500-disk-0
-These commands bring in the image block by block, and then re-assign the virtual disk that VM 500 uses to the image you just brought in, instead of the placeholder image you created in the GUI during the step prior. In order to export a VM at the command line, execute:
-  vzdump <vmID>
-  vzdump <vmID> --dumpdir /mnt/backups/machines/pve-images/<vmname>
-This will create a VM in .vma format. Then, you can extract the vma to a raw image as follows:
-  vma extract /var/lib/vz/dump/vzdump-qemu-101-2023_06_18-12_17_40.vma /mnt/backups/vma-extract
-Optionallly, you can use the python tool [[https://github.com/jancc/vma-extractor/blob/master/vma.py|vma.py]]. I don't see any value in using this, however, as it is much slower than the native tool and is also just in raw format. Example syntax is:
-  python3 vma.py vzdump-qemu-101-2023_06_18-12_17_40.vma /mnt/backups/vma
-In order to see where the virtual machine's zdevs are, run the following:
-  zfs list
-To delete one (be careful), use the following
-  zfs destroy pool/dataset
-I've migrated my Windows VM here for testing to avoid cumbersome EFI/TPM setup on virt-manager. Here's a page from ProxMux wiki on best practices for [[https://pve.proxmox.com/wiki/Windows_10_guest_best_practices|Windows VMs]]. Once thing I noticed when doing the Windows install is that PVE makes two more vdevs for the TPM / EFI, so you get something like the following in ''zfs list'':
-  vms/vm-102-disk-0
-  vms/vm-102-disk-1
-  vms/vm-102-disk-2
-Be careful not to delete those thinking wrongly that they are extraneous - those are obviously required for booting, but it might not be clear upon first look. Also, I chose to install those on the ''zpool'' and PVE's default was to store those locally - so they will likely not appear if you did not pick the option to store them on the pool.
-To verify sockets and cores
-  grep "physical id" /proc/cpuinfo | sort -u | wc -l
-Had to switch to postfix with my relay, so used satellite and:
-  [relay.haacksnetworking.org]:587
-Created bridge on pve instance with this recipe:
-  source /etc/network/interfaces.d/*
-  auto lo
-  iface lo inet loopback
-  iface enp0s31f6 inet manual
-  auto vmbr0
-  iface vmbr0 inet static
-      address 10.18.18.2/24
-      gateway 10.18.18.1
-      bridge-ports enp0s31f6
-      bridge-stp off
-      bridge-fd 0
-Make sure the PVE instance, which is also our reverse proxy has the FQDN set up in ''/etc/hosts'' properly. Additionally, ensure that your router is assigning the same matching local address (with dhcp) to the PVE as you assign within the PVE. In our situation, pve.domain.com is the proxmux host and has 80/443 passed to it by a port forward on the openWRT router. Only the openWRT router is public facing. The openWRT router is 10.13.13.1 and hands out addresses to clients on the LAN in that subnet. The PVE instance is both a reverse proxy with nginx for clients upstream on the LAN, and also a bridge for the VMs that reside on the PVE hypervisor (as per the bridge config above). Therefore, we want to ensure that both the openWRT router and the PVE instance itself have a static address.
-  sudo nano /etc/hosts
-  <10.13.13.2 pve.domain.com pve>
-  sudo nano /etc/hostname
-  <pve.domain.com>
-Set up configurations for each website on nginx as follows:
-  sudo apt install nginx
-  sudo nano /etc/nginx/sites-available/music.domain.com.conf
-Enter this in the file
-  server {
-    server_name  music.domain.com;
-    location / {
-        proxy_pass http://10.13.13.3;
-        #proxy_set_header Host music.domain.com;
-    }
-  }
-Repeat this as needed for each VM within PVE. For example, here's another entry for an instance doing nextcloud instead of music:
-  sudo nano /etc/nginx/sites-available/nextcloud.domain.com.conf
-Enter this in the file
-  server {
-    server_name  nextcloud.domain.com;
-    location / {
-        proxy_pass http://10.13.13.4;
-        #proxy_set_header Host nextcloud.domain.com;
-    }
-  }
-Once this is done, set up TLS on the PVE/RP instance as follows:
-  sudo apt install certbot letsencrypt python3-certbot-nginx
-  sudo certbot --authenticator standalone --installer nginx -d example.domain.com --pre-hook "systemctl stop nginx" --post-hook "systemctl start nginx"
-Remember, it the reverse proxy web server does not need to match the upstream web server on the LAN at the two different clients ''10.13.13.3'' and ''10.13.13.4'' because it is just a reverse proxy. It is simply fielding external url requests on port 80/443 for music.domain.com and nextcloud.domain.com and sending those requests upstream to the web servers on those clients which take over at that point. This works fine for browsing these sites when you are external, or not on the 10.13.13.0/24 LAN. But if you are within the LAN, you need to enable NAT loopback on your port forwards to 80/443 on the openWRT router. You do that by going to Firewall / Port Forward / Edit / Advanced Settings / Enable NAT Loopback. Additionally, you must add corresponding entries in hosts within openWRT for each FQDN and the corresponding local IP. To do that, you go to DHCP and DNS / Hostnames / Add. That takes care of being able to access these sites within or without the LAN 10.13.13.0/24, but there's an additional issue. From inside the LAN, Firefox has no issue accessing these sites, but Chrome will refuse access to them since the cert on the reverse proxy does not match the host serving the website content. This is related to rfc 6844 and can be handled by either making openWRT a certificate authority that has permission to do this or by installing matching certificates within the clients. I chose the latter option because I don't want to expose openWRT or tinker with it. Inside the shell in each VM, make sure to set the FQDN, add certbot, and add some header entries:
-  sudo nano /etc/hosts [inside VM1]
-  <127.0.0.1 music.domain.com music>
-  sudo nano /etc/apache2/sites-enabled/music.domain.com.conf
-  <ServerName music.domain.com>
-  <Header add myheader "music.domain.com">
-  <RequestHeader set myheader "music.domain.com">
-That takes care of VM1, now let's do VM2:
-  sudo nano /etc/hosts [inside VM2]
-  <127.0.0.1 nextcloud.domain.com nextcloud>
-  sudo nano /etc/apache2/sites-enabled/nextcloud.domain.com.conf
-  <ServerName nextcloud.domain.com>
-  <Header add myheader "nextcloud.domain.com">
-  <RequestHeader set myheader "nextcloud.domain.com">
-Once this id done, restart your web server. In my case, the VMs are using apache and only the PVE / reverse proxy server is using nginx. After restarting the web servers inside each VM, you now can create the matching certs inside each VM. Here's the commands for VM1:
-  sudo apt install certbot letsencrypt python3-certbot-apache
-  sudo certbot --authenticator standalone --installer apache -d music.example.com --pre-hook "systemctl stop apache2" --post-hook "systemctl start apache2"
-Here are the commands inside VM2:
-  sudo apt install certbot letsencrypt python3-certbot-apache
-  sudo certbot --authenticator standalone --installer apache -d nextcloud.example.com --pre-hook "systemctl stop apache2" --post-hook "systemctl start apache2"
-Once that's done, make sure that you have cron jobs set for both VMs and for the reverse proxy server / PVE instance. Just enter the following in crontab for all three:
-2 * * 1 /usr/bin/certbot renew >> /var/log/le-renew.log
-This should now make Chrome happy since the VM cert will match the cert of the reverse proxy. Why does this work you might ask? Well, when you run certbot inside the VM, it merely cares whether it can be reached externally where you've declared it to be and since it can, it creates the cert without issue. The reverse proxy / PVE instance itself is also able to handle requests for these domains so certbot likewise has no problems issuing the cert there either. This not only makes the Chrome happy, but it addresses what the Chrome developers and rfc 6844 is concerned about, namely, that without this ... connections inside the LAN could potentially be non-TLS or non-header matching. So, this is a best of both worlds.
- --- //[[jonathan@haacksnetworking.org|oemb1905]] 2023/06/19 17:40//

Haack's Wiki

User Tools

Site Tools

Differences

Page Tools