User Tools

Site Tools


computing:proxmux

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
Last revisionBoth sides next revision
computing:proxmux [2023/12/23 21:45] oemb1905computing:proxmux [2023/12/24 11:18] oemb1905
Line 11: Line 11:
 ------------------------------------------- -------------------------------------------
  
-Been testing proxmux on home back up server. The server is my old production SuperMicro, which is now used for offsite backups in the home office. I have two 6TB drives in a zfs mirror for vm spinning (no spare), and the other 6x 6TBs are in 3 two-way mirrors for actual production backups. I am using the Debian underbelly with the 3 two-way mirrors to run my normal rsnapshot version control scripts, etc., while I use one pool for testing proxmux functionality. So far everything is working fine, and it's fun to useUntil I am offering advanced business / small enterprise support though, there'not really a need for the tools. But/and, that's not the purpose - the purpose is testing for now for a later date. The first thing I tested was how to bring in an existing virtual machineTo do that, create a machine in the GUI with no OSand a trivial small HD size. Make sure the other resources match what you need for the VMThen, run these commands on the command line within the proxmux host:+Most important thing about proxmox is knowing how to get the VMs in and out of it. Let'start with getting them in. First, create an empty VM in the guithen use the following command changing 500 to match your vmid.
  
   qm importdisk 500 hub.jonathanhaack.com.qcow2 <dataset>   qm importdisk 500 hub.jonathanhaack.com.qcow2 <dataset>
-  qm set 500 --scsi0 vms:vm-500-disk-0+  qm set 500 --scsi0 vms:vm-500-disk-0 [sets boot order to make this primary]
      
-These commands bring in the image block by blockand then re-assign the virtual disk that VM 500 uses to the image you just brought ininstead of the placeholder image you created in the GUI during the step prior. In order to export a VM at the command line, execute:+Alternatelyif you just want to attach a raw image fileor VDD, to an existing proxmox VM, just do the following: 
 + 
 +  qm importdisk <vmid> obstorage.img <dataset> 
 +   
 +After that, you can add storage in hardware within the VM settings to attach this image. In order to export a VM at the command line, execute:
  
   vzdump <vmID>   vzdump <vmID>
Line 61: Line 65:
   auto vmbr0   auto vmbr0
   iface vmbr0 inet static   iface vmbr0 inet static
-      address 10.18.18.2/24 +      address 10.13.13.2/24 
-      gateway 10.18.18.1+      gateway 10.13.13.1
       bridge-ports enp0s31f6       bridge-ports enp0s31f6
       bridge-stp off       bridge-stp off
Line 77: Line 81:
  
   sudo apt install nginx   sudo apt install nginx
-  sudo nano /etc/nginx/sites-available/example.domain.com.conf+  sudo nano /etc/nginx/sites-available/music.domain.com.conf
      
 Enter this in the file Enter this in the file
  
   server {   server {
-    server_name  example.domain.com;+    server_name  music.domain.com;
     location / {     location / {
-        proxy_pass http://10.13.13.240+        proxy_pass http://10.13.13.3
-        #proxy_set_header Host example.domain.com;+        #proxy_set_header Host music.domain.com;
     }     }
   }   }
  
-Repeat this as needed for each VM within PVE. Once this is done, set up TLS on the PVE/RP instance as follows:+Repeat this as needed for each VM within PVE. For example, here's another entry for an instance doing nextcloud instead of music: 
 + 
 +  sudo nano /etc/nginx/sites-available/nextcloud.domain.com.conf 
 +   
 +Enter this in the file 
 + 
 +  server { 
 +    server_name  nextcloud.domain.com; 
 +    location / { 
 +        proxy_pass http://10.13.13.4; 
 +        #proxy_set_header Host nextcloud.domain.com; 
 +    } 
 +  } 
 + 
 +Once this is done, set up TLS on the PVE/RP instance as follows:
  
   sudo apt install certbot letsencrypt python3-certbot-nginx   sudo apt install certbot letsencrypt python3-certbot-nginx
   sudo certbot --authenticator standalone --installer nginx -d example.domain.com --pre-hook "systemctl stop nginx" --post-hook "systemctl start nginx"   sudo certbot --authenticator standalone --installer nginx -d example.domain.com --pre-hook "systemctl stop nginx" --post-hook "systemctl start nginx"
  
 +Remember, it the reverse proxy web server does not need to match the upstream web server on the LAN at the two different clients ''10.13.13.3'' and ''10.13.13.4'' because it is just a reverse proxy. It is simply fielding external url requests on port 80/443 for music.domain.com and nextcloud.domain.com and sending those requests upstream to the web servers on those clients which take over at that point. This works fine for browsing these sites when you are external, or not on the 10.13.13.0/24 LAN. But if you are within the LAN, you need to enable NAT loopback on your port forwards to 80/443 on the openWRT router. You do that by going to Firewall / Port Forward / Edit / Advanced Settings / Enable NAT Loopback. Additionally, you must add corresponding entries in hosts within openWRT for each FQDN and the corresponding local IP. To do that, you go to DHCP and DNS / Hostnames / Add. That takes care of being able to access these sites within or without the LAN 10.13.13.0/24, but there's an additional issue. From inside the LAN, Firefox has no issue accessing these sites, but Chrome will refuse access to them since the cert on the reverse proxy does not match the host serving the website content. This is related to rfc 6844 and can be handled by either making openWRT a certificate authority that has permission to do this or by installing matching certificates within the clients. I chose the latter option because I don't want to expose openWRT or tinker with it. Inside the shell in each VM, make sure to set the FQDN, add certbot, and add some header entries:
 +
 +  sudo nano /etc/hosts [inside VM1]
 +  <127.0.0.1 music.domain.com music>
 +  sudo nano /etc/apache2/sites-enabled/music.domain.com.conf
 +  <ServerName music.domain.com>
 +  <Header add myheader "music.domain.com">
 +  <RequestHeader set myheader "music.domain.com">
 +
 +That takes care of VM1, now let's do VM2:  
 +
 +  sudo nano /etc/hosts [inside VM2]
 +  <127.0.0.1 nextcloud.domain.com nextcloud>
 +  sudo nano /etc/apache2/sites-enabled/nextcloud.domain.com.conf
 +  <ServerName nextcloud.domain.com>
 +  <Header add myheader "nextcloud.domain.com">
 +  <RequestHeader set myheader "nextcloud.domain.com">
      
 +Once this id done, restart your web server. In my case, the VMs are using apache and only the PVE / reverse proxy server is using nginx. After restarting the web servers inside each VM, you now can create the matching certs inside each VM. Here's the commands for VM1:
 +
 +  sudo apt install certbot letsencrypt python3-certbot-apache
 +  sudo certbot --authenticator standalone --installer apache -d music.example.com --pre-hook "systemctl stop apache2" --post-hook "systemctl start apache2"
 +  
 +Here are the commands inside VM2:
 +
 +  sudo apt install certbot letsencrypt python3-certbot-apache
 +  sudo certbot --authenticator standalone --installer apache -d nextcloud.example.com --pre-hook "systemctl stop apache2" --post-hook "systemctl start apache2"
 +  
 +Once that's done, make sure that you have cron jobs set for both VMs and for the reverse proxy server / PVE instance. Just enter the following in crontab for all three:
 +
 +  30 2 * * 1 /usr/bin/certbot renew >> /var/log/le-renew.log
 +
 +This should now make Chrome happy since the VM cert will match the cert of the reverse proxy. Why does this work you might ask? Well, when you run certbot inside the VM, it merely cares whether it can be reached externally where you've declared it to be and since it can, it creates the cert without issue. The reverse proxy / PVE instance itself is also able to handle requests for these domains so certbot likewise has no problems issuing the cert there either. This not only makes the Chrome happy, but it addresses what the Chrome developers and rfc 6844 is concerned about, namely, that without this ... connections inside the LAN could potentially be non-TLS or non-header matching. So, this is a best of both worlds. Once this was done and I confirmed that some simple static websites were up and running and accessible externally and internally with no TLS errors on multiple browsers, etc., I then set up a WordPress instance on music.domain.com to ensure that there were no errors using some form of CMS. There were no errors. I am now migrating to this stack because it allows me to segregate services on different VMs in my home network without having to use one massive VM that has vhosts for everything.
  
 +One consideration to bear in mind is the following. My example uses one domain.com and other subdomains on that, like music.domain.com and nextcloud.domain.com. That makes it very easy to do. If you want to do this but want to do two different domains, it is certainly possible, but involves more NAT and setup on the reverse proxy and PVE instance than I a) need or b) care to do. It adds no value. I simply want separate VMs that are publicly accessible, internally accessible, TLS-secured, and separate VMs (database/stack reasons). This is now solved. 
  
  
- --- //[[jonathan@haacksnetworking.org|oemb1905]] 2023/06/19 17:40//+ --- //[[jonathan@haacksnetworking.org|oemb1905]] 2023/12/23 22:09//