This shows you the differences between two versions of the page.
Next revision | Previous revisionNext revisionBoth sides next revision | ||
computing:proxmox [2023/12/24 11:25] – created oemb1905 | computing:proxmox [2024/01/06 23:35] – oemb1905 | ||
---|---|---|---|
Line 11: | Line 11: | ||
------------------------------------------- | ------------------------------------------- | ||
- | Most important thing about proxmox | + | This tutorial has my notes for setting up Proxmox on Debian GNU/Linux. Ultimately, I set up a PVE instance on my home server as a network bridge and reverse proxy that can host multiple websites/ |
qm importdisk 500 hub.jonathanhaack.com.qcow2 < | qm importdisk 500 hub.jonathanhaack.com.qcow2 < | ||
Line 29: | Line 29: | ||
vma extract / | vma extract / | ||
| | ||
- | Optionallly, | + | If it is compressed: |
- | | + | |
- | In order to see where the virtual machine' | + | In order to see where the virtual machine' |
zfs list | zfs list | ||
Line 47: | Line 47: | ||
vms/ | vms/ | ||
| | ||
- | Be careful not to delete those thinking wrongly that they are extraneous - those are obviously required for booting, but it might not be clear upon first look. Also, I chose to install those on the '' | + | Be careful not to delete those thinking wrongly that they are extraneous - those are obviously required for booting, but it might not be clear upon first look. After getting images inside and out and spinning up the old Windows VM, I decided a year or so later that it would be worth it to spin PVE up with a network bridge and reverse proxy in order to run it to manage my home network. The home Nextcloud, airsonic, pihole, and other similar devices would be virtualized in the PVE and the PVE bridge and reverse proxy would route traffic |
- | + | ||
- | To verify | + | |
grep " | grep " | ||
| | ||
- | Had to switch | + | While installing proxmox on top of stock debian, postfix asked me how to set it up and I chose satellite system and sent outgoing email to my relay as follows: |
[relay.haacksnetworking.org]: | [relay.haacksnetworking.org]: | ||
- | Created bridge on pve instance with this recipe: | + | Once PVE is setup, log in locally and create the following inside interfaces. Make sure you are using ifupdown, not netplan. |
source / | source / | ||
Line 71: | Line 69: | ||
bridge-fd 0 | bridge-fd 0 | ||
| | ||
- | Make sure the PVE instance, which is also our reverse proxy has the FQDN set up in ''/ | + | Make sure the PVE instance, which is also our reverse proxy has the FQDN set up in ''/ |
sudo nano /etc/hosts | sudo nano /etc/hosts | ||
Line 78: | Line 76: | ||
< | < | ||
- | Set up configurations for each website on nginx as follows: | + | Inside the PVE instance, set up configurations for each website on nginx as follows: |
sudo apt install nginx | sudo apt install nginx | ||
sudo nano / | sudo nano / | ||
| | ||
- | Enter this in the file | + | Enter this server block adapted as needed |
server { | server { | ||
Line 97: | Line 95: | ||
sudo nano / | sudo nano / | ||
| | ||
- | Enter this in the file | + | And enter something like ... |
server { | server { | ||
Line 106: | Line 104: | ||
} | } | ||
} | } | ||
+ | |||
+ | To activate this server block,do as follows: | ||
+ | | ||
+ | cd / | ||
+ | ln -s ../ | ||
Once this is done, set up TLS on the PVE/RP instance as follows: | Once this is done, set up TLS on the PVE/RP instance as follows: | ||
Line 112: | Line 115: | ||
sudo certbot --authenticator standalone --installer nginx -d example.domain.com --pre-hook " | sudo certbot --authenticator standalone --installer nginx -d example.domain.com --pre-hook " | ||
- | Remember, it the reverse proxy web server does not need to match the upstream web server | + | If you want to make a proper website and/or encrypt |
- | | + | |
- | | + | |
- | sudo nano /etc/apache2/sites-enabled/music.domain.com.conf | + | |
- | < | + | proxy_pass http://10.13.13.254; |
- | < | + | #lan permit url requests |
- | < | + | allow 10.13.13.0/24; |
+ | #vpn permit url requests | ||
+ | allow 10.33.33.0/24; | ||
+ | #deny all url requests excep above | ||
+ | deny all; | ||
- | That takes care of VM1, now let's do VM2: | + | } |
+ | } | ||
- | | + | After entering the above block, make sure to run certbot again for that domain. Once that's done, you need to set up NAT loopback on the openWRT router so that the instances behind the reverse proxy / PVE instance can route back out. To do that You do that by going to Firewall / Port Forward / Edit / Advanced Settings / Enable NAT Loopback. For the devices within the LAN to locate these domains, you must enter host entries on the openWRT router and that's done in DHCP and DNS / Hostnames / Add. Within each VM, make sure to set the FQDN and install your web server of choice. Make sure to also run certbot within the VMs themselves so that matching certificates are installed. Install headers within the VMs as follows, repeat for each VM: |
- | < | + | |
- | sudo nano / | + | |
- | < | + | < |
- | <Header add myheader " | + | sudo nano / |
- | < | + | < |
+ | <Header add myheader " | ||
+ | < | ||
+ | sudo nano / | ||
+ | < | ||
+ | sudo nano / | ||
+ | < | ||
+ | <Header add myheader " | ||
+ | < | ||
| | ||
Once this id done, restart your web server. In my case, the VMs are using apache and only the PVE / reverse proxy server is using nginx. After restarting the web servers inside each VM, you now can create the matching certs inside each VM. Here's the commands for VM1: | Once this id done, restart your web server. In my case, the VMs are using apache and only the PVE / reverse proxy server is using nginx. After restarting the web servers inside each VM, you now can create the matching certs inside each VM. Here's the commands for VM1: | ||
Line 134: | Line 150: | ||
sudo apt install certbot letsencrypt python3-certbot-apache | sudo apt install certbot letsencrypt python3-certbot-apache | ||
sudo certbot --authenticator standalone --installer apache -d music.example.com --pre-hook " | sudo certbot --authenticator standalone --installer apache -d music.example.com --pre-hook " | ||
- | | ||
- | Here are the commands inside VM2: | ||
- | |||
sudo apt install certbot letsencrypt python3-certbot-apache | sudo apt install certbot letsencrypt python3-certbot-apache | ||
sudo certbot --authenticator standalone --installer apache -d nextcloud.example.com --pre-hook " | sudo certbot --authenticator standalone --installer apache -d nextcloud.example.com --pre-hook " | ||
Line 143: | Line 156: | ||
30 2 * * 1 / | 30 2 * * 1 / | ||
+ | | ||
+ | On one of my VMs, which is running apache and behind the nginx reverse proxy, I got a client sync error for the nextcloud service running on it. It was complaining about the file size. Note: the GNU/Linux clients had no such error. Anyway, the following was needed on PVE / reverse proxy configs to resolve this error: | ||
- | This should now make Chrome happy since the VM cert will match the cert of the reverse proxy. Why does this work you might ask? Well, when you run certbot inside the VM, it merely cares whether it can be reached externally where you've declared it to be and since it can, it creates the cert without issue. The reverse proxy / PVE instance itself is also able to handle requests for these domains so certbot likewise has no problems issuing the cert there either. This not only makes the Chrome happy, but it addresses what the Chrome developers and rfc 6844 is concerned about, namely, that without this ... connections inside the LAN could potentially be non-TLS or non-header matching. So, this is a best of both worlds. Once this was done and I confirmed that some simple static websites were up and running and accessible externally and internally with no TLS errors on multiple browsers, | + | sudo nano /etc/nginx.proxy_params |
+ | < | ||
+ | < | ||
+ | sudo nano /etc/ | ||
+ | < | ||
+ | < | ||
- | One consideration to bear in mind is the following. My example uses one domain.com and other subdomains on that, like music.domain.com and nextcloud.domain.com. That makes it very easy to do. If you want to do this but want to do two different domains, it is certainly possible, but involves more NAT and setup on the reverse proxy and PVE instance | + | Now the reverse proxy / PVE instance |
+ | * Physical Cores * Threads * Physical CPUs = Assignable Virtual Cores | ||
+ | * If using zfs for the storage, estimate 50% of RAM used by it | ||
+ | |||
+ | If ram is an issue, make sure to throttle arc. In my case, I let zfs run free and tightly provision the VMs. Each rig and use case will be different; adjust accordingly. Above, I already shared the command to determine your physical CPUs / sockets. Here's how to determine all three: | ||
+ | |||
+ | lscpu | grep " | ||
+ | lscpu | grep "per socket" | ||
+ | lscpu | grep " | ||
+ | CPU(s): 8 | ||
+ | Core(s) per socket: 4 | ||
+ | Socket(s): 1 | ||
+ | |||
+ | This is for an 8-core i7 Dell XPS 8900. Thus, I have 8*4*1 = 32 cores to play with. As for RAM, I have 64GB total, so subtracting 4Gb for the OS, I have 32GB left for the VMs with roughly 32GB being allocated for zfs. PVE is very strict in interpreting allocated RAM and will OOM kill VMs if it senses it is short. This is in contrast, for example, with virsh which will allow sysadmins to run how they see fit even at risk of virsh crashing. For this reason, either expand your rig, adjust arc, or tightly provision your VMs, or all three. I personally just tightly provisioned my VMs as I want zfs to use as much RAM as it needs. If you prefer to throttle arc, here's how: | ||
+ | |||
+ | sudo nano / | ||
+ | options zfs zfs_arc_max=1073741824 [1 gigabyte example] | ||
+ | update-initramfs -u -k all | ||
+ | reboot | ||
+ | |||
+ | Adjust arc throttle to fit your system. If you need to stop a system at the CLI because you can't reboot/ | ||
+ | |||
+ | / | ||
+ | | ||
+ | Next entry ... | ||
- | --- // | + | --- // |