User Tools

Site Tools


computing:virtmanagerhell

This is an old revision of the document!



  • virtmanagerhell
  • Jonathan Haack
  • Haack's Networking
  • netcmnd@jonathanhaack.com

Alright, I am completely re-writing this as the old notes were just out of date and incomplete. The title of this is virt-manager hell because virt-manager used to be a pain in the ass for me, but I now love it. Regardless, the name is staying the same. At any rate, this tutorial is for Debian based users who have physical hosts using an external IP (either self-hosted or VPS/data center), and want to allocate some other external IPs in their block to public-facing VMs. Every step except for one is command-line based, for ease, stability, and understanding - and I am not using network-manager or dhcpd at all. In my case, I am using a SuperMicro host which has 5 ethernet ports, including two which have IPMI functionality. In my case, I decided to allocate two external IPs to the physical host so in case I tinker with bridge settings (on the other interface), then I don't lose shell access to the box or have to hassle the Data Center for KVM egcy connection, etc. On ent8s0g0 I configured the base connection, and on ent8s0g1 I configured the bridge as follows:

sudo nano /etc/network/interfaces

That file should look like this (adjust to your use-case, ofc):

#eth0 (alt name ent8s0g) physical host base-connection
auto ent8s0g0
iface ent8s0f0 inet static
      address 8.25.76.160
      netmask 255.255.255.0
      gateway 8.25.76.1
      nameserver 8.8.8.8
#eth1 (alt name enp8s0g1) interface for bridge
auto enp8s0g1
iface enp8s0g1 inet manual
auto br0
iface br0 inet static
      address 8.25.76.159
      netmask 255.255.255.0
      gateway 8.25.76.1
      bridge_ports enp8s0g1
      nameserver 8.8.8.8

Once that's done, run ip a to make sure your primary interface connects upstream to the Data Center, and also make sure that the interface br0 appears at the bottom and that the secondary interface shows it as bound to the bridge in its output. Sometimes, I find that nameservers don't properly populate to resolv.conf, so I do the following so that my resolv.conf configurations stick and I don't lose upstream DNS. (Note: I do this because Debian - rightfully - still supports manual over-writing of /etc/resolv.conf.)

echo nameserver 8.8.8.8 > /etc/resolv.conf

Reboot the host and ping 8.8.8.8 and google.com to ensure you have link and upstream DNS. Next up, it is time to configure the guest / VM machine. I saw a lot of good tutorials online, but most of them got sloppy at this stage as far as interfaces and bridging was concerned, so I'll try to be clear where they were not. When you set up the new VM (not covered here), instead of relying on the NAT-based default network, change the option to “Bridge” (this is in the virt-manager GUI) and enter the name of the bridge, in my case br0. (You can also use virsh for this step, but why lol - I just use X forwarding and open the GUI.) This step connects the hypervisor NIC to the virtual switch of the bridge on the physical host. Once that's done, spin up the VM and open up the Terminal (the one inside the VM). In the VM's Terminal, configure the NIC interface as follows:

sudo nano /etc/network/interfaces

This file should look like this (adjust to your use-case - and again, this is inside the VM Terminal, and not on the Terminal of the physical host):

auto epr1
iface epr1 inet static
  address 8.25.76.158
  netmask 255.255.255.0
  gateway 8.25.76.1
  nameservers 8.8.8.8

The VM interface is listed inside the guest/VM as epr1 - but remember, that's connected to the virtual switch and bridge through the previous steps, so don't worry. After this step, restart the networking service and check to see if your IP address is assigned. Also, in my use-case my VM is Ubuntu which does not allow manual over-writing of resolv.conf, so I also add upstream DNS as follows:

sudo service networking restart
ip a
sudo apt install resolvconf
sudo nano /etc/resolvconf/resolv.conf.d/tail

Enter the name server as follows:

nameserver 8.8.8.8

At this point, I would probably reboot and then from within the VM, ping 8.8.8.8, and then ping google.com to ensure you have link and upstream DNS. Everything should be rosy ;>. Some folks might be concerned about ARP and such, but virt-manager handles that with the gateway entry combined with the bridge, so no need to alter proc and pass traffic, etc. Of course, replace Google's DNS if you so choose, but I had reliability problems with Level 3 during testing myself (sad). – Below, are notes from when I shrunk an .img file / expanded an .img file (cannot remember lol) –

Resize an .img with virt-manager, for example, expanding a 64GB disk to a 124GB disk. Start by installing the optional tools for this:

sudo apt install libguestfs-tools

After that, confirm the location of your virtual machine. You need to know the actual name of the .img file for this to work. Usually (unless it was changed by someone), this is the name that appears in the virt-manager window.

virsh dumpxml putnameofimagehere | xpath -e /domain/devices/disk/source

Navigate to the directory it spit out, and back up the image and rename it before you proceed:

cd /var/lib/libvirt/images/
sudo cp /var/lib/libvirt/images/putnameofimagehere.img /var/lib/libvirt/images/putnameofimagehere.img.bak

Once you know the name of the .img file and the location of it, prepare the virtual file systems for expansion:

virt-filesystems --long --parts --blkdevs -h -a /var/lib/libvirt/images/putnameofimagehere.img

Build a new, larger, container for the final disk, and set it aside for the merge/expansion that the tools are going to provide:

truncate -s 128G /var/lib/libvirt/images/outdisk

Now it is time to resize the existing image, using the container you just created:

virt-resize --expand /dev/sda2 /var/lib/libvirt/images/putnameofimagehere.img /var/lib/libvirt/images/outdisk

Okay, now we need to rename the new image we created to the name that virt-manager is looking for, i.e., the name of the original image. Remember, you should have the .bak file you made earlier in case this does not work. This is done as follows:

cp /var/lib/libvirt/images/putnameofimagerhere.img /var/lib/libvirt/images/putnameofimagerhere.img.old
mv /var/lib/libvirt/images/outdisk /var/lib/libvirt/images/putnameofimagehere.img

oemb1905 2021/10/29 16:39

computing/virtmanagerhell.1636352033.txt.gz · Last modified: 2021/11/08 06:13 by oemb1905