User Tools

Site Tools


computing:virtmanagerhell

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
computing:virtmanagerhell [2022/08/08 06:41] oemb1905computing:virtmanagerhell [2023/01/15 17:47] (current) oemb1905
Line 7: Line 7:
 ------------------------------------------- -------------------------------------------
  
-Alright, I am completely re-writing this as the old notes were just out of date and incomplete.  The title of this is virt-manager hell because virt-manager used to be pain in the ass for mebut I now love it.  Regardless, the name is staying the same At any rate, this tutorial is for Debian based users who have physical hosts using an external IP (either self-hosted or VPS/data center)and want to allocate some other external IPs in their block to public-facing VMs Every step except for one is command-line based, for ease, stability, and understanding - and I am not using network-manager or dhcpd at all In my case, I am using a SuperMicro host which has 5 ethernet ports, including two which have IPMI functionality.  In my case, I decided to allocate two external IPs to the physical host so in case I tinker with bridge settings (on the other interface), then I don't lose shell access to the box or have to hassle the Data Center for KVM egcy connection, etc On ent8s0g0 I configured the base connection, and on ent8s0g1 I configured the bridge as follows:+To make VM from the command linedo the followingNote that this recipe assumes you have already created your virtual switchbr0It also presumes you have already created your virtual disk, and if you have not, simply run ''sudo qemu-img create -f raw new.img 1000G''.
  
-  sudo nano /etc/network/interfaces+  sudo virt-install --name=new.img \ 
 +  --os-type=Linux \ 
 +  --os-variant=debian10 \ 
 +  --vcpu=1 \ 
 +  --ram=2048 \ 
 +  --disk path=/mnt/vms/students/new.img \ 
 +  --graphics spice \ 
 +  --location=/mnt/vms/isos/debian-11.4.0-amd64-netinst.iso \ 
 +  --network bridge:br0
  
-That file should look like this (adjust to your use-caseofc):+To clone an existing image, do the following: 
 + 
 +  virt-clone \ 
 +  --original=clean \ 
 +  --name=sequoia \ 
 +  --file=/mnt/vms/students/sequoia.img 
 + 
 +If you have a legacy image that needs to be largerthen install a few tools and the proceed to expand the virtual hard disk as follows: 
 + 
 +  apt install qemu-img kpartx 
 +  qemu-img resize debian10.img +50G
      
-  #eth0 (alt name ent8s0gphysical host base-connection +After expanding the virtual hard disk, open gparted in X passthrough / command line and expand the existing partition into as much of the the new space as you prefer. And to rename a VM (domain), do the following: 
-  auto ent8s0g0 +   
-  iface ent8s0f0 inet static +  virsh shutdown <old-name> 
-        address 8.25.76.160 +  virsh domrename <old-name> <new-name> 
-        netmask 255.255.255.0 +   
-        gateway 8.25.76.1 +To delete a domain, virt-manager uses the undefine commandTo remove the accompanying storage with it, parse the command as follows:
-        nameserver 8.8.8.8 +
-  #eth1 (alt name enp8s0g1) interface for bridge +
-  auto enp8s0g1 +
-  iface enp8s0g1 inet manual +
-  auto br0 +
-  iface br0 inet static +
-        address 8.25.76.159 +
-        netmask 255.255.255.0 +
-        gateway 8.25.76.1 +
-        bridge_ports enp8s0g1 +
-        nameserver 8.8.8.8+
  
-Once that's donerun ''ip a'' to make sure your primary interface connects upstream to the Data Centerand also make sure that the interface ''br0'' appears at the bottom and that the secondary interface shows it as bound to the bridge in its output.  Sometimes, I find that nameservers don't properly populate to resolv.conf, so I do the following so that my ''resolv.conf'' configurations stick and I don't lose upstream DNS. (NoteI do this because Debian - rightfully - still supports manual over-writing of /etc/resolv.conf.)+  virsh undefine guest1 --remove-all-storage 
 +   
 +If you need to force stop a machinevirt-manager uses the destroy command, as follows:
  
-  echo nameserver 8.8.8.8 > /etc/resolv.conf+  virsh destroy guest1 
 + 
 +I prefer the raw format (.img) but if I change my mind later, perhaps because I want snapshots, then I can easily shutdown the machine and convert the image as followsIf I change my mind, I can also go in reverse back to raw. 
 + 
 +  qemu-img convert -p -f raw -O qcow2 guest1.img guest1.qcow2 
 +  qemu-img convert -p -f qcow2 -O raw guest1.qcow2 guest1.raw
      
-Reboot the host and ping 8.8.8.8 and google.com to ensure you have link and upstream DNS.  Next up, it is time to configure the guest / VM machine.  I saw a lot of good tutorials online, but most of them got sloppy at this stage as far as interfaces and bridging was concerned, so I'll try to be clear where they were not When you set up the new VM (not covered here)instead of relying on the NAT-based default network, change the option to "Bridge" (this is in the virt-manager GUI) and enter the name of the bridge, in my case ''br0''.  (You can also use ''virsh'' for this step, but why lol - I just use X forwarding and open the GUI.)  This step connects the hypervisor NIC to the virtual switch of the bridge on the physical host.  Once that's done, spin up the VM and open up the Terminal (the one **inside** the VM) In the VM's Terminalconfigure the NIC interface as follows:+If you do end up using thisthen you will need to edit the virtual machine's xml fileTo do thatopen the file and edit the disk type. Change ''type'' and ''source file='' to match the new type and extensionIf you also moved the locationyou can edit that now as well.
  
-  sudo nano /etc/network/interfaces+  virsh edit guest1 
 +  <driver name='qemu' type='raw' cache='none'/
 +  <source file='/mnt/vms/students/guest1.qcow2'/> 
 +   
 +Another essential tool is the ability to create snapshots and, when necessary, revert to them. Here are the commands to create snapshot1 and then later revert to that specific snapshot.
  
-This file should look like this (adjust to your use-case and again, this is **inside the VM** Terminaland __not__ on the Terminal of the physical host):+  virsh snapshot-create-as guest1 snapshot1 --description "first snapshot 11-13-22" 
 +  virsh snapshot-revert guest1 snapshot1 
 +   
 +If you don't care about naming the snapshot or customizing the descriptionyou may optionally create the snapshot by excuting this:
  
-  auto epr1 +  virsh snapshot-create guest1 
-  iface epr1 inet static +   
-    address 8.25.76.158 +To create an external snapshot, append the following arguments:
-    netmask 255.255.255.0 +
-    gateway 8.25.76.1 +
-    nameservers 8.8.8.8+
  
-The VM interface is listed inside the guest/VM as epr1 but rememberthat's connected to the virtual switch and bridge through the previous stepsso don't worry.  After this step, restart the networking service and check to see if your IP address is assigned.  Also, in my use-case my VM is Ubuntu which does not allow manual over-writing of resolv.conf, so I also add upstream DNS as follows:+  virsh snapshot-create-as guest1 snapshot1 --description "first snapshot 11-13-22" --diskspec vda,snapshot=external 
 +   
 +If you want to revert to the latest snapshotthen execute this: 
  
-  sudo service networking restart +  virsh snapshot-revert guest1 --current
-  ip a +
-  sudo apt install resolvconf +
-  sudo nano /etc/resolvconf/resolv.conf.d/tail+
      
-Enter the name server as follows:+If you need to delete a particular snapshot, or delete all children of a snapshot, execute:
      
-  nameserver 8.8.8.8+  virsh snapshot-delete guest1 snapshot1 
 +  virsh snapshot-delete guest1 --current --children-only
  
-At this point, I would probably reboot and then from within the VM, ping 8.8.8.8, and then ping google.com to ensure you have link and upstream DNS.  Everything should be rosy ;> Some folks might be concerned about ARP and such, but virt-manager handles that with the gateway entry combined with the bridge, so no need to alter proc and pass traffic, etc.  Of course, replace Google's DNS if you so choose, but I had reliability problems with Level 3 during testing myself (sad).+   
 +To list all snapshots for a particular guestOS, execute this:
  
--- Below, are notes from when shrunk an .img file expanded an .img file (cannot remember lol) --+  virsh snapshot-list guest1 
 +   
 +To get information about the snapshot, these commands help: 
 + 
 +  virsh snapshot-info guest1 snapshot 
 +  virsh snapshot-dumpxml guest1 snapshot1 
 +   
 +If you need to make a live backupdo the following (Note: make sure that ''sudo apt install qemu-guest-agent'' is installed on the guest OS): 
 + 
 + 
 +  virsh domfsfreeze guest1.qcow2 
 +  qemu-img create -f qcow2 -b guest1.qcow2 snapshot.qcow2 
 +  virsh domfsthaw guest1.qcow2 
 + 
 +   
 +At times you may need to resize or gather information about a particular virtual disk. If they are in the qcow2 format, gather information as follows: 
 + 
 +  qemu-img info disk.qcow2 
 +   
 +To regain space that is being used needlessly, you can sparsify the qcow2 disk. Note that you must install virt-sparsify separately with ''sudo apt install libguestfs-tools''.  
 + 
 +  virt-sparsify --in-place disk.qcow2 
 +   
 +Most importantly, there's often a need to expand a qcow2 disk. For that, first recommend you backup the disk. Then, you can either specify the desired or target size, or you specify an increment of space to add on to the disk. 
 + 
 +  cp -ar --sparse=always disk.qcow2 disk.bk.qcow2 
 +  qemu-img resize disk.qcow2 1000G 
 +  qemu-img resize disk.qcow2 +10G 
 +   
 +Okay, so another big issue with qcow2 images is them growing over time from writes/deletes and then those blocks taking up space on the hypervisor while not actually being used on the guestOS any longerTo stop this, enable unmap on the virtIO disk in virsh, and then schedule a timer within the guest OS to trim those underlying blocks. 
 +   
 +  sudo systemctl enable fstrim.timer 
 +  sudo systemctl start fstrim.timer 
 +   
 +You can also manually run fstrim and then power down the qcow2 image and convert it. You may optionally use compression to save more space, but it takes very long. 
 + 
 +  fstrim -v / 
 +  qemu-img convert -O qcow2 guest.qcow2 guest-trimmed.qcow2 
 +  qemu-img convert -O qcow2 -c guest.qcow2 guest-trimmed.qcow2 
 +   
 +To test trimming functionality, I use a combination of ``ncdu`` which shows actually consumed space on the host OS, along with the following dd command to easily consume and delete space: 
 + 
 +  dd if=/dev/urandom of=file bs=4M count=500 
 +   
 +To create a backup volume inside a guest you create the volume, attach it, and then shell into the guest and format, mount, and create an fstab entry. First, on the hostOS: 
 +   
 +  cd /mnt/vms/backups/vm1-backup-dir/ 
 +  qemu-img create -f qcow2 vm1-backup.qcow2 32G 
 +  virsh attach-disk guestOS.qcow2 \ 
 +    --source /mnt/vms/backups/vm1-backup-dir/vm1-backup \ 
 +    --target vdb \ 
 +    --persistent 
 +   
 +Then, on the guestOS: 
 +   
 +  mkdir /mnt/backup 
 +  mkfs.ext4 /dev/vdb 
 +  mount -t auto /dev/vdb /mnt/backup 
 +  nano /etc/fstab 
 +  /dev/vdb /mnt/backup ext4 defaults, 0 0
  
-Resize an .img with virt-managerfor example, expanding a 64GB disk to a 124GB disk Start by installing the optional tools for this:+The rest from here on out is my attempt at resizing an .img virtual disk using tools exclusively from virsh / virt-manager. These are highly risky moves and totally not needed for day to day operationsIt was more of a mission I was on and based on a tutorial I used nearly 15 years ago when expanding a Windows VM I used for teaching software that was only on that VM. At any rate, I have only succeeded twice doing this, and often get confused looking at the l00ps. Proceed with caution!
      
   sudo apt install libguestfs-tools   sudo apt install libguestfs-tools
Line 142: Line 220:
  
   kpartx -d debian10.img   kpartx -d debian10.img
-   
-After messing around with this, and succeeding 1 time in resizing the drive this way, I decided that just issuing ''apt install qemu-img kpartx'' and then expanding within the VM's gparted GUI was far easier. It does require moving and/or deprecating swap (or other earlier partitions) or you cannot merge the file system. 
  
- --- //[[jonathan@haacksnetworking.org|oemb1905]] 2022/08/08 00:37//+ --- //[[jonathan@haacksnetworking.org|oemb1905]] 2023/01/15 10:31//
computing/virtmanagerhell.1659940880.txt.gz · Last modified: 2022/08/08 06:41 by oemb1905