Leaving VMware: A Practical KVM Migration Playbook

🚀 Introduction

Since Broadcom closed its acquisition of VMware, the licensing model has changed under almost every customer: perpetual licences are gone, products are sold as subscription bundles, and pricing is per core with a minimum core count per CPU. For many small and mid-sized environments, the renewal quote is the first time anyone has seriously asked: what would it take to leave?

The good news is that the hard part of virtualization — the hypervisor itself — has been a solved problem on Linux for over a decade. KVM runs a large share of the world's public cloud. What makes a VMware exit hard isn't KVM; it's the hundreds of small assumptions your VMs, network, and team have built up around ESXi.

This is the playbook we use when moving workloads to KVM, and specifically to PiVirt, our KVM-based ESXi replacement. Most of it applies to any KVM platform.

📋 Key Topics Covered

• Mapping VMware concepts to their KVM equivalents
• Building an inventory and sorting VMs into migration waves
• Preparing guests: VirtIO drivers and the initramfs trap
• Converting disks: OVA export, qemu-img, and virt-v2v
• Networking: port groups, VLANs, and renamed interfaces
• Cutover, validation, and a rollback plan you can actually use

🗺️ Mapping VMware to KVM

The first thing your team needs is a shared vocabulary. Most VMware concepts have a direct counterpart; a few don't.

📦 Step 1: Inventory and Migration Waves

You can't plan what you haven't counted. Export a full inventory — RVTools is the de facto tool for this — and capture for every VM: OS and version, vCPU and RAM, disk sizes and datastores, NICs and port groups, snapshots, and whether VMware Tools is current.

Then sort VMs into waves by risk, not by size:

🛠️ Step 2: Prepare the Guests

Most failed VM migrations aren't conversion failures. They're guests that boot on the new hypervisor and can't find their disk or network.

🪟 Windows Guests

Windows doesn't ship VirtIO drivers. If you move a Windows VM to KVM with a VirtIO disk controller and no driver installed, it blue-screens with INACCESSIBLE_BOOT_DEVICE.

Before migration, while still on ESXi:

After cutover, uninstall VMware Tools and remove the hidden "ghost" VMware NIC in Device Manager so the old IP isn't held by an adapter that no longer exists.

🐧 Linux Guests

Linux has VirtIO drivers in the kernel, but they must be available at boot. If the initramfs was built on ESXi, it may contain only the drivers needed for VMware's controllers (vmw_pvscsi, mptspi), and the guest drops to an emergency shell because it can't find its root disk.

bash
1# RHEL / Rocky / Alma: rebuild initramfs with VirtIO drivers included
2dracut -f --add-drivers "virtio_blk virtio_scsi virtio_net virtio_pci"
3
4# Debian / Ubuntu: make sure the modules are listed, then rebuild
5printf "virtio_blk\nvirtio_scsi\nvirtio_net\nvirtio_pci\n" >> /etc/initramfs-tools/modules
6update-initramfs -u

Also check /etc/fstab: entries referencing /dev/sda1 may become /dev/vda1 on VirtIO. Mounting by UUID= or LABEL= avoids this entirely.

💽 Step 3: Convert the Disks

There are three common paths, from simplest to most automated.

Export an OVA and import it. Export the VM from vCenter as an OVA/OVF package, then import it on the KVM side. PiVirt's OVA import reads the OVF descriptor — CPU, memory, disks, and network adapters — extracts the disks, and lets you map each source network to a target network during import.

Convert with qemu-img. For more control, convert VMDK disks directly:

bash
1# Convert a VMDK (flat or sparse) to QCOW2, with progress output
2qemu-img convert -p -f vmdk -O qcow2 web01-flat.vmdk web01.qcow2
3
4# Verify the result before booting it
5qemu-img info web01.qcow2
6qemu-img check web01.qcow2

Use virt-v2v. The virt-v2v tool converts a VM end to end — it can connect to vCenter or read an OVA, convert the disks, and modify the guest itself: injecting VirtIO drivers into Windows, rebuilding the Linux initramfs, and removing VMware Tools. For large Windows fleets, this saves a lot of manual preparation.

🌐 Step 4: Networking

🔌 Port Groups Become VLAN Tags

On ESXi, a VM connects to a port group, which carries a VLAN ID. On PiVirt, VM NICs attach as TAP devices to an Open vSwitch bridge, and the VLAN tag is set on that port (see Networking & SDN in the PiVirt docs). Recreate your port groups as a simple mapping: port group name → VLAN ID → OVS bridge. Get the physical uplinks trunking the same VLANs on the KVM hosts as on the ESXi hosts before Wave 1.

🏷️ Renamed Interfaces Break Static Configs

This catches nearly everyone once. Linux names interfaces after their PCI location. A VMXNET3 NIC that was ens192 on ESXi will come up with a different name on KVM, such as enp1s0, and any configuration that references ens192 silently stops applying. The VM boots fine, with no network.

Fix it before cutover:

• Netplan (Ubuntu): match the interface by MAC address instead of name.
• NetworkManager (RHEL family): bind connection profiles to the MAC address, not interface-name.
• Or keep the old name with a .link file that assigns ens192 to that MAC.

yaml
1# /etc/netplan/50-static.yaml: matches by MAC, survives the rename
2network:
3  version: 2
4  ethernets:
5    lan0:
6      match:
7        macaddress: '00:50:56:aa:bb:cc'
8      set-name: lan0
9      addresses: [10.0.20.15/24]
10      routes:
11        - to: default
12          via: 10.0.20.1

Keep the MAC address when you recreate the NIC on KVM. It avoids DHCP reservation changes, licence-server problems, and stale ARP entries on firewalls.

🔁 Step 5: Cutover and Rollback

For each VM, the cutover runbook is short and repeatable:

Rollback is fast precisely because nothing on the source was destroyed: power off the KVM copy, reconnect the source NIC, power it on.

✅ Migration Checklist

🏁 Conclusion

Leaving VMware isn't a hypervisor problem; KVM is mature, fast, and everywhere. It's a project management problem: knowing what you have, preparing guests before they move, getting networking right, and always keeping a way back.

Start small. Move a dozen low-risk VMs, fix every surprise in your runbook, then scale. Most teams find that after the first wave, the rest becomes routine, and the licensing savings start paying for the project well before it's finished.

If you want to see how PiVirt handles the KVM side — declarative YAML configuration, clustered iSCSI/OCFS2 storage, Open vSwitch networking, and QMP-based live migration — read about PiVirt, browse the PiVirt documentation, or see why PiVirt drives QEMU directly instead of using libvirt.

---

❓ Frequently Asked Questions

Q: How long does a VMware to KVM migration take?

A: The per-VM work is minutes to hours, mostly disk copy time. The project time is dominated by planning, guest preparation, and scheduling maintenance windows. A small environment of 50–100 VMs is commonly done in a few weeks once the pilot wave has refined the runbook.

Q: Can I migrate VMs without downtime?

A: Not with a standard conversion. The source VM must be shut down for a consistent final copy. You can minimise downtime by pre-copying disks and doing a short final sync, or by using application-level replication (database replicas, load-balanced web tiers) to move traffic before switching off the old VM.

Q: Why does my Windows VM blue-screen after migration?

A: Almost always INACCESSIBLE_BOOT_DEVICE because the VirtIO storage driver isn't installed. Install the drivers from the virtio-win ISO before migration, use virt-v2v to inject them, or temporarily boot with a SATA disk controller, install the drivers, then switch to VirtIO.

Q: Why does my Linux VM have no network after migration?

A: The network interface was renamed (for example, from ens192 to enp1s0) and the static configuration still references the old name. Match the interface by MAC address in Netplan or NetworkManager, or pin the old name with a systemd .link file.

Q: Do I need to convert VMDK disks to QCOW2?

A: QEMU can read VMDK files, but converting to QCOW2 is strongly recommended. QCOW2 is QEMU's native format, with better performance, snapshots, thin provisioning, and support for backing files, which PiVirt uses for fast provisioning from base images.

Q: Is KVM ready for production enterprise workloads?

A: Yes. KVM is part of the mainline Linux kernel and underpins major public clouds. The question is rarely whether KVM is ready; it's whether your management layer, storage, and operational processes are. That's the part a platform like PiVirt, or a careful in-house build, has to provide.