If you have read my previous post about my first foray into Proxmox, you know that the infrastructure of my home network is, as the Irish would say, not the best. I have been tinkering with routers and smart switches, learning about VLANs and subnets and all kinds of other things that I thought I understood, but it turns out I didn’t.
Doing stuff with server and network gear at home is a challenge because the family just doesn’t get Hardware Mode(tm). Hardware means being sequestered in the workshop, possibly interfering with our access to the Internet. I have to wait for those rare occasions when I am: 1) at home and 2) not changing diapers and 3) not asleep and 4) no one is actively using the Internet. I have been putting things in place, one piece at a time, but my progress is, well, not the best.
Part of my networking woes are design. I don’t know how to build a network for a Proxmox cluster, because I don’t know the right way to build a Proxmox cluster. I also can’t spend hours in my basement lab tinkering. I need to be upstairs with the family. So I decided to build a little portable test cluster, on my laptop, using VirtualBox.
The network design at my house looks a bit like a plate of spaghetti, with old, unmanaged switches in random spots, like meatballs. Little switches plugged into big ones. No tiers, no plan, just hearty Italian improvisimo. Last year, when I fired up two Proxmox nodes, with no consideration for what might happen… Mamma mia!It took a couple of days before the network completely crashed, and a couple of more days to figure out the problem.
The great thing about VirtualBox is that you can build Host Only Networks. A host only network behaves like a physical switch with no uplink. VirtualBox virtual machines (VMs) can talk to each other, and to the physical host without talking to the outside world. This seemed like a decent facsimile of my plan to use a small unmanaged switch to isolate cluster traffic from the rest of the network.
The other great thing about VirtualBox is that you can add lots of network interfaces to a VM in order to simulate network interactions. You can build a router using a Linux or BSD distro and use it to connect your various host only networks to a bridge into your real physical network. I tried that at first, I am not sure that it’s necessary for this exercise.
And last, but not least, VirtualBox lets you clone a VM. As in, to make a procedurally generated copy of a VM, and then start it up along side it. This is a great feature for when you are screwing up configs and installs.
It is the combination of these features that allowed me to create a little virtual lab on a PC so I could figure out how to set up all the cool stuff that Proxmox can do, and figure out what kind of network I will need for it.
Phase 1: The plan
The plan for this exercise is to figure out how to use several features of Proxmox VE. The features are as follows:
- Online Backup and Restore – Proxmox has the ability to take and store snapshots of VMs and containers. This is a great feature for a home lab where you are learning about systems and you are likely to make mistakes. Obviously, I use this feature all the time.
- Clustering – Proxmox has the ability to run multiple hosts in tandem with the ability to migrate guest VMs and Linux containers from one host to another. In theory, using a NAS as shared storage you can migrate a VM without shutting it down. Since the point of this exercise is to build Proxmox hosts and not NAS appliances, we are going to focus on offline migrations where you either suspend the host or shut it down prior to migrating.
- Storage Replication – Proxmox natively supports ZFS, and can use the ZFS Send and Receive commands to make regular copies of your VMs onto the other cluster nodes. Having a recent copy of the VM makes migrations go much faster, and saves you from losing more than a few minutes worth of data or configuration changes. I wish I had this feature working when I was building my Swedish Internet router.
- High Availability – If you have 3 or more PVE nodes in your cluster, you can set some of your VMs to automatically migrate if there is an outage on the node the VM is hosted on. The decision to migrate is based on a kind of voting system that uses a quorum to decide if a host is offline. I want to use this feature to ensure that my access devices are up and running to support my remote access shenanigans.
Phase 2: Preparation
To build the lab, you will need the following:
- A desktop or laptop computer with 2 or more cores and at least 8gb of RAM. You could probably pull this off with 4gb if you are patient. My laptop has an old dual core i5 and 8gb and it was pretty much maxed out the whole time, so your mileage may vary.
- A working OS with a web browser and SSH client. Linux would probably be best, but my laptop was running Win10pro. I recommend a tabbed SSH client capable of sending keystrokes to multiple SSH sessionsLike Moba XTerm.
- VirtualBox installed and running.
- The latest Proxmox VE ISO file.
With the plan in place, and the necessary software gathered, it’s time to proceed to Building A Proxmox Test Cluster in VirtualBox, Part 1: Building The Nodes.