Setting up Proxmox on a Poweredge R720 rack server

After getting a Poweredge R320 rack server, I kind of realised its limitations, and was hunting around for something else. Like the T420 tower server. But I kept losing to others at the online auction... but managed to land myself with another rack server. The Poweredge R720.

 

The biggest surprise when it arrived was the weight. The deliveryman didn't help to bring it in, handing it over to me at the door. And I found out why. It was a big flat box that weighed 20kg. It was a struggle to bring the box inside, given my lack of strength. Another struggle to get the rack server out of the box. Compared to the R720, the R320 was so much lighter.

I actually didn't know what I was getting, other than that it is a R720. That was all the listing said, plus some photos which don't say much. So I was kind of surprised at what it actually came with when I finally opened it up. From the photos, I knew it had a DVD drive and eight 3.5 inch bays, and the two CPU sinks implied that it had dual processors. But it actually came with dual E5-2640 processors (6 threads 12 cores each, for a total of 12 cores 24 threads) and 64 GB of RAM. Plus two 1100 W power supplies in redundancy mode, and a H710 Mini RAID controller. In addition to four full-length PCI-E slots, it also came with the optional three half-length slots. That's a total of seven PCI-E slots, although only one is a 16-lane slot, while the rest are 8 lanes. Still, not bad for 20,000 yen.

 

I had separately bought 64 GB of RAM for the R320, so I transferred that over for a total of 128 GB RAM. I am also going to swap out the processors with E5-2670v2 (10 cores 20 threads each, for a total of 20 cores 40 threads) which should be more than enough to run several virtual machines on it at the same time. I also took out the SSD and HDD that I added to the R320, which will be in storage for a while until I figure out what to do with it. (Sell? Use?) Plus I managed to get a GTX 1050Ti and a Quadro P400 (both used) for a reasonable price.

I had previously printed out adapters to mount 2.5 inch drives in 3.5 inch bays. But some of the 3.5 inch caddies of the R720 came with adapters already, which I conveniently used.

I installed the following in the R720:

2 x 500 GB SSDs for Proxmox and virtual machines

1 x 2 TB 2.5 inch HDD as additional slow storage for the virtual machines and other computers in the house to share

3 x 6 TB 3.5 inch HDD in RAIDZ configuration as NAS storage

I am using Proxmox as the hypervisor, under which I installed a TrueNAS virtual machine to handle the NAS storage (handling the 3 x 6 TB HDDs in RAIDZ). The 2 TB HDD is actually mounted as a Samba share under Proxmox so that I can share it even if the TrueNAS virtual machine is not running. I also have another Ubuntu virtual machine running a Jellyfin media server. All my media files are now in NAS and served by the Jellyfin server. The Quadro P400 is passed through to the Jellyfin server to help with hardware accelerated transcoding.

I also installed a Ubuntu MATE virtual machine, and a Windows 11 virtual machine. These are more for generic use. Or rather, the Ubuntu MATE one is for generic use. The Windows 11 one is for me to trying gaming with a virtual machine. I passed through the GTX 1050Ti and it gave reasonable performance in performance mode in Fortnite, accessed through NoMachine.

I was actually kind of surprised that the GTX 1050Ti came in such a good condition, since it was a used item bought on Yahoo! Auction at a really reasonable price (give the prices of GPU cards now). Even had the original box, manual, driver CD. Very unlike the Quadro P400 which basically came in an anti-static bag wrapped with bubble-wrap.

I also run a Python 2 script that will update the IP address of my domain names. This used to run on another computer in the house, but since the Proxmox hypervisor is likely to be operating all the time, I shifted that script to run under Proxmox at a regular interval using crontab.

I had some issues getting the Ubuntu virtual machine to use the Quadro P400 as its graphics card for virtual desktops, which I eventually managed to solve and will write about the details in another post.

 

Some technical stuff on setting up Proxmox and the virtual machines.

 

To get a coloured prompt in the terminal console, I used

cp /etc/skel/.bashrc /root/.bashrc

and then edited "/root/.bashrc" to uncomment the line

#force_color_prompt=yes

to

force_color_prompt=yes

There is a need to run

source /root/.bashrc

after that to effect the changes immediately.

 

To add a new drive, I followed this guide to extend the first SSD (/dev/sda) with another SSD (/dev/sdb). This gives me around 1 TB of space for Proxmox itself (which uses a partition of 100 GB) and virtual machines (in a thin logical volume pool called "data"). This will come in handy next time when I want to add more virtual machines, or just create addition storage to assign to existing virtual machines.

First, partition the new SSD by

cfdisk /dev/sdb

I chose "gpt", then gave it all the space to the "New" partition, "Write" to write the partition table, then "Quit" to get out of the program. In Proxmox, under "Disks", the new SSD should show with its partition (/dev/sdb and /dev/sdb1).

 

Then, create the physical volume by

pvcreate /dev/sdb1

Then, extend the existing "pve" logical volume group by

vgextend pve /dev/sdb1

Next, I used

pvs

to see how much space on the new SSD there is to assign. For me, it was 465.76 GB, so I extended the "data" thinpool in "pve" by

lvextend /dev/pve/data -L +465.76g

(Note: the "g" at the end is very important... if you miss it, it will extend by MB and not GB.)

This should extend the existing thinpool, which concludes the process.

 

Update March 19, 2022: I didn't really time it with a stopwatch, but it took about 4 minutes for the server to boot into Proxmox. Given that this server is probably 10 years old, the bootup process... is slow. Once Proxmox is booted up, the 3 VMs that I launch on boot will take another 4 to 5 minutes to fully boot up. First is the TrueNAS server since it has all the shared drives. Next is the service server that runs DNS and such. Finally is the media server which basically runs Jellyfin. So the total reboot time is actually more like 8 to 9 minutes. This is compared to less than 2 minutes for the R430 to boot up into Proxmox (no VMs started on boot, though). 

Update March 31, 2022: Here is a collection of scripts that help with Proxmox after installation.

Proxmox Helper Scripts

Proxmox Helper Scripts (GitHub repo)