Sunday, February 19, 2023
Yushui 雨水
Posted by
Teck
at
2/19/2023 01:51:00 PM
0
comments
Labels: Calligraphy
Saturday, February 18, 2023
Dynamic VFIO binding and unbinding for GPU passthrough
Identifier "layout"
Screen 0 "intel"
EndSection
Section "Device"
Identifier "intel"
Driver "modesetting"
BusID "PCI:0@0:2:0"
EndSection
Section "Screen"
Identifier "intel"
Device "intel"
EndSection
01:00.0 VGA compatible controller: NVIDIA Corporation GP107GL [Quadro P600] (rev a1)
options kvm ignore_msrs=1 report_ignored_msrs=0
options snd-hda-intel enable_msi=1
VIRSH_GPU_VIDEO=0000:01:00.0
VIRSH_GPU_AUDIO=0000:01:00.1
GPU_VIDEO_DRIVER=nvidia
GPU_AUDIO_DRIVER=snd_hda_intel
set -x
function unbind() {
busid=$1
vendor=$(cat /sys/bus/pci/devices/$busid/vendor)
device=$(cat /sys/bus/pci/devices/$busid/device)
modprobe vfio-pci #>/dev/null 2>/dev/null
if [ -e /sys/bus/pci/devices/$busid/driver ]; then
printf "Unbinding %s (%s:%s)\n" "$busid" "$vendor" "$device"
echo $busid > /sys/bus/pci/devices/$busid/driver/unbind
#echo vfio-pci > /sys/bus/pci/devices/$dev/driver/driver_override
fi
echo -n "$vendor $device" > /sys/bus/pci/drivers/vfio-pci/new_id && echo "$busid -> vfio-pci"
}
function rebind() {
busid=$1; shift
drv=$1; shift
vendor=$(cat /sys/bus/pci/devices/$busid/vendor)
device=$(cat /sys/bus/pci/devices/$busid/device)
printf "Unbinding %s (%s:%s)\n" "$busid" "$vendor" "$device"
echo -n "$vendor $device" > /sys/bus/pci/drivers/vfio-pci/remove_id
echo $busid > /sys/bus/pci/devices/$busid/driver/unbind
echo $busid > /sys/bus/pci/drivers/$drv/bind && echo "$busid -> $drv"
}
cmd=$1; shift || true
case $cmd in
unbind)
unbind $@
;;
rebind)
rebind $@
;;
*)
printf "Usage: %s [unbind <busid> | rebind <busid> <drv>]\n" "$0"
exit 1
;;
esac
Remember to make it executable by
## Load the config file
source "/root/passthrough/kvm.conf"
systemctl stop bumbleed
systemctl stop nvidia-persistenced
# Unload Nvidia
modprobe -r nvidia_drm
modprobe -r nvidia_modeset
modprobe -r drm_kms_helper
modprobe -r nvidia
modprobe -r i2c_nvidia_gpu
modprobe -r drm
modprobe -r nvidia_uvm
## Load vfio
modprobe vfio
modprobe vfio_iommu_type1
modprobe vfio_pci
modprobe vfio_virqfd
## Unbind gpu from nvidia and bind to vfio
/root/passthrough/vfio.sh unbind $VIRSH_GPU_VIDEO
/root/passthrough/vfio.sh unbind $VIRSH_GPU_AUDIO
chmod +x /root/passthrough/bind_vfio.sh
## Load the config file
source "/root/passthrough/kvm.conf"
## Unbind gpu from vfio and bind to nvidia
/root/passthrough/vfio.sh rebind $VIRSH_GPU_VIDEO $GPU_VIDEO_DRIVER
/root/passthrough/vfio.sh rebind $VIRSH_GPU_AUDIO $GPU_AUDIO_DRIVER
## Unload vfio
modprobe -r vfio_pci
modprobe -r vfio_iommu_type1
modprobe -r vfio
modprobe -r vfio_virqfd
# Reload nvidia modules
modprobe nvidia_drm
modprobe nvidia_modeset
modprobe drm_kms_helper
modprobe nvidia
modprobe drm
modprobe nvidia_uvm
systemctl start nvidia-persistenced
systemctl start bumbleed
chmod +x /root/passthrough/unbind_vfio.sh
Kernel modules: nvidia
01:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev a1)
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
11. Make sure unbinding works by running unbind_vfio.sh, after which
Kernel driver in use: nvidia
Kernel modules: nvidia
01:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev a1)
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
# Exmple hook script for PVE guests (hookscript config option)
# You can set this via pct/qm with
# pct set <vmid> -hookscript <volume-id>
# qm set <vmid> -hookscript <volume-id>
# where <volume-id> has to be an executable file in the snippets folder
# of any storage with directories e.g.:
# qm set 100 -hookscript local:snippets/hookscript.pl
use strict;
use warnings;
print "GUEST HOOK: " . join(' ', @ARGV). "\n";
# First argument is the vmid
my $vmid = shift;
# Second argument is the phase
my $phase = shift;
if ($phase eq 'pre-start') {
# First phase 'pre-start' will be executed before the guest
# is started. Exiting with a code != 0 will abort the start
print "$vmid is starting, doing preparations.\n";
system("/root/passthrough/bind_vfio.sh");
# print "preparations failed, aborting."
# exit(1);
} elsif ($phase eq 'post-start') {
# Second phase 'post-start' will be executed after the guest
# successfully started.
print "$vmid started successfully.\n";
} elsif ($phase eq 'pre-stop') {
# Third phase 'pre-stop' will be executed before stopping the guest
# via the API. Will not be executed if the guest is stopped from
# within e.g., with a 'poweroff'
print "$vmid will be stopped.\n";
} elsif ($phase eq 'post-stop') {
# Last phase 'post-stop' will be executed after the guest stopped.
# This should even be executed in case the guest crashes or stopped
# unexpectedly.
print "$vmid stopped. Doing cleanup.\n";
system("/root/passthrough/unbind_vfio.sh");
} else {
die "got unknown phase '$phase'\n";
}
exit(0);
chmod +x /var/lib/vz/snippets/gpu-hookscript.pl
Kernel driver in use: vfio-pci
Kernel modules: nvidia
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
Kernel driver in use: nvidia
Kernel modules: nvidia
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
Posted by
Teck
at
2/18/2023 01:35:00 PM
0
comments
Labels: Homelab
Friday, February 10, 2023
Installing Proxmox VE followed by Xfce desktop
deb http://deb.debian.org/debian bullseye main contrib non-free
deb-src http://deb.debian.org/debian bullseye main contrib non-free
deb-src http://security.debian.org/debian-security/ bullseye-security main
deb http://deb.debian.org/debian bullseye-updates main contrib non-free
deb-src http://deb.debian.org/debian bullseye-updates main contrib non-free
EOF
apt-get update
apt-get install nvidia-driver
Posted by
Teck
at
2/10/2023 10:11:00 AM
0
comments
Labels: Homelab
Tuesday, February 07, 2023
Comparing various ways to install a Proxmox developer workstation
Posted by
Teck
at
2/07/2023 06:10:00 PM
0
comments
Labels: Homelab
Saturday, February 04, 2023
Watching "Kin no Kuni Mizu no Kuni" (金の国 水の国)
The movie itself is 117 minutes, almost two hours. It started with a brief narration about the setting: two countries that have not been on good terms with each other for a thousand years, waging wars countless times. An ancient sign of goodwill is for Country A to send its most beautiful girl to Country B as a wife, and for Country B to send its most intelligent youth to Country A as a husband. Instead, Country A sent a kitten to be the wife of Naranbayar (a young architect from Country B), and Country B sent a puppy to be the husband of Sara (the 93rd princess of Country A, born by one of the king's concubines). Naranbayar and Saya kept their spouses' true identities a secret, knowing that their respective kings would be enraged and go to war again.
Posted by
Teck
at
2/04/2023 05:40:00 PM
0
comments
Labels: Movies
Lichun 立春
Posted by
Teck
at
2/04/2023 01:48:00 PM
0
comments
Labels: Calligraphy
Sunday, January 29, 2023
Deleting a Proxmox cluster
to see the names of the nodes.
rm -r /etc/pve/nodes/NODENAME
If pvecm delnode NODENAME fails, work around by pvecm expected 1, then try again.
Next, SSH into the node being deleted and run the following commands to stop the cluster on that node.
systemctl stop corosync
Then, delete the corosync configuration files.
rm -r /etc/corosync/*
killall pmxcfs
systemctl start pve-cluster
Remove the remaining cluster-related files on this node.
This separates the node from the cluster.
Once that has been done, my R430 and Raspberry Pi 4 are no longer in cluster configuration.
systemctl stop pve-cluster
systemctl stop corosync
Then, force the node to run in local mode.
pmxcfs -l
Then, delete all the cluster configuration files.
rm -f /etc/pve/cluster.conf /etc/pve/corosync.conf
rm -f /etc/cluster/cluster.conf /etc/corosync/corosync.conf
rm /var/lib/pve-cluster/.pmxcfs.lockfile
rm -r /etc/corosync/*
Next, restart empty cluster service.
killall pmxcfs
systemctl start pve-cluster
This separates the main node from the cluster.
Posted by
Teck
at
1/29/2023 03:05:00 PM
0
comments
Labels: Homelab
Theming XFCE to look like Linux Mint
Posted by
Teck
at
1/29/2023 02:34:00 PM
0
comments
Labels: Homelab
Saturday, January 28, 2023
Watching Suzume again (5th time)
Posted by
Teck
at
1/28/2023 09:38:00 PM
2
comments
Labels: Movies
Friday, January 27, 2023
Installing Proxmox on Linux Mint Debian Edition 5
2. Install the Proxmox VE kernel
3. Install the Proxmox VE packages
4. Remove the Debian kernel
5. Remove the os-prober package (optional)
Also, you need to set a password for root by
Dual upgrade: Proxmox 7 to 8 and Linux Mint Debian Edition 5 to 6
Posted by
Teck
at
1/27/2023 08:17:00 PM
5
comments
Labels: Homelab
Thursday, January 26, 2023
Wednesday, January 11, 2023
5th anniversary of Violet Evergarden anime series
Posted by
Teck
at
1/11/2023 12:00:00 AM
0
comments
Labels: TV, Violet Evergarden
Friday, January 06, 2023
Watching Violet Evergarden the Movie at a LIVE ZOUND cinema (15th viewing)
Posted by
Teck
at
1/06/2023 11:30:00 PM
0
comments
Labels: Movies, Violet Evergarden




















