Create a VM with GPU passthrough from noobie to noobies

Remember: To copy and paste on your console, use Shift+Ctrl+C and Shift+Ctrl+V.
You can quick-access your console in Manjaro using F12

* Special Mention: This walk-through  is based on https://www.youtube.com/watch?v=GDLAhSpkK8A video.
It didn't work as expected on my PC, so I upgraded it using the Arch VFIO-Passthrough guide and some further testing.

* Format: Commands you have to input will be marked with a letter: a) b) c), etc. Instructions with a -.

* Relevant Specs:
OS: Manjaro KDE Plasma 5.18.5
Kernel: Linux 5.7.0-3
MOBO: Gigabyte x570 Aorus Elite
CPU: AMD Ryzen 3800X
GPU 1 (For Host): GT 710
GPU 1 (For Guest): RTX 2070 Super

Step 0 - Enter BIOS and enable: SVM, IOMMU, ACS, and AER
		On my Gigabyte x570 motherboard they're located in:

SVM: Tweaker > Advanced CPU Options > SVM Mode
IOMMU: Settings > Miscellaneous > IOMMU
ACS: Settings > AMD CBS > ACS Enable
AER: Settings > AMD CBS > Enable AER Cap

Step 1 - Install packages , services and user permissions.
		Enter these commands one by one accepting when it asks for approval with the (y) key, followed by ENTER.
		If it asks you to change the name of a file, don't. Just press ENTER again and continue.

a) sudo pacman -S ovmf virt-manager libvirt firewalld qemu

b) sudo systemctl enable libvirtd

c) sudo systemctl start libvirtd

d) sudo usermod -a -G kvm,libvirt YOUR USERNAME

Step 2 - Activate IOMMU from the GRUB startup setup

a) sudo nano /etc/default/grub

		- This command will prompt a long list. Find GRUB_CMDLINE_LINUX_DEFAULT and insert the following, without the "(...)", and skipping the parts that were already there (most likely "quiet" will be pre-existent. Don't copy it again, just paste the following parts after the preexisting one). Please don't delete any other pre-existent lines:

GRUB_CMDLINE_LINUX_DEFAULT="(...) quiet amd_iommu=on iommu=pt video=efifb:off (...)"

		- Exit with Ctrl+X, save changes (with y when it asks you to), and press ENTER to continue.

b) sudo grub-mkconfig -o /boot/grub/grub.cfg

		- Reboot your pc to apply changes (You can alternatively type: sudo reboot)

Step 3 - Check to see if IOMMU option is activated & Grab IDs

a) nano iommucheck.sh

		- This command will prompt an empty page. Please paste the following scrypt inside the page;

#!/bin/bash
shopt -s nullglob
for g in /sys/kernel/iommu_groups/*; do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;

		- Exit with Ctrl+X, save changes (with y when it asks you to), and press ENTER to continue.

b) sudo chmod +x iommucheck.sh

c) ./iommucheck.sh

		- This will prompt a long list with all your devices, their controllers, and their IOMMU-assigned groups. Find inside this list the IOMMU Group of the GPU you want to pass through. You'll know you find it when you find one containing your cards name. Mine looks like:

IOMMU Group 27:
        0a:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU104 [GeForce RTX 2070 SUPER] [10de:1e84] (rev a1)
        0a:00.1 Audio device [0403]: NVIDIA Corporation TU104 HD Audio Controller [10de:10f8] (rev a1)
        0a:00.2 USB controller [0c03]: NVIDIA Corporation TU104 USB 3.1 Host Controller [10de:1ad8] (rev a1)
        0a:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU104 USB Type-C UCSI Controller [10de:1ad9] (rev a1)

		- One you find your group, take note (There is no "notepad" preinstalled in Manjaro, instead it's called "Kate") of their ID's in order separated by commas (the ID's are inside brackets []). Mine look like:

10de:1e84,10de:10f8,10de:1ad8,10de:1ad9

		- Exit (Ctrl+x)

Step 4 - Isolate the Graphics card and other devices at boot

a) sudo nano /etc/default/grub

		- This will once again prompt the GRUB. Paste the following inside GRUB_CMDLINE_LINUX_DEFAULT, after "video=efifb:off":

b) vfio-pci.ids=YOUR SAVED ID's

		- My GRUB_CMDLINE_LINUX_DEFAULT ended up looking like this, yours may be a little different, it's ok:

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt video=efifb:off vfio-pci.ids=10de:1e84,10de:10f8,10de:1ad8,10de:1ad9 apparmor=1 security=apparmor udev.log_priority=3"

		- Exit with Ctrl+X, save changes (with y when it asks you to), and press ENTER to continue.

c) sudo grub-mkconfig -o /boot/grub/grub.cfg

d) sudo nano /etc/modprobe.d/vfio.conf

		- This command will prompt an empty page. Please paste the following line inside the page;

e) options vfio-pci ids=10de:YOUR ID's

		- Exit and save changes. My file looks like this: options vfio-pci ids=10de:1e84,10de:10f8,10de:1ad8,10de:1ad9

Step 5 - Apply changes to MKINITCPIO

a) sudo nano /etc/mkinitcpio.conf

		- This command will also prompt a long list. In first place, look for MODULES="" or MODULES=() and paste the following:

b) vfio_pci vfio vfio_iommu_type1 vfio_virqfd

		- Do not confuse it with: #     MODULES=(piix ide_disk reiserfs)
			It's the line right under it. If it's not empty, paste the previous line after the pre-existing commands. The first part of my Mkinitcpio file looks like this:

# vim:set ft=sh
# MODULES
# The following modules are loaded before any boot hooks are
# run.  Advanced users may wish to specify all system modules
# in this array.  For instance:
#     MODULES=(piix ide_disk reiserfs)
MODULES="vfio_pci vfio vfio_iommu_type1 vfio_virqfd"

		- Once done, don't close the file. Scroll further down and look for HOOKS=(...) (the one without the # before it). Make sure it contains "modconf" inside it. If it's not there, put it in yourself. Mine looks like:

HOOKS="base udev autodetect modconf block keyboard keymap filesystems"

		- Exit and save changes.

Step 6 - Regenerate the initramfs

a) sudo mkinitcpio -P

		- After the command is done doing its thing, reboot your pc.

Step 7 - Verify the the configuration works

a) lspci -nnk -d ONE OF YOUR DEVICE'S ID'S

		- In my case, the command was: lspci -nnk -d 10de:1e84
			If it says "Kernel driver in use: vfio-pci", congratulations, you've isolated your GPU and assigned it to VFIO successfully. We're not done yet though.

Step 8 - Download necessary .iso files (Windows Launcher and VritIO Drivers) from the official pages:
		*VirtIO is a driver made to improve your disks performance inside the VM. To put it simply, without it the disk won't know it is working in a virtualised enviroment, so it'll operate as it normally would in a normal setup, regardless of its configuration in the host. Installing it lets it know that it is storing a VM, letting it work with the host and the VM software, instead of against them. If you don't want to install it, you don't need to, but you will lose read-write performance inside your virtualised enviroment.

https://www.microsoft.com/es-es/software-download/windows10ISO
		You'll find it in:
			Select an Option: Windows 10 > Select Language: Your Language > 64-bit Download > Save File

https://docs.fedoraproject.org/en-US/quick-docs/creating-windows-virtual-machines-using-virtio-drivers/
		Last time I checked the page was down, I uploaded my copy to: https://gofile.io/d/HYxCGo (It's the 0.1.171 version).
			You can alternatively try to download it from the web archive.
				If it weren't down at the time of you reading this, search for the "STABLE virtio-win iso" file.

		- Save them to your desired location, your VM software will need to access them later from this location. I created a new folder: /home/username/Documents/ISO/

Step 9 - Find the ID of the disks you want to passthrough to your VM.

a) cd /dev/disk/by-id

b) ls

		- This will display a list with all the Id's your disks and partitions. If you can't tell apart one from the other, look for "KDE Partition Manager" in your preinstalled apps, which will show you the name of your disks, and will tell you which is "part 1" and which is "part 2" inside it, in case you have them partitioned.

		- I will be passing through two partitions, one for the OS and one as extra storage. Note that my disks are called "Sabrent" and "ST2000DM008", hence how I know which of the Id's I needed to copy. In my case, i saved something like:

nvme-Sabrent_AA00000A000000000000-part2 (as main installation disk), and ata-ST2000DM008-0AA000_AAA00A0A-part2 (extra)

		- If you have a partitioned disk but want to pass it through as a whole, copy its ID witout the "-part1" section.

Step 10 - Enable NET Services:

a) sudo pacman -Syu

		- This will check for updates in your package manager. If it asks you to install them, do so.

b) sudo pacman -S qemu libvirt ovmf virt-manager

		- [Proceed with Installation]

c) sudo systemctl enable libvirtd.service

d) sudo systemctl start libvirtd.service

e) sudo systemctl enable virtlogd.socket

f) sudo systemctl start virtlogd.socket

g) sudo pacman -S ebtables iptables dnsmasq

		- [Proceed with Installation]

h) sudo sudo systemctl restart libvirtd

i) sudo virsh net-autostart default

		- After the command is done doing its thing, reboot your pc.

Step 11 - Launch Virtual Machine Manager and set VM:

 		- Create a new VM: File > New Virtual Machine

 		- Local install media > Forward

 		- Browse for your Windows .iso file, wherever you stored it. I did as follows:
Browse > Browse Local > /home/username/Documents/ISO/Win10_2004_EnglishInternational_x64.iso > Automatically detect from the installation media/source > Forward

				(If it says "The emulator may not have permissions for (...), press Yes)

		- On "Choose Memory and CPU settings" don't touch anything and > Forward (we will be changing them later)

		- Enable Storage for this VM (YES) > Select Custom Storage. Type the following:
 "/dev/disk/by-id/nvme-Sabrent_AA00000A000000000000-part2" > Forward

 				(Change "nvme-Sabrent_AA00000A000000000000-part2" for your main disks ID)

 		- Name the VM as you please (no sapces) and select "customize configuration" > Finish.

 Step 12 - Customize VM to preference (The following steps are recommended. This is how I set it up):

 12.1. Chipset (specially compulsory***)
 Q35 / Firmware: UEFI x86_64 (...) /OVMF.CODE.fd

 12.2. CPU's (specially compulsory***)
-CPU's: As you please
-Host CPU: host-passthrough (handwritten) or EPYC-IBPB (EPYC-IBPB Recommended, by the 23d of June there's a BUG with H-P)
-Topology: Set manually (remember, threads means "thread per core". I have an 8-core 16 thread CPU, so I'll input: Socket 1, -Cores 8, Threads 2)
-Enable available CPU security flaw: (yes)

 12.3. Memory
 As you please. I leave a minimum of 10% for the host, but normally I give it 16Gb out of 32Gb.

 12.4. Boot Options
 Enable Boot Menu (yes) > Select these, and put them as follows, using the arrows:

CDROM1 (Windows)
SATA Disk
CDROM2 (VSIO Drivers)

 12.5. Delete unnecessary devices
Press "- Remove" on: Display Spice, Tablet, Sound ich9, Console, Channel Spice, Video QXL

		- These are the ones I tend to delete, as far as I know little to no one uses these devices (+ passing through Video QXL led me to a bug). If you specifically need one of them for your VM, there is no problem if you pass it through.

 12.6. Install the VirtIO drivers (If you're using them)
Press "+ Add Hardware" > Storage > Select or Create > /home/username/Documents/ISO/virtio-win-0.1.171.iso (or wherever you saved your VirtIO .iso file) > Device Type: CDROM > Bus Type: VirtIO > Finish

 12.7. ADD YOUR PASSED-THROUGH GPU (specially compulsory***)
 +Add Hardware > PCI Host Device > Select the ones which number coincides with your card's ID's from its IOMMU Group (we saved them earlier) > Finish (repeat for all)

 12.8. ADD YOUR PERIPHERALS
 +Add Hardware > USB Host Device > Select the ones (one by one) you want to give your VM access to > Finish

 12.9. ADD YOUR EXTRA DRIVE (If you have one)
 +Add Hardware > Storage > Select or Create > /dev/disk/by-id/ata-ST2000DM008-0AA000_AAA00A0A-part2 > Disk Device > VirtIO > Finish

		- Replace "ata-ST2000DM008-0AA000_AAA00A0A-part2" with your extra-storage-disk's ID.

12.10. Assign your main installation storage as a VirtIO disk (If you're using the drivers)
Disk 1 > Advanced Options > Disk bus > VirtIO > Apply

12.11. [Just for NVIDIA GPU's, AMD is cool about it] (specially compulsory***):

		*NVIDIA sells GPU's specially marketed towards (they're pretty expensive btw, 2000USD+ the ones that I know of) virtualisation, hence them blocking their other GPU's from being able to be used in a VM. So unless you have a V100, a QUADRO RTX 6000 or 8000, a T4, an M10 or a P6 GPU, you'll have to do the following part:

		- Hide from your GPU the fact that its being virtualised:

 Overview > XML

		- Paste the following, replacing the previous part in it's place, from <hyperv> to </hyperv> .

<hyperv>
    <relaxed state="on"/>
    <vapic state="on"/>
    <spinlocks state="on" retries="8191"/>
    <vpindex state="on"/>
    <synic state="on"/>
    <stimer state="on"/>
    <vendor_id state="on" value="0123456789ab"/>
    <frequencies state="on"/>
</hyperv>
<kvm>
    <hidden state="on"/>
</kvm>

 12.12. Apply everything from before
 *** If you aren't using two displays (instead you're using two ports of the same screen), this is the moment to unplug your host's HDMI from the host GPU, and plug it into your guests GPU. After it gives visual output, you can plug it back in again. I recommend leaving the pointer on top of "Begin Installation", so you dont have to guess where to press.

 12.13. Begin Installation.