Setting up a PXE boot environment - Part 0

Upon learning that Raspberry Pi 3 and above has PXE boot capabilities, I pulled out my dust gathering, poor Raspberry Pi 3B... My motive is NOT to re-image that MicroSD card every time I want to tinker stuff, simply plug-in, power up, and viola, at least ideally.

Following the guidelines from various sources:
https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/net_tutorial.md  https://help.ubuntu.com/community/DisklessUbuntuHowto

Note that in the second howto, replace KERNEL with DEFAULT, remove APPEND and put the content of those 2 line in to one:

KERNEL DEFAULT vmlinuz-4.4.0-139-generic APPEND root=/dev/nfs ....

My restrictions:
a. I already have a router, which is a nice Ubiquiti USG-PRO-4, and it has support of PXE.
b. I don't want to use 2 Raspberry Pis, i want to setup the DHCP and TFTP on the Ubuntu VM.

I have encountered a few roadblocks initially...

1. Setup a DHCP server without interfering with my router (Ubiquiti USG-Pro-4)
2. Serve different boot config to different hosts, by their MAC addresses
3. Serve different bootloader to different hosts, by their MAC addresses

Problem 1 was solved swiftly (seemingly, maybe it'll screw my network after previous DHCP lease expiring?) from installing the isc-dhcp-server in the second howto. but at least I set it up to only work for certain MAC addresses.

Problem 2 should be fixed by the config itself, at least from this document. I have not test this yet.

Problem 3 is the one that I am struggling now, as the howto is aimed at x86 hosts, which Raspberry Pi is not. I presume it should be easily configured if the isc-dhcp-server is the master dhcp server for my home network. Or I pull out another Raspberry Pi and strictly follow the first howto, or I figure out how to use the advance DHCP options in the router.

My thought from setting these things:

1. PXE boot would speed up spinning up new metal hosts, nowadays maybe they are used for setting up Kubernetes Nodes as we move to a docker world?

2. In real world applications, if you need to setup PXE, you probably should also have a dedicated DHCP/BOOTP server, so it is easier to configure it without messing with your existing networking.

3. Also, I believe it is unlikely that in a real world to have a PXE to support heterogeneous hardware architecture as it would complicate things and make sysadmin's life harder?

On a side note, that Ubuntu VM that runs in bhyve in my 3 years old FreeNAS, is starting to look like a single point of failure.

(Developing...)