Best Design Implementation Question

[LordNex]

So I just finished upgrading my network to a proper enterprise design with a Firewalla Gold and an Aruba LAN/WLAN setup with all my traffic finally neatly tucked away in proper VLANs. I also picked up Dell PowerEdge R620 with a 40 core @3.0ghz each and 256gig of ram. It's now my main server running VMWare ESXi 7 with about 5-6 various guest servers running in it.

Im getting ready to put back in my camera facial recognition system but im wondering if I should leave it the way I have it or leverage the new server. My old setup was this

1. Home Assistant running MQTT and DoubleTake on the main server

2. Frigate running in Docker Compose on a RPI4 8gig with a Coral TPU USB Dongle for Base a object Detection and NVR

3. DeepStack running on a nVidia Jetson Nano 4gig for Facial Model Recognition

Do you think it would be better to put the TPU on the PowerEdge R620, pass it through to a new install of Ubuntu Server 20 LTS and Frigate running in Docker Compose along with all the other servers? Or should I stick with leveraging the RPI4 8gig for ease of use and all the benefits of a self contained unit?