AMD AMF Encoder freeze frames/lag spikes (v0.21.0)

[Icyberg2]

I'd end up starting with a black screen on my stream when connecting from a client device, until I either move my mouse or unless there is sound playing. (both on Android, then as well as a few friends trying to connect for some multiplayer stuff from Windows desktops.) It even gets to a point where if I'm running an emulator (Yuzu in this case), it just suddenly freezes the stream once a game is loaded and drops all my frames by 80% or higher, if it's not still frozen. For reference, here are my specs.

CPU: AMD Ryzen 7 3700X
RAM: 32GB DDR4-3200MHz
GPU: AMD Radeon RX 5700 XT
DISPLAY: 1920x1080 @144hz connected via DP (DisplayPort), clients run at 1080p 60fps and the results do not change based on resolution after extended testing.

I chalked it up to it being some sort of incompatibility with emulation, but then I recall it also happened with a few of my Steam games as well. This only happens with the AMD encoder selected, there are fewer issues using Software encoding (still gets freeze frames/lag spikes at times), but if its possible to encode via hardware, I would prefer that if possible (considering it also lowers some latency from what I saw on performance stats). My AMD AMF encoder settings go like this:

AMF Quality: Speed -- prefer speed
AMF Rate Control: cbr -- constant bitrate
AMF Usage: lowlatency - low latency (fast) (swapped from ultralowlatency thinking it would fix it, but to no avail.)
AMF Preanalysis: disabled
AMF VBAQ: enabled
AMF Coder (h264): auto