[Observers] Refactor for better FP4 support, static observers#1903

kylesayrs wants to merge 11 commits intomainvllm-project/llm-compressor:mainfrom

kylesayrs/observers-refactorvllm-project/llm-compressor:kylesayrs/observers-refactorCopy head branch name to clipboard