Is it possible to create an NF4 Model based on a GGUF model?

[softtaco1]

I’m genuinely not certain if it’s even possible, so I’m being literal when I ask: Could a NF4 model be made from a GGUF model? Would there be any benefit to it? Would it further decrease model size and VRAM requirements while still retaining enough information to be used effectively?

[pflky]

NF4 is performance optimizations, while GGUF is compression. The better method would be to first make an NF4 model, then turn it into a GGUF model, and have a system that recognizes how to handle both methods when used together.