GPU??

[bhupesh-sf]

This is cool idea, right now in the screenshot I only see CPU but not GPU. Is there any specific reason for that. Because more and more LLMs are being used on consumer machines GPU usage and even power consumption makes important metric to show.

Your thoughts?

[arinbjornk]

This is a good point. The simple answer is that GPU information needs to be handled differently based on the manufacturer (NVIDIA, AMD, Intel, etc) since each has their own proprietary APIs and monitoring tools:

• NVIDIA GPUs use NVML (NVIDIA Management Library)
• AMD GPUs use ROCm/HIP APIs
• Intel GPUs use oneAPI Level Zero

This makes it more complex than CPU monitoring which has standardised interfaces. The next step would be implementing support for the most common GPU vendors, starting with NVIDIA, followed by AMD and Intel.

Coming soon!

[bhupesh-sf]

ah understood. BTW I am apple silicon, not sure if that also counts somewhere :)