How do I get this working with google colab?

[03stevensmi]

@SayanoAI I can not select custom gguf models using the app demo!
And I currently don't own anything powerful enough to run this locally.

Only thing I own is a lenovo legion 5 laptop with
a processor that is AMD Ryzen 5 4600H (6C / 12T, 3.0 / 4.0GHz, 3MB L2 / 8MB L3),
a dedicated gpu that is NVIDIA GeForce GTX 1650 4GB GDDR6,
along with the memory being 2x 8GB SO-DIMM DDR4-3200...
and integrated AMD Radeon Graphics!

However... if this is powerful enough to run this in real-time with under 2min response time... please let me know.
(I'm using Linux mint, not windows if that matters).

If not, then how can I successfully run this in google colab with a server gui?
(The same way to how SoniTranslate does this kind of stuff):
https://github.com/R3gm/SoniTranslate/blob/main/SoniTranslate_Colab.ipynb

Please if you know how accomplish this, or if you can help with info...
Please let me know!

Thank You,
03stevensmi

[03stevensmi]

Anyone?
I even if there's someone here that can tell me whether or not
if I can run this locally using the specs mentioned above in under 2min?
That will be very helpful, but I need to know what is possible and or what's not first!

[SayanoAI]

You probably won't be able to run the latest 7B+ models without offloading to RAM. The included quantized 3B model should work for you, although you probably can't run comfyui (the image generation) at the same time...