Llama open source?

[avsk53]

I know Llama 3 is open source, but when I go to websites like replicate, it shows a price per 1 million tokens if I want to use that API. So I don't think I fully understand the concept. I am trying to develop an application that uses Llama 3, but I want to grasp the foundational knowledge first. Could someone explain this to me? And what kind of costs would be involved if I want to use llama for my application? (after calculation, per user is expected to use 15-20k tokens per day)

[phymbert]

Here is llama.cpp: an llm serving framework and http server written in c++ and licensed by MIT.
In other words if you manage to deploy it on the infrastructure for your needs, it's free.

[avsk53]

Thanks for taking the time to clear this up, but would you be able to explain it in a bit more detail? Like what exactly does Replicate do and why do they charge a price per token and how does it end up being free if I use llama.cpp? (I'm assuming there's an advantage in paying for Replicate, just trying to ascertain what that is)

[phymbert]

https://en.m.wikipedia.org/wiki/Software_as_a_service

[avsk53]

I'm well aware of what SaaS is. If you're unable to answer the specifics of my question, feel free to not respond.

[phymbert]

If you are well aware, how come you can ask this question ? they are offering LLM serving as a service. llama.cpp is a library. llama is a model.