From 31726b3db9537fd1a19ce4e6d8c948d9f696d684 Mon Sep 17 00:00:00 2001 From: Vaibhav Srivastav Date: Tue, 10 Dec 2024 14:24:00 +0100 Subject: [PATCH 1/5] Update Ollama docs. --- docs/hub/ollama.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/hub/ollama.md b/docs/hub/ollama.md index 48fd1892c..1710e3e5d 100644 --- a/docs/hub/ollama.md +++ b/docs/hub/ollama.md @@ -2,6 +2,8 @@ ![cover](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/ollama/cover.png) +🆕 You can now also run private GGUFs from the Hugging Face Hub. + Ollama is an application based on llama.cpp to interact with LLMs directly through your computer. You can use any GGUF quants created by the community ([bartowski](https://huggingface.co/bartowski), [MaziyarPanahi](https://huggingface.co/MaziyarPanahi) and [many more](https://huggingface.co/models?pipeline_tag=text-generation&library=gguf&sort=trending)) on Hugging Face directly with Ollama, without creating a new `Modelfile`. At the time of writing there are 45K public GGUF checkpoints on the Hub, you can run any of them with a single `ollama run` command. We also provide customisations like choosing quantization type, system prompt and more to improve your overall experience. Getting started is as simple as: @@ -85,6 +87,13 @@ You can optionally configure a system prompt by putting it into a new file named To change sampling parameters, create a file named `params` in the repository. The file must be in JSON format. For the list of all available parameters, please refer to [this documentation](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter). +## Run Private GGUFs from the Hugging Face Hub + +You can run private GGUFs from your personal account or from an associated organisation account in two simple steps: + +1. Copy your Ollama SSH key you can do so via: `cat ~/.ollama/id_ed25519.pub | pbcopy` +2. Add the corresponding key to your Hugging Face account by going to [your account settings](https://huggingface.co/settings/keys) and clicking on `Add new key`. +3. That's it! You can now run private GGUFs from the Hugging Face Hub: `Ollama run hf.co/{username}/{repository}`. ## References From 78a2d69e80cb867945a71ad007693910f4a7cdba Mon Sep 17 00:00:00 2001 From: Vaibhav Srivastav Date: Tue, 10 Dec 2024 16:36:55 +0100 Subject: [PATCH 2/5] Add huggingface-cli login. --- docs/hub/ollama.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/hub/ollama.md b/docs/hub/ollama.md index 1710e3e5d..46d452c66 100644 --- a/docs/hub/ollama.md +++ b/docs/hub/ollama.md @@ -89,11 +89,12 @@ To change sampling parameters, create a file named `params` in the repository. T ## Run Private GGUFs from the Hugging Face Hub -You can run private GGUFs from your personal account or from an associated organisation account in two simple steps: +You can run private GGUFs from your personal account or from an associated organisation account in three simple steps: 1. Copy your Ollama SSH key you can do so via: `cat ~/.ollama/id_ed25519.pub | pbcopy` 2. Add the corresponding key to your Hugging Face account by going to [your account settings](https://huggingface.co/settings/keys) and clicking on `Add new key`. -3. That's it! You can now run private GGUFs from the Hugging Face Hub: `Ollama run hf.co/{username}/{repository}`. +3. On terminal, run `huggingface-cli login` to authenticate with your Hugging Face account. +4. That's it! You can now run private GGUFs from the Hugging Face Hub: `Ollama run hf.co/{username}/{repository}`. ## References From 3e8581d1a95ec62a42c06f29f6a743631a935530 Mon Sep 17 00:00:00 2001 From: Vaibhav Srivastav Date: Tue, 10 Dec 2024 17:04:34 +0100 Subject: [PATCH 3/5] suggestions from review. --- docs/hub/ollama.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/hub/ollama.md b/docs/hub/ollama.md index 46d452c66..489dfc2ce 100644 --- a/docs/hub/ollama.md +++ b/docs/hub/ollama.md @@ -89,12 +89,11 @@ To change sampling parameters, create a file named `params` in the repository. T ## Run Private GGUFs from the Hugging Face Hub -You can run private GGUFs from your personal account or from an associated organisation account in three simple steps: +You can run private GGUFs from your personal account or from an associated organisation account in two simple steps: 1. Copy your Ollama SSH key you can do so via: `cat ~/.ollama/id_ed25519.pub | pbcopy` 2. Add the corresponding key to your Hugging Face account by going to [your account settings](https://huggingface.co/settings/keys) and clicking on `Add new key`. -3. On terminal, run `huggingface-cli login` to authenticate with your Hugging Face account. -4. That's it! You can now run private GGUFs from the Hugging Face Hub: `Ollama run hf.co/{username}/{repository}`. +3. That's it! You can now run private GGUFs from the Hugging Face Hub: `ollama run hf.co/{username}/{repository}`. ## References From 8d513b6504e7510bbce5fc6a82a097ec9548dd54 Mon Sep 17 00:00:00 2001 From: vb Date: Tue, 10 Dec 2024 17:04:55 +0100 Subject: [PATCH 4/5] Update docs/hub/ollama.md Co-authored-by: Julien Chaumond --- docs/hub/ollama.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hub/ollama.md b/docs/hub/ollama.md index 489dfc2ce..c8b0d107e 100644 --- a/docs/hub/ollama.md +++ b/docs/hub/ollama.md @@ -91,7 +91,7 @@ To change sampling parameters, create a file named `params` in the repository. T You can run private GGUFs from your personal account or from an associated organisation account in two simple steps: -1. Copy your Ollama SSH key you can do so via: `cat ~/.ollama/id_ed25519.pub | pbcopy` +1. Copy your Ollama SSH key, you can do so via: `cat ~/.ollama/id_ed25519.pub | pbcopy` 2. Add the corresponding key to your Hugging Face account by going to [your account settings](https://huggingface.co/settings/keys) and clicking on `Add new key`. 3. That's it! You can now run private GGUFs from the Hugging Face Hub: `ollama run hf.co/{username}/{repository}`. From dfd77e7e8db2d5f61e31077741f7db12454512ed Mon Sep 17 00:00:00 2001 From: vb Date: Tue, 10 Dec 2024 17:07:37 +0100 Subject: [PATCH 5/5] Update docs/hub/ollama.md Co-authored-by: Julien Chaumond --- docs/hub/ollama.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hub/ollama.md b/docs/hub/ollama.md index c8b0d107e..0f736653e 100644 --- a/docs/hub/ollama.md +++ b/docs/hub/ollama.md @@ -92,7 +92,7 @@ To change sampling parameters, create a file named `params` in the repository. T You can run private GGUFs from your personal account or from an associated organisation account in two simple steps: 1. Copy your Ollama SSH key, you can do so via: `cat ~/.ollama/id_ed25519.pub | pbcopy` -2. Add the corresponding key to your Hugging Face account by going to [your account settings](https://huggingface.co/settings/keys) and clicking on `Add new key`. +2. Add the corresponding key to your Hugging Face account by going to [your account settings](https://huggingface.co/settings/keys) and clicking on `Add new SSH key`. 3. That's it! You can now run private GGUFs from the Hugging Face Hub: `ollama run hf.co/{username}/{repository}`. ## References