You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/02_setting_up_the_instance.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ weight: 4
6
6
layout: learningpathall
7
7
---
8
8
9
-
In this step, you'll set up the Graviton4 instance with the tools and dependencies required to build and run the Arcee Foundation Model. This includes installing system packages and a Python environment.
9
+
In this step, you'll set up the Graviton4 instance with the tools and dependencies required to build and run the AFM-4.5B model. This includes installing system packages and a Python environment.
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/03_building_llama_cpp.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,7 @@ layout: learningpathall
7
7
---
8
8
## Build the Llama.cpp inference engine
9
9
10
-
In this step, you'll build Llama.cpp from source. Llama.cpp is a high-performance C++ implementation of the LLaMA model, optimized for inference on a range of hardware platforms,including Arm-based processors like AWS Graviton4.
10
+
In this step, you'll build Llama.cpp from source. Llama.cpp is a high-performance C++ implementation of the LLaMA model, optimized for inference on a range of hardware platforms,including Arm-based processors like AWS Graviton4.
11
11
12
12
Even though AFM-4.5B uses a custom model architecture, you can still use the standard Llama.cpp repository - Arcee AI has contributed the necessary modeling code upstream.
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/04_install_python_dependencies_for_llama_cpp.md
+3-2Lines changed: 3 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,7 +32,7 @@ This command does the following:
32
32
33
33
- Runs the activation script, which modifies your shell environment
34
34
- Updates your shell prompt to show `env-llama-cpp`, indicating the environment is active
35
-
- Updates `PATH` to use so the environment’s Python interpreter
35
+
- Updates `PATH` to use the environment’s Python interpreter
36
36
- Ensures all `pip` commands install packages into the isolated environment
37
37
38
38
## Upgrade pip to the latest version
@@ -72,7 +72,8 @@ After the installation completes, your virtual environment includes:
72
72
-**NumPy**: for numerical computations and array operations
73
73
-**Requests**: for HTTP operations and API calls
74
74
-**Other dependencies**: additional packages required by llama.cpp's Python bindings and utilities
75
-
Your environment is now ready to run Python scripts that integrate with the compiled Llama.cpp binaries
75
+
76
+
Your environment is now ready to run Python scripts that integrate with the compiled Llama.cpp binaries.
76
77
77
78
{{< notice Tip >}}
78
79
Before running any Python commands, make sure your virtual environment is activated. {{< /notice >}}
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/05_downloading_and_optimizing_afm45b.md
+6-5Lines changed: 6 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,8 @@ layout: learningpathall
8
8
9
9
In this step, you’ll download the [AFM-4.5B](https://huggingface.co/arcee-ai/AFM-4.5B) model from Hugging Face, convert it to the GGUF format for compatibility with `llama.cpp`, and generate quantized versions to optimize memory usage and improve inference speed.
10
10
11
-
**Note: if you want to skip the model optimization process, [GGUF](https://huggingface.co/arcee-ai/AFM-4.5B-GGUF) versions are available.**
11
+
{{% notice Note %}}
12
+
If you want to skip the model optimization process, [GGUF](https://huggingface.co/arcee-ai/AFM-4.5B-GGUF) versions are available. {{% /notice %}}
12
13
13
14
Make sure to activate your virtual environment before running any commands. The instructions below walk you through downloading and preparing the model for efficient use on AWS Graviton4.
0 commit comments