Llm fine tune (#109)

* feat: 💬 Chatbot for demos

* feat: 📚 Dataset generation

* feat: ⏫ Dataset augmentation

* doc: 📝 add some comments

* fix: 🐛 JSON schema validation

* doc: 📝 update readme with dataset scripts

* feat: ✨ Notebook

* fix: 🐛 end command (back slash)

* fix: 🐛 path for dataset

* fix: 🐛 change framework naming

* conf: ⚙️ fix dependencies versions

* docs: 📝 fix some typos

* fix: 🐛 fix some paths

* fix: 🐛 Some typos

* fix: 🐛 Remove explicite ID from W&B configuration

Author: philippart-s

.gitignore

@@ -55,3 +55,7 @@ kustomize
 containers-orchestration/managed-rancher/create-rancher-with-tf/variables.tf
 use-cases/create-and-use-object-storage-as-tf-backend/my-app/backend.tf
 use-cases/create-and-use-object-storage-as-tf-backend/object-storage-tf/variables.tf
+
+# LLM Fine Tune data
+ai/llm-fine-tune/dataset/docs/*
+ai/llm-fine-tune/dataset/generated/*

ai/llm-fine-tune/README.md

@@ -0,0 +1,65 @@
+# 🎯 What is the goals of this example ? 🎯
+
+This example shows you how it's very simple to fine tune a LLM with the [axolotl](https://docs.axolotl.ai/) Framework and OVHcloud [Machine Learning Services](https://www.ovhcloud.com/fr/public-cloud/ai-machine-learning/).
+
+## 📚 Prerequisites 📚
+
+ - An OVHcloud [public cloud project created](https://help.ovhcloud.com/csm/en-ie-public-cloud-compute-essential-information?id=kb_article_view&sysparm_article=KB0050387)
+ - An OVHcloud [AI Endpoints valid API Key](https://help.ovhcloud.com/csm/en-ie-public-cloud-ai-endpoints-getting-started?id=kb_article_view&sysparm_article=KB0065398) stored in an environment variable named `OVH_AI_ENDPOINTS_ACCESS_TOKEN`
+ - A valid AI Endpoint model URL stored in an environment variable named `OVH_AI_ENDPOINTS_MODEL_URL`
+ - A valid AI Endpoint model name stored in an environment variable named `OVH_AI_ENDPOINTS_MODEL_NAME`
+ - A [Hugging Face](https://huggingface.co/) account with a valid API Key
+ - Optional: 
+  - a valid Python installation
+  - a valid Docker installation
+
+## 💬 The chatbot 🤖
+
+To test the created models, you can use the chatbot in the [chatbot](./chatbot) folder.
+**⚠️ It's a simple chatbot for testing purpose only, not for real production 😉 ⚠️**
+
+The chatbot is packaged with Docker and can be built with the provided [Dockerfile](./chatbot/Dockerfile): `cd ./chatbot && docker buildx build --platform="linux/amd64"  -t <id>/fine-tune-chatbot:1.0.0 .`
+You can run the chatbot using:
+  - your local Python installation: `cd ./chatbot && pip install -r requirements.txt && python ./chatbot/chatbot.py`
+  - your local Docker installation: `cd ./chatbot && docker run -p 7860:7860 <id>/fine-tune-chatbot:1.0.0 .`
+  - using [OVHcloud AI Deploy](https://www.ovhcloud.com/fr/public-cloud/ai-deploy/):
+```bash
+ovhai app run \
+    --name fine-tune-chatbot \
+    --cpu 1 \
+    --default-http-port 7860 \
+    --env OVH_AI_ENDPOINTS_ACCESS_TOKEN=$OVH_AI_ENDPOINTS_ACCESS_TOKEN \
+    --unsecure-http \
+    my-id/fine-tune-chatbot:1.0.0
+```
+
+And you can access the chatbot by navigating to `http://127.0.0.1:7860` or using the public URL provided by OVHcloud AI Deploy.
+
+## 📚 The data generation 📚
+
+To train the model you need data.
+Data are generated from the OVHcloud AI Endpoints [official documentation](https://help.ovhcloud.com/csm/en-gb-documentation-public-cloud-ai-and-machine-learning-ai-endpoints?id=kb_browse_cat&kb_id=574a8325551974502d4c6e78b7421938&kb_category=ea1d6daa918a1a541e11d3d71f8624aa&spa=1).
+
+You have two Python scripts:
+  - one to generate valide dataset from the markdown documentation: [DatasetCreation.py](./dataset/DatasetCreation.py)
+  - one to generate synthetic data from the previous generated documentation: [DatasetAugmentation.py](./dataset/DatasetAugmentation.py) 
+
+Once you have set the environment variables (see Prerequisites section) you can run the scripts with Python : `python DatasetCreation.py`
+
+## 🏋️‍♀️ Train the model 🏋
+
+You have to create a notebook thanks to `ovhai` CLI:
+```bash
+ovhai notebook run conda jupyterlab \
+	--name axolto-llm-fine-tune \
+	--framework-version 25.3.1-py312-cudadevel128-gpu \
+	--flavor l4-1-gpu \
+	--gpu 1 \
+  --volume https://github.com/ovh/public-cloud-examples.git:/workspace/public-cloud-examples:RW \
+	--envvar HF_TOKEN=$MY_HF_TOKEN \
+	--envvar WANDB_TOKEN=$MY_WANDB_TOKEN \
+	--unsecure-http
+```
+
+To train the model please follow the steps in the [notebook](./notebook/axolto-llm-fine-tune-Meta-Llama-3.2-1B-instruct-ai-endpoints.ipynb) provided in the [notebook](./notebook/) folder.  
+You have to upload the previously generated data in the [ai-endpoints-doc](./notebook/ai-endpoints-doc/) folder.

ai/llm-fine-tune/chatbot/Dockerfile

@@ -0,0 +1,25 @@
+FROM python:3.13-slim
+
+# 📂 Working directory in the container 📂
+WORKDIR /workspace
+
+# 🐍 Copy files in /workspace 🐍
+COPY . /workspace
+
+# ⬇️ Install any needed packages specified in requirements.txt ⬇️
+RUN pip install --no-cache-dir -r requirements.txt
+
+# 🔐 Change ownership of the workspace directory to the user with UID 42420 (OVHcloud user) 🔐
+RUN chown -R 42420:42420 /workspace
+
+# ⚙️ Make port 7860 available ⚙️
+EXPOSE 7860
+
+# ⚙️ Gradio configuration to run on localhost ⚙️
+ENV GRADIO_SERVER_NAME="0.0.0.0"
+
+# 🔐 Define default value for AI Endpoints API key 🔐
+ENV OVH_AI_ENDPOINTS_ACCESS_TOKEN=$OVH_AI_ENDPOINTS_ACCESS_TOKEN
+
+# ⚡️ Run chatbot.py when the container launches ⚡️
+CMD ["python", "chatbot.py"]

ai/llm-fine-tune/chatbot/chatbot.py

@@ -0,0 +1,63 @@
+# Application to compare answers generation from OVHcloud AI Endpoints exposed model and fine tuned model.
+# ⚠️ Do not used in production!! ⚠️
+
+import gradio as gr
+import os
+
+from langchain_openai import ChatOpenAI
+from langchain_core.prompts import ChatPromptTemplate
+
+# 📜 Prompts templates 📜
+prompt_template = ChatPromptTemplate.from_messages(
+    [
+        ("system", "{system_prompt}"),
+        ("human", "{user_prompt}"),
+    ]
+)
+
+def chat(prompt, system_prompt, temperature, top_p, model_name, model_url, api_key):
+    """
+    Function to generate a chat response using the provided prompt, system prompt, temperature, top_p, model name, model URL and API key.
+    """
+
+    # ⚙️ Initialize the OpenAI model ⚙️
+    llm = ChatOpenAI(api_key=api_key, 
+                 model=model_name, 
+                 base_url=model_url,
+                 temperature=temperature,
+                 top_p=top_p
+                 )
+
+    # 📜 Apply the prompt to the model 📜
+    chain = prompt_template | llm
+    ai_msg = chain.invoke(
+        {
+            "system_prompt": system_prompt,
+            "user_prompt": prompt
+        }
+    )
+
+    # 🤖 Return answer in a compatible format for Gradio component.
+    return [{"role": "user", "content": prompt}, {"role": "assistant", "content": ai_msg.content}]
+
+# 🖥️ Main application 🖥️
+with gr.Blocks() as demo:
+    with gr.Row():
+        with gr.Column():
+            system_prompt = gr.Textbox(value="""You are a specialist on OVHcloud products.
+If you can't find any sure and relevant information about the product asked, answer with "This product doesn't exist in OVHcloud""", 
+                label="🧑‍🏫 System Prompt 🧑‍🏫")
+            temperature = gr.Slider(minimum=0.0, maximum=2.0, step=0.01, label="Temperature", value=0.5)
+            top_p = gr.Slider(minimum=0.0, maximum=1.0, step=0.01, label="Top P", value=0.0)
+            model_name = gr.Textbox(label="🧠 Model Name 🧠", value='Llama-3.1-8B-Instruct')
+            model_url = gr.Textbox(label="🔗 Model URL 🔗", value='https://oai.endpoints.kepler.ai.cloud.ovh.net/v1')
+            api_key = gr.Textbox(label="🔑 OVH AI Endpoints Access Token 🔑", value=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN"), type="password")
+
+        with gr.Column():
+            chatbot = gr.Chatbot(type="messages", label="🤖 Response 🤖")
+            prompt = gr.Textbox(label="📝 Prompt 📝", value='How many requests by minutes can I do with AI Endpoints?')
+            submit = gr.Button("Submit")
+
+    submit.click(chat, inputs=[prompt, system_prompt, temperature, top_p, model_name, model_url, api_key], outputs=chatbot)
+
+demo.launch()

ai/llm-fine-tune/chatbot/requirements.txt

@@ -0,0 +1,4 @@
+gradio==5.38.0
+langchain-openai==0.3.28
+langchain-core==0.3.69
+langchain==0.3.26

ai/llm-fine-tune/dataset/DatasetAugmentation.py

@@ -0,0 +1,110 @@
+import os
+import json
+import uuid
+from pathlib import Path
+from langchain_openai import ChatOpenAI
+from langchain.schema import HumanMessage
+from jsonschema import validate, ValidationError
+
+# 🗺️ Define the JSON schema for the response 🗺️
+message_schema = {
+    "type": "object",
+    "properties": {
+        "role": {"type": "string"},
+        "content": {"type": "string"}
+    },
+    "required": ["role", "content"]
+}
+
+response_format = {
+    "type": "json_object",
+    "json_schema": {
+        "name": "Messages",
+        "description": "A list of messages with role and content",
+        "properties": {
+            "messages": {
+                "type": "array",
+                "items": message_schema
+            }
+        }
+    }
+}
+
+# ✅ JSON validity verification ❌
+def is_valid(json_data):
+    """
+    Test the validity of the JSON data against the schema.
+    Argument:
+        json_data (dict): The JSON data to validate.  
+    Raises:
+        ValidationError: If the JSON data does not conform to the specified schema.  
+    """
+    try:
+        validate(instance=json_data, schema=response_format["json_schema"])
+        return True
+    except ValidationError as e:
+        print(f"❌ Validation error: {e}")
+        return False
+
+# ⚙️ Initialize the chat model with AI Endpoints configuration ⚙️
+chat_model = ChatOpenAI(
+    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN"),
+    base_url=os.getenv("OVH_AI_ENDPOINTS_MODEL_URL"),
+    model_name=os.getenv("OVH_AI_ENDPOINTS_MODEL_NAME"),
+    temperature=0.0
+)
+
+# 📂 Define the directory path 📂
+directory_path = "generated"
+print(f"📂 Directory path: {directory_path}")
+directory = Path(directory_path)
+
+# 🗃️ Walk through the directory and its subdirectories 🗃️
+for path in directory.rglob("*"):
+    print(f"📜 Processing file: {path}")
+    # Check if the current path is a valid file
+    if path.is_file() and path.name.__contains__ ("endpoints"):
+        # Read the raw data from the file
+        with open(path, 'r', encoding='utf-8') as file:
+            raw_data = file.read()
+
+        try:
+            json_data = json.loads(raw_data)
+        except json.JSONDecodeError:
+            print(f"❌ Failed to decode JSON from file: {path.name}")
+            continue
+
+        if not is_valid(json_data):
+            print(f"❌ Dataset non valide: {path.name}")
+            continue
+        print(f"✅ Input dataset valide: {path.name}")
+
+        user_message = HumanMessage(content=f"""
+        Given the following JSON, generate a similar JSON file where you paraphrase each question in the content attribute
+        (when the role attribute is user) and also paraphrase the value of the response to the question stored in the content attribute
+        when the role attribute is assistant.
+        The objective is to create synthetic datasets based on existing datasets.
+        I do not need to know the code to do this, but I want the resulting JSON file.
+        It is important that the term OVHcloud is present as much as possible, especially when the terms AI Endpoints are mentioned
+        either in the question or in the response.
+        There must always be a question followed by an answer, never two questions or two answers in a row.
+        It is IMPERATIVE to keep the language in English.
+        The source JSON file:
+        {raw_data}
+        """)
+
+        chat_response = chat_model.invoke([user_message], response_format=response_format)
+
+        output = chat_response.content
+
+        # Replace unauthorized characters
+        output = output.replace("\\t", " ")
+
+        generated_file_name = f"{uuid.uuid4()}_{path.name}"
+        with open(f"./generated/synthetic/{generated_file_name}", 'w', encoding='utf-8') as output_file:
+            output_file.write(output)
+
+        if not is_valid(json.loads(output)):
+            print(f"❌ ERROR: File {generated_file_name} is not valid")
+        else:
+            print(f"✅ Successfully generated file: {generated_file_name}")

ai/llm-fine-tune/dataset/DatasetCreation.py

@@ -0,0 +1,75 @@
+import os
+from pathlib import Path
+from langchain_openai import ChatOpenAI
+from langchain.schema import HumanMessage
+
+# 🗺️ Define the JSON schema for the response 🗺️
+message_schema = {
+    "type": "object",
+    "properties": {
+        "role": {"type": "string"},
+        "content": {"type": "string"}
+    },
+    "required": ["role", "content"]
+}
+
+response_format = {
+    "type": "json_object",
+    "json_schema": {
+        "name": "Messages",
+        "description": "A list of messages with role and content",
+        "properties": {
+            "messages": {
+                "type": "array",
+                "items": message_schema
+            }
+        }
+    }
+}
+
+# ⚙️ Initialize the chat model with AI Endpoints configuration ⚙️
+chat_model = ChatOpenAI(
+    api_key=os.getenv("OVH_AI_ENDPOINTS_ACCESS_TOKEN"),
+    base_url=os.getenv("OVH_AI_ENDPOINTS_MODEL_URL"),
+    model_name=os.getenv("OVH_AI_ENDPOINTS_MODEL_NAME"),
+    temperature=0.0
+)
+
+# 📂 Define the directory path 📂
+directory_path = "docs/pages/public_cloud/ai_machine_learning"
+directory = Path(directory_path)
+
+# 🗃️ Walk through the directory and its subdirectories 🗃️
+for path in directory.rglob("*"):
+    # Check if the current path is a directory
+    if path.is_dir():
+        # Get the name of the subdirectory
+        sub_directory = path.name
+
+        # Construct the path to the "guide.en-gb.md" file in the subdirectory
+        guide_file_path = path / "guide.en-gb.md"
+
+        # Check if the "guide.en-gb.md" file exists in the subdirectory
+        if "endpoints" in sub_directory and guide_file_path.exists():
+            print(f"📗 Guide processed: {sub_directory}")
+            with open(guide_file_path, 'r', encoding='utf-8') as file:
+                raw_data = file.read()
+
+            user_message = HumanMessage(content=f"""
+With the markdown following, generate a JSON file composed as follows: a list named "messages" composed of tuples with a key "role" which can have the value "user" when it's the question and "assistant" when it's the response. To split the document, base it on the markdown chapter titles to create the question, seems like a good idea.
+Keep the language English.
+I don't need to know the code to do it but I want the JSON result file.
+For the "user" field, don't just repeat the title but make a real question, for example "What are the requirements for OVHcloud AI Endpoints?"
+Be sure to add OVHcloud with AI Endpoints so that it's clear that OVHcloud creates AI Endpoints.
+Generate the entire JSON file.
+An example of what it should look like: messages [{{"role":"user", "content":"What is AI Endpoints?"}}]
+There must always be a question followed by an answer, never two questions or two answers in a row.
+The source markdown file:
+{raw_data}
+""")
+            chat_response = chat_model.invoke([user_message], response_format=response_format)
+            
+            with open(f"./generated/{sub_directory}.json", 'w', encoding='utf-8') as output_file:
+                output_file.write(chat_response.content)
+                print(f"✅ Dataset generated: ./generated/{sub_directory}.json")
+