Merge vLLM deployer project to llm-finetuning (#163)

* use the official huggingface integration

* add option to deploy to vllm

* add vllm deployer step

* fix readme and add deployment target option

* test config

* test

* fix vllm integration

* rm llm-vllm project

* uncomment config file

* Update dependencies and GCP library versions in requirements.txt

* stop overwriting the config for dataset generation

* Configure Hugging Face cache directories for dataset preparation

* Update dataset name from htahir1 to zenml namespace in configuration files

* Update README with ZenML namespace dataset and repository links

* change hf repo to zenml

* use uv

* fix syntax

* update deprecated log metadata command

---------

Co-authored-by: Alex Strick van Linschoten <[email protected]>
Co-authored-by: Alex Strick van Linschoten <[email protected]>

Author: wjayesh

databricks-production-qa-demo/steps/deployment/deployment_deploy.py

@@ -65,7 +65,7 @@ def deployment_deploy() -> (
     model_deployer = zenml_client.active_stack.model_deployer
     databricks_deployment_config = DatabricksDeploymentConfig(
         model_name=model.name,
-        model_version=model.run_metadata["model_registry_version"].value,
+        model_version=model.run_metadata["model_registry_version"],
         workload_size="Small",
         workload_type="CPU",
         scale_to_zero_enabled=True,

llm-finetuning/README.md

@@ -69,16 +69,16 @@ The three pipelines can be run using the CLI:
 
 ```shell
 # Data generation
-python run.py --feature-engineering --config <NAME_OF_CONFIG_IN_CONFIGS_FOLDER>
-python run.py --feature-engineering --config generate_code_dataset.yaml
+python run.py --feature-pipeline --config <NAME_OF_CONFIG_IN_CONFIGS_FOLDER>
+python run.py --feature-pipeline --config generate_code_dataset.yaml
 
 # Training
 python run.py --training-pipeline --config <NAME_OF_CONFIG_IN_CONFIGS_FOLDER>
 python run.py --training-pipeline --config finetune_gcp.yaml
 
 # Deployment
-python run.py --deployment-pipeline --config <NAME_OF_CONFIG_IN_CONFIGS_FOLDER>
-python run.py --deployment-pipeline --config deployment_a100.yaml
+python run.py --deploy-pipeline --config <NAME_OF_CONFIG_IN_CONFIGS_FOLDER>
+python run.py --deploy-pipeline --config deployment_a100.yaml
 ```
 
 The `feature_engineering` and `deployment` pipeline can be run simply with the `default` stack, but the training pipelines [stack](https://docs.zenml.io/user-guide/production-guide/understand-stacks) will depend on the config.
@@ -127,7 +127,7 @@ python run.py --deployment-pipeline --config deployment_a100.yaml
 
 A working prototype has been trained and deployed as of Jan 19 2024. The model is using minimal data and finetuned using QLoRA and PEFT. The model was trained using 1 A100 GPU on the cloud:
 
-- Training dataset [Link](https://huggingface.co/datasets/htahir1/zenml-codegen-v1)
+- Training dataset [Link](https://huggingface.co/datasets/zenml/zenml-codegen-v1)
 - PEFT Model [Link](https://huggingface.co/htahir1/peft-lora-zencoder15B-personal-copilot/)
 - Fully merged model (Ready to deploy on HuggingFace Inference Endpoints) [Link](https://huggingface.co/htahir1/peft-lora-zencoder15B-personal-copilot-merged)
 
@@ -147,7 +147,7 @@ The [ZenML Pro](https://zenml.io/pro) was used to manage the pipelines, models,
 
 This project recently did a [call of volunteers](https://www.linkedin.com/feed/update/urn:li:activity:7150388250178662400/). This TODO list can serve as a source of collaboration. If you want to work on any of the following, please [create an issue on this repository](https://github.com/zenml-io/zenml-projects/issues) and assign it to yourself!
 
-- [x] Create a functioning data generation pipeline (initial dataset with the core [ZenML repo](https://github.com/zenml-io/zenml) scraped and pushed [here](https://huggingface.co/datasets/htahir1/zenml-codegen-v1))
+- [x] Create a functioning data generation pipeline (initial dataset with the core [ZenML repo](https://github.com/zenml-io/zenml) scraped and pushed [here](https://huggingface.co/datasets/zenml/zenml-codegen-v1))
 - [x] Deploy the model on a HuggingFace inference endpoint and use it in the [VS Code Extension](https://github.com/huggingface/llm-vscode#installation) using a deployment pipeline.
 - [x] Create a functioning training pipeline.
 - [ ] Curate a set of 5-10 repositories that are using the ZenML latest syntax and use data generation pipeline to push dataset to HuggingFace.

llm-finetuning/configs/deployment_a10.yaml

@@ -2,6 +2,7 @@
 settings:
   docker:
     requirements: requirements.txt
+    python_package_installer: "uv"
 
 model:
   name: peft-lora-zencoder15B-personal-copilot

llm-finetuning/configs/deployment_a100.yaml

@@ -2,6 +2,7 @@
 settings:
   docker:
     requirements: requirements.txt
+    python_package_installer: "uv"
 
 model:
   name: "peft-lora-zencoder15B-personal-copilot"

llm-finetuning/configs/deployment_t4.yaml

@@ -1,6 +1,7 @@
 # environment configuration
 settings:
   docker:
+    python_package_installer: "uv"
     requirements: requirements.txt
 
 model:

llm-finetuning/configs/finetune_aws.yaml

@@ -2,11 +2,12 @@
 settings:
   docker:
     requirements: requirements.txt
+    python_package_installer: "uv"
 
 model:
   name: "peft-lora-zencoder15B-personal-copilot"
   description: "Fine-tuned `starcoder15B-personal-copilot-A100-40GB-colab` for ZenML pipelines."
-  audience: "Data Scientists / ML Engineers"  
+  audience: "Data Scientists / ML Engineers"
   use_cases: "Code Generation for ZenML MLOps pipelines."
   limitations: "There is no guarantee that this model will work for your use case. Please test it thoroughly before using it in production."
   trade_offs: "This model is optimized for ZenML pipelines. It is not optimized for other libraries."
@@ -23,13 +24,13 @@ steps:
     step_operator: sagemaker-eu
     settings:
       step_operator.sagemaker:
-        estimator_args: 
+        estimator_args:
           instance_type: "ml.p4d.24xlarge"
 
     parameters:
       args:
         model_path: "bigcode/starcoder"
-        dataset_name: "htahir1/zenml-codegen-v1"
+        dataset_name: "zenml/zenml-codegen-v1"
         subset: "data"
         data_column: "content"
         split: "train"
@@ -58,4 +59,4 @@ steps:
         use_4bit_qunatization: true
         use_nested_quant: true
         bnb_4bit_compute_dtype: "bfloat16"
-        output_peft_repo_id: "htahir1/peft-lora-zencoder15B-personal-copilot"
+        output_peft_repo_id: "zenml/peft-lora-zencoder15B-personal-copilot"

llm-finetuning/configs/finetune_gcp.yaml

@@ -2,11 +2,12 @@
 settings:
   docker:
     requirements: requirements.txt
+    python_package_installer: "uv"
 
 model:
   name: "peft-lora-zencoder15B-personal-copilot"
   description: "Fine-tuned `starcoder15B-personal-copilot-A100-40GB-colab` for ZenML pipelines."
-  audience: "Data Scientists / ML Engineers"  
+  audience: "Data Scientists / ML Engineers"
   use_cases: "Code Generation for ZenML MLOps pipelines."
   limitations: "There is no guarantee that this model will work for your use case. Please test it thoroughly before using it in production."
   trade_offs: "This model is optimized for ZenML pipelines. It is not optimized for other libraries."
@@ -29,7 +30,7 @@ steps:
     parameters:
       args:
         model_path: "bigcode/starcoder"
-        dataset_name: "htahir1/zenml-codegen-v1"
+        dataset_name: "zenml/zenml-codegen-v1"
         subset: "data"
         data_column: "content"
         split: "train"
@@ -58,4 +59,4 @@ steps:
         use_4bit_qunatization: true
         use_nested_quant: true
         bnb_4bit_compute_dtype: "bfloat16"
-        output_peft_repo_id: "htahir1/peft-lora-zencoder15B-personal-copilot"
+        output_peft_repo_id: "zenml/peft-lora-zencoder15B-personal-copilot"

llm-finetuning/configs/finetune_local.yaml

@@ -2,6 +2,7 @@
 settings:
   docker:
     requirements: requirements.txt
+    python_package_installer: "uv"
 
 model:
   name: "peft-lora-zencoder15B-personal-copilot"

llm-finetuning/configs/generate_code_dataset.yaml

@@ -1,14 +1,20 @@
 # environment configuration
 settings:
   docker:
+    python_package_installer: "uv"
     requirements: requirements.txt
+    apt_packages:
+      - git
+    environment:
+      HF_HOME: "/tmp/huggingface"
+      HF_HUB_CACHE: "/tmp/huggingface"
 
 # pipeline configuration
 parameters:
-  dataset_id: htahir1/zenml-codegen-v1
+  dataset_id: zenml/zenml-codegen-v1
 
 steps:
   mirror_repositories:
     parameters:
       repositories:
-        - zenml
+        - zenml