Refactor models.py (#209)

* add new params to readme and plan_editor

* del OpenaiHandler, OllamaHandler. local url in models -> config.toml

* remove linux-only comment for delete dir flag

* add params to readme

* rename local_url -> itmo_local_url

Author: pankovakris

README.md

@@ -187,9 +187,12 @@ documentation, see the [GitHub Action Workflow Generator README](./osa_tool/gith
 | `--base-url`         | URL of the provider compatible with API OpenAI                                      | `https://api.openai.com/v1` |
 | `--model`            | Specific LLM model to use                                                           | `gpt-3.5-turbo`             |
 | `-m`, `--mode`       | Operation mode for repository processing: `basic`, `auto` (default), or `advanced`. | `auto`                      |
-| `--delete-dir`       | Enable deleting the downloaded repository after processing (**Linux only**)         | `disabled`                  |
+| `--delete-dir`       | Enable deleting the downloaded repository after processing                          | `disabled`                  |
 | `--no-fork`          | Avoid create fork for target repository                                             | `False`                     |
 | `--no-pull-request`  | Avoid create pull request for target repository                                     | `False`                     |
+| `--top_p`            | Nucleus sampling probability                                                        | `null`                      |
+| `--temperature`      | Sampling temperature to use for the LLM output (0 = deterministic, 1 = creative).   | `null`                      |
+| `--max_tokens`       | Maximum number of tokens the model can generate in a single response                | `null`                      |
 
 To learn how to work with the interactive CLI and view descriptions of all available keys, visit
 the [CLI usage guide](./osa_tool/scheduler/README.md).

docs/index.md

@@ -148,17 +148,20 @@ Docker, ensure that you upload the PDF file to the OSA folder before building th
 
 ### Configuration
 
-| Flag                 | Description                                                                 | Default                     |
-|----------------------|-----------------------------------------------------------------------------|-----------------------------|
-| `-r`, `--repository` | URL of the GitHub repository (**Mandatory**)                                |                             |
-| `--api`              | LLM API service provider                                                    | `llama`                     |
-| `--base-url`         | URL of the provider compatible with API OpenAI                              | `https://api.openai.com/v1` |
-| `--model`            | Specific LLM model to use                                                   | `gpt-3.5-turbo`             |
-| `--article`          | Link to the pdf file of the article                                         | `None`                      |
-| `--translate-dirs`   | Enable automatic translation of the directory name into English             | `disabled`                  |
-| `--delete-dir`       | Enable deleting the downloaded repository after processing (**Linux only**) | `disabled`                  |
-| `--no-fork`          | Avoid create fork for target repository                                     | `False`                     |
-| `--no-pull-request`  | Avoid create pull request for target repository                             | `False`                     |
+| Flag                 | Description                                                                       | Default                     |
+|----------------------|-----------------------------------------------------------------------------------|-----------------------------|
+| `-r`, `--repository` | URL of the GitHub repository (**Mandatory**)                                      |                             |
+| `--api`              | LLM API service provider                                                          | `llama`                     |
+| `--base-url`         | URL of the provider compatible with API OpenAI                                    | `https://api.openai.com/v1` |
+| `--model`            | Specific LLM model to use                                                         | `gpt-3.5-turbo`             |
+| `--article`          | Link to the pdf file of the article                                               | `None`                      |
+| `--translate-dirs`   | Enable automatic translation of the directory name into English                   | `disabled`                  |
+| `--delete-dir`       | Enable deleting the downloaded repository after processing                        | `disabled`                  |
+| `--no-fork`          | Avoid create fork for target repository                                           | `False`                     |
+| `--no-pull-request`  | Avoid create pull request for target repository                                   | `False`                     |
+| `--top_p`            | Nucleus sampling probability                                                      | `null`                      |
+| `--temperature`      | Sampling temperature to use for the LLM output (0 = deterministic, 1 = creative). | `null`                      |
+| `--max_tokens`       | Maximum number of tokens the model can generate in a single response              | `null`                      |
 
 ---
 

osa_tool/config/settings.py

@@ -51,6 +51,7 @@ class ModelSettings(BaseModel):
     encoder: str
     host_name: AnyHttpUrl
     localhost: AnyHttpUrl
+    itmo_local_url: AnyHttpUrl
     model: str
     path: str
     temperature: NonNegativeFloat

osa_tool/config/settings/arguments.yaml

@@ -55,6 +55,27 @@ model:
     3. https://ollama.com/library
   default: "gpt-3.5-turbo"
 
+temperature:
+  aliases: [ "--temperature" ]
+  type: str
+  description: "Sampling temperature to use for the LLM output (0 = deterministic, 1 = creative)."
+  default: null
+  example: 0.3, 0.9
+
+tokens:
+  aliases: [ "--max-tokens" ]
+  type: str
+  description: "Maximum number of tokens the model can generate in a single response."
+  default: 4096
+  example: 256, 1024
+
+top_p:
+  aliases: [ "--top-p" ]
+  type: str
+  description: "Nucleus sampling probability (1.0 = all tokens considered)."
+  default: null
+  example: 0.8, 0.95
+
 article:
   aliases: [ "--article" ]
   type: str
@@ -82,7 +103,7 @@ convert_notebooks:
 delete_dir:
   aliases: [ "--delete-dir" ]
   type: flag
-  description: "Enable deleting the downloaded repository after processing. (Linux only)"
+  description: "Enable deleting the downloaded repository after processing."
   default: false
 
 ensure_license:

osa_tool/config/settings/config.toml

@@ -27,6 +27,7 @@ context_window = 4096
 encoder = "cl100k_base"
 host_name = "https://api.openai.com/v1"
 localhost = "http://localhost:11434/"
+itmo_local_url = "http://10.32.15.21:6672"
 model = "gpt-3.5-turbo"
 path = "generate"
 temperature = 0.05