feat: new cli for model provider and config management (#35)

Author: nabinchha

pyproject.toml

@@ -29,8 +29,10 @@ dependencies = [
   "pygments>=2.19.2",
   "pyyaml>=6.0.1",
   "python-json-logger==2.0.7",
+  "prompt-toolkit>=3.0.0",
   "requests<3,>=2.32.2",
   "rich>=13.7.1",
+  "typer>=0.12.0",
   "anyascii>=0.3.3,<1.0",
   "boto3==1.35.74",
   "datasets>=4.0.0",
@@ -52,6 +54,9 @@ dependencies = [
   "ruff==0.12.3",
 ]
 
+[project.scripts]
+data-designer = "data_designer.cli:main"
+
 [dependency-groups]
 dev = [
   "jsonpath-ng==1.5.3",

src/data_designer/cli/README.md

@@ -0,0 +1,236 @@
+# 🎨 NeMo Data Designer CLI
+
+This directory contains the Command-Line Interface (CLI) for configuring model providers and model configurations used in Data Designer.
+
+## Overview
+
+The CLI provides an interactive interface for managing:
+- **Model Providers**: LLM API endpoints (NVIDIA, OpenAI, Anthropic, custom providers)
+- **Model Configs**: Specific model configurations with inference parameters
+
+Configuration files are stored in `~/.data-designer/` by default and can be referenced by Data Designer workflows.
+
+## Architecture
+
+The CLI follows a **layered architecture** pattern, separating concerns into distinct layers:
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                         Commands                            │
+│  Entry points for CLI commands (list, providers, models)    │
+└─────────────────────────────────────────────────────────────┘
+                            │
+                            ▼
+┌─────────────────────────────────────────────────────────────┐
+│                       Controllers                           │
+│  Orchestrate user workflows and coordinate between layers   │
+└─────────────────────────────────────────────────────────────┘
+                            │
+        ┌───────────────────┴───────────────────┐
+        ▼                                       ▼
+┌──────────────────┐                 ┌──────────────────┐
+│    Services      │                 │      Forms       │
+│  Business logic  │                 │  Interactive UI  │
+└──────────────────┘                 └──────────────────┘
+        │
+        ▼
+┌──────────────────┐
+│   Repositories   │
+│  Data persistence│
+└──────────────────┘
+```
+
+### Layer Responsibilities
+
+#### 1. **Commands** (`commands/`)
+- **Purpose**: Define CLI command entry points using Typer
+- **Responsibilities**:
+  - Parse command-line arguments and options
+  - Initialize controllers with appropriate configuration
+  - Handle top-level error reporting
+- **Files**:
+  - `list.py`: List current configurations
+  - `models.py`: Configure models
+  - `providers.py`: Configure providers
+  - `reset.py`: Reset/delete configurations
+
+#### 2. **Controllers** (`controllers/`)
+- **Purpose**: Orchestrate user workflows and coordinate between services, forms, and UI
+- **Responsibilities**:
+  - Implement the main workflow logic (add, update, delete, etc.)
+  - Coordinate between services and interactive forms
+  - Handle user navigation and session state
+  - Manage associated resource deletion (e.g., deleting models when provider is deleted)
+- **Files**:
+  - `model_controller.py`: Orchestrates model configuration workflows
+  - `provider_controller.py`: Orchestrates provider configuration workflows
+
+**Key Features**:
+- **Associated Resource Management**: When deleting a provider, the controller checks for associated models and prompts the user to delete them together
+- **Interactive Navigation**: Supports add/update/delete/delete_all operations with user-friendly menus
+
+#### 3. **Services** (`services/`)
+- **Purpose**: Implement business logic and enforce domain rules
+- **Responsibilities**:
+  - Validate business rules (e.g., unique names, required fields)
+  - Implement CRUD operations with validation
+  - Coordinate between multiple repositories when needed
+  - Handle default management (e.g., default provider selection)
+- **Files**:
+  - `model_service.py`: Model configuration business logic
+  - `provider_service.py`: Provider business logic
+
+**Key Methods**:
+- `list_all()`: Get all configured items
+- `get_by_*()`: Retrieve specific items
+- `add()`: Add new item with validation
+- `update()`: Update existing item
+- `delete()`: Delete single item
+- `delete_by_aliases()`: Batch delete (models only)
+- `find_by_provider()`: Find models by provider (models only)
+- `set_default()`, `get_default()`: Manage default provider (providers only)
+
+#### 4. **Repositories** (`repositories/`)
+- **Purpose**: Handle data persistence (YAML file I/O)
+- **Responsibilities**:
+  - Load configuration from YAML files
+  - Save configuration to YAML files
+  - Check file existence
+  - Delete configuration files
+- **Files**:
+  - `base.py`: Abstract base repository with common operations
+  - `model_repository.py`: Model configuration persistence
+  - `provider_repository.py`: Provider persistence
+
+**Base Repository Pattern**:
+```python
+class ConfigRepository(ABC, Generic[T]):
+    def load(self) -> T | None: ...
+    def save(self, config: T) -> None: ...
+    def exists(self) -> bool: ...
+    def delete(self) -> None: ...
+```
+
+#### 5. **Forms** (`forms/`)
+- **Purpose**: Interactive form-based data collection from users
+- **Responsibilities**:
+  - Define form fields with validation
+  - Collect user input interactively
+  - Support navigation (back, cancel)
+  - Build configuration objects from form data
+- **Files**:
+  - `builder.py`: Abstract form builder base
+  - `field.py`: Form field types (TextField, SelectField, NumericField)
+  - `form.py`: Form container and prompt orchestration
+  - `model_builder.py`: Interactive model configuration builder
+  - `provider_builder.py`: Interactive provider configuration builder
+
+**Form Features**:
+- Field-level validation
+- Auto-completion support
+- History navigation (arrow keys)
+- Default value handling
+- Back navigation support
+
+#### 6. **UI Utilities** (`ui.py`)
+- **Purpose**: User interface utilities for terminal output and input
+- **Responsibilities**:
+  - Interactive menus with arrow key navigation
+  - Text input prompts with validation
+  - Confirmation dialogs
+  - Styled output (success, error, warning, info)
+  - Configuration preview displays
+- **Key Functions**:
+  - `select_with_arrows()`: Interactive arrow-key menu
+  - `prompt_text_input()`: Text input with validation and completion
+  - `confirm_action()`: Yes/no confirmation
+  - `print_*()`: Styled console output
+  - `display_config_preview()`: YAML preview with syntax highlighting
+
+
+## Configuration Files
+
+The CLI manages two YAML configuration files:
+
+### `~/.data-designer/model_providers.yaml`
+
+Stores model provider configurations (API endpoints):
+
+```yaml
+providers:
+  - name: nvidia
+    endpoint: https://integrate.api.nvidia.com/v1
+    provider_type: openai
+    api_key: NVIDIA_API_KEY
+  - name: openai
+    endpoint: https://api.openai.com/v1
+    provider_type: openai
+    api_key: OPENAI_API_KEY
+default: nvidia
+```
+
+### `~/.data-designer/model_configs.yaml`
+
+Stores model configurations:
+
+```yaml
+model_configs:
+  - alias: llama3-70b
+    model: meta/llama-3.1-70b-instruct
+    provider: nvidia
+    inference_parameters:
+      temperature: 0.7
+      top_p: 0.9
+      max_tokens: 2048
+      max_parallel_requests: 4
+  - alias: gpt-4
+    model: gpt-4-turbo
+    provider: openai
+    inference_parameters:
+      temperature: 0.8
+      top_p: 0.95
+      max_tokens: 4096
+```
+
+## Usage Examples
+
+### Configure Providers
+
+```bash
+# Interactive provider configuration
+data-designer config providers
+
+# Options:
+#  - Add a new provider (predefined: nvidia, openai, anthropic, or custom)
+#  - Update an existing provider
+#  - Delete a provider (with associated model cleanup)
+#  - Delete all providers
+#  - Change default provider
+```
+
+### Configure Models
+
+```bash
+# Interactive model configuration
+data-designer config models
+
+# Options:
+#  - Add a new model
+#  - Update an existing model
+#  - Delete a model
+#  - Delete all models
+```
+
+### List Configurations
+
+```bash
+# Display current configurations
+data-designer config list
+```
+
+### Reset Configurations
+
+```bash
+# Delete configuration files (with confirmation)
+data-designer config reset
+```

src/data_designer/cli/__init__.py

@@ -0,0 +1,6 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+from data_designer.cli.main import app, main
+
+__all__ = ["app", "main"]

src/data_designer/cli/commands/__init__.py

@@ -0,0 +1,2 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0

src/data_designer/cli/commands/list.py

@@ -0,0 +1,130 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+from rich.table import Table
+
+from data_designer.cli.repositories.model_repository import ModelRepository
+from data_designer.cli.repositories.provider_repository import ProviderRepository
+from data_designer.cli.ui import console, print_error, print_header, print_info, print_warning
+from data_designer.config.utils.constants import DATA_DESIGNER_HOME_DIR, NordColor
+
+
+def list_command() -> None:
+    """List current Data Designer configurations.
+
+    Returns:
+        None
+    """
+    # Determine config directory
+    print_header("Data Designer Configurations")
+    print_info(f"Configuration directory: {DATA_DESIGNER_HOME_DIR}")
+    console.print()
+
+    # Display providers
+    display_providers(ProviderRepository(DATA_DESIGNER_HOME_DIR))
+    display_models(ModelRepository(DATA_DESIGNER_HOME_DIR))
+
+
+def display_providers(provider_repo: ProviderRepository) -> None:
+    """Load and display model providers.
+
+    Args:
+        provider_repo: Provider repository
+
+    Returns:
+        None
+    """
+    try:
+        provider_registry = provider_repo.load()
+
+        if not provider_registry:
+            print_warning("Providers have not been configured. Run 'data-designer config providers' to configure them.")
+            console.print()
+            return
+
+        # Display as table
+        table = Table(title="Model Providers", border_style=NordColor.NORD8.value)
+        table.add_column("Name", style=NordColor.NORD14.value, no_wrap=True)
+        table.add_column("Endpoint", style=NordColor.NORD4.value)
+        table.add_column("Type", style=NordColor.NORD9.value, no_wrap=True)
+        table.add_column("API Key", style=NordColor.NORD7.value)
+        table.add_column("Default", style=NordColor.NORD13.value, justify="center")
+
+        default_name = provider_registry.default or provider_registry.providers[0].name
+
+        for provider in provider_registry.providers:
+            is_default = "✓" if provider.name == default_name else ""
+            api_key_display = provider.api_key or "(not set)"
+
+            # Mask actual API keys (keep env var names visible)
+            if provider.api_key and not provider.api_key.isupper():
+                api_key_display = "***" + provider.api_key[-4:] if len(provider.api_key) > 4 else "***"
+
+            table.add_row(
+                provider.name,
+                provider.endpoint,
+                provider.provider_type,
+                api_key_display,
+                is_default,
+            )
+
+        console.print(table)
+        console.print()
+    except Exception as e:
+        print_error(f"Error loading provider configuration: {e}")
+        console.print()
+
+
+def display_models(model_repo: ModelRepository) -> None:
+    """Load and display model configurations.
+
+    Args:
+        model_repo: Model repository
+
+    Returns:
+        None
+    """
+    try:
+        registry = model_repo.load()
+
+        if not registry:
+            print_warning("Models have not been configured. Run 'data-designer config models' to configure them.")
+            console.print()
+            return
+
+        # Display as table
+        table = Table(title="Model Configurations", border_style=NordColor.NORD8.value)
+        table.add_column("Alias", style=NordColor.NORD14.value, no_wrap=True)
+        table.add_column("Model ID", style=NordColor.NORD4.value)
+        table.add_column("Provider", style=NordColor.NORD9.value, no_wrap=True)
+        table.add_column("Temperature", style=NordColor.NORD15.value, justify="right")
+        table.add_column("Top P", style=NordColor.NORD15.value, justify="right")
+        table.add_column("Max Tokens", style=NordColor.NORD15.value, justify="right")
+
+        for mc in registry.model_configs:
+            # Handle distribution-based parameters
+            temp_display = (
+                f"{mc.inference_parameters.temperature:.2f}"
+                if isinstance(mc.inference_parameters.temperature, (int, float))
+                else "dist"
+            )
+            top_p_display = (
+                f"{mc.inference_parameters.top_p:.2f}"
+                if isinstance(mc.inference_parameters.top_p, (int, float))
+                else "dist"
+            )
+
+            table.add_row(
+                mc.alias,
+                mc.model,
+                mc.provider or "(default)",
+                temp_display,
+                top_p_display,
+                str(mc.inference_parameters.max_tokens) if mc.inference_parameters.max_tokens else "(none)",
+            )
+
+        console.print(table)
+        console.print()
+    except Exception as e:
+        print_error(f"Error loading model configuration: {e}")
+        console.print()