docs: llm management fix

Adding older reports tackling refactoring of the LLM management componenets of Hatchling.
Although, this will be used mostly to fix a critical issue and maybe deferred to later as a whole.

Author: LittleCoinCoin

__reports__/llm_management_fix/phase1_analysis/architectural_analysis_v0.md

@@ -0,0 +1,250 @@
+# Hatchling LLM Management System - Architectural Analysis
+
+**Version**: 0  
+**Date**: 2025-09-19  
+**Phase**: 1 - Architectural Analysis  
+**Status**: Current State Assessment Complete
+
+## Executive Summary
+
+This report provides a comprehensive architectural analysis of Hatchling's LLM model discovery, registration, and usage system. The analysis reveals significant inconsistencies in configuration priority handling, provider-specific command behaviors, and model availability assumptions that create user confusion and limit functionality in offline/restricted environments.
+
+## Current Architecture Overview
+
+### Core Components
+
+#### 1. Configuration System Architecture
+
+**Primary Components:**
+
+- `AppSettings` (singleton): Root settings aggregator with thread-safe access
+- `LLMSettings`: Provider and model configuration with environment variable defaults
+- `SettingsRegistry`: Frontend-agnostic API for settings operations with access control
+- `OllamaSettings`/`OpenAISettings`: Provider-specific configuration classes
+
+**Configuration Priority Flow:**
+
+```
+1. CLI arguments (if cli_parse_args enabled)
+2. Settings class initializer arguments  
+3. Environment variables
+4. Dotenv (.env) files
+5. Secrets directory
+6. Default field values
+```
+
+**Critical Finding**: Environment variables are loaded at class definition time via `default_factory` lambdas, creating immutable defaults that cannot be overridden by the settings system without restart.
+
+#### 2. Model Management API
+
+**ModelManagerAPI** provides static utility methods:
+
+- `check_provider_health()`: Service availability validation
+- `list_available_models()`: Cross-provider model discovery
+- `pull_model()`: Provider-specific model acquisition
+- `get_model_info()`: Individual model status checking
+
+**Provider-Specific Implementations:**
+
+- **Ollama**: Direct API calls for real model discovery and downloading
+- **OpenAI**: API-based model listing with online validation only
+
+#### 3. Command System Integration
+
+**ModelCommands** class provides CLI interface:
+
+- `llm:provider:status`: Health checking with model listing
+- `llm:model:list`: Display registered models (static list)
+- `llm:model:add`: Provider-specific model acquisition
+- `llm:model:use`: Switch active model
+- `llm:model:remove`: Remove from registered list
+
+## Identified Inconsistencies
+
+### 1. Configuration Priority Conflicts
+
+**Issue**: Environment variables loaded at import time vs runtime settings override
+
+**Evidence:**
+
+```python
+# In LLMSettings
+provider_enum: ELLMProvider = Field(
+    default_factory=lambda: LLMSettings.to_provider_enum(os.environ.get("LLM_PROVIDER", "ollama"))
+)
+```
+
+**Impact**:
+
+- Docker `.env` variables become immutable defaults
+- Settings system cannot override environment variables without restart
+- User confusion about which configuration source takes precedence
+
+### 2. Model Registration vs Availability Mismatch
+
+**Issue**: Pre-registered models may not be locally available
+
+**Evidence:**
+
+```python
+# Default models list includes llama3.2 regardless of availability
+models: List[ModelInfo] = Field(
+    default_factory=lambda: [
+        ModelInfo(name=model[1], provider=model[0], status=ModelStatus.AVAILABLE)
+        for model in LLMSettings.extract_provider_model_list(
+            os.environ.get("LLM_MODELS", "") if os.environ.get("LLM_MODELS") 
+            else "[(ollama, llama3.2), (openai, gpt-4.1-nano)]"
+        )
+    ]
+)
+```
+
+**Impact**:
+
+- Models marked as `AVAILABLE` may not exist locally
+- No validation of model availability at startup
+- Users expect registered models to work out-of-the-box
+
+### 3. Provider-Specific Command Inconsistencies
+
+**Issue**: `llm:model:add` behaves differently across providers
+
+**Ollama Behavior:**
+
+- Downloads models via `client.pull()` with progress tracking
+- Requires internet connectivity and Ollama service
+- Fails in offline environments even with local models
+
+**OpenAI Behavior:**
+
+- Validates model existence via API call
+- No actual "download" operation
+- Requires API key and internet connectivity
+
+**Impact**:
+
+- Inconsistent user experience across providers
+- Offline environments cannot add locally available Ollama models
+- Command name implies downloading but behavior varies
+
+## Architecture Assessment
+
+### Strengths
+
+1. **Modular Design**: Clear separation between configuration, model management, and UI layers
+2. **Provider Registry Pattern**: Extensible system for adding new LLM providers
+3. **Comprehensive Settings System**: Rich configuration management with access levels
+4. **Async Support**: Proper async/await patterns for I/O operations
+
+### Critical Weaknesses
+
+1. **Configuration Immutability**: Environment variables locked at import time
+2. **Availability Assumptions**: No validation of model accessibility
+3. **Provider Inconsistency**: Different behaviors for same operations
+4. **Offline Limitations**: Cannot discover or register local models without internet
+
+### Technical Debt
+
+1. **Singleton Pattern Complexity**: Thread-safe singleton with reset capabilities adds complexity
+2. **Mixed Responsibilities**: ModelManagerAPI combines discovery, health checking, and downloading
+3. **Static Model Lists**: `llm:model:list` shows registered models, not discovered ones
+4. **Error Handling Gaps**: Limited graceful degradation for offline scenarios
+
+## Industry Standards Analysis
+
+### Configuration Management Best Practices
+
+**Standard Pattern**: Configuration precedence should be:
+
+1. Command-line arguments (highest)
+2. Environment variables  
+3. Configuration files
+4. Defaults (lowest)
+
+**Hatchling Gap**: Environment variables are treated as defaults rather than overrides.
+
+### Multi-Provider LLM Management Patterns
+
+**Industry Standard**: Unified interface with provider-specific implementations hidden from users.
+
+**Examples from Research:**
+
+- **LiteLLM**: Provides unified API across providers with consistent behavior
+- **Pydantic Settings**: Clear precedence rules with runtime override capability
+- **AWS Multi-Provider Gateway**: Consistent operations regardless of backend provider
+
+**Hatchling Gap**: Provider-specific behaviors leak through to user interface.
+
+### Offline Environment Support
+
+**Standard Pattern**: Graceful degradation with local discovery fallbacks.
+
+**Best Practices:**
+
+- Detect offline state and adjust behavior
+- Provide local model discovery mechanisms
+- Cache model metadata for offline access
+- Clear user feedback about connectivity requirements
+
+**Hatchling Gap**: Hard dependency on internet connectivity for basic operations.
+
+## Recommended Architecture Improvements
+
+### 1. Configuration System Redesign
+
+**Objective**: Implement proper configuration precedence with runtime override capability
+
+**Approach**:
+
+- Move environment variable reading to settings initialization
+- Implement lazy evaluation for configuration values
+- Add configuration source tracking and override mechanisms
+
+### 2. Unified Model Lifecycle Management
+
+**Objective**: Consistent behavior across providers with clear separation of concerns
+
+**Approach**:
+
+- Abstract model operations (discover, validate, acquire, remove)
+- Provider-specific implementations behind unified interface
+- Separate local discovery from remote operations
+
+### 3. Offline-First Design
+
+**Objective**: Full functionality in restricted environments with graceful online enhancement
+
+**Approach**:
+
+- Local model discovery as primary mechanism
+- Online validation as enhancement, not requirement
+- Clear user feedback about connectivity state and capabilities
+
+## Next Steps
+
+This analysis provides the foundation for Phase 2 (Test Suite Development). The identified inconsistencies and architectural gaps will be addressed through:
+
+1. **Test-Driven Development**: Comprehensive tests defining expected behavior
+2. **Configuration System Refactoring**: Proper precedence implementation
+3. **Provider Interface Standardization**: Unified command behavior
+4. **Offline Capability Implementation**: Local discovery and validation
+
+## Appendix: Component Interaction Diagram
+
+```
+┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
+│   CLI Commands  │───▶│  ModelManagerAPI │───▶│ Provider Registry│
+└─────────────────┘    └──────────────────┘    └─────────────────┘
+         │                       │                       │
+         ▼                       ▼                       ▼
+┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
+│ Settings System │    │   Configuration  │    │ LLM Providers   │
+│   (Registry)    │◀───│     Sources      │    │ (Ollama/OpenAI) │
+└─────────────────┘    └──────────────────┘    └─────────────────┘
+```
+
+**Key Interaction Issues:**
+
+- Configuration sources bypass settings system precedence
+- Model commands don't validate against actual availability
+- Provider implementations have inconsistent interfaces