Update AI Model from Granite 3.3 to Granite 4

[ajbozarth]

Note: This issue description was AI-generated. Please review, validate, and adjust the details as needed.

Background
The project currently uses IBM's Granite 3.3:8b model for AI-powered features. Granite 4 offers improved performance, better reasoning capabilities, and enhanced context handling.

Current State

• Model specified in api/chat-agent/agents.yaml: granite3.3:8b
• Model specified in api/codegen-agent/agents.yaml: granite3.3:8b
• Ollama used as the local LLM provider
• Maestro framework orchestrates AI workflows
• README.md references granite3.3:8b in setup instructions

Goals

1. Upgrade to Granite 4 model
2. Maintain backward compatibility during transition
3. Update documentation and setup instructions
4. Validate performance improvements
5. Update any model-specific configurations

Implementation Plan

Phase 1: Research & Preparation

• Identify Granite 4 model variants and sizes
• Review Granite 4 release notes and breaking changes
• Determine optimal model size for use case
• Check Ollama compatibility with Granite 4
• Review Maestro framework compatibility

Phase 2: Configuration Updates

• Update api/chat-agent/agents.yaml

  spec:
    model: granite4:8b  # or appropriate variant

• Update api/codegen-agent/agents.yaml
• Update README.md installation instructions
• Update AGENTS.md documentation
• Update CONTRIBUTING.md setup guide

Phase 3: Testing & Validation

• Test chat agent functionality
• Test code generation agent
• Validate RAG (Retrieval-Augmented Generation) performance
• Compare response quality with Granite 3.3
• Test with various quantum computing queries
• Validate code generation accuracy

Phase 4: Performance Benchmarking

• Measure response latency
• Evaluate context window utilization
• Test with complex quantum algorithms
• Compare token usage and costs
• Document performance improvements

Phase 5: Documentation & Rollout

• Update all documentation
• Create migration guide for users
• Update Docker images
• Update Helm chart configurations
• Announce upgrade in release notes

Files to Update

• api/chat-agent/agents.yaml (line 8)
• api/codegen-agent/agents.yaml (line 21)
• README.md (lines 98, 144)
• AGENTS.md (line 25, 98, 104-105)
• CONTRIBUTING.md (setup instructions)
• .env.local.template (if model-specific configs exist)

Acceptance Criteria

• All agent configurations updated to Granite 4
• Documentation reflects new model version
• All tests pass with new model
• Performance benchmarks documented
• Migration guide created
• No regression in functionality
• Ollama pull commands updated

Technical Considerations

• Granite 4 may have different context window sizes
• Token limits might differ from Granite 3.3
• Prompt engineering may need adjustment
• Consider maintaining Granite 3.3 as fallback option
• Update any model-specific temperature/parameter settings
• Verify embedding model compatibility (nomic-embed-text)

Rollback Plan

• Keep Granite 3.3 configuration in comments
• Document rollback procedure
• Maintain both models in Ollama during transition
• Create feature flag for model selection

Resources

• Ollama Model Library
• Granite Model Documentation
• Maestro Framework