Merge pull request #48 from jongalloway/copilot/fix-37

🛠️ Implement Advanced Tool System with Search, Details, Compare, Ensemble, and Recipe Tools

Author: jongalloway

.github/custom-instructions.md

@@ -29,38 +29,44 @@ This document provides guidance for AI assistants working with the NLWebNet code
 ## Architecture Patterns
 
 ### 1. Minimal API Approach
+
 - **Primary approach**: Uses **Minimal APIs** for modern, lightweight endpoints
 - **Legacy support**: Traditional controllers still present but being phased out
 - Endpoints are organized in static classes: `AskEndpoints`, `McpEndpoints`
 - Extension methods provide clean endpoint mapping: `app.MapNLWebNet()`
 - Both `app.MapNLWebNet()` (minimal APIs) and `app.MapNLWebNetControllers()` (legacy) available
 
 ### 2. Dependency Injection
+
 - Follows standard .NET DI patterns
 - Extension methods for service registration: `services.AddNLWebNet()`
 - Interface-based design with clear service contracts
 - Supports custom backend implementations via generics
 
 ### 3. Service Layer Architecture
-```
+
+```text
 Controllers/Endpoints → Services → Data Backends
                      ↘ MCP Integration
 ```
 
 Key interfaces:
+
 - `INLWebService` - Main orchestration service
 - `IDataBackend` - Data retrieval abstraction
 - `IQueryProcessor` - Query processing logic
 - `IResultGenerator` - Response generation
 - `IMcpService` - Model Context Protocol integration
 
 ### 4. Configuration Pattern
+
 - Uses `NLWebOptions` for strongly-typed configuration
 - Supports configuration binding from `appsettings.json`
 - User secrets for sensitive data (API keys)
 - Environment-specific configurations
 
 ### 5. Architectural Migration
+
 **Important**: The project is transitioning from traditional MVC controllers to Minimal APIs.
 
 - **Preferred**: Use `Endpoints/` classes with static mapping methods
@@ -71,6 +77,7 @@ Key interfaces:
 ## Code Conventions
 
 ### Naming and Structure
+
 - **Namespace**: All code under `NLWebNet` namespace
 - **Models**: Request/response DTOs with JSON serialization attributes
 - **Services**: Interface + implementation pattern (`IService` → `Service`)
@@ -79,14 +86,16 @@ Key interfaces:
 - **Controllers**: Traditional MVC controllers (legacy, being phased out)
 
 ### C# Style Guidelines
+
 - **Nullable reference types** enabled (`<Nullable>enable</Nullable>`)
 - **Implicit usings** enabled for common namespaces
 - **XML documentation** for all public APIs
 - **Data annotations** for request validation
 - **JSON property names** in snake_case for protocol compliance
 
 ### File Organization
-```
+
+```text
 src/NLWebNet/
 ├── Models/          # Request/response DTOs
 ├── Services/        # Business logic interfaces/implementations
@@ -100,58 +109,72 @@ src/NLWebNet/
 ## NLWeb Protocol Implementation
 
 ### Core Concepts
+
 - **Three query modes**: `List`, `Summarize`, `Generate`
 - **Streaming support**: Real-time response delivery
 - **Query context**: Previous queries and decontextualization
 - **Site filtering**: Subset targeting within data backends
 
 ### Request/Response Flow
+
 1. Validate incoming `NLWebRequest`
 2. Process query through `IQueryProcessor`
 3. Retrieve data via `IDataBackend`
 4. Generate response using `IResultGenerator`
 5. Return `NLWebResponse` with results
 
 ### MCP Integration
+
 - Supports core MCP methods: `list_tools`, `list_prompts`, `call_tool`, `get_prompt`
 - Parallel endpoint structure: `/ask` and `/mcp` with shared logic
 - Tool and prompt template management
 
 ## Testing Approach
 
 ### Unit Testing
+
 - **39 unit tests** with 100% pass rate (current standard)
 - **NSubstitute** for mocking dependencies
 - **MSTest** framework with `[TestMethod]` attributes
 - Focus on service layer and business logic testing
 
 ### Integration Testing
+
 - **Manual testing** preferred over automated integration tests
 - **Comprehensive guides** in `/doc/manual-testing-guide.md`
 - **Sample requests** in `/doc/sample-requests.http` for IDE testing
 - **Demo application** for end-to-end validation
 
 ### Testing Conventions
+
 - Test classes named `[ClassUnderTest]Tests`
 - Arrange-Act-Assert pattern
 - Mock external dependencies (AI services, data backends)
 - Test both success and error scenarios
 
 ## Development Practices
 
+### Pre-Commit Requirements
+
+- **Code formatting**: Run `dotnet format ./NLWebNet.sln` before each commit to ensure consistent code style
+- **Automated formatting**: GitHub Actions will also auto-format code on push to feature branches
+
 ### CI/CD Pipeline
+
 - **GitHub Actions** for automated builds and testing
 - **NuGet package** generation and validation
 - **Release automation** with version tagging
 - **Security scanning** for vulnerable dependencies
 
 ### Build and Packaging
+
 - **Deterministic builds** for reproducible packages
 - **Symbol packages** (.snupkg) for debugging support
 - **Source Link** integration for GitHub source navigation
 - **Package validation** scripts for quality assurance
 
 ### Documentation Standards
+
 - **Comprehensive README** with usage examples
 - **XML documentation** for all public APIs
 - **OpenAPI specification** generated automatically
@@ -160,12 +183,14 @@ src/NLWebNet/
 ## AI Service Integration
 
 ### Microsoft.Extensions.AI Pattern
+
 - Use `Microsoft.Extensions.AI` abstractions for AI service integration
 - Support multiple AI providers (Azure OpenAI, OpenAI API)
 - Configuration-driven AI service selection
 - Async/await patterns for AI service calls
 
 ### Error Handling
+
 - **Graceful degradation** when AI services are unavailable
 - **Fallback responses** for service failures
 - **Proper exception handling** with meaningful error messages
@@ -174,20 +199,23 @@ src/NLWebNet/
 ## Common Patterns and Best Practices
 
 ### When Adding New Features
+
 1. **Start with interfaces** - Define contracts before implementations
 2. **Add configuration options** to `NLWebOptions` if needed
 3. **Include unit tests** - Maintain the 100% pass rate standard
 4. **Update documentation** - XML docs and README as appropriate
 5. **Consider MCP integration** - How does this fit with MCP protocol?
 
 ### When Modifying Endpoints
+
 1. **Use Minimal APIs** - Prefer `Endpoints/` classes over `Controllers/`
 2. **Maintain protocol compliance** - Follow NLWeb specification
 3. **Add OpenAPI documentation** - Use `.WithSummary()` and `.WithDescription()`
 4. **Include error responses** - Proper status codes and problem details
 5. **Test with sample requests** - Update manual testing guides
 
 ### When Adding Dependencies
+
 1. **Prefer Microsoft.Extensions.*** - Use standard .NET abstractions
 2. **Check for existing alternatives** - Avoid duplicate functionality
 3. **Update project files** - Include in main library and test projects
@@ -196,19 +224,22 @@ src/NLWebNet/
 ## Limitations and Current Implementation Status
 
 ### Current Implementation Status
+
 - **Early alpha prerelease** - Core functionality implemented, not yet production ready
 - **Mock data backend** as default - Real data source integrations can be implemented via `IDataBackend`
 - **Basic AI integration** - Extensible via Microsoft.Extensions.AI patterns
 - **Authentication framework** - Ready for implementation based on application requirements
 - **Code quality standards** - Production-level code quality maintained throughout development
 
 ### Performance Considerations
+
 - **Streaming responses** for better perceived performance
 - **Async/await** throughout for scalability
 - **Minimal allocations** where possible
 - **Configuration caching** for frequently accessed settings
 
 ### Deployment Considerations
+
 - **Requires .NET 9.0** - Latest framework dependency for modern features
 - **Early prerelease status** - Not yet ready for production deployment
 - **Production-quality code** - Library being developed with production standards
@@ -217,10 +248,11 @@ src/NLWebNet/
 ## When to Seek Clarification
 
 Ask for guidance when:
+
 - **Breaking changes** to public APIs are needed
 - **New external dependencies** are required
 - **Significant architectural changes** are proposed
 - **Protocol compliance** questions arise
 - **Production deployment** patterns need to be established
 
-Remember: This library is being developed with production-quality standards, though it is currently in early prerelease and not yet ready for production use. All code additions and edits should maintain production-level quality as the project works toward its goal of becoming a production-ready NLWeb protocol implementation for .NET applications.
+Remember: This library is being developed with production-quality standards, though it is currently in early prerelease and not yet ready for production use. All code additions and edits should maintain production-level quality as the project works toward its goal of becoming a production-ready NLWeb protocol implementation for .NET applications.

doc/advanced-tool-system-guide.md

@@ -0,0 +1,139 @@
+# Advanced Tool System Usage Guide
+
+The Advanced Tool System provides enhanced query capabilities through specialized tool handlers that route queries to appropriate processors based on intent analysis.
+
+## Configuration
+
+Enable the tool system in your configuration:
+
+```json
+{
+  "NLWebNet": {
+    "ToolSelectionEnabled": true,
+    "DefaultMode": "List"
+  }
+}
+```
+
+## Available Tools
+
+### 1. Search Tool (`search`)
+Enhanced keyword and semantic search with result optimization.
+
+**Example Queries:**
+- "search for REST API documentation"
+- "find information about microservices"
+- "locate best practices for authentication"
+
+**Features:**
+- Query optimization (removes redundant search terms)
+- Enhanced relevance scoring
+- Result sorting and filtering
+
+### 2. Details Tool (`details`)
+Retrieves comprehensive information about specific named items.
+
+**Example Queries:**
+- "tell me about GraphQL"
+- "what is Docker?"
+- "describe OAuth 2.0"
+- "information about React hooks"
+
+**Features:**
+- Subject extraction from natural language queries
+- Detailed information focus
+- Comprehensive result filtering
+
+### 3. Compare Tool (`compare`)
+Side-by-side comparison of two items or technologies.
+
+**Example Queries:**
+- "compare React vs Vue"
+- "difference between REST and GraphQL"
+- "Node.js versus Python for backend"
+
+**Features:**
+- Automatic item extraction from comparison queries
+- Structured comparison results
+- Side-by-side analysis
+
+### 4. Ensemble Tool (`ensemble`)
+Creates cohesive sets of related recommendations.
+
+**Example Queries:**
+- "recommend a full-stack JavaScript development setup"
+- "suggest tools for DevOps pipeline"
+- "I need a complete testing framework"
+
+**Features:**
+- Multi-category recommendations
+- Coherent item grouping
+- Thematic organization
+
+### 5. Recipe Tool (`recipe`)
+Specialized for cooking, recipes, and food-related queries.
+
+**Example Queries:**
+- "substitute eggs in baking"
+- "what goes with grilled salmon?"
+- "recipe for chocolate chip cookies"
+
+**Features:**
+- Ingredient substitution guidance
+- Food pairing suggestions
+- Cooking technique advice
+
+## Integration
+
+The tool system integrates automatically with your existing NLWebNet setup:
+
+```csharp
+// Add NLWebNet services (includes tool system)
+services.AddNLWebNet(options =>
+{
+    options.ToolSelectionEnabled = true;
+});
+```
+
+## Backward Compatibility
+
+When `ToolSelectionEnabled = false`, the system uses the standard query processing pipeline, maintaining full backward compatibility.
+
+## Query Processing Flow
+
+1. **Query Analysis**: The `IToolSelector` analyzes query intent
+2. **Tool Selection**: Appropriate tool is selected based on keywords and patterns  
+3. **Tool Execution**: The `IToolExecutor` routes to the selected tool handler
+4. **Result Enhancement**: Each tool applies specialized processing
+5. **Response Generation**: Results are formatted and returned
+
+## Custom Tool Development
+
+To create custom tools, implement the `IToolHandler` interface:
+
+```csharp
+public class CustomToolHandler : BaseToolHandler
+{
+    public override string ToolType => "custom";
+    
+    public override async Task<NLWebResponse> ExecuteAsync(
+        NLWebRequest request, 
+        CancellationToken cancellationToken = default)
+    {
+        // Custom tool logic here
+        return CreateSuccessResponse(request, results, processingTime);
+    }
+    
+    public override bool CanHandle(NLWebRequest request)
+    {
+        // Determine if this tool can handle the request
+        return request.Query.Contains("custom");
+    }
+}
+```
+
+Register your custom tool:
+
+```csharp
+services.AddScoped<IToolHandler, CustomToolHandler>();
+```