diff --git a/.github/custom-instructions.md b/.github/custom-instructions.md index f37f01f..8558dac 100644 --- a/.github/custom-instructions.md +++ b/.github/custom-instructions.md @@ -29,6 +29,7 @@ This document provides guidance for AI assistants working with the NLWebNet code ## Architecture Patterns ### 1. Minimal API Approach + - **Primary approach**: Uses **Minimal APIs** for modern, lightweight endpoints - **Legacy support**: Traditional controllers still present but being phased out - Endpoints are organized in static classes: `AskEndpoints`, `McpEndpoints` @@ -36,18 +37,21 @@ This document provides guidance for AI assistants working with the NLWebNet code - Both `app.MapNLWebNet()` (minimal APIs) and `app.MapNLWebNetControllers()` (legacy) available ### 2. Dependency Injection + - Follows standard .NET DI patterns - Extension methods for service registration: `services.AddNLWebNet()` - Interface-based design with clear service contracts - Supports custom backend implementations via generics ### 3. Service Layer Architecture -``` + +```text Controllers/Endpoints → Services → Data Backends ↘ MCP Integration ``` Key interfaces: + - `INLWebService` - Main orchestration service - `IDataBackend` - Data retrieval abstraction - `IQueryProcessor` - Query processing logic @@ -55,12 +59,14 @@ Key interfaces: - `IMcpService` - Model Context Protocol integration ### 4. Configuration Pattern + - Uses `NLWebOptions` for strongly-typed configuration - Supports configuration binding from `appsettings.json` - User secrets for sensitive data (API keys) - Environment-specific configurations ### 5. Architectural Migration + **Important**: The project is transitioning from traditional MVC controllers to Minimal APIs. - **Preferred**: Use `Endpoints/` classes with static mapping methods @@ -71,6 +77,7 @@ Key interfaces: ## Code Conventions ### Naming and Structure + - **Namespace**: All code under `NLWebNet` namespace - **Models**: Request/response DTOs with JSON serialization attributes - **Services**: Interface + implementation pattern (`IService` → `Service`) @@ -79,6 +86,7 @@ Key interfaces: - **Controllers**: Traditional MVC controllers (legacy, being phased out) ### C# Style Guidelines + - **Nullable reference types** enabled (`enable`) - **Implicit usings** enabled for common namespaces - **XML documentation** for all public APIs @@ -86,7 +94,8 @@ Key interfaces: - **JSON property names** in snake_case for protocol compliance ### File Organization -``` + +```text src/NLWebNet/ ├── Models/ # Request/response DTOs ├── Services/ # Business logic interfaces/implementations @@ -100,12 +109,14 @@ src/NLWebNet/ ## NLWeb Protocol Implementation ### Core Concepts + - **Three query modes**: `List`, `Summarize`, `Generate` - **Streaming support**: Real-time response delivery - **Query context**: Previous queries and decontextualization - **Site filtering**: Subset targeting within data backends ### Request/Response Flow + 1. Validate incoming `NLWebRequest` 2. Process query through `IQueryProcessor` 3. Retrieve data via `IDataBackend` @@ -113,6 +124,7 @@ src/NLWebNet/ 5. Return `NLWebResponse` with results ### MCP Integration + - Supports core MCP methods: `list_tools`, `list_prompts`, `call_tool`, `get_prompt` - Parallel endpoint structure: `/ask` and `/mcp` with shared logic - Tool and prompt template management @@ -120,18 +132,21 @@ src/NLWebNet/ ## Testing Approach ### Unit Testing + - **39 unit tests** with 100% pass rate (current standard) - **NSubstitute** for mocking dependencies - **MSTest** framework with `[TestMethod]` attributes - Focus on service layer and business logic testing ### Integration Testing + - **Manual testing** preferred over automated integration tests - **Comprehensive guides** in `/doc/manual-testing-guide.md` - **Sample requests** in `/doc/sample-requests.http` for IDE testing - **Demo application** for end-to-end validation ### Testing Conventions + - Test classes named `[ClassUnderTest]Tests` - Arrange-Act-Assert pattern - Mock external dependencies (AI services, data backends) @@ -139,19 +154,27 @@ src/NLWebNet/ ## Development Practices +### Pre-Commit Requirements + +- **Code formatting**: Run `dotnet format ./NLWebNet.sln` before each commit to ensure consistent code style +- **Automated formatting**: GitHub Actions will also auto-format code on push to feature branches + ### CI/CD Pipeline + - **GitHub Actions** for automated builds and testing - **NuGet package** generation and validation - **Release automation** with version tagging - **Security scanning** for vulnerable dependencies ### Build and Packaging + - **Deterministic builds** for reproducible packages - **Symbol packages** (.snupkg) for debugging support - **Source Link** integration for GitHub source navigation - **Package validation** scripts for quality assurance ### Documentation Standards + - **Comprehensive README** with usage examples - **XML documentation** for all public APIs - **OpenAPI specification** generated automatically @@ -160,12 +183,14 @@ src/NLWebNet/ ## AI Service Integration ### Microsoft.Extensions.AI Pattern + - Use `Microsoft.Extensions.AI` abstractions for AI service integration - Support multiple AI providers (Azure OpenAI, OpenAI API) - Configuration-driven AI service selection - Async/await patterns for AI service calls ### Error Handling + - **Graceful degradation** when AI services are unavailable - **Fallback responses** for service failures - **Proper exception handling** with meaningful error messages @@ -174,6 +199,7 @@ src/NLWebNet/ ## Common Patterns and Best Practices ### When Adding New Features + 1. **Start with interfaces** - Define contracts before implementations 2. **Add configuration options** to `NLWebOptions` if needed 3. **Include unit tests** - Maintain the 100% pass rate standard @@ -181,6 +207,7 @@ src/NLWebNet/ 5. **Consider MCP integration** - How does this fit with MCP protocol? ### When Modifying Endpoints + 1. **Use Minimal APIs** - Prefer `Endpoints/` classes over `Controllers/` 2. **Maintain protocol compliance** - Follow NLWeb specification 3. **Add OpenAPI documentation** - Use `.WithSummary()` and `.WithDescription()` @@ -188,6 +215,7 @@ src/NLWebNet/ 5. **Test with sample requests** - Update manual testing guides ### When Adding Dependencies + 1. **Prefer Microsoft.Extensions.*** - Use standard .NET abstractions 2. **Check for existing alternatives** - Avoid duplicate functionality 3. **Update project files** - Include in main library and test projects @@ -196,6 +224,7 @@ src/NLWebNet/ ## Limitations and Current Implementation Status ### Current Implementation Status + - **Early alpha prerelease** - Core functionality implemented, not yet production ready - **Mock data backend** as default - Real data source integrations can be implemented via `IDataBackend` - **Basic AI integration** - Extensible via Microsoft.Extensions.AI patterns @@ -203,12 +232,14 @@ src/NLWebNet/ - **Code quality standards** - Production-level code quality maintained throughout development ### Performance Considerations + - **Streaming responses** for better perceived performance - **Async/await** throughout for scalability - **Minimal allocations** where possible - **Configuration caching** for frequently accessed settings ### Deployment Considerations + - **Requires .NET 9.0** - Latest framework dependency for modern features - **Early prerelease status** - Not yet ready for production deployment - **Production-quality code** - Library being developed with production standards @@ -217,10 +248,11 @@ src/NLWebNet/ ## When to Seek Clarification Ask for guidance when: + - **Breaking changes** to public APIs are needed - **New external dependencies** are required - **Significant architectural changes** are proposed - **Protocol compliance** questions arise - **Production deployment** patterns need to be established -Remember: This library is being developed with production-quality standards, though it is currently in early prerelease and not yet ready for production use. All code additions and edits should maintain production-level quality as the project works toward its goal of becoming a production-ready NLWeb protocol implementation for .NET applications. \ No newline at end of file +Remember: This library is being developed with production-quality standards, though it is currently in early prerelease and not yet ready for production use. All code additions and edits should maintain production-level quality as the project works toward its goal of becoming a production-ready NLWeb protocol implementation for .NET applications. diff --git a/doc/advanced-tool-system-guide.md b/doc/advanced-tool-system-guide.md new file mode 100644 index 0000000..1875b78 --- /dev/null +++ b/doc/advanced-tool-system-guide.md @@ -0,0 +1,139 @@ +# Advanced Tool System Usage Guide + +The Advanced Tool System provides enhanced query capabilities through specialized tool handlers that route queries to appropriate processors based on intent analysis. + +## Configuration + +Enable the tool system in your configuration: + +```json +{ + "NLWebNet": { + "ToolSelectionEnabled": true, + "DefaultMode": "List" + } +} +``` + +## Available Tools + +### 1. Search Tool (`search`) +Enhanced keyword and semantic search with result optimization. + +**Example Queries:** +- "search for REST API documentation" +- "find information about microservices" +- "locate best practices for authentication" + +**Features:** +- Query optimization (removes redundant search terms) +- Enhanced relevance scoring +- Result sorting and filtering + +### 2. Details Tool (`details`) +Retrieves comprehensive information about specific named items. + +**Example Queries:** +- "tell me about GraphQL" +- "what is Docker?" +- "describe OAuth 2.0" +- "information about React hooks" + +**Features:** +- Subject extraction from natural language queries +- Detailed information focus +- Comprehensive result filtering + +### 3. Compare Tool (`compare`) +Side-by-side comparison of two items or technologies. + +**Example Queries:** +- "compare React vs Vue" +- "difference between REST and GraphQL" +- "Node.js versus Python for backend" + +**Features:** +- Automatic item extraction from comparison queries +- Structured comparison results +- Side-by-side analysis + +### 4. Ensemble Tool (`ensemble`) +Creates cohesive sets of related recommendations. + +**Example Queries:** +- "recommend a full-stack JavaScript development setup" +- "suggest tools for DevOps pipeline" +- "I need a complete testing framework" + +**Features:** +- Multi-category recommendations +- Coherent item grouping +- Thematic organization + +### 5. Recipe Tool (`recipe`) +Specialized for cooking, recipes, and food-related queries. + +**Example Queries:** +- "substitute eggs in baking" +- "what goes with grilled salmon?" +- "recipe for chocolate chip cookies" + +**Features:** +- Ingredient substitution guidance +- Food pairing suggestions +- Cooking technique advice + +## Integration + +The tool system integrates automatically with your existing NLWebNet setup: + +```csharp +// Add NLWebNet services (includes tool system) +services.AddNLWebNet(options => +{ + options.ToolSelectionEnabled = true; +}); +``` + +## Backward Compatibility + +When `ToolSelectionEnabled = false`, the system uses the standard query processing pipeline, maintaining full backward compatibility. + +## Query Processing Flow + +1. **Query Analysis**: The `IToolSelector` analyzes query intent +2. **Tool Selection**: Appropriate tool is selected based on keywords and patterns +3. **Tool Execution**: The `IToolExecutor` routes to the selected tool handler +4. **Result Enhancement**: Each tool applies specialized processing +5. **Response Generation**: Results are formatted and returned + +## Custom Tool Development + +To create custom tools, implement the `IToolHandler` interface: + +```csharp +public class CustomToolHandler : BaseToolHandler +{ + public override string ToolType => "custom"; + + public override async Task ExecuteAsync( + NLWebRequest request, + CancellationToken cancellationToken = default) + { + // Custom tool logic here + return CreateSuccessResponse(request, results, processingTime); + } + + public override bool CanHandle(NLWebRequest request) + { + // Determine if this tool can handle the request + return request.Query.Contains("custom"); + } +} +``` + +Register your custom tool: + +```csharp +services.AddScoped(); +``` \ No newline at end of file diff --git a/src/NLWebNet/Extensions/ServiceCollectionExtensions.cs b/src/NLWebNet/Extensions/ServiceCollectionExtensions.cs index 6029d21..f267de5 100644 --- a/src/NLWebNet/Extensions/ServiceCollectionExtensions.cs +++ b/src/NLWebNet/Extensions/ServiceCollectionExtensions.cs @@ -9,6 +9,7 @@ using NLWebNet.Health; using NLWebNet.RateLimiting; using NLWebNet.Metrics; +using NLWebNet.Extensions; using System.Diagnostics; namespace NLWebNet; @@ -38,6 +39,9 @@ public static IServiceCollection AddNLWebNet(this IServiceCollection services, A services.AddScoped(); services.AddScoped(); + // Register Advanced Tool System (idempotent) + services.AddAdvancedToolSystem(); + // Register MCP services services.AddScoped(); // Register default data backend (can be overridden) services.AddScoped(); @@ -79,6 +83,9 @@ public static IServiceCollection AddNLWebNet(this IServiceCollecti services.AddScoped(); services.AddScoped(); + // Register Advanced Tool System + services.AddAdvancedToolSystem(); + // Register MCP services services.AddScoped(); @@ -155,6 +162,10 @@ public static IServiceCollection AddNLWebNetMultiBackend(this IServiceCollection services.AddScoped(); services.AddScoped(); + + // Register Advanced Tool System + services.AddAdvancedToolSystem(); + services.AddScoped(provider => { var options = provider.GetRequiredService>(); diff --git a/src/NLWebNet/Extensions/ToolSystemServiceCollectionExtensions.cs b/src/NLWebNet/Extensions/ToolSystemServiceCollectionExtensions.cs new file mode 100644 index 0000000..c05b380 --- /dev/null +++ b/src/NLWebNet/Extensions/ToolSystemServiceCollectionExtensions.cs @@ -0,0 +1,34 @@ +using Microsoft.Extensions.DependencyInjection; +using NLWebNet.Services; + +namespace NLWebNet.Extensions; + +/// +/// Extension methods for registering the Advanced Tool System services. +/// +public static class ToolSystemServiceCollectionExtensions +{ + /// + /// Adds the Advanced Tool System services to the dependency injection container. + /// This includes all tool handlers and the tool executor. + /// + /// The service collection + /// The service collection for chaining + public static IServiceCollection AddAdvancedToolSystem(this IServiceCollection services) + { + // Register the tool executor + services.AddScoped(); + + // Register all tool handlers + services.AddScoped(); + services.AddScoped(); + services.AddScoped(); + services.AddScoped(); + services.AddScoped(); + + // Register tool definition loader (already exists but ensure it's registered) + services.AddScoped(); + + return services; + } +} \ No newline at end of file diff --git a/src/NLWebNet/Services/BaseToolHandler.cs b/src/NLWebNet/Services/BaseToolHandler.cs new file mode 100644 index 0000000..4017b51 --- /dev/null +++ b/src/NLWebNet/Services/BaseToolHandler.cs @@ -0,0 +1,124 @@ +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using NLWebNet.Models; + +namespace NLWebNet.Services; + +/// +/// Base implementation for tool handlers providing common functionality. +/// +public abstract class BaseToolHandler : IToolHandler +{ + protected readonly ILogger Logger; + protected readonly NLWebOptions Options; + protected readonly IQueryProcessor QueryProcessor; + protected readonly IResultGenerator ResultGenerator; + + protected BaseToolHandler( + ILogger logger, + IOptions options, + IQueryProcessor queryProcessor, + IResultGenerator resultGenerator) + { + Logger = logger ?? throw new ArgumentNullException(nameof(logger)); + Options = options?.Value ?? throw new ArgumentNullException(nameof(options)); + QueryProcessor = queryProcessor ?? throw new ArgumentNullException(nameof(queryProcessor)); + ResultGenerator = resultGenerator ?? throw new ArgumentNullException(nameof(resultGenerator)); + } + + /// + public abstract string ToolType { get; } + + /// + public abstract Task ExecuteAsync(NLWebRequest request, CancellationToken cancellationToken = default); + + /// + public virtual bool CanHandle(NLWebRequest request) + { + // Base implementation - can handle if not null and has a query + return request?.Query != null && !string.IsNullOrWhiteSpace(request.Query); + } + + /// + public virtual int GetPriority(NLWebRequest request) + { + // Default priority - can be overridden by specific handlers + return 50; + } + + /// + /// Creates a standard error response for tool execution failures. + /// + /// The original request + /// The error message + /// Optional exception details + /// Error response + protected NLWebResponse CreateErrorResponse(NLWebRequest request, string errorMessage, Exception? exception = null) + { + Logger.LogError(exception, "Tool '{ToolType}' error for request {QueryId}: {ErrorMessage}", + ToolType, request.QueryId, errorMessage); + + return new NLWebResponse + { + QueryId = request.QueryId ?? string.Empty, + Query = request.Query, + Mode = request.Mode, + Results = new List + { + new NLWebResult + { + Name = "Tool Error", + Description = errorMessage, + Url = string.Empty, + Site = "System", + Score = 0.0 + } + }, + Error = errorMessage, + ProcessingTimeMs = 0, + Timestamp = DateTimeOffset.UtcNow + }; + } + + /// + /// Creates a standard success response template. + /// + /// The original request + /// The results to include + /// Processing time in milliseconds + /// Success response + protected NLWebResponse CreateSuccessResponse(NLWebRequest request, IList results, long processingTimeMs) + { + return new NLWebResponse + { + QueryId = request.QueryId ?? string.Empty, + Query = request.Query, + Mode = request.Mode, + Results = results, + Error = null, // Success means no error + ProcessingTimeMs = processingTimeMs, + Timestamp = DateTimeOffset.UtcNow + }; + } + + /// + /// Creates a tool result with proper property mapping. + /// + /// The name/title of the result + /// The description/summary of the result + /// The URL + /// The site identifier + /// The relevance score + /// A properly formatted NLWebResult + protected NLWebResult CreateToolResult(string name, string description, string url = "", string site = "", double score = 1.0) + { + return new NLWebResult + { + Name = name, + Description = description, + Url = url, + Site = string.IsNullOrEmpty(site) ? ToolType : site, + Score = score + }; + } +} \ No newline at end of file diff --git a/src/NLWebNet/Services/CompareToolHandler.cs b/src/NLWebNet/Services/CompareToolHandler.cs new file mode 100644 index 0000000..d433817 --- /dev/null +++ b/src/NLWebNet/Services/CompareToolHandler.cs @@ -0,0 +1,191 @@ +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using NLWebNet.Models; +using System.Diagnostics; +using System.Text.RegularExpressions; + +namespace NLWebNet.Services; + +/// +/// Tool handler for side-by-side comparison of two items. +/// Analyzes queries to identify comparison subjects and provides structured comparison results. +/// +public class CompareToolHandler : BaseToolHandler +{ + public CompareToolHandler( + ILogger logger, + IOptions options, + IQueryProcessor queryProcessor, + IResultGenerator resultGenerator) + : base(logger, options, queryProcessor, resultGenerator) + { + } + + /// + public override string ToolType => "compare"; + + /// + public override async Task ExecuteAsync(NLWebRequest request, CancellationToken cancellationToken = default) + { + var stopwatch = Stopwatch.StartNew(); + + try + { + Logger.LogDebug("Executing compare tool for query: {Query}", request.Query); + + // Extract the items to compare from the query + var comparisonItems = ExtractComparisonItems(request.Query); + if (comparisonItems.Item1 == null || comparisonItems.Item2 == null) + { + return CreateErrorResponse(request, "Could not identify two items to compare"); + } + + Logger.LogDebug("Comparing '{Item1}' vs '{Item2}'", comparisonItems.Item1, comparisonItems.Item2); + + // Create comparison query + var comparisonQuery = $"{comparisonItems.Item1} vs {comparisonItems.Item2} comparison differences"; + + // Generate comparison results + var searchResults = await ResultGenerator.GenerateListAsync(comparisonQuery, request.Site, cancellationToken); + var resultsList = searchResults.ToList(); + + // Create structured comparison results + var comparisonResults = CreateComparisonResults(resultsList, comparisonItems.Item1, comparisonItems.Item2); + + stopwatch.Stop(); + + var response = CreateSuccessResponse(request, comparisonResults, stopwatch.ElapsedMilliseconds); + response.ProcessedQuery = comparisonQuery; + response.Summary = $"Comparison completed between '{comparisonItems.Item1}' and '{comparisonItems.Item2}'"; + + Logger.LogDebug("Compare tool completed in {ElapsedMs}ms", stopwatch.ElapsedMilliseconds); + + return response; + } + catch (Exception ex) + { + stopwatch.Stop(); + return CreateErrorResponse(request, $"Compare tool execution failed: {ex.Message}", ex); + } + } + + /// + public override bool CanHandle(NLWebRequest request) + { + if (!base.CanHandle(request)) + return false; + + var query = request.Query?.ToLowerInvariant() ?? string.Empty; + + // Can handle queries that contain comparison keywords + var compareKeywords = new[] + { + "compare", "vs", "versus", "difference", "differences", + "contrast", "better", "worse", "pros and cons", "which is better" + }; + + return compareKeywords.Any(keyword => query.Contains(keyword)); + } + + /// + public override int GetPriority(NLWebRequest request) + { + var query = request.Query?.ToLowerInvariant() ?? string.Empty; + + // Higher priority for explicit comparison requests + if (query.Contains(" vs ") || query.Contains(" versus ") || query.StartsWith("compare")) + return 95; + + // High priority for difference queries + if (query.Contains("difference") || query.Contains("contrast")) + return 85; + + // Medium priority for other comparison-related queries + return 70; + } + + /// + /// Extracts the two items to compare from the query. + /// + private (string? Item1, string? Item2) ExtractComparisonItems(string query) + { + if (string.IsNullOrWhiteSpace(query)) + return (null, null); + + var queryLower = query.ToLowerInvariant().Trim(); + + // Common comparison patterns + var patterns = new[] + { + // "compare A vs B" or "compare A versus B" + @"compare\s+(.+?)\s+(?:vs|versus)\s+(.+)", + // "A vs B" or "A versus B" + @"(.+?)\s+(?:vs|versus)\s+(.+)", + // "difference between A and B" + @"difference\s+between\s+(.+?)\s+and\s+(.+)", + }; + + foreach (var pattern in patterns) + { + var match = Regex.Match(queryLower, pattern, RegexOptions.IgnoreCase); + if (match.Success && match.Groups.Count > 2) + { + var item1 = match.Groups[1].Value.Trim(); + var item2 = match.Groups[2].Value.Trim(); + + if (!string.IsNullOrWhiteSpace(item1) && !string.IsNullOrWhiteSpace(item2)) + { + return (item1, item2); + } + } + } + + return (null, null); + } + + /// + /// Creates structured comparison results. + /// + private IList CreateComparisonResults(IList results, string item1, string item2) + { + var comparisonResults = new List(); + + // Create summary comparison result + comparisonResults.Add(CreateToolResult( + $"Comparison: {item1} vs {item2}", + $"Side-by-side comparison analysis of {item1} and {item2}", + "", + "Compare", + 1.0 + )); + + // Add relevant comparison results + var relevantResults = results + .Where(r => IsRelevantForComparison(r, item1, item2)) + .Take(8) + .ToList(); + + foreach (var result in relevantResults) + { + comparisonResults.Add(CreateToolResult( + $"[Compare] {result.Name}", + result.Description, + result.Url, + result.Site ?? "Compare", + result.Score + )); + } + + return comparisonResults; + } + + /// + /// Checks if a result is relevant for comparison. + /// + private bool IsRelevantForComparison(NLWebResult result, string item1, string item2) + { + var text = $"{result.Name} {result.Description}".ToLowerInvariant(); + return text.Contains(item1.ToLowerInvariant()) || text.Contains(item2.ToLowerInvariant()) || + text.Contains("compare") || text.Contains("difference") || text.Contains("versus"); + } +} \ No newline at end of file diff --git a/src/NLWebNet/Services/DetailsToolHandler.cs b/src/NLWebNet/Services/DetailsToolHandler.cs new file mode 100644 index 0000000..0c613fe --- /dev/null +++ b/src/NLWebNet/Services/DetailsToolHandler.cs @@ -0,0 +1,223 @@ +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using NLWebNet.Models; +using System.Diagnostics; +using System.Text.RegularExpressions; + +namespace NLWebNet.Services; + +/// +/// Tool handler for retrieving specific information about named items. +/// Focuses on providing detailed, comprehensive information about specific entities. +/// +public class DetailsToolHandler : BaseToolHandler +{ + public DetailsToolHandler( + ILogger logger, + IOptions options, + IQueryProcessor queryProcessor, + IResultGenerator resultGenerator) + : base(logger, options, queryProcessor, resultGenerator) + { + } + + /// + public override string ToolType => "details"; + + /// + public override async Task ExecuteAsync(NLWebRequest request, CancellationToken cancellationToken = default) + { + var stopwatch = Stopwatch.StartNew(); + + try + { + Logger.LogDebug("Executing details tool for query: {Query}", request.Query); + + // Extract the subject entity from the query + var subject = ExtractSubject(request.Query); + if (string.IsNullOrWhiteSpace(subject)) + { + return CreateErrorResponse(request, "Could not identify the subject to get details about"); + } + + Logger.LogDebug("Extracted subject '{Subject}' from query", subject); + + // Create details-focused query + var detailsQuery = $"{subject} overview definition explanation details"; + + // Generate detailed results + var searchResults = await ResultGenerator.GenerateListAsync(detailsQuery, request.Site, cancellationToken); + var resultsList = searchResults.ToList(); + + // Enhance results for details focus + var detailsResults = EnhanceDetailsResults(resultsList, subject); + + stopwatch.Stop(); + + var response = CreateSuccessResponse(request, detailsResults, stopwatch.ElapsedMilliseconds); + response.ProcessedQuery = detailsQuery; + response.Summary = $"Details retrieved for '{subject}' - found {detailsResults.Count} detailed results"; + + Logger.LogDebug("Details tool completed in {ElapsedMs}ms for subject '{Subject}'", + stopwatch.ElapsedMilliseconds, subject); + + return response; + } + catch (Exception ex) + { + stopwatch.Stop(); + return CreateErrorResponse(request, $"Details tool execution failed: {ex.Message}", ex); + } + } + + /// + public override bool CanHandle(NLWebRequest request) + { + if (!base.CanHandle(request)) + return false; + + var query = request.Query?.ToLowerInvariant() ?? string.Empty; + + // Can handle queries that ask for details, information, or descriptions + var detailsKeywords = new[] + { + "details", "information about", "tell me about", "describe", + "what is", "explain", "definition of", "overview of" + }; + + return detailsKeywords.Any(keyword => query.Contains(keyword)); + } + + /// + public override int GetPriority(NLWebRequest request) + { + var query = request.Query?.ToLowerInvariant() ?? string.Empty; + + // Higher priority for explicit details requests + if (query.StartsWith("tell me about") || query.StartsWith("what is") || query.Contains("details about")) + return 90; + + // Medium-high priority for informational queries + if (query.Contains("information about") || query.Contains("describe")) + return 75; + + // Default priority for details-related queries + return 65; + } + + /// + /// Extracts the subject entity from a details query. + /// + private string ExtractSubject(string query) + { + if (string.IsNullOrWhiteSpace(query)) + return string.Empty; + + var queryLower = query.ToLowerInvariant().Trim(); + + // Common patterns for details queries + var patterns = new[] + { + @"(?:tell me about|information about|details about|describe)\s+(.+)", + @"(?:what is|what are)\s+(.+)", + @"(?:explain|definition of|overview of)\s+(.+)", + }; + + foreach (var pattern in patterns) + { + var match = Regex.Match(queryLower, pattern, RegexOptions.IgnoreCase); + if (match.Success && match.Groups.Count > 1) + { + return match.Groups[1].Value.Trim(); + } + } + + // If no pattern matches, return the whole query + return query; + } + + /// + /// Enhances results to focus on detailed information. + /// + private IList EnhanceDetailsResults(IList results, string subject) + { + if (results == null || !results.Any()) + return Array.Empty(); + + // Filter and rank results by their detail relevance + var detailResults = results + .Select(r => new { Result = r, Score = CalculateDetailsRelevance(r, subject) }) + .Where(x => x.Score > 0) + .OrderByDescending(x => x.Score) + .Take(10) // Focus on top detailed results + .Select(x => x.Result) + .ToList(); + + // Enhance results with details-specific formatting + foreach (var result in detailResults) + { + // Enhance name to indicate it's a details result + if (!string.IsNullOrEmpty(result.Name) && !result.Name.ToLowerInvariant().Contains("details")) + { + result.Name = $"Details: {result.Name}"; + } + + // Set site to indicate details processing + if (string.IsNullOrEmpty(result.Site)) + { + result.Site = "Details"; + } + } + + return detailResults; + } + + /// + /// Calculates how relevant a result is for providing details about the subject. + /// + private double CalculateDetailsRelevance(NLWebResult result, string subject) + { + if (result == null || string.IsNullOrWhiteSpace(subject)) + return result?.Score ?? 0.0; + + double score = result.Score; + var subjectLower = subject.ToLowerInvariant(); + var subjectTerms = subjectLower.Split(' ', StringSplitOptions.RemoveEmptyEntries); + + // Check if result contains comprehensive information + var detailsIndicators = new[] { "overview", "introduction", "definition", "explanation", "guide", "about" }; + + // Name relevance with details indicators + if (!string.IsNullOrEmpty(result.Name)) + { + var nameLower = result.Name.ToLowerInvariant(); + + // High score for exact subject match in name + if (subjectTerms.All(term => nameLower.Contains(term))) + score += 5.0; + + // Bonus for details indicators + foreach (var indicator in detailsIndicators) + { + if (nameLower.Contains(indicator)) + score += 2.0; + } + } + + // Description relevance + if (!string.IsNullOrEmpty(result.Description)) + { + var descriptionLower = result.Description.ToLowerInvariant(); + + // Score for subject terms in description + var matchingTerms = subjectTerms.Count(term => descriptionLower.Contains(term)); + score += matchingTerms * 1.5; + + // Bonus for comprehensive description (longer, more detailed) + if (result.Description.Length > 100) + score += 1.0; + } + + return score; + } +} \ No newline at end of file diff --git a/src/NLWebNet/Services/EnsembleToolHandler.cs b/src/NLWebNet/Services/EnsembleToolHandler.cs new file mode 100644 index 0000000..0b79671 --- /dev/null +++ b/src/NLWebNet/Services/EnsembleToolHandler.cs @@ -0,0 +1,134 @@ +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using NLWebNet.Models; +using System.Diagnostics; + +namespace NLWebNet.Services; + +/// +/// Tool handler for creating cohesive sets of related items. +/// Handles queries like "Give me an appetizer, main and dessert for an Italian dinner" +/// or "I'm visiting Seattle for a day - suggest museums and nearby restaurants". +/// +public class EnsembleToolHandler : BaseToolHandler +{ + public EnsembleToolHandler( + ILogger logger, + IOptions options, + IQueryProcessor queryProcessor, + IResultGenerator resultGenerator) + : base(logger, options, queryProcessor, resultGenerator) + { + } + + /// + public override string ToolType => "ensemble"; + + /// + public override async Task ExecuteAsync(NLWebRequest request, CancellationToken cancellationToken = default) + { + var stopwatch = Stopwatch.StartNew(); + + try + { + Logger.LogDebug("Executing ensemble tool for query: {Query}", request.Query); + + // Create ensemble-focused query + var ensembleQuery = $"{request.Query} recommendations suggestions set"; + + // Generate ensemble results + var searchResults = await ResultGenerator.GenerateListAsync(ensembleQuery, request.Site, cancellationToken); + var resultsList = searchResults.ToList(); + + // Create ensemble response + var ensembleResults = CreateEnsembleResults(resultsList, request.Query); + + stopwatch.Stop(); + + var response = CreateSuccessResponse(request, ensembleResults, stopwatch.ElapsedMilliseconds); + response.ProcessedQuery = ensembleQuery; + response.Summary = $"Ensemble recommendations created for your request"; + + Logger.LogDebug("Ensemble tool completed in {ElapsedMs}ms", stopwatch.ElapsedMilliseconds); + + return response; + } + catch (Exception ex) + { + stopwatch.Stop(); + return CreateErrorResponse(request, $"Ensemble tool execution failed: {ex.Message}", ex); + } + } + + /// + public override bool CanHandle(NLWebRequest request) + { + if (!base.CanHandle(request)) + return false; + + var query = request.Query?.ToLowerInvariant() ?? string.Empty; + + // Can handle queries that ask for recommendations, suggestions, or sets of items + var ensembleKeywords = new[] + { + "recommend", "suggest", "give me", "plan", "set of", "ensemble", + "what should i", "help me choose", "i need", "looking for" + }; + + return ensembleKeywords.Any(keyword => query.Contains(keyword)); + } + + /// + public override int GetPriority(NLWebRequest request) + { + var query = request.Query?.ToLowerInvariant() ?? string.Empty; + + // Higher priority for explicit ensemble requests + if (query.Contains("give me") && (query.Contains(" and ") || query.Contains(", "))) + return 90; + + // High priority for planning requests + if (query.StartsWith("plan") || query.Contains("help me plan")) + return 85; + + // Medium-high priority for recommendation requests with multiple items + if ((query.Contains("recommend") || query.Contains("suggest")) && + (query.Contains(" and ") || query.Contains(", "))) + return 80; + + // Default priority for ensemble-related queries + return 65; + } + + /// + /// Creates ensemble results from search results. + /// + private IList CreateEnsembleResults(IList results, string originalQuery) + { + var ensembleResults = new List(); + + // Create ensemble overview + ensembleResults.Add(CreateToolResult( + "Curated Ensemble Recommendations", + $"A carefully selected collection of recommendations based on your request: {originalQuery}", + "", + "Ensemble", + 1.0 + )); + + // Add categorized results + var categorizedResults = results + .Take(10) + .Select((result, index) => CreateToolResult( + $"[Option {index + 1}] {result.Name}", + result.Description, + result.Url, + result.Site ?? "Ensemble", + result.Score + )) + .ToList(); + + ensembleResults.AddRange(categorizedResults); + return ensembleResults; + } +} \ No newline at end of file diff --git a/src/NLWebNet/Services/IToolExecutor.cs b/src/NLWebNet/Services/IToolExecutor.cs new file mode 100644 index 0000000..25ea847 --- /dev/null +++ b/src/NLWebNet/Services/IToolExecutor.cs @@ -0,0 +1,78 @@ +using Microsoft.Extensions.Logging; +using NLWebNet.Models; + +namespace NLWebNet.Services; + +/// +/// Interface for executing tools based on tool selection results. +/// +public interface IToolExecutor +{ + /// + /// Executes the appropriate tool handler for the given request and selected tool. + /// + /// The NLWeb request to process + /// The tool selected by the tool selector + /// Cancellation token for async operations + /// The processed response from the tool handler + Task ExecuteToolAsync(NLWebRequest request, string selectedTool, CancellationToken cancellationToken = default); + + /// + /// Gets all available tool handlers. + /// + /// Collection of available tool handlers + IEnumerable GetAvailableTools(); +} + +/// +/// Implementation of tool execution logic that routes requests to appropriate tool handlers. +/// +public class ToolExecutor : IToolExecutor +{ + private readonly IEnumerable _toolHandlers; + private readonly ILogger _logger; + + public ToolExecutor(IEnumerable toolHandlers, ILogger logger) + { + _toolHandlers = toolHandlers ?? throw new ArgumentNullException(nameof(toolHandlers)); + _logger = logger ?? throw new ArgumentNullException(nameof(logger)); + } + + /// + public async Task ExecuteToolAsync(NLWebRequest request, string selectedTool, CancellationToken cancellationToken = default) + { + _logger.LogDebug("Executing tool '{Tool}' for request {QueryId}", selectedTool, request.QueryId); + + var handler = _toolHandlers + .Where(h => h.ToolType.Equals(selectedTool, StringComparison.OrdinalIgnoreCase)) + .Where(h => h.CanHandle(request)) + .OrderByDescending(h => h.GetPriority(request)) + .FirstOrDefault(); + + if (handler == null) + { + _logger.LogWarning("No handler found for tool '{Tool}' that can process request {QueryId}", selectedTool, request.QueryId); + throw new InvalidOperationException($"No handler available for tool '{selectedTool}'"); + } + + _logger.LogDebug("Using handler {HandlerType} for tool '{Tool}'", handler.GetType().Name, selectedTool); + + try + { + var response = await handler.ExecuteAsync(request, cancellationToken); + _logger.LogDebug("Tool '{Tool}' execution completed for request {QueryId}", selectedTool, request.QueryId); + return response; + } + catch (Exception ex) + { + _logger.LogError(ex, "Tool '{Tool}' execution failed for request {QueryId}", selectedTool, request.QueryId); + throw; + } + } + + /// + public IEnumerable GetAvailableTools() + { + return _toolHandlers.ToList(); + } +} \ No newline at end of file diff --git a/src/NLWebNet/Services/IToolHandler.cs b/src/NLWebNet/Services/IToolHandler.cs new file mode 100644 index 0000000..da34e88 --- /dev/null +++ b/src/NLWebNet/Services/IToolHandler.cs @@ -0,0 +1,38 @@ +using NLWebNet.Models; + +namespace NLWebNet.Services; + +/// +/// Base interface for all tool handlers in the Advanced Tool System. +/// Tool handlers process specific types of queries based on intent analysis. +/// +public interface IToolHandler +{ + /// + /// The tool type this handler supports (e.g., "search", "details", "compare", "ensemble", "recipe"). + /// + string ToolType { get; } + + /// + /// Executes the tool functionality for the given request. + /// + /// The NLWeb request to process + /// Cancellation token for async operations + /// The processed response + Task ExecuteAsync(NLWebRequest request, CancellationToken cancellationToken = default); + + /// + /// Determines if this tool handler can process the given request. + /// + /// The request to analyze + /// True if this handler can process the request, false otherwise + bool CanHandle(NLWebRequest request); + + /// + /// Gets the priority of this handler for the given request. + /// Higher values indicate higher priority. + /// + /// The request to analyze + /// Priority value (0-100) + int GetPriority(NLWebRequest request); +} \ No newline at end of file diff --git a/src/NLWebNet/Services/NLWebService.cs b/src/NLWebNet/Services/NLWebService.cs index e2f4273..c612308 100644 --- a/src/NLWebNet/Services/NLWebService.cs +++ b/src/NLWebNet/Services/NLWebService.cs @@ -14,6 +14,8 @@ public class NLWebService : INLWebService private readonly IResultGenerator _resultGenerator; private readonly IDataBackend? _dataBackend; private readonly IBackendManager? _backendManager; + private readonly IToolSelector? _toolSelector; + private readonly IToolExecutor? _toolExecutor; private readonly ILogger _logger; private readonly NLWebOptions _options; @@ -51,6 +53,48 @@ public NLWebService( _options = options?.Value ?? throw new ArgumentNullException(nameof(options)); } + /// + /// Constructor for single-backend mode with tool support. + /// + public NLWebService( + IQueryProcessor queryProcessor, + IResultGenerator resultGenerator, + IDataBackend dataBackend, + IToolSelector toolSelector, + IToolExecutor toolExecutor, + ILogger logger, + IOptions options) + { + _queryProcessor = queryProcessor ?? throw new ArgumentNullException(nameof(queryProcessor)); + _resultGenerator = resultGenerator ?? throw new ArgumentNullException(nameof(resultGenerator)); + _dataBackend = dataBackend ?? throw new ArgumentNullException(nameof(dataBackend)); + _toolSelector = toolSelector ?? throw new ArgumentNullException(nameof(toolSelector)); + _toolExecutor = toolExecutor ?? throw new ArgumentNullException(nameof(toolExecutor)); + _logger = logger ?? throw new ArgumentNullException(nameof(logger)); + _options = options?.Value ?? throw new ArgumentNullException(nameof(options)); + } + + /// + /// Constructor for multi-backend mode with tool support. + /// + public NLWebService( + IQueryProcessor queryProcessor, + IResultGenerator resultGenerator, + IBackendManager backendManager, + IToolSelector toolSelector, + IToolExecutor toolExecutor, + ILogger logger, + IOptions options) + { + _queryProcessor = queryProcessor ?? throw new ArgumentNullException(nameof(queryProcessor)); + _resultGenerator = resultGenerator ?? throw new ArgumentNullException(nameof(resultGenerator)); + _backendManager = backendManager ?? throw new ArgumentNullException(nameof(backendManager)); + _toolSelector = toolSelector ?? throw new ArgumentNullException(nameof(toolSelector)); + _toolExecutor = toolExecutor ?? throw new ArgumentNullException(nameof(toolExecutor)); + _logger = logger ?? throw new ArgumentNullException(nameof(logger)); + _options = options?.Value ?? throw new ArgumentNullException(nameof(options)); + } + /// public async Task ProcessRequestAsync(NLWebRequest request, CancellationToken cancellationToken = default) { @@ -72,6 +116,14 @@ public async Task ProcessRequestAsync(NLWebRequest request, Cance var processedQuery = await _queryProcessor.ProcessQueryAsync(request, cancellationToken); _logger.LogDebug("ProcessQueryAsync complete for QueryId={QueryId}", queryId); + // Check if tool execution is available and enabled + var toolResponse = await TryExecuteToolAsync(request, queryId, cancellationToken); + if (toolResponse != null) + { + return toolResponse; + } + + _logger.LogDebug("Using standard processing pipeline for QueryId={QueryId}", queryId); _logger.LogDebug("Calling GenerateListAsync for QueryId={QueryId}", queryId); var searchResults = await _resultGenerator.GenerateListAsync(processedQuery, request.Site, cancellationToken); var resultsList = searchResults.ToList(); @@ -150,6 +202,58 @@ private async IAsyncEnumerable ProcessRequestStreamInternalAsync( NLWebRequest request, string queryId, [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + // Check if tool execution is available and enabled first + if (_toolSelector != null && _toolExecutor != null && _options.ToolSelectionEnabled) + { + _logger.LogDebug("[StreamInternal] Tool execution enabled, checking if tool selection is needed for QueryId={QueryId}", queryId); + + if (_toolSelector.ShouldSelectTool(request)) + { + _logger.LogDebug("[StreamInternal] Tool selection needed for QueryId={QueryId}", queryId); + + var toolResponse = await TryExecuteToolAsync(request, queryId, cancellationToken); + if (toolResponse != null) + { + yield return toolResponse; + yield break; + } + // If tool execution failed, fall through to standard processing + } + } + + // Standard processing pipeline + await foreach (var response in ProcessStandardStreamingAsync(request, queryId, cancellationToken)) + { + yield return response; + } + } + + private async Task TryExecuteToolAsync(NLWebRequest request, string queryId, CancellationToken cancellationToken) + { + try + { + var selectedTool = await _toolSelector!.SelectToolAsync(request, cancellationToken); + if (!string.IsNullOrEmpty(selectedTool)) + { + _logger.LogInformation("[StreamInternal] Tool '{Tool}' selected for QueryId={QueryId}, executing tool", selectedTool, queryId); + + var toolResponse = await _toolExecutor!.ExecuteToolAsync(request, selectedTool, cancellationToken); + _logger.LogInformation("[StreamInternal] Tool execution completed for QueryId={QueryId} with tool '{Tool}'", queryId, selectedTool); + return toolResponse; + } + } + catch (Exception toolEx) + { + _logger.LogError(toolEx, "[StreamInternal] Tool execution failed for QueryId={QueryId}, falling back to standard processing", queryId); + } + return null; + } + + private async IAsyncEnumerable ProcessStandardStreamingAsync( + NLWebRequest request, + string queryId, + [EnumeratorCancellation] CancellationToken cancellationToken = default) { IAsyncEnumerable? responseStream = null; Exception? processingException = null; @@ -158,6 +262,7 @@ private async IAsyncEnumerable ProcessRequestStreamInternalAsync( bool hadException = false; try { + _logger.LogDebug("[StreamInternal] Using standard processing pipeline for QueryId={QueryId}", queryId); _logger.LogDebug("[StreamInternal] Calling ProcessQueryAsync for QueryId={QueryId}", queryId); var processedQuery = await _queryProcessor.ProcessQueryAsync(request, cancellationToken); _logger.LogDebug("[StreamInternal] ProcessQueryAsync complete for QueryId={QueryId}", queryId); diff --git a/src/NLWebNet/Services/RecipeToolHandler.cs b/src/NLWebNet/Services/RecipeToolHandler.cs new file mode 100644 index 0000000..19052fc --- /dev/null +++ b/src/NLWebNet/Services/RecipeToolHandler.cs @@ -0,0 +1,163 @@ +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using NLWebNet.Models; +using System.Diagnostics; + +namespace NLWebNet.Services; + +/// +/// Tool handler for recipe-related queries including ingredient substitutions and accompaniment suggestions. +/// Handles cooking, recipe, and food-related queries with specialized knowledge. +/// +public class RecipeToolHandler : BaseToolHandler +{ + public RecipeToolHandler( + ILogger logger, + IOptions options, + IQueryProcessor queryProcessor, + IResultGenerator resultGenerator) + : base(logger, options, queryProcessor, resultGenerator) + { + } + + /// + public override string ToolType => "recipe"; + + /// + public override async Task ExecuteAsync(NLWebRequest request, CancellationToken cancellationToken = default) + { + var stopwatch = Stopwatch.StartNew(); + + try + { + Logger.LogDebug("Executing recipe tool for query: {Query}", request.Query); + + // Create recipe-focused query + var recipeQuery = $"{request.Query} recipe cooking instructions"; + + // Generate recipe results + var searchResults = await ResultGenerator.GenerateListAsync(recipeQuery, request.Site, cancellationToken); + var resultsList = searchResults.ToList(); + + // Create recipe-specific results + var recipeResults = CreateRecipeResults(resultsList, request.Query); + + stopwatch.Stop(); + + var response = CreateSuccessResponse(request, recipeResults, stopwatch.ElapsedMilliseconds); + response.ProcessedQuery = recipeQuery; + response.Summary = $"Recipe information and cooking guidance provided"; + + Logger.LogDebug("Recipe tool completed in {ElapsedMs}ms", stopwatch.ElapsedMilliseconds); + + return response; + } + catch (Exception ex) + { + stopwatch.Stop(); + return CreateErrorResponse(request, $"Recipe tool execution failed: {ex.Message}", ex); + } + } + + /// + public override bool CanHandle(NLWebRequest request) + { + if (!base.CanHandle(request)) + return false; + + var query = request.Query?.ToLowerInvariant() ?? string.Empty; + + // Can handle recipe, cooking, and food-related queries + var recipeKeywords = new[] + { + "recipe", "cooking", "cook", "ingredient", "substitute", "substitution", + "bake", "baking", "preparation", "kitchen", "culinary", "food", + "accompaniment", "side dish", "pair with", "serve with", "goes with" + }; + + return recipeKeywords.Any(keyword => query.Contains(keyword)); + } + + /// + public override int GetPriority(NLWebRequest request) + { + var query = request.Query?.ToLowerInvariant() ?? string.Empty; + + // Higher priority for specific recipe operations + if (query.Contains("substitute") || query.Contains("substitution")) + return 95; + + if (query.Contains("recipe for") || query.Contains("how to cook")) + return 90; + + if (query.Contains("serve with") || query.Contains("goes with")) + return 85; + + // Medium priority for general cooking queries + return 70; + } + + /// + /// Creates recipe-specific results from search results. + /// + private IList CreateRecipeResults(IList results, string originalQuery) + { + var recipeResults = new List(); + + // Determine query type and create appropriate header + var queryType = DetermineQueryType(originalQuery); + recipeResults.Add(CreateToolResult( + $"Recipe Guide: {queryType}", + $"Culinary information and guidance for: {originalQuery}", + "", + "Recipe", + 1.0 + )); + + // Add recipe-specific results + var relevantResults = results + .Where(r => IsRelevantForRecipe(r, originalQuery)) + .Take(8) + .Select(result => CreateToolResult( + $"[{queryType}] {result.Name}", + result.Description, + result.Url, + result.Site ?? "Recipe", + result.Score + )) + .ToList(); + + recipeResults.AddRange(relevantResults); + return recipeResults; + } + + /// + /// Determines the type of recipe query. + /// + private string DetermineQueryType(string query) + { + var queryLower = query.ToLowerInvariant(); + + if (queryLower.Contains("substitute")) + return "Substitution"; + if (queryLower.Contains("serve with") || queryLower.Contains("goes with")) + return "Pairing"; + if (queryLower.Contains("recipe")) + return "Recipe"; + if (queryLower.Contains("cook") || queryLower.Contains("bake")) + return "Technique"; + + return "Cooking"; + } + + /// + /// Checks if a result is relevant for recipe queries. + /// + private bool IsRelevantForRecipe(NLWebResult result, string originalQuery) + { + var text = $"{result.Name} {result.Description}".ToLowerInvariant(); + var recipeTerms = new[] { "recipe", "cooking", "food", "ingredient", "kitchen", "culinary" }; + + return recipeTerms.Any(term => text.Contains(term)); + } +} \ No newline at end of file diff --git a/src/NLWebNet/Services/SearchToolHandler.cs b/src/NLWebNet/Services/SearchToolHandler.cs new file mode 100644 index 0000000..d6ba1de --- /dev/null +++ b/src/NLWebNet/Services/SearchToolHandler.cs @@ -0,0 +1,180 @@ +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using NLWebNet.Models; +using System.Diagnostics; + +namespace NLWebNet.Services; + +/// +/// Tool handler for enhanced keyword and semantic search operations. +/// This is an upgrade of the current search capability with enhanced features. +/// +public class SearchToolHandler : BaseToolHandler +{ + public SearchToolHandler( + ILogger logger, + IOptions options, + IQueryProcessor queryProcessor, + IResultGenerator resultGenerator) + : base(logger, options, queryProcessor, resultGenerator) + { + } + + /// + public override string ToolType => "search"; + + /// + public override async Task ExecuteAsync(NLWebRequest request, CancellationToken cancellationToken = default) + { + var stopwatch = Stopwatch.StartNew(); + + try + { + Logger.LogDebug("Executing search tool for query: {Query}", request.Query); + + // Process query for enhanced search + var processedQuery = await QueryProcessor.ProcessQueryAsync(request, cancellationToken); + var enhancedQuery = await OptimizeSearchQuery(processedQuery, cancellationToken); + + // Generate search results using the existing result generator + var searchResults = await ResultGenerator.GenerateListAsync(enhancedQuery, request.Site, cancellationToken); + var resultsList = searchResults.ToList(); + + // Enhance results for search-specific improvements + var enhancedResults = EnhanceSearchResults(resultsList, request.Query); + + stopwatch.Stop(); + + var response = CreateSuccessResponse(request, enhancedResults, stopwatch.ElapsedMilliseconds); + response.ProcessedQuery = enhancedQuery; + response.Summary = $"Enhanced search completed - found {enhancedResults.Count} results"; + + Logger.LogDebug("Search tool completed in {ElapsedMs}ms with {ResultCount} results", + stopwatch.ElapsedMilliseconds, enhancedResults.Count); + + return response; + } + catch (Exception ex) + { + stopwatch.Stop(); + return CreateErrorResponse(request, $"Search tool execution failed: {ex.Message}", ex); + } + } + + /// + public override bool CanHandle(NLWebRequest request) + { + if (!base.CanHandle(request)) + return false; + + // Search tool can handle most general queries + // It's designed as a fallback for general search operations + return true; + } + + /// + public override int GetPriority(NLWebRequest request) + { + var query = request.Query?.ToLowerInvariant() ?? string.Empty; + + // Higher priority for explicit search terms + if (ContainsSearchKeywords(query)) + return 80; + + // Medium priority for general queries (search is often the default) + return 60; + } + + /// + /// Optimizes the search query for better results. + /// + private Task OptimizeSearchQuery(string query, CancellationToken cancellationToken) + { + // Basic query optimization - in production this could use ML models + var optimized = query.Trim(); + + // Remove redundant search terms + var searchTerms = new[] { "search for", "find", "look for", "locate" }; + foreach (var term in searchTerms) + { + if (optimized.StartsWith(term, StringComparison.OrdinalIgnoreCase)) + { + optimized = optimized.Substring(term.Length).Trim(); + break; + } + } + + return Task.FromResult(optimized); + } + + /// + /// Enhances search results with search-specific improvements. + /// + private IList EnhanceSearchResults(IList results, string originalQuery) + { + if (results == null || !results.Any()) + return Array.Empty(); + + // Sort results by relevance (simple implementation) + var sortedResults = results + .OrderByDescending(r => CalculateSearchRelevance(r, originalQuery)) + .ToList(); + + // Add search-specific metadata + foreach (var result in sortedResults) + { + if (string.IsNullOrEmpty(result.Site)) + { + result.Site = "Search"; + } + } + + return sortedResults; + } + + /// + /// Calculates search relevance score for a result. + /// + private double CalculateSearchRelevance(NLWebResult result, string query) + { + if (result == null || string.IsNullOrWhiteSpace(query)) + return result?.Score ?? 0.0; + + double score = result.Score; + var queryLower = query.ToLowerInvariant(); + var queryTerms = queryLower.Split(' ', StringSplitOptions.RemoveEmptyEntries); + + // Name relevance (higher weight) + if (!string.IsNullOrEmpty(result.Name)) + { + var nameLower = result.Name.ToLowerInvariant(); + foreach (var term in queryTerms) + { + if (nameLower.Contains(term)) + score += 3.0; + } + } + + // Description relevance (medium weight) + if (!string.IsNullOrEmpty(result.Description)) + { + var descriptionLower = result.Description.ToLowerInvariant(); + foreach (var term in queryTerms) + { + if (descriptionLower.Contains(term)) + score += 2.0; + } + } + + return score; + } + + /// + /// Checks if the query contains explicit search keywords. + /// + private static bool ContainsSearchKeywords(string query) + { + var searchKeywords = new[] { "search", "find", "look for", "locate", "discover" }; + return searchKeywords.Any(keyword => query.Contains(keyword)); + } +} \ No newline at end of file diff --git a/tests/NLWebNet.Tests/Services/ToolExecutorTests.cs b/tests/NLWebNet.Tests/Services/ToolExecutorTests.cs new file mode 100644 index 0000000..f0132ae --- /dev/null +++ b/tests/NLWebNet.Tests/Services/ToolExecutorTests.cs @@ -0,0 +1,121 @@ +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using Microsoft.VisualStudio.TestTools.UnitTesting; +using NLWebNet.Extensions; +using NLWebNet.Models; +using NLWebNet.Services; + +namespace NLWebNet.Tests.Services; + +[TestClass] +public class ToolExecutorTests +{ + private ServiceProvider _serviceProvider = null!; + private IToolExecutor _toolExecutor = null!; + + [TestInitialize] + public void Setup() + { + var services = new ServiceCollection(); + + // Add logging + services.AddLogging(builder => builder.AddConsole()); + + // Configure NLWebNet options + services.Configure(options => + { + options.ToolSelectionEnabled = true; + options.DefaultMode = QueryMode.List; + }); + + // Add NLWebNet services with tool system + services.AddNLWebNet(); + + _serviceProvider = services.BuildServiceProvider(); + _toolExecutor = _serviceProvider.GetRequiredService(); + } + + [TestCleanup] + public void Cleanup() + { + _serviceProvider?.Dispose(); + } + + [TestMethod] + public void GetAvailableTools_ShouldReturnAllToolHandlers() + { + // Act + var tools = _toolExecutor.GetAvailableTools().ToList(); + + // Assert + Assert.IsTrue(tools.Count > 0, "Should have at least one tool handler"); + + var toolTypes = tools.Select(t => t.ToolType).ToList(); + Assert.IsTrue(toolTypes.Contains("search"), "Should include search tool"); + Assert.IsTrue(toolTypes.Contains("details"), "Should include details tool"); + Assert.IsTrue(toolTypes.Contains("compare"), "Should include compare tool"); + Assert.IsTrue(toolTypes.Contains("ensemble"), "Should include ensemble tool"); + Assert.IsTrue(toolTypes.Contains("recipe"), "Should include recipe tool"); + } + + [TestMethod] + public async Task ExecuteToolAsync_WithSearchTool_ShouldReturnResponse() + { + // Arrange + var request = new NLWebRequest + { + Query = "search for information about APIs", + Mode = QueryMode.List, + QueryId = "test-search-001" + }; + + // Act + var response = await _toolExecutor.ExecuteToolAsync(request, "search"); + + // Assert + Assert.IsNotNull(response); + Assert.AreEqual(request.QueryId, response.QueryId); + Assert.AreEqual(request.Query, response.Query); + Assert.IsNotNull(response.Results); + } + + [TestMethod] + public async Task ExecuteToolAsync_WithDetailsQuery_ShouldReturnDetailsResponse() + { + // Arrange + var request = new NLWebRequest + { + Query = "tell me about REST APIs", + Mode = QueryMode.List, + QueryId = "test-details-001" + }; + + // Act + var response = await _toolExecutor.ExecuteToolAsync(request, "details"); + + // Assert + Assert.IsNotNull(response); + Assert.AreEqual(request.QueryId, response.QueryId); + Assert.IsNotNull(response.Results); + // The mock backend may return empty results, but the response should be processed by details tool + Assert.IsTrue(response.Summary?.Contains("Details") == true || response.Summary?.Contains("details") == true, + "Should be processed by details tool"); + } + + [TestMethod] + public async Task ExecuteToolAsync_WithInvalidTool_ShouldThrowException() + { + // Arrange + var request = new NLWebRequest + { + Query = "test query", + Mode = QueryMode.List, + QueryId = "test-invalid-001" + }; + + // Act & Assert + await Assert.ThrowsExceptionAsync( + () => _toolExecutor.ExecuteToolAsync(request, "nonexistent-tool")); + } +} \ No newline at end of file