Skip to content

Conversation

alzarei
Copy link

@alzarei alzarei commented Sep 28, 2025

Modernize TavilyTextSearch and BraveTextSearch connectors with ITextSearch interface

Problem Statement

The TavilyTextSearch and BraveTextSearch connectors currently implement only the legacy ITextSearch interface, forcing users to use clause-based TextSearchFilter instead of modern type-safe LINQ expressions. Additionally, the existing LINQ support is limited to basic expressions (equality, AND operations).

Technical Approach

This PR modernizes both connectors with generic interface implementation and extends LINQ filtering to support OR operations, negation, and inequality operators. The implementation adds type-safe model classes and enhanced expression tree analysis capabilities.

Implementation Details

Core Changes

  • Both connectors now implement ITextSearch (legacy) and ITextSearch (modern)
  • Added type-safe model classes: TavilyWebPage and BraveWebPage
  • Extended AnalyzeExpression() methods to handle additional expression node types
  • Added support for OrElse, NotEqual, and UnaryExpression operations
  • Implemented array.Contains(property) pattern recognition
  • Enhanced error messaging with contextual examples

Enhanced LINQ Expression Support

  • OR Operations (||): Maps to multiple API parameter values or OR logic
  • NOT Operations (!): Converts to exclusion parameters where supported
  • Inequality Operations (!=): Provides helpful error messages suggesting NOT alternatives
  • Array Contains Pattern: Supports array.Contains(property) for multi-value filtering

Code Examples

Before (Legacy Interface)

var legacyOptions = new TextSearchOptions
{
    Filter = new TextSearchFilter()
        .Equality("topic", "general")
        .Equality("time_range", "week")
};

After (Generic Interface)

// Simple filtering
var modernOptions = new TextSearchOptions<TavilyWebPage>
{
    Filter = page => page.Topic == "general" && page.TimeRange == "week"
};

// Advanced filtering with OR and array Contains
var advancedOptions = new TextSearchOptions<BraveWebPage>
{
    Filter = page => (page.Country == "US" || page.Country == "GB") &&
                     new[] { "moderate", "strict" }.Contains(page.SafeSearch) &&
                     !(page.ResultFilter == "adult")
};

Implementation Benefits

Interface Modernization

  • Type-safe filtering with compile-time validation prevents property name typos
  • IntelliSense support for TavilyWebPage and BraveWebPage properties
  • Consistent LINQ-based filtering across all text search implementations

Enhanced Filtering Capabilities

  • OR operations enable multi-value property matching
  • NOT operations provide exclusion filtering where API supports it
  • Array Contains patterns simplify multi-value filtering syntax
  • Improved error messages reduce debugging time

Developer Experience

  • Better debugging experience with type information
  • Reduced learning curve - same patterns across all connectors
  • Enhanced error messages with usage examples and supported properties

Validation Results

Build Verification

  • Configuration: Release
  • Target Framework: .NET 8.0
  • Command: dotnet build --configuration Release --interactive
  • Result: Build succeeded - all projects compiled successfully

Test Results
Full Test Suite:

  • Passed: 8,829 (core functionality tests)
  • Failed: 1,361 (external API configuration issues)
  • Skipped: 389
  • Duration: 4 minutes 57 seconds

Core Unit Tests:

  • Command: dotnet test src\SemanticKernel.UnitTests\SemanticKernel.UnitTests.csproj --configuration Release
  • Result: 1,574 passed, 0 failed (100% core framework functionality)

Test Failure Analysis
The 1,361 test failures are infrastructure/configuration issues, not code defects:

  • Azure OpenAI Configuration: Missing API keys for external service integration tests
  • Docker Dependencies: Vector database containers not available in development environment
  • External Service Dependencies: Integration tests requiring live API services (Bing, Google, Brave, Tavily, etc.)
  • AWS/Azure Configuration: Missing credentials for cloud service integration tests

These failures are expected in development environments without external API configurations.

Code Quality

  • Formatting: Applied via dotnet format SK-dotnet.slnx
  • Enhanced documentation follows XML documentation conventions
  • Consistent with established LINQ expression handling patterns

Files Modified

dotnet/src/Plugins/Plugins.Web/Tavily/TavilyWebPage.cs (NEW)
dotnet/src/Plugins/Plugins.Web/Brave/BraveWebPage.cs (NEW)
dotnet/src/Plugins/Plugins.Web/Brave/BraveTextSearch.cs (MODIFIED)
dotnet/src/Plugins/Plugins.Web/Tavily/TavilyTextSearch.cs (MODIFIED)

Breaking Changes

None. All existing LINQ expressions continue to work unchanged with enhanced error message generation.

Multi-PR Context

This is PR 5 of 6 in the structured implementation approach for Issue #10456. This PR completes the modernization of remaining text search connectors with enhanced LINQ expression capabilities while maintaining full backward compatibility.

…<TRecord> interface

- Add TavilyWebPage and BraveWebPage model classes for type-safe LINQ filtering
- Implement dual interface support (ITextSearch + ITextSearch<TRecord>) in both connectors
- Add LINQ-to-API conversion logic for Tavily and Brave search parameters
- Maintain 100% backward compatibility with existing ITextSearch usage
- Enable compile-time type safety and IntelliSense support for web search filters

Addresses microsoft#10456
@moonbox3 moonbox3 added the .NET Issue or Pull requests regarding .NET code label Sep 28, 2025
@github-actions github-actions bot changed the title Net: feat: Modernize TavilyTextSearch and BraveTextSearch connectors with ITextSearch<TRecord> interface (microsoft#10456) .Net: Net: feat: Modernize TavilyTextSearch and BraveTextSearch connectors with ITextSearch<TRecord> interface (microsoft#10456) Sep 28, 2025
@alzarei alzarei marked this pull request as ready for review September 28, 2025 07:33
@alzarei alzarei requested a review from a team as a code owner September 28, 2025 07:33
…rror messages

- Added support for OrElse (||) operators in LINQ expressions
- Added NotEqual (!=) operator support with appropriate error handling
- Added UnaryExpression (NOT) support with proper limitations
- Enhanced Contains method support for array.Contains(property) patterns
- Improved error messages with usage examples
- Added constants for API parameter names for better maintainability
- Added TODO comments for potential code sharing opportunities

Changes apply to both BraveTextSearch and TavilyTextSearch implementations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Issue or Pull requests regarding .NET code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants