Skip to content

Latest commit

 

History

History
354 lines (243 loc) · 12.7 KB

test-generation.md

File metadata and controls

354 lines (243 loc) · 12.7 KB

Test Generation Pipeline Documentation

Overview

The Test Generation Pipeline is a sophisticated multi-stage system within ISC-CodeConnect that creates comprehensive Apex test classes and Jest tests for Lightning Web Components. The pipeline employs a staged approach with specialized agents working collaboratively to produce high-quality, context-aware test implementations.

Pipeline Architecture

Apex Test Generation Pipeline (5-Stage Process)

The Apex test generation follows a structured pipeline that builds upon each stage:

1. Test Planner → 2. Setup Generator → 3. Stub Generator → 4. Test Method Generator → 5. Test Finalizer

Jest Test Generation (Streamlined Process)

Jest test generation uses a more streamlined approach optimized for frontend component testing:

Jest Test Generator → Test Explanation (Optional)

Apex Test Generation Stages

Stage 1: Test Planner Agent

async def plan_apex_test_implementation(
    state: AgentState, config=RunnableConfig
) -> Command[Literal["test_setup_generator"]]

Purpose: Creates comprehensive test strategy and implementation plan

Responsibilities:

  • Strategy Development: Analyzes code structure to create testing strategy
  • Scenario Identification: Identifies 2-4 key test scenarios covering main functionality
  • Mocking Strategy: Determines appropriate mocking approach based on code dependencies
  • Data Requirements: Plans mock data strategy using context patterns
  • Method Prioritization: Groups methods into batches for efficient generation

Key Features:

  • Context-Aware Planning: Leverages available Salesforce context patterns
  • Dependency Analysis: Identifies database operations, SOQL queries, and external dependencies
  • User Intent Focus: Tailors test plan to specific user requirements
  • Batch Processing: Organizes methods into logical batches (2 methods per batch)

Output: Structured test plan with verification strategies and method batches

Stage 2: Setup Generator Agent

async def generate_test_setup(
    state: AgentState, config=RunnableConfig
) -> Command[Literal["test_stub_generator"]]

Purpose: Generates test class skeleton with setup infrastructure

Responsibilities:

  • Class Declaration: Creates test class with proper @isTest annotation
  • Mock Infrastructure: Sets up Query.setMock and DML.setMock frameworks
  • Test Data Fields: Creates static fields for mock data and tracking
  • Helper Methods: Implements setup methods following context patterns
  • Dependency Preparation: Prepares for stub injection and dependency management

Advanced Mocking Capabilities:

  • QueryMock Implementation: For SOQL query mocking
  • DmlMocks Integration: For database operation tracking
  • HttpCalloutMock Setup: For external service mocking
  • StubHelper Preparation: For dependency injection mocking

Output: Complete test class setup with mocking infrastructure

Stage 3: Stub Generator Agent

async def generate_stubs(
    state: AgentState, config=RunnableConfig
) -> Command[Literal["test_method_generator"]]

Purpose: Creates stub implementations for external dependencies

Responsibilities:

  • Dependency Analysis: Identifies classes requiring stub implementations
  • Stub Creation: Generates StubHelper-based mock implementations
  • Method Behavior: Configures stub behavior for test scenarios
  • Integration Points: Ensures stubs integrate with setup infrastructure
  • Service Mocking: Creates mocks for Service, Manager, Helper, and Controller classes

Stub Detection Logic:

  • Constructor Analysis: Identifies dependency injection patterns
  • Instance Variable Detection: Finds private dependencies requiring stubs
  • Service Pattern Recognition: Detects common service types needing mocking
  • Explicit Request Handling: Responds to user requests for specific stubbing

Output: Complete stub implementations integrated with test setup

Stage 4: Test Method Generator Agent

async def generate_test_methods(
    state: AgentState, config=RunnableConfig
) -> Command[Literal["test_method_generator", "finalize_apex_test_code"]]

Purpose: Generates actual test methods with proper assertions

Responsibilities:

  • Batch Processing: Generates test methods for current batch (2 methods at a time)
  • Assertion Implementation: Uses GherkinAssertionBuilder for Given/When/Then structure
  • Mock Integration: Integrates with setup and stub infrastructure
  • Verification Logic: Implements appropriate verification strategies
  • Edge Case Coverage: Creates test methods for important edge cases

Testing Patterns:

  • Mocking Approach: Uses Query.setMock and DML.setMock for verification
  • Database Approach: Direct database state verification (when appropriate)
  • Stub Verification: Validates stub method calls and parameters
  • Error Scenario Testing: Covers exception handling and edge cases

Method Structure:

@isTest
static void testMethodName_Scenario() {
    // Given
    assertion.setGiven('clear description based on context patterns');
    // Setup code using previous stage outputs

    // When
    assertion.setWhen('description of the action being tested');
    Test.startTest();
    // Test execution code using mocks
    Test.stopTest();

    // Then
    assertion.setThen('description of the expected outcome');
    // Verification code integrated with previous stages
}

Output: Complete test methods for current batch

Stage 5: Test Finalizer Agent

async def finalize_apex_test_code(state: AgentState, config=RunnableConfig) -> Dict

Purpose: Assembles complete test class and provides summary

Responsibilities:

  • Code Assembly: Combines setup, stubs, and methods into complete class
  • Class Completion: Adds closing braces and final formatting
  • Quality Validation: Ensures code completeness and syntax correctness
  • Summary Generation: Creates conversational summary of implementation
  • Integration Verification: Confirms all stages integrated properly

Final Output: Complete, executable test class with comprehensive summary

Jest Test Generation

Jest Test Generator Agent

async def generate_jest_tests(state: AgentState, config=None) -> Dict

Purpose: Creates comprehensive Jest tests for Lightning Web Components

Responsibilities:

  • Component Analysis: Analyzes LWC HTML, JavaScript, and CSS
  • Test Scenario Creation: Identifies component behaviors requiring testing
  • Mock Strategy: Implements mocking for Apex calls and wire adapters
  • Interaction Testing: Tests user interactions and component events
  • Accessibility Testing: Includes accessibility validation where appropriate

Key Features:

  • Template Testing: Tests component rendering and template logic
  • Event Testing: Validates component event handling
  • Wire Adapter Mocking: Mocks Salesforce wire adapters
  • Apex Method Mocking: Creates mocks for backend service calls
  • Error Scenario Testing: Tests component error handling

Jest Test Explanation Agent

async def explain_jest_tests(state: AgentState, config=RunnableConfig) -> Dict

Purpose: Explains existing Jest test implementations

Responsibilities:

  • Test Analysis: Analyzes existing Jest test code
  • Pattern Explanation: Explains testing patterns and strategies
  • Component Context: Provides context about component under test
  • Best Practices: Highlights testing best practices demonstrated

Advanced Pipeline Features

Context Integration

RAG-Aware Testing:

  • Salesforce Object Integration: Uses actual Salesforce objects from context
  • Field Mapping: Leverages real field names and relationships
  • Pattern Reuse: Follows established testing patterns from codebase
  • API Consistency: Maintains consistency with existing code patterns

Context Processing:

  • Multi-Collection Retrieval: Searches across multiple vector stores
  • Pattern Recognition: Identifies relevant testing patterns
  • Code Relationship Analysis: Understands relationships between classes
  • Best Practice Application: Applies proven patterns to new tests

Intelligent Mocking Strategies

Automatic Mocking Detection:

  • Database Operation Analysis: Detects SOQL queries and DML operations
  • External Dependency Identification: Finds HTTP callouts and service dependencies
  • Dependency Injection Recognition: Identifies constructor and setter injection
  • Service Pattern Detection: Recognizes Service, Manager, Helper patterns

Mocking Implementation Approaches:

  1. Query Mocking:
QueryMock queryMock = new QueryMock(mockRecords);
Query.setMock(queryMock);
  1. DML Mocking:
DmlMocks.TrackRecords dmlMock = new DmlMocks.TrackRecords();
Dml.setMock(dmlMock);
// Verification: dmlMock.updates.size()
  1. StubHelper Integration:
SomeService mockService = (SomeService)StubHelper.createStub(SomeService.class);
StubHelper.stub(mockService, 'methodName').returns(expectedValue);

Error Handling and Recovery

Pipeline Resilience:

  • Stage Isolation: Failures in one stage don't cascade to others
  • Graceful Degradation: Continues with available components on partial failures
  • Error Context Preservation: Maintains detailed error information for debugging
  • Fallback Strategies: Provides alternative generation approaches

Quality Assurance:

  • Code Validation: Syntax checking and completeness verification
  • Integration Testing: Ensures stages work together properly
  • Pattern Compliance: Validates adherence to Salesforce best practices
  • Context Consistency: Maintains consistency with existing codebase patterns

Performance Optimization

Batch Processing

Method Batching:

  • Optimal Batch Size: Processes 2 methods per batch for balance
  • Memory Management: Prevents memory overflow with large classes
  • Progress Tracking: Provides clear progress indicators
  • Parallel Opportunities: Enables potential parallelization

Context Management

Efficient Context Usage:

  • Chunk Limiting: Processes top 5 relevant chunks for performance
  • Context Caching: Reuses processed context across stages
  • Token Management: Optimizes prompt length for LLM efficiency
  • Parallel Processing: Concurrent context processing where possible

Integration with System Architecture

Router Integration

Pipeline Activation:

  • Automatic Detection: Router detects test creation/modification requests
  • Context Preparation: Retrieves relevant test patterns and examples
  • State Management: Maintains comprehensive state across pipeline stages
  • Result Integration: Seamlessly integrates with overall response flow

State Management

Cross-Stage State:

  • Cumulative Building: Each stage builds upon previous outputs
  • Context Preservation: Maintains conversation and analysis context
  • Progress Tracking: Tracks completion across all stages
  • Error State Management: Handles and recovers from stage failures

Best Practices

For Test Generation

  1. Context Utilization: Always leverage available Salesforce context
  2. Mocking Strategy: Choose appropriate mocking approach based on dependencies
  3. User Intent Focus: Tailor tests to specific user requirements
  4. Pattern Consistency: Follow established testing patterns from codebase

For System Integration

  1. State Completeness: Ensure all required state is available for pipeline
  2. Error Handling: Implement robust error recovery at each stage
  3. Performance Monitoring: Track pipeline performance and optimization opportunities
  4. Quality Validation: Validate generated tests for completeness and correctness

Future Enhancements

Planned Improvements

  1. Parallel Stage Processing: Execute independent stages concurrently
  2. Adaptive Batch Sizing: Dynamic batch sizing based on complexity
  3. Quality Metrics: Automated quality assessment of generated tests
  4. Pattern Learning: Learn from successful test patterns for improvement

Advanced Features

  1. Test Coverage Analysis: Analyze and optimize test coverage
  2. Performance Testing: Generate performance-focused test scenarios
  3. Integration Testing: Create tests for cross-component integration
  4. Regression Testing: Generate regression test suites automatically

The Test Generation Pipeline represents a sophisticated approach to automated test creation, combining intelligent analysis, context awareness, and proven testing patterns to generate high-quality, maintainable test code that follows Salesforce best practices while addressing specific user requirements.