Feature Request: Adopt Mollusk's Validation System and Low-Level Control Features

[brimigs]

Summary

Mollusk is test harness for solana programs (https://github.com/anza-xyz/mollusk) that offers a lot of great functionality. I would love to see LiteSVM offer something similar.

Motivation

While liteSVM excels at transaction simulation and state management, Mollusk offers superior features for:

• Precise validation through its Check system
• Low-level control via custom syscalls
• Compute unit benchmarking with dedicated tools
• Instruction-level testing without transaction overhead

Adding these capabilities would make liteSVM a complete testing solution covering both unit and integration testing needs.

Proposed Features

1. Check-Based Validation System

Current State (liteSVM)

// Manual assertions after transaction
let result = svm.send_transaction(tx);
assert!(result.is_ok());

let account = svm.get_account(&counter).unwrap();
assert_eq!(account.lamports, expected_lamports);
assert_eq!(account.data[8], expected_value);
assert_eq!(account.owner, expected_owner);

Proposed API (Inspired by Mollusk)

// Declarative validation with rich checks
svm.send_and_validate_transaction(
    tx,
    &[
        Check::success(),
        Check::compute_units(450),
        Check::account(&counter)
            .lamports(expected_lamports)
            .data_slice(8, &[expected_value])
            .owner(&program_id)
            .rent_exempt()
            .build(),
        Check::logs_contain("Program log: Counter incremented"),
    ],
);

Implementation Details

pub enum Check {
    /// Validate transaction succeeded
    Success,
    
    /// Validate transaction failed with specific error
    Err(TransactionError),
    
    /// Validate compute units consumed
    ComputeUnits(u64),
    
    /// Validate execution time
    ExecutionTime(Duration),
    
    /// Validate specific account state
    Account(AccountCheck),
    
    /// Validate all accounts are rent exempt
    AllRentExempt,
    
    /// Validate transaction logs
    LogsContain(&str),
    
    /// Validate return data
    ReturnData(&[u8]),
    
    /// Custom validation function
    Custom(Box<dyn Fn(&TransactionResult) -> bool>),
}

pub struct AccountCheck {
    pubkey: Pubkey,
    lamports: Option<u64>,
    data: Option<Vec<u8>>,
    data_slice: Option<(usize, Vec<u8>)>,
    owner: Option<Pubkey>,
    executable: Option<bool>,
    rent_exempt: Option<bool>,
    closed: bool,
}

impl AccountCheck {
    pub fn new(pubkey: &Pubkey) -> Self { ... }
    
    pub fn lamports(mut self, lamports: u64) -> Self {
        self.lamports = Some(lamports);
        self
    }
    
    pub fn data(mut self, data: &[u8]) -> Self {
        self.data = Some(data.to_vec());
        self
    }
    
    pub fn data_slice(mut self, offset: usize, data: &[u8]) -> Self {
        self.data_slice = Some((offset, data.to_vec()));
        self
    }
    
    pub fn closed(mut self) -> Self {
        self.closed = true;
        self
    }
    
    pub fn build(self) -> Check {
        Check::Account(self)
    }
}

2. Custom Syscall Support

Current State (liteSVM)

// No support for custom syscalls - cannot test programs that use them

Proposed API

use litesvm::syscall::{SyscallRegistry, CustomSyscall};

// Define custom syscall
struct CheckSpreadSyscall;

impl CustomSyscall for CheckSpreadSyscall {
    fn call(
        &self,
        invoke_context: &mut InvokeContext,
        args: &[u64],
        memory_mapping: &mut MemoryMapping,
    ) -> Result<u64, Error> {
        // Custom implementation
        let spread = calculate_spread();
        
        // Write result to memory
        let ptr = memory_mapping.map(AccessType::Store, args[0], 8)?;
        unsafe {
            std::ptr::write(ptr as *mut u64, spread);
        }
        
        Ok(0)
    }
}

// Register with liteSVM
let mut svm = LiteSVM::new();
svm.register_syscall("sol_check_spread", Box::new(CheckSpreadSyscall))?;

// Now programs using this syscall will work
let result = svm.send_transaction(tx);

Use Cases

• Testing programs with custom syscalls
• Mocking external dependencies
• Simulating cross-program calls
• Testing error conditions

3. Instruction-Level Processing

Current State (liteSVM)

// Must create full transaction even for single instruction
let tx = Transaction::new_signed_with_payer(&[ix], Some(&payer), &[&payer], blockhash);
let result = svm.send_transaction(tx);

Proposed API

// Process single instruction without transaction overhead
let result = svm.process_instruction(
    &instruction,
    &[
        (account1_pubkey, account1),
        (account2_pubkey, account2),
    ],
);

// Process instruction chain
let results = svm.process_instruction_chain(
    &[instruction1, instruction2, instruction3],
    &accounts,
);

// With validation
svm.process_and_validate_instruction(
    &instruction,
    &accounts,
    &[Check::success(), Check::compute_units(300)],
);

4. Compute Unit Benchmarking

Proposed API

use litesvm::bencher::ComputeUnitBencher;

// Benchmark individual instructions
let bencher = ComputeUnitBencher::new(svm);

bencher
    .bench("initialize", &init_instruction, &init_accounts)
    .bench("increment", &increment_instruction, &increment_accounts)
    .bench("decrement", &decrement_instruction, &decrement_accounts)
    .must_pass(true)
    .output_markdown("benches/compute_units.md")
    .execute();

// Output markdown table:
// | Operation  | CUs   | Delta |
// |------------|-------|-------|
// | initialize | 450   | --    |
// | increment  | 372   | -78   |
// | decrement  | 372   | 0     |

5. Fixture Support

Proposed API

// Generate fixtures from tests
std::env::set_var("LITESVM_DUMP_FIXTURES", "./fixtures");
let result = svm.send_transaction(tx); // Automatically saves fixture

// Load and execute fixtures
let fixture = Fixture::load("./fixtures/test1.json")?;
let result = svm.process_fixture(&fixture);

// Validate against expected fixture output
svm.process_and_validate_fixture(
    &fixture,
    &[
        FixtureCheck::ProgramResult,
        FixtureCheck::ReturnData,
        FixtureCheck::AccountChanges,
    ],
);

6. Enhanced State Management Options

Proposed API

// Stateless mode (like Mollusk)
let svm = LiteSVM::stateless();
let result = svm.process_instruction(&ix, &accounts);
// State changes don't persist

// Explicit state management
let mut state = AccountStore::new();
state.insert(pubkey1, account1);

let svm = LiteSVM::with_state(state);
let result = svm.process_instruction(&ix);
// Access modified state
let modified_account = svm.state.get(&pubkey1);

Implementation Plan

Phase 1: Validation System (High Priority)

• Implement Check enum and validation logic
• Add AccountCheck builder
• Create send_and_validate_transaction method
• Add validation for logs and return data

Phase 2: Instruction Processing (High Priority)

• Add process_instruction method
• Implement process_instruction_chain
• Add validation variants for instruction processing
• Optimize for performance without transaction overhead

Phase 3: Custom Syscalls (Medium Priority)

• Design syscall registry system
• Implement CustomSyscall trait
• Add memory mapping utilities
• Create examples for common syscalls

Phase 4: Benchmarking Tools (Medium Priority)

• Create ComputeUnitBencher struct
• Implement markdown output
• Add delta tracking between runs
• Create CLI tool for benchmarking

Phase 5: Advanced Features (Low Priority)

• Fixture generation and loading
• Stateless mode option
• Cross-implementation fixture format
• Integration with fuzzing tools

Benefits

For Unit Testing

// Before: 20+ lines with manual assertions
// After: 5 lines with declarative validation
svm.process_and_validate_instruction(
    &instruction,
    &accounts,
    &[Check::success(), Check::account(&pubkey).lamports(1000).build()],
);

For Performance Testing

// Automatic compute unit tracking and regression detection
bencher.bench("critical_operation", &ix, &accounts);
// Generates report showing performance changes

For Complex Programs

// Test programs with custom syscalls
svm.register_syscall("sol_custom_op", CustomOp::new());
// Previously untestable programs now work

Comparison with Current State

Feature	Current liteSVM	With Proposed Changes	Mollusk
Validation API	Manual assertions	Rich Check system	✓ Check system
Custom Syscalls	❌ Not supported	✓ Full support	✓ Full support
Instruction Processing	Via transactions only	✓ Direct processing	✓ Direct processing
Compute Benchmarking	Manual	✓ Built-in bencher	✓ Built-in bencher
Fixtures	❌ Not supported	✓ Generate/load	✓ Generate/load
State Management	Always stateful	✓ Configurable	Stateless by default

Backwards Compatibility

All proposed features would be additive:

• Existing send_transaction API remains unchanged
• New validation methods are optional
• Instruction processing is additional API
• Custom syscalls are opt-in

Performance Considerations

• Instruction processing would be 30-50% faster than transaction processing
• Validation checks add minimal overhead (< 5%)
• Custom syscalls have same performance as native syscalls
• Stateless mode reduces memory usage for unit tests

Use Case Examples

Example 1: Testing Compute-Intensive Operations

let bencher = ComputeUnitBencher::new(svm);
for size in [10, 100, 1000, 10000] {
    let (ix, accounts) = create_sort_instruction(size);
    bencher.bench(&format!("sort_{}", size), &ix, &accounts);
}
bencher.output_markdown("benches/sort_performance.md");

Example 2: Testing Custom Syscalls

// Program uses custom syscall for randomness
svm.register_syscall("sol_get_random", RandomnessSyscall::new(seed));

// Test different random scenarios
for seed in test_seeds {
    svm.update_syscall("sol_get_random", RandomnessSyscall::new(seed));
    let result = svm.process_instruction(&ix, &accounts);
    validate_random_behavior(result, seed);
}

Example 3: Precise State Validation

svm.process_and_validate_instruction(
    &complex_instruction,
    &accounts,
    &[
        Check::success(),
        Check::compute_units(5000),
        Check::account(&vault)
            .lamports(1_000_000)
            .data_slice(0, &[1, 2, 3, 4]) // Check header
            .data_slice(100, &[5, 6, 7, 8]) // Check specific field
            .owner(&program_id)
            .rent_exempt()
            .build(),
        Check::logs_contain("Vault updated successfully"),
        Check::return_data(&[0, 0, 0, 1]), // Success code
    ],
);

Success Metrics

1. API Completeness: Feature parity with Mollusk for unit testing
2. Performance: Instruction processing 30-50% faster than transactions
3. Adoption: Used for both unit and integration tests
4. Developer Satisfaction: Reduced test verbosity by 40%

Questions for Maintainers

1. Should validation be a separate crate (litesvm-check) or built-in?
2. Is there interest in maintaining fixture compatibility with Mollusk/Firedancer?
3. Should custom syscalls use the same API as Agave or simplified version?
4. Would you prefer trait-based or enum-based validation checks?

Conclusion

These features would give liteSVM users the best of both worlds: liteSVM's excellent transaction simulation and state management combined with Mollusk's precise validation and low-level control. This would make liteSVM the only testing framework needed for Solana development, suitable for everything from unit tests to complex integration scenarios.

The proposed changes maintain liteSVM's ease of use while adding optional power features for advanced users. Developers could start with simple transaction testing and gradually adopt more sophisticated features as needed.

---

[kevinheavey]

1. The first example is much better!

I don't want to learn a special API that does the same thing as assert_eq!(account.lamports, expected_lamports); for no clear benefit. Nor do I want to encourage the proliferation of such an API.

2. Perhaps, would need to see how much complexity custom syscalls would add

3. Meh / maybe

I am generally a fan of having one way to do things. Maybe if there's a significant performance benefit? We'd have to compare to doing send_transaction after disabling sigverify and transaction history, both of which litesvm already supports.

Definitely not a fan of process_instruction_chain, at that point just make a transaction.

4. I don't think we need an opinionated way to store compute unit consumption.

Not seeing anything special here. LiteSVM is not a markdown document generator. The API seems worse than just checking the compute units consumed by a transaction.

5. It's unclear what this is, which is strong evidence that it's not something we want in the API

6. We have stateless mode at home

We already have simulate_transaction. And get_account and set_account are already pretty explicit state management

Sorry for being a little brusque, I am just passionate about API bloat

[stegaBOB]

We currently use both litesvm and mollusk. With mollusk, unit testing instructions or multiple instructions directly without having to deal with signers or any of the transaction api shenanigans has significantly sped up our workflow. Performance isn't actually a factor for us.

IFAICT, the only added benefit I can think of for testing transactions vs groups of instructions directly is making sure everything fits in a transaction (although this could totally be tested with just groups of instructions too!!), or potentially integration testing higher level client code. The only extra data a transaction has is just adding sigs (which are specified in the ix account meta, and if that were to mismatch the validator would reject the txn anyway), a recent blockhash (which we'd only ever want to test with valid ones anyway for programs), and listing the account keys and num signers in the message header (which is literally just a compressed version of a list of uncompiled instructions).

[stegaBOB]

Hey @kevinheavey just bumping this 😅