SPEC-20: Typed Signatures System

DSPy-inspired typed signatures for rlm-core

Status: Implemented in rlm-core runtime (typed runtime protocol + deterministic composition validation + derive type-inference refinements) Created: 2026-01-20 Epic: loop-zcx (DSPy-Inspired RLM Improvements) Tasks: loop-d75, loop-jqo, loop-9l6, loop-bzz

Overview

Implement DSPy-style typed signatures that enable composable modules, automatic output validation, and optimization. This is the foundational system upon which module composition and BootstrapFewShot optimization depend.

Implementation Snapshot (2026-02-20)

Section	Status	Runtime Evidence
SPEC-20.01 Signature trait	Implemented	`rlm-core/src/signature/mod.rs`
SPEC-20.02 Field specification model	Implemented	`rlm-core/src/signature/types.rs`
SPEC-20.03 Runtime validation	Implemented (input pre-validation + output validation path)	`validate_fields` in `rlm-core/src/signature/validation.rs`, `Predict::forward` in `rlm-core/src/module/predict.rs`
SPEC-20.04 Derive macro attributes	Implemented (including explicit enum values)	`rlm-core-derive/src/lib.rs`
SPEC-20.05 Type inference	Implemented	Primitive/List/Option plus reference/slice/array inference in `rlm-core-derive/src/lib.rs`; explicit enum via `#[field(enum_values = \"...\")]`
SPEC-20.07..20.10 SUBMIT + REPL protocol	Implemented	`rlm-core/python/rlm_repl/sandbox.py`, `rlm-core/python/rlm_repl/main.py`, `rlm-core/src/repl.rs`
SPEC-20.11 Module trait	Implemented	`rlm-core/src/module/mod.rs`
SPEC-20.12 Predict wrapper	Implemented (with deterministic pre-exec input validation)	`rlm-core/src/module/predict.rs`
SPEC-20.13 Composition validation	Implemented	Deterministic direct-field compatibility checks (`validate_direct_field_mapping`) + validation-backed decode path in `rlm-core/src/module/compose.rs`

Requirements

SPEC-20.01: Signature Trait

The core trait defining typed LLM I/O contracts.

pub trait Signature: Send + Sync + 'static {
    /// Input type (must be serializable)
    type Inputs: Serialize + DeserializeOwned + Clone + Send + Sync;
    /// Output type (must be serializable)
    type Outputs: Serialize + DeserializeOwned + Clone + Send + Sync;

    /// Task instructions for the LLM
    fn instructions() -> &'static str;

    /// Input field specifications
    fn input_fields() -> Vec<FieldSpec>;

    /// Output field specifications
    fn output_fields() -> Vec<FieldSpec>;

    /// Generate prompt from inputs
    fn to_prompt(inputs: &Self::Inputs) -> String {
        // Default implementation using field specs
    }

    /// Parse outputs from LLM response
    fn from_response(response: &str) -> Result<Self::Outputs, ParseError>;
}

Acceptance Criteria:

Trait compiles and is object-safe where possible
Default to_prompt generates structured prompt
Default from_response parses JSON or structured text

SPEC-20.02: Field Specification

Metadata for input and output fields.

pub struct FieldSpec {
    /// Field name (matches struct field)
    pub name: String,
    /// Field type for validation
    pub field_type: FieldType,
    /// Human-readable description (for prompt generation)
    pub description: String,
    /// Optional display prefix/label
    pub prefix: Option<String>,
    /// Whether field is required
    pub required: bool,
    /// Default value (JSON) if not required
    pub default: Option<serde_json::Value>,
}

pub enum FieldType {
    String,
    Integer,
    Float,
    Boolean,
    List(Box<FieldType>),
    Object(Vec<FieldSpec>),
    Enum(Vec<String>),  // Allowed values
    Custom(String),     // Custom type name
}

Acceptance Criteria:

FieldType covers common Rust types
Nested types (List, Object) work recursively
Enum validates against allowed values

SPEC-20.03: Signature Validation

Runtime validation of inputs and outputs.

pub trait SignatureValidator {
    /// Validate inputs before execution
    fn validate_inputs<S: Signature>(inputs: &S::Inputs) -> Result<(), ValidationError>;

    /// Validate outputs before returning
    fn validate_outputs<S: Signature>(outputs: &S::Outputs) -> Result<(), ValidationError>;
}

pub enum ValidationError {
    MissingField { field: String },
    TypeMismatch { field: String, expected: FieldType, got: String },
    EnumInvalid { field: String, value: String, allowed: Vec<String> },
    ConstraintViolated { field: String, constraint: String },
    Custom(String),
}

Acceptance Criteria:

Inputs validated before execution
Outputs validated before returning to caller
Clear error messages with field context

SPEC-20.04: Derive Macro Attributes

Proc macro for automatic Signature implementation.

#[derive(Signature)]
#[signature(instructions = "Analyze code for security vulnerabilities")]
pub struct AnalyzeCode {
    // Inputs
    #[input(desc = "Source code to analyze")]
    code: String,

    #[input(desc = "Programming language", prefix = "Language")]
    language: String,

    // Outputs
    #[output(desc = "List of vulnerabilities found")]
    vulnerabilities: Vec<String>,

    #[output(desc = "Overall severity", prefix = "Severity")]
    severity: Severity,
}

Attributes:

#[derive(Signature)] - Generate Signature impl
#[signature(instructions = "...")] - Set instructions text
#[input(desc = "...", prefix = "...")] - Mark as input field
#[output(desc = "...", prefix = "...")] - Mark as output field
#[field(required = false, default = "...")] - Optional field config
#[field(enum_values = "a,b,c")] - Explicit enum value list for field validation

Acceptance Criteria:

Macro generates correct Signature impl
All attributes parsed correctly
Compile-time errors for invalid usage

SPEC-20.05: Type Inference

Automatic FieldType inference from Rust types.

Rust Type	FieldType
`String`, `&str`	`FieldType::String`
`i8`..`i128`, `u8`..`u128`, `isize`, `usize`	`FieldType::Integer`
`f32`, `f64`	`FieldType::Float`
`bool`	`FieldType::Boolean`
`Vec<T>`	`FieldType::List(T)`
`Option<T>`	Same as T, but `required = false`
`&T`	Same as `T`
`[T]`, `[T; N]`	`FieldType::List(T)`
`String` + `#[field(enum_values = "...")]`	`FieldType::Enum(values)`
Rust enum type without `enum_values` annotation	`FieldType::Custom(type_name)` (current behavior)
Other	`FieldType::Custom(type_name)`

Acceptance Criteria:

All primitive types inferred correctly
Generic types (Vec, Option, references/slices/arrays) handled
Custom types fall back to Custom variant

SPEC-20.06: Compile-Time Validation

Errors at compile time for invalid signatures.

Condition	Error Message
No `#[input]` fields	"Signature must have at least one input field"
No `#[output]` fields	"Signature must have at least one output field"
Field without attribute	"Field 'X' must be marked with #[input] or #[output]"
Invalid attribute syntax	"Invalid attribute: expected #[input(desc = "...")]"

Acceptance Criteria:

All invalid usages produce compile errors
Error messages are helpful and actionable

SPEC-20.07: SUBMIT Function

Python REPL function for structured output termination.

# In REPL sandbox (rlm_repl.sandbox.Sandbox._submit)
def SUBMIT(outputs: dict) -> NoReturn:
    """
    Terminate execution and return validated outputs.

    Args:
        outputs: Dictionary matching signature output fields

    Behavior:
        - Validates against active signature registration
        - Stores structured submit_result payload
        - Terminates current execution via internal control-flow signal
    """

Behavior:

SUBMIT() immediately terminates current execution
Validates all required output fields present
Validates field types match signature
Includes submit_result in execute response to Rust orchestrator

Acceptance Criteria:

SUBMIT() terminates execution immediately
Validation against registered signature
Clear errors for missing/invalid fields

SPEC-20.08: SUBMIT Behavior

Detailed SUBMIT semantics.

pub enum SubmitResult {
    /// Successful submission with validated outputs
    Success {
        outputs: serde_json::Value,
        metrics: Option<SubmitMetrics>,
    },
    /// Validation failed
    ValidationError {
        errors: Vec<SubmitError>,
        original_outputs: Option<serde_json::Value>,
    },
    /// Execution completed without SUBMIT
    NotSubmitted {
        reason: String,
    },
}

Rules:

SUBMIT() MUST terminate current execution immediately
SUBMIT() MUST validate all required output fields present
SUBMIT() MUST validate field types match signature
SUBMIT() MUST return SubmitResult to Rust side
Multiple SUBMIT() calls: only first is processed

Acceptance Criteria:

Immediate termination on SUBMIT()
All validation rules enforced
Multiple calls handled gracefully

SPEC-20.09: Validation Errors

Error types for SUBMIT validation.

pub enum SubmitError {
    MissingField {
        field: String,
        expected_type: FieldType,
    },
    TypeMismatch {
        field: String,
        expected: FieldType,
        got: String,
        value_preview: String,  // First 100 chars
    },
    EnumInvalid {
        field: String,
        value: String,
        allowed: Vec<String>,
    },
    ValidationFailed {
        field: String,
        reason: String,
    },
}

impl SubmitError {
    pub fn to_user_message(&self) -> String {
        // Human-readable error message
    }
}

Acceptance Criteria:

All error variants have clear messages
Value preview helps debugging
Errors are actionable

SPEC-20.10: REPL Protocol Extension

JSON-RPC protocol for signature registration and execute-response submit payload.

// Register signature before execution
{
    "jsonrpc": "2.0",
    "method": "register_signature",
    "params": {
        "output_fields": [
            {"name": "vulnerabilities", "type": "list", "required": true},
            {"name": "severity", "type": "enum", "values": ["low", "medium", "high", "critical"]}
        ]
    },
    "id": 1
}

// Optional signature cleanup
{
    "jsonrpc": "2.0",
    "method": "clear_signature",
    "params": null,
    "id": 2
}

// Execute response carrying SUBMIT result
{
    "jsonrpc": "2.0",
    "result": {
        "success": true,
        "result": null,
        "stdout": "",
        "stderr": "",
        "error": null,
        "error_type": null,
        "execution_time_ms": 12.3,
        "pending_operations": [],
        "submit_result": {
            "status": "success",
            "outputs": {
                "vulnerabilities": ["SQL injection"],
                "severity": "high"
            }
        }
    },
    "id": 3
}

Acceptance Criteria:

register_signature method implemented
clear_signature method implemented
Execute responses include optional submit_result payload for SUBMIT outcomes

SPEC-20.11: Module Trait

Composable module abstraction.

pub trait Module: Send + Sync {
    type Signature: Signature;

    /// Execute the module with inputs
    async fn forward(
        &self,
        inputs: <Self::Signature as Signature>::Inputs
    ) -> Result<<Self::Signature as Signature>::Outputs, ModuleError>;

    /// Get all predictors in this module (for optimization)
    fn predictors(&self) -> Vec<&dyn Predictor>;

    /// Set LLM client for all predictors
    fn set_lm(&mut self, lm: Arc<dyn LLMClient>);

    /// Get current LLM client
    fn get_lm(&self) -> Option<Arc<dyn LLMClient>>;
}

Acceptance Criteria:

Module trait is object-safe where needed
Predictors discoverable for optimization
LM propagation works

SPEC-20.12: Predict Wrapper

Basic predictor that executes a signature.

pub struct Predict<S: Signature> {
    signature: PhantomData<S>,
    lm: Option<Arc<dyn LLMClient>>,
    demonstrations: Vec<Demonstration<S>>,
    config: PredictConfig,
}

pub struct PredictConfig {
    pub temperature: f64,
    pub max_tokens: Option<u32>,
    pub stop_sequences: Vec<String>,
}

pub struct Demonstration<S: Signature> {
    pub inputs: S::Inputs,
    pub outputs: S::Outputs,
}

impl<S: Signature> Module for Predict<S> {
    type Signature = S;

    async fn forward(&self, inputs: S::Inputs) -> Result<S::Outputs, ModuleError> {
        // 1. Generate prompt from signature + inputs + demonstrations
        // 2. Call LLM
        // 3. Parse response into S::Outputs
        // 4. Validate outputs
    }
}

Acceptance Criteria:

Predict generates correct prompts
Demonstrations injected as few-shot examples
Config options respected

SPEC-20.13: Composition Validation

Compile-time and runtime composition checks.

/// Compose two modules where output of A matches input of B
pub fn compose<A, B>(a: A, b: B) -> Composed<A, B>
where
    A: Module,
    B: Module,
    // Compile-time: A's output type must match B's input type
    <A::Signature as Signature>::Outputs: Into<<B::Signature as Signature>::Inputs>,
{
    Composed { a, b }
}

Runtime Validation:

Verify output field names match input field names
Verify output types are compatible with input types
Propagate LM to all sub-modules

Acceptance Criteria:

Type-safe composition at compile time
Runtime validation for dynamic cases
LM propagation through composition

File Locations

Component	Location
Signature trait	`rlm-core/src/signature/mod.rs`
FieldSpec, FieldType	`rlm-core/src/signature/types.rs`
Validation	`rlm-core/src/signature/validation.rs`
SUBMIT result/error types	`rlm-core/src/signature/submit.rs`
Derive macro	`rlm-core-derive/src/lib.rs`
Module trait	`rlm-core/src/module/mod.rs`
Predict wrapper	`rlm-core/src/module/predict.rs`
REPL SUBMIT runtime	`rlm-core/python/rlm_repl/sandbox.py`
REPL protocol handlers	`rlm-core/python/rlm_repl/main.py`
REPL protocol schema	`rlm-core/python/rlm_repl/protocol.py`
Rust REPL client bridge	`rlm-core/src/repl.rs`

Test Plan

Test	Description	Spec
`signature::tests::derive_tests::*`	Derive macro and signature behavior (including optional-list/array type-inference coverage)	SPEC-20.04, SPEC-20.05
`signature::validation::tests::*`	Validation behavior and error paths	SPEC-20.03, SPEC-20.09
`signature::submit::tests::*`	SubmitResult/SubmitError serialization and semantics	SPEC-20.08, SPEC-20.09
`tests/test_repl.py::TestReplServer::test_submit_*`	Python SUBMIT scenarios (success + validation failures)	SPEC-20.07, SPEC-20.09, SPEC-20.10
`repl::tests::test_submit_result_roundtrip_*` (ignored)	Rust/Python end-to-end submit_result roundtrip scenarios	SPEC-20.08, SPEC-20.10
`module::compose::tests::*`	Module composition runtime mapping validation + deterministic error behavior	SPEC-20.13
`module::predict::tests::*`	Predict wrapper behavior and prompt/input formatting	SPEC-20.12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SPEC-20: Typed Signatures System

Overview

Implementation Snapshot (2026-02-20)

Requirements

SPEC-20.01: Signature Trait

SPEC-20.02: Field Specification

SPEC-20.03: Signature Validation

SPEC-20.04: Derive Macro Attributes

SPEC-20.05: Type Inference

SPEC-20.06: Compile-Time Validation

SPEC-20.07: SUBMIT Function

SPEC-20.08: SUBMIT Behavior

SPEC-20.09: Validation Errors

SPEC-20.10: REPL Protocol Extension

SPEC-20.11: Module Trait

SPEC-20.12: Predict Wrapper

SPEC-20.13: Composition Validation

File Locations

Test Plan

References

FilesExpand file tree

SPEC-20-typed-signatures.md

Latest commit

History

SPEC-20-typed-signatures.md

File metadata and controls

SPEC-20: Typed Signatures System

Overview

Implementation Snapshot (2026-02-20)

Requirements

SPEC-20.01: Signature Trait

SPEC-20.02: Field Specification

SPEC-20.03: Signature Validation

SPEC-20.04: Derive Macro Attributes

SPEC-20.05: Type Inference

SPEC-20.06: Compile-Time Validation

SPEC-20.07: SUBMIT Function

SPEC-20.08: SUBMIT Behavior

SPEC-20.09: Validation Errors

SPEC-20.10: REPL Protocol Extension

SPEC-20.11: Module Trait

SPEC-20.12: Predict Wrapper

SPEC-20.13: Composition Validation

File Locations

Test Plan

References