docs: improved documentation

Author: evilsocket

docs/concepts.md

@@ -23,6 +23,25 @@ using:
   - shell
 ```
 
+#### Agent Configuration Fields
+
+- **`agent`**: The system prompt defining the agent's role
+- **`task`**: The objective or behavior (can use Jinja2 templating)
+- **`generator`**: Override the default LLM model
+- **`reasoning`**: Enable reasoning for supported models (`low`, `medium`, or `high`)
+- **`description`**: Human-readable description of the agent
+- **`version`**: Version string for your agent
+- **`requires`**: Minimum Nerve version requirement (e.g., `">=1.2.0"`)
+- **`defaults`**: Pre-set values for variables that can be overridden via CLI
+- **`limits`**: Execution constraints:
+  - `max_steps`: Maximum number of steps
+  - `max_cost`: Maximum cost in USD
+  - `timeout`: Timeout in seconds
+- **`jail`**: Restrict namespace access to specific filesystem paths
+- **`using`**: List of built-in namespaces to import
+- **`mcp`**: MCP server configurations
+- **`tools`**: Custom tool definitions
+
 #### Enable Reasoning
 
 For models supporting reasoning, you can add a `reasoning` field to enable it, with a value that can either be `low`, `medium` or `high`.
@@ -35,7 +54,7 @@ Nerve supports [Jinja2](https://jinja.palletsprojects.com/) templating for dynam
 - Reference built-in variables like `{{ CURRENT_DATE }}` or `{{ LOCAL_IP }}`
 
 ### Tools
-Tools extend the agent’s capabilities. They can be:
+Tools extend the agent's capabilities. They can be:
 - **Shell commands** (interpolated into a shell script)
 - **Python functions** (via annotated `tools.py` files)
 - **Remote tools via MCP** (from another Nerve instance or a compatible server)
@@ -52,6 +71,34 @@ tools:
     tool: curl wttr.in/{{ city }}
 ```
 
+#### Tool Configuration Properties
+
+Tools can have additional properties:
+```yaml
+tools:
+  - name: process_data
+    description: Process the input data
+    complete_task: true  # Marks task as complete when this tool returns
+    print: true  # Print tool output to console  
+    mime: "application/json"  # Specify output format
+    arguments:
+      - name: data
+        description: Input data
+    tool: |  # For shell-based tools, the command template
+      echo "Processing: {{ data }}"
+```
+
+#### Auto-loading Tools
+If a `tools.py` file exists in the agent's directory, it's automatically loaded without needing to specify it in the YAML configuration.
+
+#### Tools in Templates
+You can call tools directly from Jinja2 templates in prompts:
+```yaml
+task: |
+  Process this data: {{ get_data_tool() }}
+  And analyze the results.
+```
+
 ### Workflows
 A **workflow** is a YAML file that chains multiple agents sequentially. Each agent in the pipeline can:
 - Use a different model
@@ -107,4 +154,3 @@ graph TD
 ```
 
 This loop continues until the task is complete, failed, or times out.
-

docs/evaluation.md

@@ -20,6 +20,13 @@ nerve eval path/to/evaluation --output results.json
 ```
 Each case is passed to the agent, and results (e.g., completion, duration, output) are saved.
 
+### Trace Integration
+Evaluation runs can automatically generate trace files for debugging:
+```bash
+nerve eval path/to/evaluation --output results.json --trace eval-trace.jsonl
+```
+This captures all events during evaluation for analysis, including tool calls, variable changes, and execution flow.
+
 
 ## 🗂 Case Formats
 Nerve supports three evaluation case formats:
@@ -91,4 +98,3 @@ Results are written to a `.json` file with details like:
 - [concepts.md](concepts.md#evaluation)
 - [index.md](index.md): CLI usage
 - [mcp.md](mcp.md): when using remote agents or tools in evaluation
-

docs/index.md

@@ -70,6 +70,39 @@ Run it with:
 nerve run new-agent --url cnn.com
 ```
 
+#### Full Configuration Example
+```yaml
+# Complete agent configuration with all available fields
+agent: You are a helpful assistant.
+task: Make an HTTP request to {{ url }}
+generator: openai/gpt-4o  # optional, override default generator
+reasoning: medium  # optional: low, medium, or high for reasoning models
+description: "This agent makes HTTP requests"  # optional description
+version: "1.0.0"  # optional version string
+requires: ">=1.2.0"  # optional minimum Nerve version requirement
+
+# Set default values for variables
+defaults:
+  url: "https://example.com"
+  timeout: 30
+
+# Restrict namespace access to specific paths
+jail:
+  filesystem:
+    - "/allowed/path"
+    - "{{ target_dir }}"
+
+# Set execution limits
+limits:
+  max_steps: 100
+  max_cost: 5.0
+  timeout: 300  # seconds
+
+using:
+  - shell
+  - task
+```
+
 #### Enable Reasoning
 
 For models supporting reasoning, you can add a `reasoning` field to enable it, with a value that can either be `low`, `medium` or `high`.
@@ -168,10 +201,10 @@ Run in interactive step-by-step mode:
 nerve run agent -i
 ```
 Available commands:
-- `step`, `s`, or `Enter`: one step
-- `continue`, `c`: run till done
-- `view`, `v`: view current state
-- `quit`, `q`, `exit`: exit
+- `step`, `s`, or `Enter`: Execute one step
+- `continue`, `c`: Run until task completion
+- `view`, `v`: View current state and variables
+- `quit`, `q`, `exit`: Exit the agent
 
 ### 🎥 Record & Replay
 Record sessions:
@@ -184,6 +217,13 @@ nerve play trace.jsonl
 nerve play trace.jsonl -f  # fast-forward
 ```
 
+### 📊 Trace Files
+Trace files are JSONL (JSON Lines) format with one event per line:
+```bash
+nerve run agent --trace trace.jsonl
+```
+Events include: `task_started`, `step_started`, `tool_called`, `variable_change`, `task_complete`, and more. This is useful for debugging agent behavior and analyzing execution flow.
+
 ### 🛠 Adding Tools
 See [concepts.md](concepts.md#tools) for details.
 
@@ -288,4 +328,4 @@ nerve run <agent-name> --litellm-tracing langfuse
 - [mcp.md](mcp.md): Advanced agent integration with MCP
 - `namespaces.md`: (Coming soon)
 
-Run `nerve -h` to explore all commands and flags.
+Run `nerve -h` to explore all commands and flags.

docs/mcp.md

@@ -37,6 +37,47 @@ mcp:
     url: stream://localhost:8080/example
 ```
 
+### MCP Server Configuration Options
+
+Full configuration options for MCP servers:
+
+```yaml
+mcp:
+  my_server:
+    # For stdio-based servers (process communication)
+    command: python  # Command to execute
+    args: ["server.py"]  # Arguments for the command
+    env:  # Environment variables
+      API_KEY: "secret"
+      DEBUG: "true"
+    session_timeout: 5  # Connection timeout in seconds (default: 5)
+    
+  # For SSE (Server-Sent Events)
+  sse_server:
+    url: http://localhost:9090/  # SSE endpoint
+    headers:  # Optional HTTP headers
+      Authorization: "Bearer token"
+      X-Custom-Header: "value"
+    timeout: 5  # Connection timeout (default: 5)
+    read_timeout: 300  # Read timeout in seconds (default: 5 minutes)
+    
+  # For Streamable HTTP
+  stream_server:
+    url: stream://localhost:8080/example  # Prefix with stream://
+    headers:  # Optional HTTP headers
+      Authorization: "Bearer token"
+    timeout: 5  # Connection timeout
+    read_timeout: 300  # Read timeout
+```
+
+### Transport Mechanisms
+
+Nerve supports three transport mechanisms for MCP:
+
+- **stdio**: Process-based communication (default). The MCP server runs as a subprocess and communicates via standard input/output.
+- **SSE**: Server-Sent Events over HTTP. Connect to an HTTP endpoint that streams events.
+- **Streamable HTTP**: HTTP with streaming responses. Similar to SSE but uses a different protocol (prefix URL with `stream://`).
+
 You can connect to any of the [publicly available MCP servers](https://github.com/punkpeye/awesome-mcp-servers), or define your own custom tools.
 
 ## 🖧 MCP Server
@@ -115,4 +156,3 @@ For full examples, see the [mcp-recipe](https://github.com/evilsocket/nerve/tree
 - [concepts.md](concepts.md#mcp-model-context-protocol) — overview of how MCP fits into Nerve's architecture
 - [index.md](index.md) — quick usage examples
 - [workflows.md](workflows.md) — for linear pipelines (MCP enables more complex ones)
-

docs/namespaces.md

@@ -2,6 +2,15 @@
 
 Nerve offers a rich set of predefined tools, organized in namespaces, that the agent can import [via the `using` directive](index.md#usage). This page contains the list of namespaces available in Nerve, with the descriptive prompt that will be provided to the model.
 
+> [!NOTE]
+> The `jail` directive can restrict filesystem and filesystem_w namespaces to specific paths:
+> ```yaml
+> jail:
+>   filesystem: ["/allowed/path", "/another/allowed/path"]
+>   filesystem_w: ["/writable/path", "{{ dynamic_path }}"]
+> ```
+> Variables can be used in jail paths for dynamic configuration.
+
 ## 🔧 anytool
 
 Let the agent create its own tools in Python.
@@ -334,4 +343,3 @@ Provides tools for getting the current date and time and waiting for a given num
 * `seconds` <i>(<class 'int'>)</i>: The number of seconds to wait
 
 </details>
-

docs/workflows.md

@@ -116,6 +116,28 @@ tools:
           [ ... ]
 ```
 
+## 🔄 State and Variables
+
+### Variable Passing
+Variables are shared across all agents in a workflow. Each agent can:
+- Read variables set by previous agents
+- Set new variables for subsequent agents
+- Override existing variables
+
+### Default Values
+Each agent can define defaults that apply when variables aren't set:
+```yaml
+defaults:
+  temperature: 0.7
+  max_tokens: 1000
+```
+
+### Variable Interpolation
+All text fields support Jinja2 templating:
+```yaml
+task: Process {{ food }} with {{ ingredients }}
+```
+
 ## 📎 Notes
 - Agents receive inputs from previous agents via templating variables (e.g., `{{ ingredients }}`)
 - Each tool must call `complete_task: true` to advance the workflow
@@ -125,4 +147,3 @@ tools:
 - [concepts.md](concepts.md#workflows)
 - [mcp.md](mcp.md): for building advanced orchestrations
 - [index.md](index.md): CLI usage overview
-