diff --git a/CLAUDE.md b/CLAUDE.md
index 93b271f..08646eb 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -7,8 +7,15 @@ This is a **beancount language server** implementation written in Rust that prov
This is a Rust workspace with the following structure:
- **`crates/lsp/`** - Main Rust language server implementation
+ - **`src/checkers/`** - Bean-check validation implementations (Strategy pattern)
+ - **`mod.rs`** - Strategy trait and factory pattern
+ - **`system_call.rs`** - Traditional bean-check binary execution
+ - **`python.rs`** - Python script integration via subprocess
+ - **`pyo3_embedded.rs`** - PyO3 embedded Python integration
+ - **`types.rs`** - Shared data structures
- **`vscode/`** - VSCode extension (TypeScript)
- **`python/`** - Python utilities for beancount integration
+ - **`bean_check.py`** - Enhanced Python validation script with JSON output
- **Root workspace** - Cargo workspace configuration
## Key Files
@@ -27,9 +34,15 @@ This is a Rust workspace with the following structure:
# Build the language server
cargo build
+# Build with PyO3 embedded Python support (experimental)
+cargo build --features python-embedded
+
# Run tests with coverage
cargo llvm-cov --all-features --locked --workspace --lcov --output-path lcov.info -- --include-ignored
+# Run tests with PyO3 feature
+cargo test --features python-embedded
+
# Format code
cargo fmt
@@ -83,11 +96,24 @@ nix flake check
### Language Server Features
-- **Diagnostics** - Provided via beancount Python integration
+- **Diagnostics** - Multi-method validation system with pluggable checkers
+ - **System Call** - Traditional bean-check binary execution (default)
+ - **Python Script** - Enhanced Python script with JSON output (experimental)
+ - **PyO3 Embedded** - Direct Python library integration (experimental)
- **Formatting** - Generates edits similar to bean-format
- **Completions** - Shows completions for Payees, Accounts, Dates
- **Future planned**: definitions, folding, hover, rename
+### Bean-check Integration Architecture
+
+The diagnostics system uses a Strategy pattern with three implementations:
+
+1. **SystemCallChecker** - Executes bean-check binary via subprocess and parses stderr
+2. **PythonChecker** - Runs Python script that outputs structured JSON
+3. **PyO3EmbeddedChecker** - Calls beancount library directly via embedded Python
+
+Factory pattern in `checkers/mod.rs` handles creation based on configuration.
+
### Key Dependencies
- **tree-sitter-beancount** - Parsing via tree-sitter
@@ -95,13 +121,19 @@ nix flake check
- **ropey** - Efficient text rope data structure
- **tracing** - Structured logging
- **anyhow** / **thiserror** - Error handling
-- **regex** - Pattern matching
+- **regex** - Pattern matching for error parsing
- **chrono** - Date/time handling
+- **serde** / **serde_json** - JSON serialization for Python integration
+- **pyo3** - Python integration (optional, feature-gated)
## Configuration
Language server accepts configuration via LSP initialization:
- **journal_file** - Path to main beancount journal file
+- **bean_check.method** - Validation method: "system" (default), "python-script", or "python-embedded"
+- **bean_check.bean_check_cmd** - Path to bean-check binary (for system method)
+- **bean_check.python_cmd** - Path to Python executable (for Python methods)
+- **bean_check.python_script** - Path to Python validation script (for python-script method)
## Testing
@@ -146,7 +178,9 @@ Supports multiple editors:
- **Add new LSP feature**: Modify `crates/lsp/src/handlers.rs` and related provider files
- **Update completions**: Modify `crates/lsp/src/providers/completion.rs`
-- **Add diagnostics**: Integrate with beancount via `python/bean_check.py`
+- **Add new checker method**: Implement `BeancountChecker` trait in `crates/lsp/src/checkers/`
+- **Modify bean-check integration**: Update checker implementations or factory in `crates/lsp/src/checkers/`
+- **Enhance Python validation**: Modify `python/bean_check.py` script
- **Update VSCode extension**: Modify files in `vscode/src/`
## External Dependencies
diff --git a/Cargo.lock b/Cargo.lock
index 6ceee95..1fe86ac 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -110,6 +110,7 @@ dependencies = [
"lsp-server",
"lsp-types",
"nucleo",
+ "pyo3",
"regex",
"ropey",
"serde",
@@ -477,6 +478,12 @@ dependencies = [
"hashbrown 0.15.4",
]
+[[package]]
+name = "indoc"
+version = "2.0.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f4c7245a08504955605670dbf141fceab975f15ca21570696aebe9d2e71576bd"
+
[[package]]
name = "insta"
version = "1.41.1"
@@ -604,6 +611,15 @@ version = "2.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f665ee40bc4a3c5590afb1e9677db74a508659dfd71e126420da8274909a0167"
+[[package]]
+name = "memoffset"
+version = "0.9.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a"
+dependencies = [
+ "autocfg",
+]
+
[[package]]
name = "nu-ansi-term"
version = "0.46.0"
@@ -707,6 +723,12 @@ version = "0.2.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8afb450f006bf6385ca15ef45d71d2288452bc3683ce2e2cacc0d18e4be60b58"
+[[package]]
+name = "portable-atomic"
+version = "1.11.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483"
+
[[package]]
name = "proc-macro2"
version = "1.0.89"
@@ -716,6 +738,68 @@ dependencies = [
"unicode-ident",
]
+[[package]]
+name = "pyo3"
+version = "0.25.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "8970a78afe0628a3e3430376fc5fd76b6b45c4d43360ffd6cdd40bdde72b682a"
+dependencies = [
+ "indoc",
+ "libc",
+ "memoffset",
+ "once_cell",
+ "portable-atomic",
+ "pyo3-build-config",
+ "pyo3-ffi",
+ "pyo3-macros",
+ "unindent",
+]
+
+[[package]]
+name = "pyo3-build-config"
+version = "0.25.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "458eb0c55e7ece017adeba38f2248ff3ac615e53660d7c71a238d7d2a01c7598"
+dependencies = [
+ "once_cell",
+ "target-lexicon",
+]
+
+[[package]]
+name = "pyo3-ffi"
+version = "0.25.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "7114fe5457c61b276ab77c5055f206295b812608083644a5c5b2640c3102565c"
+dependencies = [
+ "libc",
+ "pyo3-build-config",
+]
+
+[[package]]
+name = "pyo3-macros"
+version = "0.25.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8725c0a622b374d6cb051d11a0983786448f7785336139c3c94f5aa6bef7e50"
+dependencies = [
+ "proc-macro2",
+ "pyo3-macros-backend",
+ "quote",
+ "syn",
+]
+
+[[package]]
+name = "pyo3-macros-backend"
+version = "0.25.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4109984c22491085343c05b0dbc54ddc405c3cf7b4374fc533f5c3313a572ccc"
+dependencies = [
+ "heck",
+ "proc-macro2",
+ "pyo3-build-config",
+ "quote",
+ "syn",
+]
+
[[package]]
name = "quote"
version = "1.0.35"
@@ -875,9 +959,9 @@ dependencies = [
[[package]]
name = "serde_json"
-version = "1.0.140"
+version = "1.0.142"
source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "20068b6e96dc6c9bd23e01df8827e6c7e1f2fddd43c21810382803c136b99373"
+checksum = "030fedb782600dcbd6f02d479bf0d817ac3bb40d644745b769d6a96bc3afc5a7"
dependencies = [
"indexmap",
"itoa",
@@ -962,6 +1046,12 @@ dependencies = [
"unicode-ident",
]
+[[package]]
+name = "target-lexicon"
+version = "0.13.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "e502f78cdbb8ba4718f566c418c52bc729126ffd16baee5baa718cf25dd5a69a"
+
[[package]]
name = "tempfile"
version = "3.10.1"
@@ -1188,6 +1278,12 @@ version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f6ccf251212114b54433ec949fd6a7841275f9ada20dddd2f29e9ceea4501493"
+[[package]]
+name = "unindent"
+version = "0.2.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3"
+
[[package]]
name = "url"
version = "2.5.2"
diff --git a/README.md b/README.md
index e0353d1..a962bf3 100644
--- a/README.md
+++ b/README.md
@@ -12,13 +12,13 @@ A [Language Server Protocol](https://microsoft.github.io/language-server-protoco
### 🚀 Currently Implemented
-| LSP Feature | Description | Status |
-|-------------|-------------|---------|
-| **Completions** | Smart autocompletion for accounts, payees, dates, narration, tags, links, and transaction types | ✅ |
-| **Diagnostics** | Real-time error checking and validation via beancount Python integration | ✅ |
-| **Formatting** | Document formatting compatible with `bean-format`, with support for prefix-width, num-width, and currency-column options | ✅ |
-| **Rename** | Rename symbols across files | ✅ |
-| **References** | Find all references to accounts, payees, etc. | ✅ |
+| LSP Feature | Description | Status |
+| --------------- | ------------------------------------------------------------------------------------------------------------------------ | ------ |
+| **Completions** | Smart autocompletion for accounts, payees, dates, narration, tags, links, and transaction types | ✅ |
+| **Diagnostics** | Real-time error checking and validation via beancount Python integration | ✅ |
+| **Formatting** | Document formatting compatible with `bean-format`, with support for prefix-width, num-width, and currency-column options | ✅ |
+| **Rename** | Rename symbols across files | ✅ |
+| **References** | Find all references to accounts, payees, etc. | ✅ |
### 📋 Completion Types
@@ -32,17 +32,17 @@ A [Language Server Protocol](https://microsoft.github.io/language-server-protoco
### 🔮 Planned Features
-| LSP Feature | Description | Priority |
-|-------------|-------------|----------|
-| **Hover** | Show account balances, transaction details, account metadata | High |
-| **Go to Definition** | Jump to account/payee/commodity definitions | High |
-| **Document Symbols** | Outline view showing accounts, transactions, and structure | High |
-| **Folding Ranges** | Fold transactions, account hierarchies, and multi-line entries | Medium |
-| **Semantic Highlighting** | Advanced syntax highlighting with semantic information | Medium |
-| **Code Actions** | Quick fixes, refactoring, auto-balance transactions | Medium |
-| **Inlay Hints** | Show computed balances, exchange rates, running totals | Low |
-| **Signature Help** | Help with transaction syntax and directive parameters | Low |
-| **Workspace Symbols** | Find accounts, payees, commodities across all files | Low |
+| LSP Feature | Description | Priority |
+| ------------------------- | -------------------------------------------------------------- | -------- |
+| **Hover** | Show account balances, transaction details, account metadata | High |
+| **Go to Definition** | Jump to account/payee/commodity definitions | High |
+| **Document Symbols** | Outline view showing accounts, transactions, and structure | High |
+| **Folding Ranges** | Fold transactions, account hierarchies, and multi-line entries | Medium |
+| **Semantic Highlighting** | Advanced syntax highlighting with semantic information | Medium |
+| **Code Actions** | Quick fixes, refactoring, auto-balance transactions | Medium |
+| **Inlay Hints** | Show computed balances, exchange rates, running totals | Low |
+| **Signature Help** | Help with transaction syntax and directive parameters | Low |
+| **Workspace Symbols** | Find accounts, payees, commodities across all files | Low |
## 📦 Installation
@@ -57,8 +57,9 @@ cargo install beancount-language-server
Download the latest release for your platform from the [releases page](https://github.com/polarmutex/beancount-language-server/releases).
**Supported Platforms:**
+
- Linux (x86_64, aarch64, loongarch64)
-- macOS (x86_64, aarch64)
+- macOS (x86_64, aarch64)
- Windows (x86_64)
### Method 3: Homebrew (macOS/Linux)
@@ -85,7 +86,12 @@ nix develop
```bash
git clone https://github.com/polarmutex/beancount-language-server.git
cd beancount-language-server
+
+# Standard build
cargo build --release
+
+# Build with PyO3 embedded Python support (experimental)
+cargo build --release --features python-embedded
```
The binary will be available at `target/release/beancount-language-server`.
@@ -93,16 +99,25 @@ The binary will be available at `target/release/beancount-language-server`.
## 🔧 Requirements
### Required
+
- **Beancount**: Install the Python beancount package for diagnostics
- ```bash
- pip install beancount
- ```
+ ```bash
+ pip install beancount
+ ```
### Optional
+
- **Bean-format**: The language server includes built-in formatting that's fully compatible with bean-format. Installing bean-format is optional for comparison or standalone use
- ```bash
- pip install bean-format
- ```
+ ```bash
+ pip install bean-format
+ ```
+
+### Experimental Features
+
+- **PyO3 Embedded Python**: For improved performance, build with embedded Python support
+ ```bash
+ cargo build --features python-embedded
+ ```
## ⚙️ Configuration
@@ -110,41 +125,117 @@ The language server accepts configuration via LSP initialization options:
```json
{
- "journal_file": "/path/to/main.beancount",
- "formatting": {
- "prefix_width": 30,
- "num_width": 10,
- "currency_column": 60,
- "account_amount_spacing": 2,
- "number_currency_spacing": 1
- }
+ "journal_file": "/path/to/main.beancount",
+ "bean_check": {
+ "method": "system",
+ "bean_check_cmd": "bean-check"
+ },
+ "formatting": {
+ "prefix_width": 30,
+ "num_width": 10,
+ "currency_column": 60,
+ "account_amount_spacing": 2,
+ "number_currency_spacing": 1
+ }
}
```
### Configuration Options
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `journal_file` | string | Path to the main beancount journal file | None |
+| Option | Type | Description | Default |
+| -------------- | ------ | --------------------------------------- | ------- |
+| `journal_file` | string | Path to the main beancount journal file | None |
+
+### Bean-check Configuration
+
+| Option | Type | Description | Default |
+| --------------------------- | ------ | ------------------------------------------------------------------ | ------------------------ |
+| `bean_check.method` | string | Validation method: "system", "python-script", or "python-embedded" | "system" |
+| `bean_check.bean_check_cmd` | string | Path to bean-check binary (for "system" method) | "bean-check" |
+| `bean_check.python_cmd` | string | Path to Python executable (for Python methods) | "python3" |
+| `bean_check.python_script` | string | Path to Python validation script (for "python-script" method) | "./python/bean_check.py" |
+
+#### Bean-check Methods
+
+The language server supports three different methods for validating beancount files:
+
+**System Method** (default):
+
+- Uses the traditional `bean-check` binary via subprocess
+- Fastest startup time, lower memory usage
+- Requires `bean-check` binary to be installed and available in PATH
+- Compatible with all existing bean-check installations
+
+**Python Script Method** (experimental):
+
+- Executes a Python script that uses the beancount library directly
+- Provides structured JSON output for better error handling
+- Supports both validation errors and flagged entry detection
+- Requires Python with beancount library installed
+
+**Python Embedded Method** (experimental):
+
+- Uses PyO3 to embed Python directly in the Rust process
+- Highest performance with no subprocess overhead
+- Best error handling and flagged entry support
+- Requires compilation with `python-embedded` feature
+- Must have beancount library available to embedded Python
+
+#### Configuration Examples
+
+**Traditional system call approach:**
+
+```json
+{
+ "bean_check": {
+ "method": "system",
+ "bean_check_cmd": "/usr/local/bin/bean-check"
+ }
+}
+```
+
+**Python script with custom paths:**
+
+```json
+{
+ "bean_check": {
+ "method": "python-script",
+ "python_cmd": "/usr/bin/python3",
+ "python_script": "./python/bean_check.py"
+ }
+}
+```
+
+**Embedded Python (requires python-embedded feature):**
+
+```json
+{
+ "bean_check": {
+ "method": "python-embedded"
+ }
+}
+```
### Formatting Options
-| Option | Type | Description | Default | Bean-format Equivalent |
-|--------|------|-------------|---------|----------------------|
-| `prefix_width` | number | Fixed width for account names (overrides auto-detection) | Auto-calculated | `--prefix-width` (`-w`) |
-| `num_width` | number | Fixed width for number alignment (overrides auto-detection) | Auto-calculated | `--num-width` (`-W`) |
-| `currency_column` | number | Align currencies at this specific column | None (right-align) | `--currency-column` (`-c`) |
-| `account_amount_spacing` | number | Minimum spaces between account names and amounts | 2 | N/A |
-| `number_currency_spacing` | number | Number of spaces between number and currency | 1 | N/A |
+| Option | Type | Description | Default | Bean-format Equivalent |
+| ------------------------- | ------ | ----------------------------------------------------------- | ------------------ | -------------------------- |
+| `prefix_width` | number | Fixed width for account names (overrides auto-detection) | Auto-calculated | `--prefix-width` (`-w`) |
+| `num_width` | number | Fixed width for number alignment (overrides auto-detection) | Auto-calculated | `--num-width` (`-W`) |
+| `currency_column` | number | Align currencies at this specific column | None (right-align) | `--currency-column` (`-c`) |
+| `account_amount_spacing` | number | Minimum spaces between account names and amounts | 2 | N/A |
+| `number_currency_spacing` | number | Number of spaces between number and currency | 1 | N/A |
#### Formatting Modes
**Default Mode** (no `currency_column` specified):
+
- Accounts are left-aligned
- Numbers are right-aligned with consistent end positions
- Behaves like `bean-format` with no special options
**Currency Column Mode** (`currency_column` specified):
+
- Currencies are aligned at the specified column
- Numbers are positioned to place currencies at the target column
- Equivalent to `bean-format --currency-column N`
@@ -152,53 +243,59 @@ The language server accepts configuration via LSP initialization options:
#### Examples
**Basic formatting with auto-detection:**
+
```json
{
- "formatting": {}
+ "formatting": {}
}
```
**Fixed prefix width (like `bean-format -w 25`):**
+
```json
{
- "formatting": {
- "prefix_width": 25
- }
+ "formatting": {
+ "prefix_width": 25
+ }
}
```
**Currency column alignment (like `bean-format -c 60`):**
+
```json
{
- "formatting": {
- "currency_column": 60
- }
+ "formatting": {
+ "currency_column": 60
+ }
}
```
**Number-currency spacing control:**
+
```json
{
- "formatting": {
- "number_currency_spacing": 2
- }
+ "formatting": {
+ "number_currency_spacing": 2
+ }
}
```
This controls the whitespace between numbers and currency codes:
+
- `0`: No space (`100.00USD`)
- `1`: Single space (`100.00 USD`) - default
- `2`: Two spaces (`100.00 USD`)
**Combined options:**
+
```json
{
- "formatting": {
- "prefix_width": 30,
- "currency_column": 65,
- "account_amount_spacing": 3,
- "number_currency_spacing": 1
- }
+ "formatting": {
+ "prefix_width": 30,
+ "currency_column": 65,
+ "account_amount_spacing": 3,
+ "number_currency_spacing": 1
+ }
}
```
@@ -208,16 +305,20 @@ This controls the whitespace between numbers and currency codes:
1. Install the [Beancount extension](https://marketplace.visualstudio.com/items?itemName=polarmutex.beancount-langserver) from the marketplace
2. Configure in `settings.json`:
- ```json
- {
- "beancount.journal_file": "/path/to/main.beancount",
- "beancount.formatting": {
- "prefix_width": 30,
- "currency_column": 60,
- "number_currency_spacing": 1
- }
- }
- ```
+ ```json
+ {
+ "beancount.journal_file": "/path/to/main.beancount",
+ "beancount.bean_check": {
+ "method": "python-embedded",
+ "python_cmd": "python3"
+ },
+ "beancount.formatting": {
+ "prefix_width": 30,
+ "currency_column": 60,
+ "number_currency_spacing": 1
+ }
+ }
+ ```
### Neovim
@@ -229,6 +330,11 @@ local lspconfig = require('lspconfig')
lspconfig.beancount.setup({
init_options = {
journal_file = "/path/to/main.beancount",
+ bean_check = {
+ method = "python-script",
+ python_cmd = "python3",
+ python_script = "./python/bean_check.py",
+ },
formatting = {
prefix_width = 30,
currency_column = 60,
@@ -239,6 +345,7 @@ lspconfig.beancount.setup({
```
**File type detection**: Ensure beancount files are detected. Add to your config:
+
```lua
vim.filetype.add({
extension = {
@@ -260,6 +367,10 @@ args = ["--stdio"]
[language-server.beancount-language-server.config]
journal_file = "/path/to/main.beancount"
+[language-server.beancount-language-server.config.bean_check]
+method = "system"
+bean_check_cmd = "bean-check"
+
[language-server.beancount-language-server.config.formatting]
prefix_width = 30
currency_column = 60
@@ -285,6 +396,7 @@ Using [lsp-mode](https://github.com/emacs-lsp/lsp-mode):
:server-id 'beancount-language-server
:initialization-options
(lambda () (list :journal_file "/path/to/main.beancount"
+ :bean_check '(:method "python-embedded")
:formatting '(:prefix_width 30 :currency_column 60 :number_currency_spacing 1))))))
```
@@ -300,6 +412,10 @@ if executable('beancount-language-server')
\ 'allowlist': ['beancount'],
\ 'initialization_options': {
\ 'journal_file': '/path/to/main.beancount',
+ \ 'bean_check': {
+ \ 'method': 'system',
+ \ 'bean_check_cmd': 'bean-check'
+ \ },
\ 'formatting': {
\ 'prefix_width': 30,
\ 'currency_column': 60,
@@ -315,23 +431,24 @@ endif
Using [LSP](https://packagecontrol.io/packages/LSP):
Add to LSP settings:
+
```json
{
- "clients": {
- "beancount-language-server": {
- "enabled": true,
- "command": ["beancount-language-server"],
- "selector": "source.beancount",
- "initializationOptions": {
- "journal_file": "/path/to/main.beancount",
- "formatting": {
- "prefix_width": 30,
- "currency_column": 60,
- "number_currency_spacing": 1
+ "clients": {
+ "beancount-language-server": {
+ "enabled": true,
+ "command": ["beancount-language-server"],
+ "selector": "source.beancount",
+ "initializationOptions": {
+ "journal_file": "/path/to/main.beancount",
+ "formatting": {
+ "prefix_width": 30,
+ "currency_column": 60,
+ "number_currency_spacing": 1
+ }
+ }
}
- }
}
- }
}
```
@@ -355,7 +472,8 @@ Add to LSP settings:
- **LSP Server**: Main Rust application handling LSP protocol
- **Tree-sitter Parser**: Fast, incremental parsing of Beancount syntax
- **Completion Engine**: Smart autocompletion with context awareness
-- **Diagnostic Provider**: Integration with beancount Python for validation
+- **Diagnostic Provider**: Multi-method validation system with pluggable checkers
+- **Bean-check Integration**: Three validation methods (system, python-embedded)
- **Formatter**: Code formatting fully compatible with bean-format, supporting prefix-width, num-width, and currency-column options
### Project Structure
@@ -366,9 +484,13 @@ beancount-language-server/
│ ├── src/
│ │ ├── handlers.rs # LSP request/notification handlers
│ │ ├── providers/ # Feature providers (completion, diagnostics, etc.)
+│ │ ├── checkers/ # Bean-check validation implementations
+│ │ │ ├── mod.rs # Strategy trait and factory pattern
+│ │ │ ├── system_call.rs # Traditional bean-check binary
+│ │ │ ├── pyo3_embedded.rs # PyO3 embedded Python
+│ │ │ └── types.rs # Shared data structures
│ │ └── server.rs # Core LSP server logic
├── vscode/ # VS Code extension
-├── python/ # Python integration utilities
└── flake.nix # Nix development environment
```
@@ -383,11 +505,13 @@ beancount-language-server/
### Development Environment
**Using Nix (Recommended):**
+
```bash
nix develop
```
**Manual Setup:**
+
```bash
# Install Rust dependencies
cargo build
@@ -408,6 +532,9 @@ cargo test
# Run with coverage
cargo llvm-cov --all-features --locked --workspace --lcov --output-path lcov.info
+# Run tests with PyO3 feature
+cargo test --features python-embedded
+
# Run specific test
cargo test test_completion
```
@@ -430,9 +557,9 @@ cargo fmt -- --check
1. **Make changes** to the Rust code
2. **Test locally** with `cargo test`
3. **Run LSP server** in development mode:
- ```bash
- cargo run --bin beancount-language-server
- ```
+ ```bash
+ cargo run --bin beancount-language-server
+ ```
4. **Test with editor** by configuring it to use the local binary
### VS Code Extension Development
@@ -450,20 +577,22 @@ The project uses [cargo-dist](https://opensource.axo.dev/cargo-dist/) for automa
1. **Tag a release**: `git tag v1.0.0 && git push --tags`
2. **GitHub Actions** automatically builds and publishes:
- - Binaries for all supported platforms
- - Crates.io release
- - GitHub release with assets
+ - Binaries for all supported platforms
+ - Crates.io release
+ - GitHub release with assets
## 🤝 Contributing
Contributions are welcome! Here are some ways to help:
### 🐛 Bug Reports
+
- Search existing issues first
- Include beancount file examples that trigger the bug
- Provide editor and OS information
### 💡 Feature Requests
+
- Check the [planned features](#-planned-features) list
- Describe the use case and expected behavior
- Consider the LSP specification constraints
@@ -481,6 +610,7 @@ Contributions are welcome! Here are some ways to help:
### 🎯 Good First Issues
Look for issues labeled `good-first-issue`:
+
- Add new completion types
- Improve error messages
- Add editor configuration examples
@@ -508,4 +638,4 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
Happy Beancounting! 📊✨
-
\ No newline at end of file
+
diff --git a/beancount-language-server.log b/beancount-language-server.log
new file mode 100644
index 0000000..cf9e153
--- /dev/null
+++ b/beancount-language-server.log
@@ -0,0 +1,19 @@
+2025-07-27T17:47:09.835410Z INFO ThreadId(01) Starting beancount-language-server v1.4.1
+2025-07-27T17:47:09.835450Z INFO ThreadId(01) beancount-language-server started
+2025-07-27T19:17:36.369616Z INFO ThreadId(01) Starting beancount-language-server v1.4.1
+2025-07-27T19:17:36.369687Z DEBUG ThreadId(01) Command line args: stdio=true, log_level=Some("debug")
+2025-07-27T19:17:36.369726Z INFO ThreadId(01) beancount-language-server started
+2025-07-27T19:17:36.369746Z DEBUG ThreadId(01) Setting up stdio connections
+2025-07-27T19:17:36.369862Z DEBUG ThreadId(01) Waiting for client initialization
+2025-07-27T19:17:53.455800Z INFO ThreadId(01) Starting beancount-language-server v1.4.1
+2025-07-27T19:17:53.455872Z INFO ThreadId(01) beancount-language-server started
+2025-07-27T19:19:05.776081Z INFO ThreadId(01) Starting beancount-language-server v1.4.1
+2025-07-27T19:19:05.776151Z DEBUG ThreadId(01) Command line args: stdio=true, log_to_file=true, log_level=None
+2025-07-27T19:19:05.776191Z INFO ThreadId(01) beancount-language-server started
+2025-07-27T19:19:05.776205Z DEBUG ThreadId(01) Setting up stdio connections
+2025-07-27T19:19:05.776317Z DEBUG ThreadId(01) Waiting for client initialization
+2025-07-27T19:19:14.817888Z INFO ThreadId(01) Starting beancount-language-server v1.4.1
+2025-07-27T19:19:14.817958Z DEBUG ThreadId(01) Command line args: stdio=true, log_to_file=true, log_level=Some("trace")
+2025-07-27T19:19:14.817998Z INFO ThreadId(01) beancount-language-server started
+2025-07-27T19:19:14.818019Z DEBUG ThreadId(01) Setting up stdio connections
+2025-07-27T19:19:14.818161Z DEBUG ThreadId(01) Waiting for client initialization
diff --git a/crates/lsp/Cargo.toml b/crates/lsp/Cargo.toml
index 07f2a8d..f010c6e 100644
--- a/crates/lsp/Cargo.toml
+++ b/crates/lsp/Cargo.toml
@@ -50,6 +50,9 @@ dashmap = "6.1"
bytes = "1.10"
linked-list = "0.1"
+# Python integration (optional)
+pyo3 = { version = "0.25", features = ["auto-initialize"], optional = true }
+
# Logging
tracing-subscriber = { version = "0.3", default-features = false, features = [
"env-filter",
@@ -71,3 +74,7 @@ dist = true
[package.metadata.release]
tag = true
+
+[features]
+default = []
+python-embedded = ["pyo3"]
diff --git a/crates/lsp/src/checkers/mod.rs b/crates/lsp/src/checkers/mod.rs
new file mode 100644
index 0000000..73b1aa6
--- /dev/null
+++ b/crates/lsp/src/checkers/mod.rs
@@ -0,0 +1,142 @@
+use anyhow::Result;
+use std::path::{Path, PathBuf};
+use tracing::debug;
+
+pub mod pyo3_embedded;
+pub mod system_call;
+pub mod types;
+
+pub use pyo3_embedded::PyO3EmbeddedChecker;
+pub use system_call::SystemCallChecker;
+pub use types::*;
+
+/// Trait for different bean-check execution strategies.
+///
+/// This allows the language server to support multiple ways of running
+/// bean-check validation, including system calls and Python integration.
+pub trait BeancountChecker: Send + Sync {
+ /// Execute bean-check validation on the given journal file.
+ ///
+ /// # Arguments
+ /// * `journal_file` - Path to the main beancount journal file to validate
+ ///
+ /// # Returns
+ /// * `Ok(BeancountCheckResult)` - Validation results with errors and flagged entries
+ /// * `Err(anyhow::Error)` - Execution error (command not found, parsing failed, etc.)
+ fn check(&self, journal_file: &Path) -> Result;
+
+ /// Get a human-readable name for this checker implementation.
+ /// Used for logging and debugging purposes.
+ fn name(&self) -> &'static str;
+
+ /// Check if this checker implementation is available on the current system.
+ /// For example, system call checker needs bean-check binary, Python checker needs Python.
+ fn is_available(&self) -> bool;
+}
+
+/// Configuration for bean-check execution method selection.
+#[derive(Debug, Clone)]
+pub enum BeancountCheckMethod {
+ /// Use system call to execute bean-check binary (traditional approach)
+ SystemCall,
+ /// Use embedded Python via PyO3 to call beancount library directly (best performance)
+ PythonEmbedded,
+}
+
+impl Default for BeancountCheckMethod {
+ fn default() -> Self {
+ // Default to system call for backward compatibility
+ Self::SystemCall
+ }
+}
+
+/// Configuration options for bean-check execution.
+#[derive(Debug, Clone)]
+pub struct BeancountCheckConfig {
+ /// Which execution method to use
+ pub method: BeancountCheckMethod,
+ /// Path to bean-check executable (for SystemCall method)
+ pub bean_check_cmd: PathBuf,
+ /// Path to Python executable (for Python method)
+ pub python_cmd: PathBuf,
+ /// Path to the Python script (for Python method)
+ pub python_script_path: PathBuf,
+}
+
+impl Default for BeancountCheckConfig {
+ fn default() -> Self {
+ Self {
+ method: BeancountCheckMethod::default(),
+ bean_check_cmd: PathBuf::from("bean-check"),
+ python_cmd: PathBuf::from("python3"),
+ python_script_path: PathBuf::from("python/bean_check.py"),
+ }
+ }
+}
+
+/// Factory function to create a checker based on configuration.
+///
+/// # Arguments
+/// * `config` - Configuration specifying which checker to create and its options
+///
+/// # Returns
+/// * Boxed trait object implementing BeancountChecker
+pub fn create_checker(config: &BeancountCheckConfig) -> Box {
+ debug!("Creating bean checker with method: {:?}", config.method);
+
+ let checker: Box = match config.method {
+ BeancountCheckMethod::SystemCall => {
+ debug!(
+ "Creating SystemCallChecker with command: {}",
+ config.bean_check_cmd.display()
+ );
+ Box::new(SystemCallChecker::new(config.bean_check_cmd.clone()))
+ }
+ BeancountCheckMethod::PythonEmbedded => {
+ debug!("Creating PyO3EmbeddedChecker");
+ Box::new(PyO3EmbeddedChecker::new())
+ }
+ };
+
+ debug!(
+ "Created checker: {}, availability: {}",
+ checker.name(),
+ checker.is_available()
+ );
+ checker
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn test_default_config() {
+ let config = BeancountCheckConfig::default();
+ assert!(matches!(config.method, BeancountCheckMethod::SystemCall));
+ assert_eq!(config.bean_check_cmd, PathBuf::from("bean-check"));
+ assert_eq!(config.python_cmd, PathBuf::from("python3"));
+ }
+
+ #[test]
+ fn test_factory_system_call() {
+ let config = BeancountCheckConfig::default();
+ let checker = create_checker(&config);
+ assert_eq!(checker.name(), "SystemCall");
+ }
+
+ #[test]
+ fn test_factory_python_embedded() {
+ let config = BeancountCheckConfig {
+ method: BeancountCheckMethod::PythonEmbedded,
+ ..Default::default()
+ };
+ let checker = create_checker(&config);
+
+ #[cfg(feature = "python-embedded")]
+ assert_eq!(checker.name(), "PyO3Embedded");
+
+ #[cfg(not(feature = "python-embedded"))]
+ assert_eq!(checker.name(), "PyO3Embedded (disabled)");
+ }
+}
diff --git a/crates/lsp/src/checkers/pyo3_embedded.rs b/crates/lsp/src/checkers/pyo3_embedded.rs
new file mode 100644
index 0000000..1cabab7
--- /dev/null
+++ b/crates/lsp/src/checkers/pyo3_embedded.rs
@@ -0,0 +1,615 @@
+#[cfg(feature = "python-embedded")]
+use super::types::*;
+#[cfg(feature = "python-embedded")]
+use super::BeancountChecker;
+#[cfg(feature = "python-embedded")]
+use anyhow::{Context, Result};
+#[cfg(feature = "python-embedded")]
+use pyo3::prelude::*;
+#[cfg(feature = "python-embedded")]
+use pyo3::types::{PyDict, PyList, PyString};
+#[cfg(feature = "python-embedded")]
+use std::path::{Path, PathBuf};
+#[cfg(feature = "python-embedded")]
+use std::sync::OnceLock;
+#[cfg(feature = "python-embedded")]
+use tracing::{debug, warn};
+
+#[cfg(feature = "python-embedded")]
+/// Bean-check implementation using embedded Python via PyO3.
+///
+/// This approach embeds the Python interpreter directly into the Rust process
+/// and calls the beancount library functions directly without subprocess overhead.
+/// Provides the best performance and error handling of all implementations.
+#[derive(Debug)]
+pub struct PyO3EmbeddedChecker {
+ /// Whether to cache the Python code compilation (future optimization)
+ _cache_compiled_code: bool,
+}
+
+/// Cache for the beancount.loader module to avoid repeated imports
+#[cfg(feature = "python-embedded")]
+static BEANCOUNT_LOADER_CACHE: OnceLock> = OnceLock::new();
+
+#[cfg(feature = "python-embedded")]
+impl PyO3EmbeddedChecker {
+ /// Create a new PyO3 embedded checker.
+ pub fn new() -> Self {
+ Self {
+ _cache_compiled_code: true,
+ }
+ }
+
+ /// Execute beancount validation using embedded Python.
+ fn execute_beancount_check(&self, journal_file: &Path) -> Result {
+ debug!(
+ "PyO3EmbeddedChecker: starting embedded beancount check for file: {}",
+ journal_file.display()
+ );
+
+ // Check if file exists before proceeding
+ if !journal_file.exists() {
+ warn!(
+ "PyO3EmbeddedChecker: journal file does not exist: {}",
+ journal_file.display()
+ );
+ return Err(anyhow::anyhow!(
+ "Journal file does not exist: {}",
+ journal_file.display()
+ ));
+ }
+
+ debug!("PyO3EmbeddedChecker: entering Python GIL context");
+ Python::with_gil(|py| {
+ debug!("PyO3EmbeddedChecker: successfully acquired Python GIL");
+
+ // Import required Python modules (cached for performance)
+ debug!("PyO3EmbeddedChecker: attempting to import beancount.loader");
+ let beancount_loader = py
+ .import("beancount.loader")
+ .context("Failed to import beancount.loader - ensure beancount is installed")?;
+ debug!("PyO3EmbeddedChecker: successfully imported beancount.loader");
+
+ // Convert file path to Python string
+ debug!("PyO3EmbeddedChecker: converting file path to Python string");
+ let file_path_str = journal_file
+ .to_str()
+ .context("File path contains invalid UTF-8")?;
+ let py_file_path = PyString::new(py, file_path_str);
+ debug!(
+ "PyO3EmbeddedChecker: created Python string for file path: {}",
+ file_path_str
+ );
+
+ // Call beancount.loader.load_file(file_path)
+ debug!(
+ "PyO3EmbeddedChecker: calling beancount.loader.load_file with path: {}",
+ file_path_str
+ );
+ let load_result = beancount_loader
+ .call_method1("load_file", (py_file_path,))
+ .context("Failed to call beancount.loader.load_file")?;
+ debug!("PyO3EmbeddedChecker: beancount.loader.load_file completed successfully");
+
+ // Extract the tuple (entries, errors, options)
+ debug!("PyO3EmbeddedChecker: extracting results tuple from load_file");
+ let (entries, errors, _options): (Bound, Bound, Bound) =
+ load_result
+ .extract()
+ .context("Failed to extract load_file result tuple")?;
+
+ debug!(
+ "PyO3EmbeddedChecker: extracted {} entries, {} errors from beancount",
+ entries.len(),
+ errors.len()
+ );
+
+ // Process errors (pre-allocate capacity for performance)
+ debug!(
+ "PyO3EmbeddedChecker: processing {} beancount errors",
+ errors.len()
+ );
+ let mut beancount_errors = Vec::with_capacity(errors.len());
+ for (error_index, error_obj) in errors.iter().enumerate() {
+ debug!(
+ "PyO3EmbeddedChecker: processing error {} of {}",
+ error_index + 1,
+ errors.len()
+ );
+ match self.extract_error_info(py, &error_obj, journal_file) {
+ Ok(error) => {
+ debug!(
+ "PyO3EmbeddedChecker: successfully extracted error: {}:{} - {}",
+ error.file.display(),
+ error.line,
+ error.message
+ );
+ beancount_errors.push(error);
+ }
+ Err(e) => {
+ warn!(
+ "PyO3EmbeddedChecker: failed to extract error info for error {}: {}",
+ error_index + 1,
+ e
+ );
+ // Add a generic error as fallback
+ beancount_errors.push(BeancountError::new(
+ journal_file.to_path_buf(),
+ 0,
+ format!("Failed to process beancount error: {e}"),
+ ));
+ }
+ }
+ }
+
+ // Process flagged entries (estimate capacity and optimize early returns)
+ debug!(
+ "PyO3EmbeddedChecker: processing {} entries for flags",
+ entries.len()
+ );
+ let mut flagged_entries = Vec::new();
+ for (entry_index, entry_obj) in entries.iter().enumerate() {
+ debug!(
+ "PyO3EmbeddedChecker: processing entry {} of {} for flags",
+ entry_index + 1,
+ entries.len()
+ );
+ match self.extract_flagged_entry_info(py, &entry_obj) {
+ Ok(Some(flagged)) => {
+ debug!(
+ "PyO3EmbeddedChecker: found flagged entry: {}:{} - {}",
+ flagged.file.display(),
+ flagged.line,
+ flagged.message
+ );
+ flagged_entries.push(flagged);
+ }
+ Ok(None) => {
+ // Not flagged, skip silently for performance
+ }
+ Err(e) => {
+ debug!("PyO3EmbeddedChecker: failed to extract flagged entry info for entry {}: {}", entry_index + 1, e);
+ // Non-critical, continue processing
+ }
+ }
+ }
+
+ debug!(
+ "PyO3EmbeddedChecker: processing complete - {} errors, {} flagged entries found",
+ beancount_errors.len(),
+ flagged_entries.len()
+ );
+
+ let result = BeancountCheckResult {
+ errors: beancount_errors,
+ flagged_entries,
+ };
+
+ debug!(
+ "PyO3EmbeddedChecker: returning result with {} errors and {} flagged entries",
+ result.errors.len(),
+ result.flagged_entries.len()
+ );
+
+ Ok(result)
+ })
+ }
+
+ /// Extract error information from a Python error object.
+ fn extract_error_info(
+ &self,
+ _py: Python,
+ error_obj: &Bound<'_, PyAny>,
+ default_file: &Path,
+ ) -> Result {
+ // Get the error source (filename and line number)
+ let source = error_obj
+ .getattr("source")
+ .context("Error object missing 'source' attribute")?;
+
+ let (filename, line_number) = if source.is_none() {
+ // No source information, use defaults
+ (default_file.to_path_buf(), 0)
+ } else {
+ // Source is a dictionary, not an object with attributes
+ let filename_attr = source
+ .get_item("filename")
+ .and_then(|f| f.extract::())
+ .unwrap_or_else(|_| default_file.to_string_lossy().to_string());
+
+ let line_number = source
+ .get_item("lineno")
+ .and_then(|l| l.extract::())
+ .unwrap_or(0);
+
+ (PathBuf::from(filename_attr), line_number)
+ };
+
+ // Get the error message
+ let message = error_obj
+ .getattr("message")
+ .and_then(|m| m.extract::())
+ .or_else(|_| error_obj.str().map(|s| s.to_string()))
+ .unwrap_or_else(|_| "Unknown beancount error".to_string());
+
+ Ok(BeancountError::new(filename, line_number, message))
+ }
+
+ /// Extract flagged entry information from a Python entry object.
+ fn extract_flagged_entry_info(
+ &self,
+ _py: Python,
+ entry_obj: &Bound<'_, PyAny>,
+ ) -> Result> {
+ // Check if the entry has a 'flag' attribute (fast early return)
+ let flag = match entry_obj.getattr("flag") {
+ Ok(flag_obj) => match flag_obj.extract::() {
+ Ok(flag_str) => Some(flag_str),
+ Err(_) => return Ok(None), // Fast exit for non-string flags
+ },
+ Err(_) => return Ok(None), // Fast exit for entries without flags
+ };
+
+ // Only process entries with review flags (fast filter)
+ let flag_str = match flag.as_deref() {
+ Some("!") => "Transaction flagged for review",
+ Some(_) => return Ok(None), // Fast exit for non-review flags
+ None => return Ok(None), // Fast exit for no flag
+ };
+
+ // Get metadata for file and line information
+ let meta = entry_obj
+ .getattr("meta")
+ .context("Entry missing 'meta' attribute")?;
+
+ let filename = meta
+ .get_item("filename")
+ .and_then(|f| f.extract::())
+ .unwrap_or_else(|_| "".to_string());
+
+ let line_number = meta
+ .get_item("lineno")
+ .and_then(|l| l.extract::())
+ .unwrap_or(0);
+
+ debug!(
+ "PyO3EmbeddedChecker: creating flagged entry for {}:{} - {}",
+ filename, line_number, flag_str
+ );
+
+ Ok(Some(FlaggedEntry::new(
+ PathBuf::from(filename),
+ line_number,
+ flag_str.to_string(),
+ )))
+ }
+}
+
+#[cfg(feature = "python-embedded")]
+impl Default for PyO3EmbeddedChecker {
+ fn default() -> Self {
+ Self::new()
+ }
+}
+
+#[cfg(feature = "python-embedded")]
+impl BeancountChecker for PyO3EmbeddedChecker {
+ fn check(&self, journal_file: &Path) -> Result {
+ debug!(
+ "PyO3EmbeddedChecker::check() called for file: {}",
+ journal_file.display()
+ );
+
+ // Check availability first (cached for performance)
+ if !self.is_available() {
+ warn!("PyO3EmbeddedChecker: checker is not available - beancount library cannot be imported");
+ return Err(anyhow::anyhow!(
+ "PyO3EmbeddedChecker is not available - beancount library cannot be imported"
+ ));
+ }
+
+ debug!("PyO3EmbeddedChecker: availability confirmed, proceeding with check");
+
+ match self.execute_beancount_check(journal_file) {
+ Ok(result) => {
+ debug!("PyO3EmbeddedChecker::check() completed successfully with {} errors and {} flagged entries",
+ result.errors.len(), result.flagged_entries.len());
+ Ok(result)
+ }
+ Err(e) => {
+ warn!("PyO3EmbeddedChecker::check() failed: {}", e);
+ Err(e).context("PyO3 embedded beancount check failed")
+ }
+ }
+ }
+
+ fn name(&self) -> &'static str {
+ "PyO3Embedded"
+ }
+
+ fn is_available(&self) -> bool {
+ debug!("PyO3EmbeddedChecker::is_available() checking beancount availability");
+
+ // Use cached result if available for performance
+ let cache_result = BEANCOUNT_LOADER_CACHE.get_or_init(|| {
+ Python::with_gil(|py| {
+ debug!("PyO3EmbeddedChecker: trying to import beancount.loader in GIL context");
+ match py.import("beancount.loader") {
+ Ok(_) => {
+ debug!("PyO3EmbeddedChecker: successfully imported beancount.loader");
+ Ok(())
+ }
+ Err(e) => {
+ warn!(
+ "PyO3EmbeddedChecker: failed to import beancount.loader: {}",
+ e
+ );
+ Err(e.to_string())
+ }
+ }
+ })
+ });
+
+ let available = cache_result.is_ok();
+ debug!(
+ "PyO3EmbeddedChecker::is_available() returning: {}",
+ available
+ );
+ available
+ }
+}
+
+// Provide a stub implementation when the feature is not enabled
+#[cfg(not(feature = "python-embedded"))]
+pub struct PyO3EmbeddedChecker;
+
+#[cfg(not(feature = "python-embedded"))]
+impl PyO3EmbeddedChecker {
+ pub fn new() -> Self {
+ Self
+ }
+}
+
+#[cfg(not(feature = "python-embedded"))]
+impl Default for PyO3EmbeddedChecker {
+ fn default() -> Self {
+ Self::new()
+ }
+}
+
+#[cfg(not(feature = "python-embedded"))]
+impl crate::checkers::BeancountChecker for PyO3EmbeddedChecker {
+ fn check(
+ &self,
+ _journal_file: &std::path::Path,
+ ) -> anyhow::Result {
+ Err(anyhow::anyhow!(
+ "PyO3 embedded checker not available - compile with 'python-embedded' feature"
+ ))
+ }
+
+ fn name(&self) -> &'static str {
+ "PyO3Embedded (disabled)"
+ }
+
+ fn is_available(&self) -> bool {
+ false
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use crate::checkers::BeancountChecker;
+
+ #[test]
+ fn test_pyo3_checker_creation() {
+ let checker = PyO3EmbeddedChecker::new();
+
+ #[cfg(feature = "python-embedded")]
+ assert_eq!(checker.name(), "PyO3Embedded");
+
+ #[cfg(not(feature = "python-embedded"))]
+ assert_eq!(checker.name(), "PyO3Embedded (disabled)");
+ }
+
+ #[cfg(feature = "python-embedded")]
+ #[test]
+ fn test_pyo3_checker_availability() {
+ let checker = PyO3EmbeddedChecker::new();
+ // Note: This test will pass/fail based on whether beancount is installed
+ // In CI/CD, we'd want to control this with test environment setup
+ let _is_available = checker.is_available();
+ // Don't assert specific value since it depends on environment
+ }
+
+ #[cfg(not(feature = "python-embedded"))]
+ #[test]
+ fn test_pyo3_checker_disabled() {
+ let checker = PyO3EmbeddedChecker::new();
+ assert!(!checker.is_available());
+ assert_eq!(checker.name(), "PyO3Embedded (disabled)");
+ }
+
+ #[cfg(feature = "python-embedded")]
+ #[test]
+ fn test_pyo3_checker_with_valid_file() {
+ use std::fs;
+ use tempfile::TempDir;
+
+ let checker = PyO3EmbeddedChecker::new();
+
+ // Only run if beancount is available
+ if !checker.is_available() {
+ return;
+ }
+
+ let temp_dir = TempDir::new().expect("Failed to create temp directory");
+ let file_path = temp_dir.path().join("test.beancount");
+ let content = "2023-01-01 open Assets:Cash";
+ fs::write(&file_path, content).expect("Failed to write temp file");
+
+ let result = checker.check(&file_path);
+
+ // Should succeed (exact content depends on beancount validation)
+ match result {
+ Ok(_check_result) => {
+ // Basic validation that we got a result
+ // (actual counts depend on beancount validation behavior)
+ }
+ Err(e) => {
+ // If beancount is not properly configured, that's OK for this test
+ eprintln!(
+ "Beancount check failed (possibly due to environment): {e}"
+ );
+ }
+ }
+ }
+
+ #[cfg(feature = "python-embedded")]
+ #[test]
+ fn test_pyo3_checker_with_flagged_entry() {
+ use std::fs;
+ use tempfile::TempDir;
+
+ let checker = PyO3EmbeddedChecker::new();
+
+ // Only run if beancount is available
+ if !checker.is_available() {
+ return;
+ }
+
+ let temp_dir = TempDir::new().expect("Failed to create temp directory");
+ let file_path = temp_dir.path().join("test.beancount");
+ let content =
+ "2023-01-01 ! \"Flagged transaction\"\n Assets:Cash 100 USD\n Expenses:Food";
+ fs::write(&file_path, content).expect("Failed to write temp file");
+
+ let result = checker.check(&file_path);
+
+ match result {
+ Ok(check_result) => {
+ // Should have at least the flagged entry
+ // (May also have validation errors depending on beancount setup)
+ println!(
+ "Flagged entries found: {}",
+ check_result.flagged_entries.len()
+ );
+ println!("Errors found: {}", check_result.errors.len());
+ }
+ Err(e) => {
+ // If beancount is not properly configured, that's OK for this test
+ eprintln!(
+ "Beancount check failed (possibly due to environment): {e}"
+ );
+ }
+ }
+ }
+
+ #[cfg(feature = "python-embedded")]
+ #[test]
+ fn test_pyo3_checker_ignores_cleared_transactions() {
+ use std::fs;
+ use tempfile::TempDir;
+
+ let checker = PyO3EmbeddedChecker::new();
+
+ // Only run if beancount is available
+ if !checker.is_available() {
+ return;
+ }
+
+ let temp_dir = TempDir::new().expect("Failed to create temp directory");
+ let file_path = temp_dir.path().join("test.beancount");
+ // Use "*" flag which should NOT be reported as flagged entry
+ let content =
+ "2023-01-01 * \"Cleared transaction\"\n Assets:Cash 100 USD\n Expenses:Food";
+ fs::write(&file_path, content).expect("Failed to write temp file");
+
+ let result = checker.check(&file_path);
+
+ match result {
+ Ok(check_result) => {
+ // Should NOT have any flagged entries (cleared transactions should be ignored)
+ println!(
+ "Flagged entries found: {}",
+ check_result.flagged_entries.len()
+ );
+ println!("Errors found: {}", check_result.errors.len());
+ // Note: We don't assert specific counts since this depends on beancount environment
+ // but the test documents the expected behavior
+ }
+ Err(e) => {
+ // If beancount is not properly configured, that's OK for this test
+ eprintln!(
+ "Beancount check failed (possibly due to environment): {e}"
+ );
+ }
+ }
+ }
+
+ #[cfg(feature = "python-embedded")]
+ #[test]
+ fn test_pyo3_checker_line_number_fix() {
+ use std::fs;
+ use tempfile::TempDir;
+
+ let checker = PyO3EmbeddedChecker::new();
+
+ // Only run if beancount is available
+ if !checker.is_available() {
+ return;
+ }
+
+ let temp_dir = TempDir::new().expect("Failed to create temp directory");
+ let file_path = temp_dir.path().join("test.beancount");
+
+ // Create content with errors on specific lines
+ let content = r#"2023-01-01 open Assets:Cash USD
+
+invalid syntax on line 3
+
+2023-01-05 * "Valid transaction"
+ Assets:Cash 100.00 USD
+ Expenses:Food -100.00 USD
+
+another error on line 9
+
+2023-01-10 txn "Invalid transaction - missing account"
+ Assets:Cash 50.00 USD"#;
+
+ fs::write(&file_path, content).expect("Failed to write temp file");
+
+ let result = checker.check(&file_path);
+
+ match result {
+ Ok(check_result) => {
+ println!("Found {} errors:", check_result.errors.len());
+ for (i, error) in check_result.errors.iter().enumerate() {
+ println!(
+ " Error {}: {}:{} - {}",
+ i,
+ error.file.display(),
+ error.line,
+ error.message
+ );
+ }
+
+ // Check if we have proper line numbers (not all 0 or 1)
+ let has_proper_line_numbers = check_result.errors.iter().any(|e| e.line > 1);
+ println!("Has proper line numbers: {has_proper_line_numbers}");
+
+ if !has_proper_line_numbers {
+ eprintln!("WARNING: Line number fix may not be working - all errors showing line 0 or 1");
+ } else {
+ println!("SUCCESS: Line number fix is working correctly");
+ }
+ }
+ Err(e) => {
+ eprintln!(
+ "Beancount check failed (possibly due to environment): {e}"
+ );
+ }
+ }
+ }
+}
diff --git a/crates/lsp/src/checkers/system_call.rs b/crates/lsp/src/checkers/system_call.rs
new file mode 100644
index 0000000..1b65175
--- /dev/null
+++ b/crates/lsp/src/checkers/system_call.rs
@@ -0,0 +1,295 @@
+use super::types::*;
+use super::BeancountChecker;
+use anyhow::{Context, Result};
+use std::path::{Path, PathBuf};
+use std::process::Command;
+use std::sync::OnceLock;
+use tracing::debug;
+
+/// Static regex for parsing bean-check error output.
+/// Pattern: "file:line: error_message"
+/// Compiled once at startup for optimal performance.
+static ERROR_LINE_REGEX: OnceLock = OnceLock::new();
+
+fn get_error_line_regex() -> &'static regex::Regex {
+ ERROR_LINE_REGEX.get_or_init(|| {
+ regex::Regex::new(r"^([^:]+):(\d+):\s*(.*)$").expect("Failed to compile error line regex")
+ })
+}
+
+/// Bean-check implementation using system calls to execute the bean-check binary.
+///
+/// This is the traditional approach that executes bean-check as a subprocess
+/// and parses its stderr output for error messages.
+#[derive(Debug, Clone)]
+pub struct SystemCallChecker {
+ /// Path to the bean-check executable
+ bean_check_cmd: PathBuf,
+}
+
+impl SystemCallChecker {
+ /// Create a new system call checker with the specified bean-check command path.
+ pub fn new(bean_check_cmd: PathBuf) -> Self {
+ Self { bean_check_cmd }
+ }
+
+ /// Parse bean-check stderr output into structured errors.
+ fn parse_stderr_output(&self, stderr: &[u8], root_journal_file: &Path) -> Vec {
+ let stderr_str = match std::str::from_utf8(stderr) {
+ Ok(s) => s,
+ Err(e) => {
+ debug!("Failed to parse bean-check stderr as UTF-8: {}", e);
+ return Vec::new();
+ }
+ };
+
+ let mut errors = Vec::new();
+ let regex = get_error_line_regex();
+
+ for line in stderr_str.lines() {
+ debug!("Processing error line: {}", line);
+
+ if let Some(caps) = regex.captures(line) {
+ debug!(
+ "Parsed error: file={}, line={}, message={}",
+ &caps[1], &caps[2], &caps[3]
+ );
+
+ // Parse line number (1-based) and handle special cases
+ let line_number = match caps[2].parse::() {
+ Ok(0) => 0, // Keep as 0 for file-level errors
+ Ok(line) => line, // Keep 1-based for consistency
+ Err(e) => {
+ debug!("Failed to parse line number '{}': {}", &caps[2], e);
+ continue;
+ }
+ };
+
+ // Convert file path string to PathBuf
+ let file_path_str = &caps[1];
+ let parsed_line_number = caps[2].parse::().unwrap_or(1);
+ let file_path = if parsed_line_number == 0 {
+ // File-level error: use root journal file
+ root_journal_file.to_path_buf()
+ } else {
+ // Line-specific error: use the file mentioned in the error
+ match PathBuf::from(file_path_str).canonicalize() {
+ Ok(path) => path,
+ Err(_) => {
+ // Fallback to raw path if canonicalization fails
+ PathBuf::from(file_path_str)
+ }
+ }
+ };
+
+ let error = BeancountError::new(file_path, line_number, caps[3].trim().to_string());
+ errors.push(error);
+ }
+ }
+
+ errors
+ }
+}
+
+impl BeancountChecker for SystemCallChecker {
+ fn check(&self, journal_file: &Path) -> Result {
+ debug!(
+ "SystemCallChecker: executing bean-check on {}",
+ journal_file.display()
+ );
+ debug!(
+ "SystemCallChecker: using command {}",
+ self.bean_check_cmd.display()
+ );
+
+ let output = Command::new(&self.bean_check_cmd)
+ .arg(journal_file)
+ .output()
+ .context(format!(
+ "Failed to execute bean-check command: {}",
+ self.bean_check_cmd.display()
+ ))?;
+
+ debug!(
+ "SystemCallChecker: command executed, status: {}",
+ output.status
+ );
+ debug!("SystemCallChecker: stderr length: {}", output.stderr.len());
+
+ let errors = if !output.status.success() {
+ debug!("SystemCallChecker: parsing error output");
+ self.parse_stderr_output(&output.stderr, journal_file)
+ } else {
+ debug!("SystemCallChecker: no errors found");
+ Vec::new()
+ };
+
+ debug!("SystemCallChecker: found {} errors", errors.len());
+
+ Ok(BeancountCheckResult {
+ errors,
+ flagged_entries: Vec::new(), // System call checker doesn't handle flagged entries
+ })
+ }
+
+ fn name(&self) -> &'static str {
+ "SystemCall"
+ }
+
+ fn is_available(&self) -> bool {
+ // Try to run bean-check with --help to see if it's available
+ Command::new(&self.bean_check_cmd)
+ .arg("--help")
+ .output()
+ .map(|output| output.status.success())
+ .unwrap_or(false)
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use std::fs;
+ use tempfile::TempDir;
+
+ /// Helper to create a temporary beancount file for testing
+ fn create_temp_beancount_file(content: &str) -> (TempDir, PathBuf) {
+ let temp_dir = TempDir::new().expect("Failed to create temp directory");
+ let file_path = temp_dir.path().join("test.beancount");
+ fs::write(&file_path, content).expect("Failed to write temp file");
+ (temp_dir, file_path)
+ }
+
+ /// Helper to create a mock bean-check command that always succeeds
+ fn create_mock_bean_check_success() -> PathBuf {
+ #[cfg(unix)]
+ {
+ PathBuf::from("/bin/true")
+ }
+ #[cfg(windows)]
+ {
+ PathBuf::from("cmd")
+ }
+ }
+
+ /// Helper to create a mock bean-check command that always fails
+ fn create_mock_bean_check_failure() -> PathBuf {
+ #[cfg(unix)]
+ {
+ PathBuf::from("/bin/false")
+ }
+ #[cfg(windows)]
+ {
+ PathBuf::from("cmd")
+ }
+ }
+
+ #[test]
+ fn test_system_call_checker_new() {
+ let cmd = PathBuf::from("bean-check");
+ let checker = SystemCallChecker::new(cmd.clone());
+ assert_eq!(checker.bean_check_cmd, cmd);
+ assert_eq!(checker.name(), "SystemCall");
+ }
+
+ #[test]
+ fn test_system_call_checker_success() {
+ let (_temp_dir, file_path) = create_temp_beancount_file("2023-01-01 open Assets:Cash");
+ let checker = SystemCallChecker::new(create_mock_bean_check_success());
+
+ let result = checker.check(&file_path);
+ // Some systems might not have /bin/true, so just check it doesn't panic
+ match result {
+ Ok(check_result) => {
+ // If successful, should have no errors (since /bin/true outputs nothing)
+ assert_eq!(check_result.errors.len(), 0);
+ assert_eq!(check_result.flagged_entries.len(), 0);
+ }
+ Err(_) => {
+ // If the mock command fails, that's OK for this test environment
+ // The test verifies the function handles commands gracefully
+ }
+ }
+ }
+
+ #[test]
+ fn test_system_call_checker_failure() {
+ let (_temp_dir, file_path) = create_temp_beancount_file("invalid content");
+ let checker = SystemCallChecker::new(create_mock_bean_check_failure());
+
+ let result = checker.check(&file_path);
+ // Some systems might not have /bin/false, so handle both cases
+ match result {
+ Ok(check_result) => {
+ // If command succeeds but returns failure status, should have no parsed errors
+ // (since /bin/false doesn't output structured bean-check errors)
+ assert_eq!(check_result.errors.len(), 0);
+ }
+ Err(_) => {
+ // If the mock command fails to execute, that's OK for this test environment
+ // The test verifies the function handles command failures gracefully
+ }
+ }
+ }
+
+ #[test]
+ fn test_system_call_checker_invalid_command() {
+ let (_temp_dir, file_path) = create_temp_beancount_file("test content");
+ let checker = SystemCallChecker::new(PathBuf::from("/nonexistent/command"));
+
+ let result = checker.check(&file_path);
+ assert!(result.is_err()); // Should fail to execute
+ }
+
+ #[test]
+ fn test_parse_stderr_output() {
+ let checker = SystemCallChecker::new(PathBuf::from("bean-check"));
+ let stderr = b"/path/to/file.beancount:123: Test error message\nanother/file.beancount:456: Another error";
+ let root_file = PathBuf::from("/root/main.beancount");
+
+ let errors = checker.parse_stderr_output(stderr, &root_file);
+ assert_eq!(errors.len(), 2);
+
+ assert_eq!(errors[0].line, 123);
+ assert_eq!(errors[0].message, "Test error message");
+ assert_eq!(errors[1].line, 456);
+ assert_eq!(errors[1].message, "Another error");
+ }
+
+ #[test]
+ fn test_parse_stderr_output_line_zero() {
+ let checker = SystemCallChecker::new(PathBuf::from("bean-check"));
+ let stderr = b":0: Missing Commodity directive for 'USD'";
+ let root_file = PathBuf::from("/root/main.beancount");
+
+ let errors = checker.parse_stderr_output(stderr, &root_file);
+ assert_eq!(errors.len(), 1);
+ assert_eq!(errors[0].line, 0);
+ assert_eq!(errors[0].file, root_file);
+ assert_eq!(errors[0].message, "Missing Commodity directive for 'USD'");
+ }
+
+ #[test]
+ fn test_error_line_regex() {
+ let regex = get_error_line_regex();
+
+ // Valid formats
+ assert!(regex.is_match("/path/to/file.beancount:123: Error message"));
+ assert!(regex.is_match("relative/path.beancount:1: Another error"));
+ assert!(regex.is_match("file.beancount:0: File-level error"));
+
+ // Invalid formats
+ assert!(!regex.is_match("no colon separator"));
+ assert!(!regex.is_match("file.beancount: missing line number"));
+ assert!(!regex.is_match("file.beancount:not_a_number: invalid line"));
+
+ // Test capture groups
+ if let Some(caps) = regex.captures("/path/file.beancount:42: Test error message") {
+ assert_eq!(&caps[1], "/path/file.beancount");
+ assert_eq!(&caps[2], "42");
+ assert_eq!(&caps[3], "Test error message");
+ } else {
+ panic!("Regex should match valid error format");
+ }
+ }
+}
diff --git a/crates/lsp/src/checkers/types.rs b/crates/lsp/src/checkers/types.rs
new file mode 100644
index 0000000..d0d0379
--- /dev/null
+++ b/crates/lsp/src/checkers/types.rs
@@ -0,0 +1,151 @@
+use std::path::PathBuf;
+
+/// Result of bean-check validation containing both errors and flagged entries.
+#[derive(Debug, Clone, PartialEq)]
+pub struct BeancountCheckResult {
+ /// Validation errors from bean-check (syntax errors, semantic errors, etc.)
+ pub errors: Vec,
+ /// Entries marked with flags like '!' for review
+ pub flagged_entries: Vec,
+}
+
+impl BeancountCheckResult {
+ /// Create a new empty result.
+ pub fn new() -> Self {
+ Self {
+ errors: Vec::new(),
+ flagged_entries: Vec::new(),
+ }
+ }
+
+ /// Create a result with only errors.
+ pub fn with_errors(errors: Vec) -> Self {
+ Self {
+ errors,
+ flagged_entries: Vec::new(),
+ }
+ }
+
+ /// Create a result with only flagged entries.
+ pub fn with_flagged_entries(flagged_entries: Vec) -> Self {
+ Self {
+ errors: Vec::new(),
+ flagged_entries,
+ }
+ }
+
+ /// Check if the result contains any errors or flagged entries.
+ pub fn has_issues(&self) -> bool {
+ !self.errors.is_empty() || !self.flagged_entries.is_empty()
+ }
+
+ /// Get total number of issues (errors + flagged entries).
+ pub fn issue_count(&self) -> usize {
+ self.errors.len() + self.flagged_entries.len()
+ }
+}
+
+impl Default for BeancountCheckResult {
+ fn default() -> Self {
+ Self::new()
+ }
+}
+
+/// Represents a validation error from bean-check.
+#[derive(Debug, Clone, PartialEq)]
+pub struct BeancountError {
+ /// File where the error occurred
+ pub file: PathBuf,
+ /// Line number (1-based, 0 for file-level errors)
+ pub line: u32,
+ /// Error message
+ pub message: String,
+}
+
+impl BeancountError {
+ /// Create a new beancount error.
+ pub fn new(file: PathBuf, line: u32, message: String) -> Self {
+ Self {
+ file,
+ line,
+ message,
+ }
+ }
+
+ /// Check if this is a file-level error (line 0).
+ pub fn is_file_level(&self) -> bool {
+ self.line == 0
+ }
+}
+
+/// Represents an entry flagged for review (e.g., with '!' flag).
+#[derive(Debug, Clone, PartialEq)]
+pub struct FlaggedEntry {
+ /// File containing the flagged entry
+ pub file: PathBuf,
+ /// Line number (1-based)
+ pub line: u32,
+ /// Message describing the flag
+ pub message: String,
+}
+
+impl FlaggedEntry {
+ /// Create a new flagged entry.
+ pub fn new(file: PathBuf, line: u32, message: String) -> Self {
+ Self {
+ file,
+ line,
+ message,
+ }
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn test_beancount_check_result_empty() {
+ let result = BeancountCheckResult::new();
+ assert!(!result.has_issues());
+ assert_eq!(result.issue_count(), 0);
+ }
+
+ #[test]
+ fn test_beancount_check_result_with_errors() {
+ let errors = vec![BeancountError::new(
+ PathBuf::from("test.beancount"),
+ 5,
+ "Syntax error".to_string(),
+ )];
+ let result = BeancountCheckResult::with_errors(errors);
+ assert!(result.has_issues());
+ assert_eq!(result.issue_count(), 1);
+ }
+
+ #[test]
+ fn test_beancount_check_result_with_flagged() {
+ let flagged = vec![FlaggedEntry::new(
+ PathBuf::from("test.beancount"),
+ 10,
+ "Flagged for review".to_string(),
+ )];
+ let result = BeancountCheckResult::with_flagged_entries(flagged);
+ assert!(result.has_issues());
+ assert_eq!(result.issue_count(), 1);
+ }
+
+ #[test]
+ fn test_beancount_error_file_level() {
+ let error = BeancountError::new(
+ PathBuf::from("test.beancount"),
+ 0,
+ "File-level error".to_string(),
+ );
+ assert!(error.is_file_level());
+
+ let line_error =
+ BeancountError::new(PathBuf::from("test.beancount"), 5, "Line error".to_string());
+ assert!(!line_error.is_file_level());
+ }
+}
diff --git a/crates/lsp/src/config.rs b/crates/lsp/src/config.rs
index 71af26f..e688b33 100644
--- a/crates/lsp/src/config.rs
+++ b/crates/lsp/src/config.rs
@@ -1,3 +1,4 @@
+use crate::checkers::{BeancountCheckConfig, BeancountCheckMethod};
use anyhow::Result;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
@@ -7,6 +8,7 @@ pub struct Config {
pub root_file: PathBuf,
pub journal_root: Option,
pub formatting: FormattingConfig,
+ pub bean_check: BeancountCheckConfig,
}
#[derive(Debug, Clone)]
@@ -56,6 +58,7 @@ impl Config {
root_file,
journal_root: None,
formatting: FormattingConfig::default(),
+ bean_check: BeancountCheckConfig::default(),
}
}
pub fn update(&mut self, json: serde_json::Value) -> Result<()> {
@@ -91,6 +94,22 @@ impl Config {
self.formatting.indent_width = Some(indent_width);
}
}
+
+ // Update bean-check configuration
+ if let Some(bean_check) = beancount_lsp_settings.bean_check {
+ if let Some(method) = bean_check.method {
+ self.bean_check.method = method;
+ }
+ if let Some(bean_check_cmd) = bean_check.bean_check_cmd {
+ self.bean_check.bean_check_cmd = PathBuf::from(bean_check_cmd);
+ }
+ if let Some(python_cmd) = bean_check.python_cmd {
+ self.bean_check.python_cmd = PathBuf::from(python_cmd);
+ }
+ if let Some(python_script_path) = bean_check.python_script_path {
+ self.bean_check.python_script_path = PathBuf::from(python_script_path);
+ }
+ }
}
Ok(())
@@ -101,6 +120,7 @@ impl Config {
pub struct BeancountLspOptions {
pub journal_file: Option,
pub formatting: Option,
+ pub bean_check: Option,
}
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
@@ -124,6 +144,54 @@ pub struct FormattingOptions {
pub indent_width: Option,
}
+#[derive(Debug, Clone, Default, Deserialize, Serialize)]
+pub struct BeancountCheckOptions {
+ /// Method for bean-check execution: "system" or "python"
+ #[serde(with = "bean_check_method_serde")]
+ pub method: Option,
+ /// Path to bean-check executable (for system method)
+ pub bean_check_cmd: Option,
+ /// Path to Python executable (for python method)
+ pub python_cmd: Option,
+ /// Path to Python script (for python method)
+ pub python_script_path: Option,
+}
+
+// Custom serde module for BeancountCheckMethod
+mod bean_check_method_serde {
+ use super::BeancountCheckMethod;
+ use serde::{Deserialize, Deserializer, Serialize, Serializer};
+
+ pub fn serialize(
+ value: &Option,
+ serializer: S,
+ ) -> Result
+ where
+ S: Serializer,
+ {
+ match value {
+ Some(BeancountCheckMethod::SystemCall) => "system".serialize(serializer),
+ Some(BeancountCheckMethod::PythonEmbedded) => "python-embedded".serialize(serializer),
+ None => serializer.serialize_none(),
+ }
+ }
+
+ pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error>
+ where
+ D: Deserializer<'de>,
+ {
+ let value: Option = Option::deserialize(deserializer)?;
+ match value.as_deref() {
+ Some("system") => Ok(Some(BeancountCheckMethod::SystemCall)),
+ Some("python-embedded") | Some("pyo3") => {
+ Ok(Some(BeancountCheckMethod::PythonEmbedded))
+ }
+ Some(_) => Ok(None), // Invalid method, ignore gracefully
+ None => Ok(None),
+ }
+ }
+}
+
#[cfg(test)]
mod tests {
use super::*;
diff --git a/crates/lsp/src/lib.rs b/crates/lsp/src/lib.rs
index 9c6b929..8069bbf 100644
--- a/crates/lsp/src/lib.rs
+++ b/crates/lsp/src/lib.rs
@@ -1,5 +1,6 @@
mod beancount_data;
mod capabilities;
+pub mod checkers;
mod config;
mod dispatcher;
pub mod document;
diff --git a/crates/lsp/src/main.rs b/crates/lsp/src/main.rs
index f8a5185..e47a8d8 100644
--- a/crates/lsp/src/main.rs
+++ b/crates/lsp/src/main.rs
@@ -10,7 +10,7 @@ fn main() {
let matches = Command::new("beancount-language-server")
.args(&[
arg!(--stdio "specifies to use stdio to communicate with lsp"),
- arg!(--log "write log to file"),
+ arg!(--log [LOG_LEVEL] "write log to file with optional level (trace, debug, info, warn, error)"),
arg!(version: -v --version),
])
.get_matches();
@@ -20,17 +20,19 @@ fn main() {
return;
}
- let log_to_file = matches.get_flag("log");
- setup_logging(log_to_file);
+ let log_to_file = matches.contains_id("log");
+ let log_level = matches.get_one::("log");
+ setup_logging(log_to_file, log_level);
tracing::info!(
"Starting beancount-language-server v{}",
env!("CARGO_PKG_VERSION")
);
tracing::debug!(
- "Command line args: stdio={}, log_to_file={}",
+ "Command line args: stdio={}, log_to_file={}, log_level={:?}",
matches.get_flag("stdio"),
- log_to_file
+ log_to_file,
+ log_level
);
match beancount_language_server::run_server() {
@@ -44,8 +46,19 @@ fn main() {
}
}
-fn setup_logging(file: bool) {
- let file = if file {
+fn setup_logging(log_to_file: bool, log_level_arg: Option<&String>) {
+ let level = match log_level_arg {
+ Some(level_str) => parse_log_level(level_str),
+ None => {
+ if log_to_file {
+ LevelFilter::DEBUG // Default level when logging to file
+ } else {
+ LevelFilter::INFO // Default level when logging to stderr
+ }
+ }
+ };
+
+ let file = if log_to_file {
match fs::OpenOptions::new()
.create(true)
.append(true)
@@ -69,7 +82,7 @@ fn setup_logging(file: bool) {
None => BoxMakeWriter::new(io::stderr),
};
- let filter = EnvFilter::default().add_directive(Directive::from(LevelFilter::INFO));
+ let filter = EnvFilter::default().add_directive(Directive::from(level));
tracing_subscriber::fmt()
.with_env_filter(filter)
.with_writer(writer)
@@ -78,3 +91,18 @@ fn setup_logging(file: bool) {
.with_level(true)
.init();
}
+
+fn parse_log_level(level_str: &str) -> LevelFilter {
+ match level_str.to_lowercase().as_str() {
+ "trace" => LevelFilter::TRACE,
+ "debug" => LevelFilter::DEBUG,
+ "info" => LevelFilter::INFO,
+ "warn" => LevelFilter::WARN,
+ "error" => LevelFilter::ERROR,
+ "off" => LevelFilter::OFF,
+ _ => {
+ eprintln!("Invalid log level '{level_str}'. Using 'info' as default. Valid levels: trace, debug, info, warn, error, off");
+ LevelFilter::INFO
+ }
+ }
+}
diff --git a/crates/lsp/src/providers/completion.rs b/crates/lsp/src/providers/completion.rs
index 1cf7995..a75f616 100644
--- a/crates/lsp/src/providers/completion.rs
+++ b/crates/lsp/src/providers/completion.rs
@@ -259,7 +259,7 @@ fn determine_completion_context(
// Don't use the file node - it's too generic
if node.kind() == "file" {
// If we only found the file node, manually search for a more specific context
- find_context_by_manual_search(tree, cursor)
+ find_context_by_manual_search(tree, cursor, content)
} else {
analyze_node_context(tree, content, node, cursor)
}
@@ -279,6 +279,7 @@ fn determine_completion_context(
fn find_context_by_manual_search(
tree: &tree_sitter::Tree,
cursor: tree_sitter::Point,
+ content: &ropey::Rope,
) -> CompletionContext {
debug!("Manual search for context at {:?}", cursor);
@@ -298,7 +299,7 @@ fn find_context_by_manual_search(
// Check if cursor is within this transaction
if cursor.row >= start.row && cursor.row <= end.row {
debug!("Found transaction containing cursor!");
- return analyze_transaction_context(child, cursor);
+ return analyze_transaction_context(child, cursor, content);
}
}
}
@@ -330,7 +331,7 @@ fn analyze_node_context(
// We're in a transaction
"transaction" => {
debug!("Found transaction context");
- return analyze_transaction_context(n, cursor);
+ return analyze_transaction_context(n, cursor, _content);
}
// We're in a posting within a transaction
"posting" => {
@@ -371,6 +372,7 @@ fn analyze_node_context(
fn analyze_transaction_context(
node: tree_sitter::Node,
cursor: tree_sitter::Point,
+ content: &ropey::Rope,
) -> CompletionContext {
let mut walker = node.walk();
let children: Vec<_> = node.children(&mut walker).collect();
@@ -409,7 +411,44 @@ fn analyze_transaction_context(
}
// We're somewhere in a transaction but not in a specific field
- // Likely in the posting area - prioritize account completion
+ // Try to determine if we're in payee/narration position by analyzing the line content
+ let line_text = content.line(cursor.row).to_string();
+
+ // Check if this line contains "txn" and we're after it
+ if line_text.contains("txn") {
+ if let Some(txn_pos) = line_text.find("txn") {
+ if cursor.column > txn_pos + 3 { // After "txn "
+ // Count quotes to determine if we're in payee or narration position
+ let before_cursor = &line_text[..std::cmp::min(cursor.column, line_text.len())];
+ let completed_quotes = before_cursor.matches("\"").count();
+
+ // Check if we already have a complete payee (closed quotes)
+ let has_complete_payee = before_cursor.contains("\"") &&
+ before_cursor.split("\"").filter(|s| !s.trim().is_empty()).count() >= 1 &&
+ completed_quotes >= 2;
+
+ if has_complete_payee {
+ // We have a completed payee field, so this should be narration
+ return CompletionContext {
+ structure_type: StructureType::Transaction,
+ expected_next: vec![ExpectedType::Narration],
+ prefix: String::new(),
+ parent_context: Some("transaction".to_string()),
+ };
+ } else {
+ // Default to narration for backward compatibility
+ return CompletionContext {
+ structure_type: StructureType::Transaction,
+ expected_next: vec![ExpectedType::Narration],
+ prefix: String::new(),
+ parent_context: Some("transaction".to_string()),
+ };
+ }
+ }
+ }
+ }
+
+ // Default to account completion if we can't determine payee/narration context
CompletionContext {
structure_type: StructureType::Transaction,
expected_next: vec![ExpectedType::Account], // Focus on accounts in posting area
@@ -528,11 +567,33 @@ fn complete_based_on_context(
'#' => return complete_tag(beancount_data),
'^' => return complete_link(beancount_data),
'"' => {
+ // When quote is triggered, try to be smart about payee vs narration
let line_text = content.line(cursor_point.row).to_string();
- return complete_narration_with_quotes(
+ // Smart detection: if this looks like it could be payee position, prioritize payee completion
+ if line_text.contains("txn") {
+ if let Some(txn_pos) = line_text.find("txn") {
+ let after_txn = &line_text[txn_pos + 3..];
+ let quote_count = after_txn.matches('"').count();
+
+ // If we're right after txn with no quotes, likely payee position
+ if quote_count == 0 && cursor_point.column <= txn_pos + 4 + after_txn.trim_start().len() {
+ return complete_payee_with_full_context(
+ beancount_data,
+ &context.prefix,
+ trigger_character,
+ Some(&line_text),
+ cursor_point.column
+ );
+ }
+ }
+ }
+
+ // Default to narration completion for other cases
+ return complete_narration_with_quotes_context(
beancount_data,
&line_text,
cursor_point.column,
+ true, // triggered_by_quote = true
);
}
_ => {} // Continue with context-based completion
@@ -554,7 +615,7 @@ fn complete_based_on_context(
let line_text = content.line(cursor_point.row).to_string();
complete_narration_with_quotes(beancount_data, &line_text, cursor_point.column)
}
- ExpectedType::Payee => complete_payee(beancount_data, &context.prefix),
+ ExpectedType::Payee => complete_payee_with_context(beancount_data, &context.prefix, trigger_character),
ExpectedType::Tag => complete_tag(beancount_data),
ExpectedType::Link => complete_link(beancount_data),
ExpectedType::TransactionKind => complete_kind(),
@@ -585,7 +646,7 @@ fn complete_based_on_context(
.unwrap_or_default()
}
ExpectedType::Payee => {
- complete_payee(beancount_data.clone(), &context.prefix)?.unwrap_or_default()
+ complete_payee_with_context(beancount_data.clone(), &context.prefix, trigger_character)?.unwrap_or_default()
}
ExpectedType::Tag => complete_tag(beancount_data.clone())?.unwrap_or_default(),
ExpectedType::Link => complete_link(beancount_data.clone())?.unwrap_or_default(),
@@ -681,10 +742,22 @@ fn complete_flag() -> Result>> {
Ok(Some(items))
}
-/// Complete payee names from previous transactions
-fn complete_payee(
+/// Complete payee names from previous transactions with context awareness
+fn complete_payee_with_context(
beancount_data: HashMap>,
prefix: &str,
+ trigger_character: Option,
+) -> Result>> {
+ complete_payee_with_full_context(beancount_data, prefix, trigger_character, None, 0)
+}
+
+/// Complete payee names with full context including line text for quote detection
+fn complete_payee_with_full_context(
+ beancount_data: HashMap>,
+ prefix: &str,
+ trigger_character: Option,
+ line_text: Option<&str>,
+ cursor_char: usize,
) -> Result>> {
let mut payees = std::collections::HashSet::new();
@@ -700,21 +773,49 @@ fn complete_payee(
}
}
+ // Determine if we should add quotes based on trigger
+ let triggered_by_quote = trigger_character == Some('"');
+
+ // Check if there's already a closing quote after the cursor (for triggered by quote case)
+ let has_closing_quote = if triggered_by_quote && line_text.is_some() {
+ line_text.unwrap().chars().skip(cursor_char).any(|c| c == '"')
+ } else {
+ false
+ };
+
let items: Vec = payees
.into_iter()
.filter(|payee| prefix.is_empty() || payee.to_lowercase().contains(&prefix.to_lowercase()))
- .map(|payee| lsp_types::CompletionItem {
- label: payee.clone(),
- detail: Some("Payee".to_string()),
- kind: Some(lsp_types::CompletionItemKind::TEXT),
- insert_text: Some(format!("\"{payee}\"")),
- ..Default::default()
+ .map(|payee| {
+ let (label, insert_text) = if triggered_by_quote {
+ // When triggered by quote, user already typed opening quote
+ let insert_text = if has_closing_quote {
+ // Closing quote exists, just insert content
+ payee.clone()
+ } else {
+ // No closing quote, add it
+ format!("{}\"", payee)
+ };
+ (payee.clone(), insert_text)
+ } else {
+ // Normal completion includes quotes in both label and insert text
+ (format!("\"{}\"", payee), format!("\"{}\"", payee))
+ };
+
+ lsp_types::CompletionItem {
+ label,
+ detail: Some("Payee".to_string()),
+ kind: Some(lsp_types::CompletionItemKind::TEXT),
+ insert_text: Some(insert_text),
+ ..Default::default()
+ }
})
.collect();
Ok(if items.is_empty() { None } else { Some(items) })
}
+
pub(crate) fn complete_date() -> anyhow::Result>> {
debug!("providers::completion::date");
let today = chrono::offset::Local::now().naive_local().date();
@@ -810,6 +911,15 @@ fn complete_narration_with_quotes(
data: HashMap>,
line_text: &str,
cursor_char: usize,
+) -> anyhow::Result>> {
+ complete_narration_with_quotes_context(data, line_text, cursor_char, false)
+}
+
+fn complete_narration_with_quotes_context(
+ data: HashMap>,
+ line_text: &str,
+ cursor_char: usize,
+ triggered_by_quote: bool,
) -> anyhow::Result>> {
debug!("providers::completion::narration");
@@ -823,16 +933,28 @@ fn complete_narration_with_quotes(
let mut completions = Vec::new();
for data in data.values() {
for txn_string in data.get_narration() {
- let insert_text = if has_closing_quote {
+ let (label, insert_text) = if triggered_by_quote {
+ // When triggered by quote, user already typed opening quote
+ let clean_text = txn_string.trim_matches('"');
+ let insert_text = if has_closing_quote {
+ // Closing quote exists, just insert content
+ clean_text.to_string()
+ } else {
+ // No closing quote, add it
+ format!("{}\"", clean_text)
+ };
+ (clean_text.to_string(), insert_text)
+ } else if has_closing_quote {
// Remove the quotes from the stored string and don't add closing quote
- txn_string.trim_matches('"').to_string()
+ let clean_text = txn_string.trim_matches('"');
+ (txn_string.clone(), clean_text.to_string())
} else {
// Keep the full quoted string as stored
- txn_string.clone()
+ (txn_string.clone(), txn_string.clone())
};
completions.push(lsp_types::CompletionItem {
- label: txn_string.clone(),
+ label,
detail: Some("Beancount Narration".to_string()),
kind: Some(lsp_types::CompletionItemKind::ENUM),
insert_text: Some(insert_text),
@@ -848,7 +970,61 @@ fn complete_account_with_prefix(
prefix: &str,
) -> anyhow::Result>> {
debug!("providers::completion::account with prefix: '{}'", prefix);
- complete_account_internal(data, prefix, true) // true = triggered by colon, use filtering
+ complete_account_internal_colon_triggered(data, prefix)
+}
+
+fn complete_account_internal_colon_triggered(
+ data: HashMap>,
+ prefix: &str,
+) -> anyhow::Result>> {
+ debug!(
+ "providers::completion::account colon-triggered with prefix: '{}'",
+ prefix
+ );
+ let mut completions = Vec::new();
+
+ for data in data.values() {
+ let accounts: Vec = data.get_accounts().into_iter().collect();
+
+ // Find accounts that start with the prefix
+ let matching_accounts: Vec = accounts
+ .into_iter()
+ .filter(|account| account.starts_with(prefix))
+ .collect();
+
+ // Extract the parts after the prefix
+ for account in matching_accounts {
+ if let Some(suffix) = account.strip_prefix(prefix) {
+ // Remove leading colon if present
+ let suffix = suffix.strip_prefix(':').unwrap_or(suffix);
+
+ // Only show the next segment (up to the next colon, if any)
+ let next_segment = if let Some(colon_pos) = suffix.find(':') {
+ &suffix[..colon_pos]
+ } else {
+ suffix
+ };
+
+ // Skip empty segments and avoid duplicates
+ if !next_segment.is_empty() {
+ let completion_text = next_segment.to_string();
+
+ // Check if we already have this completion
+ if !completions
+ .iter()
+ .any(|item: &lsp_types::CompletionItem| item.label == completion_text)
+ {
+ completions.push(create_completion_item(completion_text, 1.0));
+ }
+ }
+ }
+ }
+ }
+
+ // Sort completions alphabetically
+ completions.sort_by(|a, b| a.label.cmp(&b.label));
+
+ Ok(Some(completions))
}
fn complete_account_internal(
@@ -1540,10 +1716,10 @@ mod tests {
assert_eq!(
items,
[lsp_types::CompletionItem {
- label: String::from("\"Test Co\""),
+ label: String::from("Test Co"), // No quotes in label when triggered by quote
kind: Some(lsp_types::CompletionItemKind::ENUM),
detail: Some(String::from("Beancount Narration")),
- insert_text: Some(String::from("\"Test Co\"")), // No closing quote exists, so keep full quoted string
+ insert_text: Some(String::from("Test Co\"")), // Add closing quote when triggered by quote and no closing quote exists
..Default::default()
},]
)
@@ -1570,9 +1746,10 @@ mod tests {
.unwrap_or_default();
// New intelligent system provides narration completions after payee
assert!(!items.is_empty());
- assert!(items.iter().any(|item| item.label == "\"Foo Bar\""));
+ assert!(items.iter().any(|item| item.label == "Foo Bar")); // No quotes when triggered by quote
}
+
#[test]
fn handle_narration_completion_with_existing_closing_quote() {
let fixure = r#"
@@ -1597,7 +1774,7 @@ mod tests {
assert!(!items.is_empty());
let test_co_completion = items
.iter()
- .find(|item| item.label == "\"Test Co\"")
+ .find(|item| item.label == "Test Co") // No quotes in label when triggered by quote
.unwrap();
assert_eq!(
test_co_completion.insert_text,
@@ -1606,7 +1783,7 @@ mod tests {
let foo_bar_completion = items
.iter()
- .find(|item| item.label == "\"Foo Bar\"")
+ .find(|item| item.label == "Foo Bar") // No quotes in label when triggered by quote
.unwrap();
assert_eq!(
foo_bar_completion.insert_text,
@@ -1636,10 +1813,10 @@ mod tests {
assert_eq!(
items,
[lsp_types::CompletionItem {
- label: String::from("\"Foo Bar\""),
+ label: String::from("Foo Bar"), // No quotes in label when triggered by quote
kind: Some(lsp_types::CompletionItemKind::ENUM),
detail: Some(String::from("Beancount Narration")),
- insert_text: Some(String::from("\"Foo Bar\"")), // Keep full quotes since no closing quote
+ insert_text: Some(String::from("Foo Bar\"")), // Add closing quote when triggered by quote and no closing quote exists
..Default::default()
},]
)
@@ -1695,16 +1872,16 @@ mod tests {
.unwrap_or_default();
assert_eq!(items.len(), 2);
- // Should have both Assets accounts
+ // Should have both Assets accounts parts after the colon
let labels: Vec<&String> = items.iter().map(|item| &item.label).collect();
- assert!(labels.contains(&&"Assets:Test".to_string()));
- assert!(labels.contains(&&"Assets:Checking".to_string()));
+ assert!(labels.contains(&&"Test".to_string()));
+ assert!(labels.contains(&&"Checking".to_string()));
// Check properties of all items
for item in &items {
assert_eq!(item.kind, Some(lsp_types::CompletionItemKind::ENUM));
assert_eq!(item.detail, Some("Beancount Account".to_string()));
- assert!(item.label.starts_with("Assets:"));
+ // Note: labels now contain only the part after the colon, not the full account name
}
}
@@ -2073,32 +2250,28 @@ include "accounts1.bean"
// Should include accounts from both files
let labels: Vec<&String> = items.iter().map(|item| &item.label).collect();
- // Accounts from main file
+ // Account parts after "Expenses:" from main file
assert!(
- labels.contains(&&"Expenses:Food".to_string()),
- "Should include Expenses:Food from main file"
+ labels.contains(&&"Food".to_string()),
+ "Should include Food from main file"
);
assert!(
- labels.contains(&&"Expenses:Transport".to_string()),
- "Should include Expenses:Transport from main file"
+ labels.contains(&&"Transport".to_string()),
+ "Should include Transport from main file"
);
- // Accounts from included file - this is what was missing!
+ // Account parts after "Expenses:" from included file - this is what was missing!
assert!(
- labels.contains(&&"Expenses:Included1".to_string()),
- "Should include Expenses:Included1 from included file"
+ labels.contains(&&"Included1".to_string()),
+ "Should include Included1 from included file"
);
assert!(
- labels.contains(&&"Expenses:Included2".to_string()),
- "Should include Expenses:Included2 from included file"
+ labels.contains(&&"Included2".to_string()),
+ "Should include Included2 from included file"
);
- // Should have 4 Expenses accounts total
- let expenses_count = labels
- .iter()
- .filter(|label| label.starts_with("Expenses:"))
- .count();
- assert_eq!(expenses_count, 4, "Should have 4 Expenses accounts total");
+ // Should have 4 account parts total (after "Expenses:")
+ assert_eq!(labels.len(), 4, "Should have 4 account parts total");
}
#[test]
@@ -2141,35 +2314,96 @@ include "accounts1.bean"
let labels: Vec = items.iter().map(|item| item.label.clone()).collect();
- // Should include accounts from both files
+ // Should include account parts after "Expenses:" from both files
assert!(
- labels.contains(&"Expenses:Food".to_string()),
- "Should include Expenses:Food from main file"
+ labels.contains(&"Food".to_string()),
+ "Should include Food from main file"
);
assert!(
- labels.contains(&"Expenses:Transport".to_string()),
- "Should include Expenses:Transport from main file"
+ labels.contains(&"Transport".to_string()),
+ "Should include Transport from main file"
);
assert!(
- labels.contains(&"Expenses:Included1".to_string()),
- "Should include Expenses:Included1 from included file"
+ labels.contains(&"Included1".to_string()),
+ "Should include Included1 from included file"
);
assert!(
- labels.contains(&"Expenses:Included2".to_string()),
- "Should include Expenses:Included2 from included file"
+ labels.contains(&"Included2".to_string()),
+ "Should include Included2 from included file"
);
- // Should have 4 Expenses accounts total
- let expenses_count = labels
- .iter()
- .filter(|label| label.starts_with("Expenses:"))
- .count();
+ // Should have 4 account parts total (after "Expenses:")
assert_eq!(
- expenses_count, 4,
- "Should have 4 Expenses accounts total: {labels:?}"
+ labels.len(),
+ 4,
+ "Should have 4 account parts total: {labels:?}"
);
}
+ #[test]
+ fn test_colon_triggered_completion_behavior() {
+ // Test that colon-triggered completion returns only parts after the colon
+ let fixture = r#"
+%! /main.beancount
+2023-10-01 open Assets:Checking:Personal USD
+2023-10-01 open Assets:Checking:Business USD
+2023-10-01 open Assets:Savings:Emergency USD
+2023-10-01 open Expenses:Food:Groceries USD
+2023-10-01 open Expenses:Food:Restaurants USD
+2023-10-01 txn "Test transaction"
+ Assets:Checking:
+ |
+ ^
+"#;
+ let test_state = TestState::new(fixture).unwrap();
+ let cursor = test_state.cursor().unwrap();
+ let items = completion(test_state.snapshot, Some(':'), cursor)
+ .unwrap()
+ .unwrap_or_default();
+
+ let labels: Vec<&String> = items.iter().map(|item| &item.label).collect();
+
+ // Should return only the parts after "Assets:Checking:"
+ assert_eq!(items.len(), 2);
+ assert!(labels.contains(&&"Personal".to_string()));
+ assert!(labels.contains(&&"Business".to_string()));
+
+ // Should NOT contain full account paths
+ assert!(!labels.contains(&&"Assets:Checking:Personal".to_string()));
+ assert!(!labels.contains(&&"Assets:Checking:Business".to_string()));
+ }
+
+ #[test]
+ fn test_top_level_colon_completion() {
+ // Test completion at top level (e.g., typing "Assets:")
+ let fixture = r#"
+%! /main.beancount
+2023-10-01 open Assets:Checking USD
+2023-10-01 open Assets:Savings USD
+2023-10-01 open Expenses:Food USD
+2023-10-01 txn "Test transaction"
+ Assets:
+ |
+ ^
+"#;
+ let test_state = TestState::new(fixture).unwrap();
+ let cursor = test_state.cursor().unwrap();
+ let items = completion(test_state.snapshot, Some(':'), cursor)
+ .unwrap()
+ .unwrap_or_default();
+
+ let labels: Vec<&String> = items.iter().map(|item| &item.label).collect();
+
+ // Should return only the parts after "Assets:"
+ assert_eq!(items.len(), 2);
+ assert!(labels.contains(&&"Checking".to_string()));
+ assert!(labels.contains(&&"Savings".to_string()));
+
+ // Should NOT contain full account paths or accounts from other hierarchies
+ assert!(!labels.contains(&&"Assets:Checking".to_string()));
+ assert!(!labels.contains(&&"Food".to_string()));
+ }
+
#[test]
fn test_nucleo_fuzzy_matching() {
use crate::providers::completion::fuzzy_search_accounts;
diff --git a/crates/lsp/src/providers/diagnostics.rs b/crates/lsp/src/providers/diagnostics.rs
index 5730502..0e7f770 100644
--- a/crates/lsp/src/providers/diagnostics.rs
+++ b/crates/lsp/src/providers/diagnostics.rs
@@ -1,9 +1,8 @@
use crate::beancount_data::BeancountData;
+use crate::checkers::{BeancountChecker, BeancountError, FlaggedEntry};
use std::collections::HashMap;
-use std::path::Path;
-use std::path::PathBuf;
-use std::process::Command;
-use std::sync::{Arc, OnceLock};
+use std::path::{Path, PathBuf};
+use std::sync::Arc;
#[cfg(test)]
use tempfile;
use tracing::debug;
@@ -27,191 +26,172 @@ impl Default for DiagnosticData {
}
}
-/// Static regex for parsing bean-check error output.
-/// Pattern: "file:line: error_message"
-/// Compiled once at startup for optimal performance.
-static ERROR_LINE_REGEX: OnceLock = OnceLock::new();
-
-fn get_error_line_regex() -> &'static regex::Regex {
- ERROR_LINE_REGEX.get_or_init(|| {
- regex::Regex::new(r"^([^:]+):(\d+):\s*(.*)$").expect("Failed to compile error line regex")
- })
-}
-
/// Provider function for LSP `textDocument/publishDiagnostics`.
///
/// This function collects diagnostics from two sources:
-/// 1. External bean-check command output (syntax/semantic errors)
+/// 1. Bean-check validation (via configurable checker implementation)
/// 2. Internal flagged entries from parsed beancount data (warnings)
///
/// # Arguments
/// * `beancount_data` - Parsed beancount data containing flagged entries
-/// * `bean_check_cmd` - Path to the bean-check executable
+/// * `checker` - Bean-check implementation (system call or Python)
/// * `root_journal_file` - Main beancount file to validate
///
/// # Returns
/// HashMap mapping file paths to their diagnostic messages
///
/// # Performance Notes
-/// - Runs external bean-check synchronously (consider async in future)
-/// - Parses stderr output line by line for memory efficiency
-/// - Uses static regex compilation for optimal parsing performance
+/// - Checker execution depends on implementation (system call vs Python)
+/// - Combines results from checker with internal flagged entry analysis
+/// - Uses structured error types for better error handling
pub fn diagnostics(
beancount_data: HashMap>,
- bean_check_cmd: &Path,
+ checker: &dyn BeancountChecker,
root_journal_file: &Path,
) -> HashMap> {
tracing::info!("Starting diagnostics for: {}", root_journal_file.display());
- tracing::debug!("Using bean-check command: {}", bean_check_cmd.display());
+ tracing::debug!("Using checker: {}", checker.name());
tracing::debug!(
"Processing beancount data for {} files",
beancount_data.len()
);
- // Execute bean-check command and capture output
- // TODO: Consider adding timeout to prevent hanging on large files
- let output = match Command::new(bean_check_cmd).arg(root_journal_file).output() {
- Ok(output) => {
- tracing::debug!("bean-check command executed successfully");
- tracing::debug!("bean-check exit status: {}", output.status);
- if !output.stderr.is_empty() {
- tracing::debug!(
- "bean-check stderr: {}",
- String::from_utf8_lossy(&output.stderr)
- );
- }
- output
+ // Execute bean-check validation using the configured checker
+ tracing::debug!(
+ "Calling checker.check() with file: {}",
+ root_journal_file.display()
+ );
+ let check_result = match checker.check(root_journal_file) {
+ Ok(result) => {
+ tracing::debug!(
+ "Bean-check {} completed: {} errors, {} flagged entries",
+ checker.name(),
+ result.errors.len(),
+ result.flagged_entries.len()
+ );
+ result
}
Err(e) => {
- tracing::error!("Failed to execute bean-check command: {}", e);
- tracing::warn!("Continuing with flagged entries only");
+ tracing::error!("Bean-check {} execution failed: {}", checker.name(), e);
+ tracing::warn!("Continuing with flagged entries from parsed data only");
+
// Continue processing in tests to allow testing of flagged entries
- // Don't return early in tests - continue to process flagged entries
#[cfg(not(test))]
- return HashMap::new();
+ {
+ let mut diagnostics_map = HashMap::new();
+ merge_flagged_entries_from_parsed_data(&mut diagnostics_map, beancount_data);
+ return diagnostics_map;
+ }
#[cfg(test)]
{
- // In tests, create a fake successful output so we can test flagged entries
- // We'll use a simple approach - just continue processing with empty bean-check data
- std::process::Output {
- status: std::process::Command::new("true")
- .status()
- .unwrap_or_else(|_| {
- // Fallback if 'true' command fails
- std::process::Command::new("echo").status().unwrap()
- }),
- stdout: Vec::new(),
- stderr: Vec::new(),
- }
+ // In tests, create an empty result so we can test flagged entries
+ Default::default()
}
}
};
- debug!(
- "bean-check output status: {}, stderr lines: {}",
- output.status,
- std::str::from_utf8(&output.stderr)
- .map(|s| s.lines().count())
- .unwrap_or(0)
- );
- // Parse bean-check output for error diagnostics
- let bean_check_diags = if !output.status.success() {
- debug!("Parsing bean-check error output");
+ // Convert checker errors to LSP diagnostics
+ let mut diagnostics_map = convert_errors_to_diagnostics(check_result.errors);
- // Parse stderr output as UTF-8
- let stderr_str = match std::str::from_utf8(&output.stderr) {
- Ok(s) => s,
- Err(e) => {
- debug!("Failed to parse bean-check stderr as UTF-8: {}", e);
- return HashMap::new();
- }
+ // Add flagged entries from checker (if supported by implementation)
+ merge_flagged_entries_from_checker(&mut diagnostics_map, check_result.flagged_entries);
+
+ // Add diagnostics for flagged entries from parsed beancount data
+ // (These are additional to any flagged entries returned by the checker)
+ merge_flagged_entries_from_parsed_data(&mut diagnostics_map, beancount_data);
+
+ debug!("Generated diagnostics for {} files", diagnostics_map.len());
+ diagnostics_map
+}
+
+/// Convert checker errors to LSP diagnostic format.
+fn convert_errors_to_diagnostics(
+ errors: Vec,
+) -> HashMap> {
+ let mut diagnostics_map: HashMap> = HashMap::new();
+
+ for error in errors {
+ // Convert 1-based line numbers to 0-based for LSP (except for line 0 which stays 0)
+ let line_number = if error.line == 0 {
+ 0
+ } else {
+ error.line.saturating_sub(1)
+ };
+
+ let position = lsp_types::Position {
+ line: line_number,
+ character: 0, // Start of line (bean-check doesn't provide column info)
+ };
+
+ let diagnostic = lsp_types::Diagnostic {
+ range: lsp_types::Range {
+ start: position,
+ end: position, // Point diagnostic
+ },
+ message: error.message,
+ severity: Some(lsp_types::DiagnosticSeverity::ERROR),
+ source: Some("bean-check".to_string()),
+ code: None,
+ code_description: None,
+ tags: None,
+ related_information: None,
+ data: None,
};
- let mut diagnostics_map: HashMap> = HashMap::new();
-
- // Process each line of stderr output
- for line in stderr_str.lines() {
- debug!("Processing error line: {}", line);
-
- // Try to parse the line as a structured error message
- if let Some(caps) = get_error_line_regex().captures(line) {
- debug!(
- "Parsed error: file={}, line={}, message={}",
- &caps[1], &caps[2], &caps[3]
- );
-
- // Parse line number (1-based) and convert to 0-based for LSP
- // Special case: line 0 from bean-check indicates a file-level error
- let line_number = match caps[2].parse::() {
- Ok(0) => 0, // Keep as 0 for file-level errors
- Ok(line) => line.saturating_sub(1), // Convert to 0-based
- Err(e) => {
- debug!("Failed to parse line number '{}': {}", &caps[2], e);
- continue;
- }
- };
-
- let position = lsp_types::Position {
- line: line_number,
- character: 0, // Start of line (bean-check doesn't provide column info)
- };
-
- // Convert file path string to PathBuf
- // Bean-check outputs paths in a consistent format that we can parse directly
- // For line 0 errors (file-level), use the root journal file
- let file_path_str = &caps[1];
- let parsed_line_number = caps[2].parse::().unwrap_or(1);
- let file_path = if parsed_line_number == 0 {
- // File-level error: use root journal file
- root_journal_file.to_path_buf()
- } else {
- // Line-specific error: use the file mentioned in the error
- match PathBuf::from(file_path_str).canonicalize() {
- Ok(path) => path,
- Err(_) => {
- // Fallback to raw path if canonicalization fails
- PathBuf::from(file_path_str)
- }
- }
- };
-
- // Create diagnostic with error severity
- let diagnostic = lsp_types::Diagnostic {
- range: lsp_types::Range {
- start: position,
- end: position, // Point diagnostic (no column info from bean-check)
- },
- message: caps[3].trim().to_string(),
- severity: Some(lsp_types::DiagnosticSeverity::ERROR),
- source: Some("bean-check".to_string()),
- code: None, // Bean-check doesn't provide error codes
- code_description: None,
- tags: None,
- related_information: None,
- data: None,
- };
-
- // Add diagnostic to the appropriate file's diagnostic list
- diagnostics_map
- .entry(file_path)
- .or_default()
- .push(diagnostic);
- }
- }
diagnostics_map
- } else {
- debug!("bean-check completed successfully with no errors");
- HashMap::new()
- };
+ .entry(error.file)
+ .or_default()
+ .push(diagnostic);
+ }
- // Combine bean-check diagnostics with flagged entry diagnostics
- let mut combined_diagnostics: HashMap> = bean_check_diags;
+ diagnostics_map
+}
- // Merge bean-check errors into the result map
- // (This step is now redundant since we're using bean_check_diags directly,
- // but kept for clarity and future extensibility)
- // Add diagnostics for flagged entries (marked with ! or * flags)
+/// Merge flagged entries from checker into diagnostics map.
+fn merge_flagged_entries_from_checker(
+ diagnostics_map: &mut HashMap>,
+ flagged_entries: Vec,
+) {
+ for entry in flagged_entries {
+ // Convert 1-based line numbers to 0-based for LSP
+ let line_number = if entry.line == 0 {
+ 0
+ } else {
+ entry.line.saturating_sub(1)
+ };
+
+ let position = lsp_types::Position {
+ line: line_number,
+ character: 0,
+ };
+
+ let diagnostic = lsp_types::Diagnostic {
+ range: lsp_types::Range {
+ start: position,
+ end: position,
+ },
+ message: entry.message,
+ severity: Some(lsp_types::DiagnosticSeverity::WARNING),
+ source: Some("bean-check".to_string()),
+ code: Some(lsp_types::NumberOrString::String(
+ "flagged-entry".to_string(),
+ )),
+ ..lsp_types::Diagnostic::default()
+ };
+
+ diagnostics_map
+ .entry(entry.file)
+ .or_default()
+ .push(diagnostic);
+ }
+}
+
+/// Merge flagged entries from parsed beancount data into diagnostics map.
+fn merge_flagged_entries_from_parsed_data(
+ diagnostics_map: &mut HashMap>,
+ beancount_data: HashMap>,
+) {
for (file_path, data) in beancount_data.iter() {
for flagged_entry in &data.flagged_entries {
let position = lsp_types::Position {
@@ -233,19 +213,12 @@ pub fn diagnostics(
..lsp_types::Diagnostic::default()
};
- // Add flagged entry diagnostic to the file's diagnostic list
- combined_diagnostics
+ diagnostics_map
.entry(file_path.clone())
.or_default()
.push(diagnostic);
}
}
-
- debug!(
- "Generated diagnostics for {} files",
- combined_diagnostics.len()
- );
- combined_diagnostics
}
#[cfg(test)]
@@ -319,12 +292,15 @@ mod tests {
#[test]
fn test_diagnostics_no_errors() {
+ use crate::checkers::SystemCallChecker;
+
let (_temp_dir, file_path) =
create_temp_beancount_file("2023-01-01 open Assets:Cash\n2023-01-01 close Assets:Cash");
let beancount_data = HashMap::new();
let mock_bean_check = create_mock_bean_check_success();
+ let checker = SystemCallChecker::new(mock_bean_check);
- let result = diagnostics(beancount_data, &mock_bean_check, &file_path);
+ let result = diagnostics(beancount_data, &checker, &file_path);
assert!(
result.is_empty(),
@@ -334,12 +310,14 @@ mod tests {
#[test]
fn test_diagnostics_bean_check_errors() {
+ use crate::checkers::SystemCallChecker;
+
let (_temp_dir, file_path) = create_temp_beancount_file("invalid beancount syntax");
let beancount_data = HashMap::new();
-
let mock_bean_check = create_mock_bean_check_with_errors();
+ let checker = SystemCallChecker::new(mock_bean_check);
- let result = diagnostics(beancount_data, &mock_bean_check, &file_path);
+ let result = diagnostics(beancount_data, &checker, &file_path);
// Since /bin/false doesn't output structured errors, we expect empty result
// but the test verifies that the function handles command failures gracefully
@@ -351,14 +329,16 @@ mod tests {
#[test]
fn test_diagnostics_flagged_entries() {
+ use crate::checkers::SystemCallChecker;
+
let flagged_content =
"2023-01-01 ! \"Flagged transaction\"\n Assets:Cash 100 USD\n Expenses:Food";
let (_temp_dir, file_path) = create_temp_beancount_file(flagged_content);
let beancount_data = create_mock_beancount_data_with_flags(&file_path, flagged_content);
-
let mock_bean_check = create_mock_bean_check_success();
+ let checker = SystemCallChecker::new(mock_bean_check);
- let result = diagnostics(beancount_data, &mock_bean_check, &file_path);
+ let result = diagnostics(beancount_data, &checker, &file_path);
assert!(
!result.is_empty(),
@@ -396,13 +376,15 @@ mod tests {
#[test]
fn test_diagnostics_combined_errors_and_flags() {
+ use crate::checkers::SystemCallChecker;
+
let flagged_content = "2023-01-01 ! \"Test\"\n Assets:Cash\n Expenses:Food";
let (_temp_dir, file_path) = create_temp_beancount_file(flagged_content);
let beancount_data = create_mock_beancount_data_with_flags(&file_path, flagged_content);
-
let mock_bean_check = create_mock_bean_check_with_errors();
+ let checker = SystemCallChecker::new(mock_bean_check);
- let result = diagnostics(beancount_data, &mock_bean_check, &file_path);
+ let result = diagnostics(beancount_data, &checker, &file_path);
assert!(
!result.is_empty(),
@@ -431,11 +413,14 @@ mod tests {
#[test]
fn test_diagnostics_invalid_bean_check_command() {
+ use crate::checkers::SystemCallChecker;
+
let (_temp_dir, file_path) = create_temp_beancount_file("test content");
let beancount_data = HashMap::new();
let invalid_command = PathBuf::from("/nonexistent/command/that/does/not/exist");
+ let checker = SystemCallChecker::new(invalid_command);
- let result = diagnostics(beancount_data, &invalid_command, &file_path);
+ let result = diagnostics(beancount_data, &checker, &file_path);
assert!(
result.is_empty(),
@@ -445,12 +430,14 @@ mod tests {
#[test]
fn test_diagnostics_malformed_error_output() {
+ use crate::checkers::SystemCallChecker;
+
let (_temp_dir, file_path) = create_temp_beancount_file("test content");
let beancount_data = HashMap::new();
-
let mock_bean_check = create_mock_bean_check_with_errors();
+ let checker = SystemCallChecker::new(mock_bean_check);
- let result = diagnostics(beancount_data, &mock_bean_check, &file_path);
+ let result = diagnostics(beancount_data, &checker, &file_path);
// Should handle command failures gracefully (no panics)
assert!(
@@ -461,6 +448,8 @@ mod tests {
#[test]
fn test_diagnostics_multiple_files() {
+ use crate::checkers::SystemCallChecker;
+
let (_temp_dir1, file_path1) = create_temp_beancount_file("content1");
let (_temp_dir2, file_path2) = create_temp_beancount_file("content2");
@@ -473,8 +462,9 @@ mod tests {
beancount_data.extend(create_mock_beancount_data_with_flags(&file_path2, content2));
let mock_bean_check = create_mock_bean_check_success();
+ let checker = SystemCallChecker::new(mock_bean_check);
- let result = diagnostics(beancount_data, &mock_bean_check, &file_path1);
+ let result = diagnostics(beancount_data, &checker, &file_path1);
// Should have diagnostics for files with flagged entries
assert!(
@@ -492,8 +482,8 @@ mod tests {
#[test]
fn test_error_line_regex() {
- // Test the static regex directly
- let regex = get_error_line_regex();
+ // Test the regex pattern directly
+ let regex = regex::Regex::new(r"^([^:]+):(\d+):\s*(.*)$").unwrap();
// Valid error formats
assert!(regex.is_match("/path/to/file.beancount:123: Error message"));
@@ -518,11 +508,14 @@ mod tests {
#[test]
fn test_diagnostics_empty_beancount_data() {
+ use crate::checkers::SystemCallChecker;
+
let (_temp_dir, file_path) = create_temp_beancount_file("empty");
let beancount_data = HashMap::new(); // No beancount data
let mock_bean_check = create_mock_bean_check_success();
+ let checker = SystemCallChecker::new(mock_bean_check);
- let result = diagnostics(beancount_data, &mock_bean_check, &file_path);
+ let result = diagnostics(beancount_data, &checker, &file_path);
assert!(
result.is_empty(),
@@ -532,12 +525,14 @@ mod tests {
#[test]
fn test_diagnostic_position_conversion() {
+ use crate::checkers::SystemCallChecker;
+
let (_temp_dir, file_path) = create_temp_beancount_file("test");
let beancount_data = HashMap::new();
-
let mock_bean_check = create_mock_bean_check_success();
+ let checker = SystemCallChecker::new(mock_bean_check);
- let result = diagnostics(beancount_data, &mock_bean_check, &file_path);
+ let result = diagnostics(beancount_data, &checker, &file_path);
// Since we're not testing actual bean-check error parsing here,
// we just verify that the function works without crashing
@@ -550,7 +545,7 @@ mod tests {
#[test]
fn test_error_line_regex_with_line_zero() {
- let regex = get_error_line_regex();
+ let regex = regex::Regex::new(r"^([^:]+):(\d+):\s*(.*)$").unwrap();
// Test line 0 format (file-level errors)
assert!(regex.is_match(":0: Missing Commodity directive for 'HFCGX'"));
diff --git a/crates/lsp/src/providers/text_document.rs b/crates/lsp/src/providers/text_document.rs
index a0d1053..b7bd761 100644
--- a/crates/lsp/src/providers/text_document.rs
+++ b/crates/lsp/src/providers/text_document.rs
@@ -1,4 +1,5 @@
use crate::beancount_data::BeancountData;
+use crate::checkers::create_checker;
use crate::document::Document;
use crate::providers::diagnostics;
use crate::server::LspServerState;
@@ -283,7 +284,22 @@ fn handle_diagnostics(
uri: lsp_types::Uri,
) -> Result<()> {
tracing::debug!("text_document::handle_diagnostics");
- let bean_check_cmd = &PathBuf::from("bean-check");
+
+ // Create the appropriate checker based on configuration
+ tracing::debug!(
+ "Bean check configuration: method={:?}, bean_check_cmd={}, python_cmd={}, python_script={}",
+ snapshot.config.bean_check.method,
+ snapshot.config.bean_check.bean_check_cmd.display(),
+ snapshot.config.bean_check.python_cmd.display(),
+ snapshot.config.bean_check.python_script_path.display()
+ );
+
+ let checker = create_checker(&snapshot.config.bean_check);
+ tracing::debug!(
+ "Using checker: {}, available: {}",
+ checker.name(),
+ checker.is_available()
+ );
sender
.send(Task::Progress(ProgressMsg::BeanCheck { done: 0, total: 1 }))
@@ -296,8 +312,11 @@ fn handle_diagnostics(
uri.to_file_path().unwrap_or_default()
};
- let diags =
- diagnostics::diagnostics(snapshot.beancount_data, bean_check_cmd, &root_journal_path);
+ let diags = diagnostics::diagnostics(
+ snapshot.beancount_data,
+ checker.as_ref(),
+ &root_journal_path,
+ );
sender
.send(Task::Progress(ProgressMsg::BeanCheck { done: 1, total: 1 }))
diff --git a/docs/brownfield-architecture.md b/docs/brownfield-architecture.md
new file mode 100644
index 0000000..ff43487
--- /dev/null
+++ b/docs/brownfield-architecture.md
@@ -0,0 +1,408 @@
+# Beancount Language Server Brownfield Architecture Document
+
+## Introduction
+
+This document captures the CURRENT STATE of the Beancount Language Server codebase, including architecture patterns, technical decisions, and real-world implementation details. It serves as a reference for AI agents working on enhancements and maintenance.
+
+### Document Scope
+
+Comprehensive documentation of the entire beancount-language-server system, including the Rust LSP server, VSCode extension, Python integration components, and development infrastructure.
+
+### Change Log
+
+| Date | Version | Description | Author |
+|------|---------|-------------|--------|
+| 2025-01-09 | 1.0 | Initial brownfield analysis | Claude |
+
+## Quick Reference - Key Files and Entry Points
+
+### Critical Files for Understanding the System
+
+- **Main Entry**: `crates/lsp/src/main.rs` - CLI argument parsing and logging setup
+- **Core LSP Logic**: `crates/lsp/src/lib.rs` - Server initialization and main event loop
+- **Configuration**: `crates/lsp/src/config.rs` - Runtime configuration management
+- **LSP Handlers**: `crates/lsp/src/handlers.rs` - Request/response handlers for LSP protocol
+- **Core Providers**: `crates/lsp/src/providers/` - Feature implementations (completion, diagnostics, formatting)
+- **Bean-check Strategy**: `crates/lsp/src/checkers/mod.rs` - Pluggable validation architecture
+- **VSCode Extension**: `vscode/src/extension.ts` - Client-side LSP integration
+
+### Key Algorithms and Business Logic
+
+- **Diagnostics Provider**: `crates/lsp/src/providers/diagnostics.rs` - Multi-method beancount validation
+- **Completion Engine**: `crates/lsp/src/providers/completion.rs` - Context-aware autocompletion
+- **Formatting Logic**: `crates/lsp/src/providers/formatting.rs` - Bean-format compatible formatting
+- **Tree-sitter Integration**: `crates/lsp/src/treesitter_utils.rs` - AST parsing utilities
+
+## High Level Architecture
+
+### Technical Summary
+
+This is a **Language Server Protocol (LSP) implementation** written in Rust that provides rich editing features for Beancount accounting files. The system follows a plugin-based architecture for beancount validation with three different execution strategies.
+
+**Core Value Proposition**: Brings modern IDE features (completions, diagnostics, formatting, references) to the Beancount plain-text accounting ecosystem through LSP protocol compliance.
+
+### Actual Tech Stack (from package.json/Cargo.toml)
+
+| Category | Technology | Version | Notes |
+|----------|------------|---------|--------|
+| **Core Runtime** | Rust | 1.75.0+ | Stable toolchain, edition 2021 |
+| **LSP Framework** | lsp-server | 0.7 | LSP protocol implementation |
+| **LSP Types** | lsp-types | 0.97 | LSP data structures and protocol |
+| **Text Processing** | ropey | 1.6 | Efficient rope data structure |
+| **Parsing** | tree-sitter-beancount | 2.4.1 | Grammar-based parsing |
+| **Pattern Matching** | regex | 1.0 | Error message parsing |
+| **Python Integration** | pyo3 | 0.25 | Embedded Python (optional feature) |
+| **Threading** | crossbeam-channel | 0.5 | Message passing between threads |
+| **JSON/Config** | serde + serde_json | 1.0 | Configuration and data serialization |
+| **CLI** | clap | 4.0 | Command line argument parsing |
+| **Logging** | tracing + tracing-subscriber | 0.3 | Structured logging framework |
+| **Error Handling** | anyhow + thiserror | 1.0/2.0 | Error propagation and custom errors |
+| **VSCode Extension** | TypeScript | 4.6.3 | Client-side LSP integration |
+| **VSCode LSP Client** | vscode-languageclient | 8.0.0-next.14 | LSP protocol client |
+
+### Repository Structure Reality Check
+
+- **Type**: Hybrid workspace (Rust workspace + NPM package for VSCode extension)
+- **Package Manager**: Cargo (Rust) + NPM (VSCode extension)
+- **Build System**: Cargo + cargo-dist for releases, esbuild for VSCode extension
+- **Notable**: Nix flake for reproducible development environment
+
+## Source Tree and Module Organization
+
+### Project Structure (Actual)
+
+```text
+beancount-language-server/
+├── crates/lsp/ # Main Rust LSP server implementation
+│ ├── src/
+│ │ ├── main.rs # CLI entry point with logging setup
+│ │ ├── lib.rs # Core LSP server logic and initialization
+│ │ ├── server.rs # LSP server state management and message loop
+│ │ ├── handlers.rs # LSP request/notification handlers
+│ │ ├── config.rs # Configuration management and defaults
+│ │ ├── providers/ # LSP feature implementations
+│ │ │ ├── completion.rs # Autocompletion engine
+│ │ │ ├── diagnostics.rs # Multi-method validation provider
+│ │ │ ├── formatting.rs # Bean-format compatible formatter
+│ │ │ ├── references.rs # Find references implementation
+│ │ │ └── text_document.rs # Document lifecycle management
+│ │ ├── checkers/ # Pluggable bean-check validation strategies
+│ │ │ ├── mod.rs # Strategy trait and factory pattern
+│ │ │ ├── system_call.rs # Traditional bean-check subprocess
+│ │ │ ├── pyo3_embedded.rs # Embedded Python validation (feature-gated)
+│ │ │ └── types.rs # Shared validation data structures
+│ │ ├── document.rs # Document representation and management
+│ │ ├── forest.rs # Multi-document forest management
+│ │ ├── beancount_data.rs # Beancount-specific data extraction
+│ │ └── treesitter_utils.rs # Tree-sitter parsing utilities
+│ ├── tests/ # Integration and unit tests (insta snapshots)
+│ └── Cargo.toml # Package manifest with optional features
+├── vscode/ # VS Code extension (TypeScript)
+│ ├── src/
+│ │ ├── extension.ts # Main extension entry point
+│ │ ├── config.ts # Configuration management
+│ │ ├── persistent_state.ts # Client-side state persistence
+│ │ ├── semantic_tokens.ts # Semantic highlighting (tree-sitter)
+│ │ └── util.ts # Utility functions
+│ ├── package.json # Extension manifest and dependencies
+│ └── language-configuration.json # Beancount language definition
+├── python/ # Python integration utilities
+│ └── bean_check.py # Enhanced validation script with JSON output
+├── docs/ # Additional documentation
+│ └── completion-system.md # Completion system architecture
+├── flake.nix # Nix development environment with Crane
+├── Cargo.toml # Workspace configuration
+└── cliff.toml # git-cliff changelog configuration
+```
+
+### Key Modules and Their Purpose
+
+**Core LSP Infrastructure**:
+- **Server State Management**: `src/server.rs` - Manages LSP connection state, document forest, configuration
+- **Message Dispatcher**: `src/dispatcher.rs` - Routes LSP messages to appropriate handlers
+- **Document Forest**: `src/forest.rs` - Multi-file document management with tree-sitter integration
+
+**Feature Providers** (implements LSP capabilities):
+- **Completion Engine**: `src/providers/completion.rs` - Context-aware completions for accounts, payees, dates
+- **Diagnostics Provider**: `src/providers/diagnostics.rs` - Pluggable validation with multiple backends
+- **Formatting Provider**: `src/providers/formatting.rs` - Bean-format compatible with configuration options
+- **References Provider**: `src/providers/references.rs` - Find all references across files
+
+**Validation Architecture** (Strategy pattern):
+- **Strategy Interface**: `src/checkers/mod.rs` - BeancountChecker trait and factory
+- **System Call Checker**: `src/checkers/system_call.rs` - Traditional bean-check subprocess execution
+- **Embedded Python**: `src/checkers/pyo3_embedded.rs` - Direct Python library integration (optional)
+
+**Configuration System**:
+- **Config Management**: `src/config.rs` - JSON-based configuration with defaults and validation
+- **LSP Integration**: Configuration passed via LSP initialization options
+
+## Data Models and APIs
+
+### Core Data Structures
+
+**Document Representation**:
+- **Document**: `src/document.rs` - Individual beancount file with rope-based text storage
+- **BeancountData**: `src/beancount_data.rs` - Extracted semantic information (accounts, payees, etc.)
+- **Forest**: `src/forest.rs` - Multi-document management with include file resolution
+
+**Validation Types**:
+- **BeancountCheckResult**: `src/checkers/types.rs` - Validation results with errors and flagged entries
+- **BeancountError**: Structured error representation with file/line information
+- **FlaggedEntry**: Warning-level issues found in beancount files
+
+### Configuration Schema
+
+See `crates/lsp/src/config.rs` for complete configuration structure:
+
+```rust
+pub struct Config {
+ pub journal_file: Option,
+ pub bean_check_config: BeancountCheckConfig,
+ pub formatting_options: Option,
+ // ... other fields
+}
+```
+
+**Bean-check Configuration**: Three validation methods with different performance/accuracy tradeoffs
+**Formatting Options**: Bean-format compatibility with prefix_width, num_width, currency_column support
+
+### LSP Protocol Implementation
+
+**Supported LSP Features**:
+- `textDocument/completion` - Context-aware autocompletion
+- `textDocument/publishDiagnostics` - Multi-method validation
+- `textDocument/formatting` - Document formatting
+- `textDocument/references` - Find all references
+- `textDocument/rename` - Symbol renaming across files
+
+**VSCode Extension Integration**:
+- **Language Configuration**: `.beancount` and `.bean` file association
+- **Tree-sitter Grammar**: Client-side syntax highlighting
+- **Configuration Bridge**: Maps VSCode settings to LSP initialization options
+
+## Technical Debt and Known Issues
+
+### Architectural Strengths
+
+1. **Clean Strategy Pattern**: Checker architecture is well-designed and extensible
+2. **Proper LSP Implementation**: Follows LSP protocol correctly with good client compatibility
+3. **Tree-sitter Integration**: Fast, incremental parsing with proper AST handling
+4. **Feature Flag System**: Optional PyO3 integration properly feature-gated
+5. **Comprehensive Testing**: Good test coverage with snapshot testing (insta)
+
+### Areas for Improvement
+
+1. **Python Script Method**: The python-script checker method (`python/bean_check.py`) is basic and could use better error handling
+2. **Configuration Validation**: Limited validation of user-provided configuration options
+3. **Error Recovery**: Some error conditions could have more graceful degradation
+4. **Caching Strategy**: Diagnostic results and completion data could benefit from caching
+5. **Performance Monitoring**: No built-in performance metrics or profiling capabilities
+
+### Technical Constraints
+
+1. **Python Dependency**: Diagnostics require Python beancount library to be installed
+2. **Tree-sitter Version**: Tied to specific tree-sitter-beancount grammar version
+3. **LSP Protocol Limits**: Some advanced features constrained by LSP specification
+4. **Single-threaded Processing**: Most operations are single-threaded (could benefit from parallelization)
+
+## Integration Points and External Dependencies
+
+### External Services and Tools
+
+| Service/Tool | Purpose | Integration Type | Key Files |
+|--------------|---------|------------------|-----------|
+| **beancount (Python)** | Validation and parsing | System command / Python import | `checkers/system_call.rs`, `checkers/pyo3_embedded.rs` |
+| **bean-check** | Traditional validation | System command execution | `checkers/system_call.rs` |
+| **tree-sitter-beancount** | AST parsing | Direct library integration | `treesitter_utils.rs` |
+| **VSCode** | Editor integration | LSP protocol | `vscode/src/extension.ts` |
+
+### Internal Integration Points
+
+**LSP Protocol Compliance**:
+- **Message Handling**: Bidirectional JSON-RPC over stdio/TCP
+- **Document Lifecycle**: Proper handling of open/change/save/close events
+- **Configuration Management**: LSP initialization options and workspace/did_change_configuration
+
+**Multi-File Processing**:
+- **Include Resolution**: Automatic discovery and processing of included beancount files
+- **Cross-File References**: Find references and rename operations across file boundaries
+- **Forest Management**: Efficient tracking of document dependencies and changes
+
+**Validation Pipeline**:
+- **Strategy Selection**: Factory pattern for choosing validation method
+- **Error Aggregation**: Combining results from multiple validation sources
+- **Incremental Updates**: Re-validation on file changes with proper debouncing
+
+## Development and Deployment
+
+### Local Development Setup
+
+**Using Nix (Recommended)**:
+```bash
+nix develop # Enters development shell with all dependencies
+```
+
+**Manual Setup**:
+```bash
+# Install Rust toolchain
+rustup install stable
+# Install Python dependencies
+pip install beancount # Required for diagnostics
+# Build language server
+cargo build --release
+# VSCode extension development
+cd vscode && npm install && npm run build
+```
+
+### Build and Deployment Process
+
+**Release Process** (automated via GitHub Actions):
+- **cargo-dist**: Cross-platform binary builds for Linux (x86_64, aarch64, loongarch64), macOS (x86_64, aarch64), Windows (x86_64)
+- **Deployment Targets**: GitHub releases, Crates.io, Homebrew, Nix packages
+- **VSCode Extension**: Manual packaging via `npm run package` → VSIX file
+
+**Development Commands**:
+```bash
+# Core development
+cargo build # Standard build
+cargo build --features python-embedded # With embedded Python
+cargo test # Run all tests
+cargo clippy --all-targets # Linting
+cargo fmt # Code formatting
+
+# VSCode extension
+cd vscode
+npm run build # Build extension
+npm run watch # Watch mode
+npm run package # Create VSIX package
+```
+
+### Quality Assurance
+
+**Testing Strategy**:
+- **Unit Tests**: Comprehensive test coverage with `cargo test`
+- **Integration Tests**: End-to-end LSP functionality testing
+- **Snapshot Testing**: Using `insta` for output validation
+- **Feature Testing**: Optional PyO3 feature tested separately
+
+**CI/CD Pipeline** (GitHub Actions):
+- **Continuous Integration**: Format, clippy, tests on multiple OS/Rust versions
+- **Security**: CodeQL analysis for vulnerability scanning
+- **Release Automation**: Automated binary builds and publishing
+- **Quality Gates**: All checks must pass before merge
+
+## Current Feature Implementation Status
+
+### ✅ Fully Implemented
+
+| Feature | Implementation Status | Performance Notes |
+|---------|----------------------|-------------------|
+| **Completions** | Production ready | Fast with tree-sitter parsing |
+| **Diagnostics** | Multiple backends available | Depends on chosen validation method |
+| **Formatting** | Bean-format compatible | Configurable width/alignment options |
+| **References** | Cross-file support | Efficient with forest management |
+| **Rename** | Symbol renaming | Works across file boundaries |
+
+### 📋 Planned Features (from README)
+
+| Feature | Priority | Implementation Complexity |
+|---------|----------|---------------------------|
+| **Hover** | High | Medium (requires balance computation) |
+| **Go to Definition** | High | Low (similar to references) |
+| **Document Symbols** | High | Medium (requires semantic analysis) |
+| **Folding Ranges** | Medium | Low (tree-sitter based) |
+| **Semantic Highlighting** | Medium | Medium (requires token classification) |
+| **Code Actions** | Medium | High (requires transaction analysis) |
+
+## Configuration and Customization
+
+### Runtime Configuration
+
+**Configuration Sources** (in precedence order):
+1. LSP initialization options (primary)
+2. Default configuration values
+3. Environment-based detection (PATH for executables)
+
+**Key Configuration Areas**:
+- **Journal File**: Path to main beancount file for validation
+- **Validation Method**: Choice between system/python-script/python-embedded
+- **Formatting Options**: Width settings and alignment preferences
+- **Logging**: File-based logging with configurable levels
+
+### Editor Integration Status
+
+**Production Ready**:
+- **VSCode**: Full extension with marketplace publication
+- **Neovim**: Well-documented nvim-lspconfig integration
+- **Helix**: Configuration examples provided
+
+**Community Supported**:
+- **Emacs**: lsp-mode integration documented
+- **Vim**: vim-lsp configuration examples
+- **Sublime Text**: LSP package integration
+
+## Performance Characteristics
+
+### Current Performance Profile
+
+**Startup Time**: Sub-second initialization for most projects
+**Memory Usage**: Moderate (depends on project size and validation method)
+**Validation Speed**:
+- System call: Fastest startup, moderate validation speed
+- Python embedded: Higher memory but fastest validation
+- Python script: Slowest but most flexible
+
+**Scalability Limits**:
+- **File Count**: Tested with projects up to 50+ files
+- **File Size**: Individual files up to several MB handled efficiently
+- **Concurrent Operations**: Single-threaded processing with proper debouncing
+
+### Optimization Opportunities
+
+1. **Parallel Processing**: Validation and completion could be parallelized
+2. **Incremental Parsing**: More granular tree-sitter update tracking
+3. **Result Caching**: Cache validation results and completion data
+4. **Memory Optimization**: More efficient document storage for large projects
+
+## Appendix - Useful Commands and Scripts
+
+### Development Workflow
+
+```bash
+# Quick development cycle
+cargo watch -x 'test' # Auto-test on changes
+cargo run -- --stdio # Run server locally
+cargo run -- --log debug # Run with debug logging
+
+# Release preparation
+git cliff # Generate changelog
+cargo dist build # Test release build
+cargo dist plan # Review release plan
+
+# Nix-based development
+nix flake check # Run all checks
+nix build # Build with nix
+nix develop # Enter dev environment
+```
+
+### Debugging and Troubleshooting
+
+**Log Analysis**:
+- **Server Logs**: `beancount-language-server.log` (when --log flag used)
+- **Debug Mode**: Set log level to `debug` or `trace` for detailed output
+- **VSCode Logs**: Check Output panel → Beancount Language Server
+
+**Common Issues**:
+- **Bean-check Not Found**: Ensure `bean-check` binary is in PATH or configure path
+- **Python Import Errors**: Verify beancount library is installed in correct Python environment
+- **Performance Issues**: Consider switching to python-embedded method for large projects
+- **Configuration Problems**: Check LSP initialization options format and values
+
+**Testing Specific Features**:
+```bash
+# Test specific validation methods
+cargo test --features python-embedded # Test PyO3 integration
+cargo test system_call # Test system call validation
+cargo test completion # Test completion engine
+```
\ No newline at end of file
diff --git a/docs/prd.md b/docs/prd.md
new file mode 100644
index 0000000..9eb2778
--- /dev/null
+++ b/docs/prd.md
@@ -0,0 +1,333 @@
+# Beancount Language Server Brownfield Enhancement PRD
+
+## Intro Project Analysis and Context
+
+### SCOPE ASSESSMENT
+
+This PRD addresses SIGNIFICANT enhancements to the existing beancount language server that require comprehensive planning and multiple coordinated stories. The project already has a mature, production-ready foundation that supports substantial architectural improvements.
+
+### Existing Project Overview
+
+**Analysis Source:** Document-project output available at `docs/brownfield-architecture.md`
+
+**Current Project State:**
+Production-ready Language Server Protocol (LSP) implementation written in Rust that provides rich editing features for Beancount accounting files. The system includes:
+
+- Multi-method validation system (system call, Python script, PyO3 embedded)
+- Complete LSP feature set (completions, diagnostics, formatting, references, rename)
+- VSCode extension with marketplace publication
+- Cross-platform deployment (Linux, macOS, Windows)
+- Nix-based development environment
+- Comprehensive CI/CD pipeline
+
+### Available Documentation Analysis
+
+✅ **Using existing project analysis from document-project output**
+
+Key documents available from document-project:
+- ✅ Tech Stack Documentation (comprehensive)
+- ✅ Source Tree/Architecture (detailed module breakdown)
+- ✅ API Documentation (LSP protocol implementation)
+- ✅ External API Documentation (beancount Python integration)
+- ✅ Technical Debt Documentation (identified constraints and improvements)
+- ✅ Performance Characteristics (current benchmarks)
+- ✅ Development Workflow (complete setup instructions)
+
+### Enhancement Scope Definition
+
+**Enhancement Type:**
+- ✅ New Feature Addition
+- ✅ Major Feature Modification
+- ✅ Performance/Scalability Improvements
+
+**Enhancement Description:**
+Expand the beancount language server with additional LSP capabilities (hover, go-to-definition, document symbols, etc.), enhance existing features (completions, diagnostics, formatting), and significantly improve diagnostic performance through optimization and potentially new validation strategies.
+
+**Impact Assessment:**
+- ✅ Significant Impact (substantial existing code changes)
+- ✅ Major Impact (architectural changes required for performance improvements)
+
+### Goals and Background Context
+
+**Goals:**
+- Implement missing LSP features to provide comprehensive IDE experience
+- Enhance existing LSP features with better accuracy and user experience
+- Dramatically improve diagnostic performance for better real-time feedback
+- Maintain backward compatibility and existing feature quality
+- Leverage existing clean architecture for sustainable expansion
+
+**Background Context:**
+The beancount language server has established itself as a production-ready LSP implementation with solid fundamentals. However, users need more comprehensive IDE features to match modern development experiences. Current diagnostic performance, while functional, could be significantly improved for larger projects. The existing pluggable architecture and clean codebase provide an excellent foundation for these enhancements without compromising stability.
+
+**Change Log:**
+| Change | Date | Version | Description | Author |
+|--------|------|---------|-------------|--------|
+| Initial PRD Creation | 2025-01-09 | 1.0 | Comprehensive LSP enhancement planning | PM John |
+
+## Requirements
+
+### Functional
+
+**FR1:** The language server shall implement hover support showing account balances, transaction details, and metadata when hovering over beancount elements
+
+**FR2:** The language server shall provide go-to-definition functionality for navigating to account/payee/commodity definitions across files
+
+**FR3:** The language server shall implement document symbols support for outline views showing accounts, transactions, and file structure
+
+**FR4:** The language server shall add folding ranges capability for collapsing transactions, account hierarchies, and multi-line entries
+
+**FR5:** The language server shall implement semantic highlighting providing enhanced syntax coloring with semantic information
+
+**FR6:** The existing completion system shall be enhanced with better context awareness and additional completion types (commodities, tags, links)
+
+**FR7:** The existing formatting system shall support additional configuration options and improved algorithm efficiency
+
+**FR8:** The diagnostics system shall implement performance optimizations including caching, incremental validation, and parallel processing
+
+**FR9:** The language server shall add a new high-performance validation method optimized for real-time diagnostics
+
+**FR10:** All new features shall integrate seamlessly with the existing tree-sitter parsing and document forest management
+
+### Non Functional
+
+**NFR1:** Enhancement must maintain existing performance characteristics for current features and not exceed memory usage by more than 30%
+
+**NFR2:** New diagnostic optimizations must achieve at least 50% performance improvement for projects with 20+ files
+
+**NFR3:** All new LSP features must respond within 200ms for typical beancount files (up to 1000 lines)
+
+**NFR4:** The enhanced system must maintain 99.9% backward compatibility with existing editor configurations
+
+**NFR5:** Code quality standards must be maintained with comprehensive test coverage (>80%) for all new features
+
+**NFR6:** New features must support all existing platforms (Linux x86_64/aarch64, macOS x86_64/aarch64, Windows x86_64)
+
+### Compatibility Requirements
+
+**CR1:** All existing LSP protocol implementations must remain fully functional without configuration changes
+
+**CR2:** The current three-method validation system (system call, Python script, PyO3 embedded) must be preserved and enhanced
+
+**CR3:** VSCode extension and all documented editor integrations must continue working without modification
+
+**CR4:** Existing configuration schema must remain valid with new options being additive only
+
+## Technical Constraints and Integration Requirements
+
+### Existing Technology Stack
+
+**Languages**: Rust 1.75.0+ (edition 2021), Python 3.x (for beancount integration), TypeScript 4.6.3 (VSCode extension)
+
+**Frameworks**: LSP Server 0.7 + LSP Types 0.97 (LSP protocol), Tree-sitter 2.4.1 (parsing), PyO3 0.25 (optional Python embedding)
+
+**Database**: File-based (beancount plain text files with include resolution)
+
+**Infrastructure**: Cross-platform binary distribution via cargo-dist, Nix flake development environment, GitHub Actions CI/CD
+
+**External Dependencies**: Beancount Python library (required), bean-check binary (optional), tree-sitter-beancount grammar
+
+### Integration Approach
+
+**Database Integration Strategy**: Enhance existing file-based document forest management with caching layers for computed data (balances, symbol tables). Maintain compatibility with beancount include file resolution.
+
+**API Integration Strategy**: Extend existing LSP protocol handlers with new capabilities while preserving current request/response patterns. Add new providers following established provider pattern in `src/providers/`.
+
+**Frontend Integration Strategy**: Leverage existing tree-sitter parsing infrastructure for new semantic analysis. Enhance existing completion and diagnostic providers with optimized algorithms and caching.
+
+**Testing Integration Strategy**: Extend existing insta snapshot testing for new features. Add performance benchmarks for diagnostic improvements. Maintain existing test coverage standards.
+
+### Code Organization and Standards
+
+**File Structure Approach**: Follow established provider pattern - new LSP features as modules in `src/providers/`, shared utilities in existing utility modules. Performance optimizations integrated into existing checker system.
+
+**Naming Conventions**: Maintain existing Rust naming conventions and LSP protocol naming. New diagnostic methods follow established `BeancountChecker` trait pattern.
+
+**Coding Standards**: Adhere to existing cargo fmt + clippy standards. Maintain existing error handling patterns with anyhow/thiserror. Continue structured logging with tracing framework.
+
+**Documentation Standards**: Follow existing inline documentation style. Update README.md feature tables. Add performance benchmarks to documentation.
+
+### Deployment and Operations
+
+**Build Process Integration**: Utilize existing cargo-dist cross-platform builds. Maintain optional PyO3 feature compilation. Ensure new features work in Nix development environment.
+
+**Deployment Strategy**: Leverage existing release automation via GitHub Actions. Maintain backward compatibility for existing installations. Update VSCode extension as needed for new features.
+
+**Monitoring and Logging**: Enhance existing tracing-based logging with performance metrics. Add diagnostic timing information. Maintain existing log level configurability.
+
+**Configuration Management**: Extend existing JSON-based LSP configuration schema additively. Maintain existing default values and backward compatibility.
+
+### Risk Assessment and Mitigation
+
+**Technical Risks**:
+- Performance optimizations may introduce complexity that affects maintainability
+- New LSP features could impact single-threaded processing limitations noted in technical debt
+- Tree-sitter parsing overhead for semantic analysis features may affect responsiveness
+
+**Integration Risks**:
+- Changes to diagnostic system could affect existing three-method validation strategy
+- New caching layers may introduce state management complexity
+- Enhanced features may stress existing document forest management under high file counts
+
+**Deployment Risks**:
+- Additional dependencies or optional features may complicate cross-platform builds
+- Performance optimizations may behave differently across target platforms
+- VSCode extension updates may require marketplace approval delays
+
+**Mitigation Strategies**:
+- Implement performance features behind feature flags for gradual rollout
+- Maintain existing validation methods while adding optimized variants
+- Use comprehensive benchmarking to validate performance improvements across platforms
+- Design caching as optional enhancement that degrades gracefully
+- Implement thorough integration testing with existing editor configurations
+
+## Epic and Story Structure
+
+### Epic Approach
+
+**Epic Structure Decision**: Single comprehensive epic with rationale: Maintains architectural coherence while enabling systematic development that builds optimizations first, then features, ensuring each story benefits from previous improvements.
+
+## Epic 1: Comprehensive LSP Enhancement
+
+**Epic Goal**: Transform the beancount language server into a comprehensive, high-performance IDE experience by implementing missing LSP features, optimizing diagnostic performance, and enhancing existing capabilities while maintaining full backward compatibility.
+
+**Integration Requirements**: All enhancements must integrate seamlessly with existing tree-sitter parsing, document forest management, and three-method validation architecture. Performance improvements must be measurable and not compromise existing functionality.
+
+### Story 1.1: Diagnostic Performance Foundation
+
+As a **beancount developer**,
+I want **significantly faster diagnostic feedback with caching and optimization**,
+so that **I can work efficiently with larger beancount projects without waiting for validation**.
+
+#### Acceptance Criteria
+
+1. **Diagnostic caching system**: Implement result caching that invalidates appropriately on file changes
+2. **Incremental validation**: Only re-validate changed files and their dependencies
+3. **Performance benchmarks**: Achieve 50%+ performance improvement for projects with 20+ files
+4. **Memory efficiency**: Caching system uses <30% additional memory overhead
+5. **Configurable optimization**: Users can enable/disable optimization features via configuration
+
+#### Integration Verification
+
+**IV1**: All existing validation methods (system call, Python script, PyO3 embedded) continue to function correctly with caching layer
+**IV2**: Existing diagnostic accuracy is maintained - no false positives or missed errors introduced
+**IV3**: VSCode extension and all editor integrations continue to receive diagnostics without modification
+
+### Story 1.2: Enhanced Symbol Analysis Infrastructure
+
+As a **beancount developer**,
+I want **improved symbol extraction and analysis for accounts, payees, and commodities**,
+so that **new LSP features have accurate data to work with**.
+
+#### Acceptance Criteria
+
+1. **Symbol database**: Comprehensive extraction of accounts, payees, commodities, tags, and links with metadata
+2. **Cross-file resolution**: Symbol references resolved across included files using existing forest management
+3. **Incremental updates**: Symbol database updates efficiently when files change
+4. **Memory efficiency**: Symbol data structures optimized for lookup performance
+5. **API consistency**: Symbol data accessible through consistent internal APIs
+
+#### Integration Verification
+
+**IV1**: Existing completion system continues to work and benefits from enhanced symbol data
+**IV2**: Document forest management integrates seamlessly with new symbol extraction
+**IV3**: Tree-sitter parsing performance is not significantly impacted by additional analysis
+
+### Story 1.3: Hover Information Support
+
+As a **beancount user**,
+I want **informative hover tooltips showing account details, transaction context, and computed information**,
+so that **I can understand my beancount data without navigating away from my current location**.
+
+#### Acceptance Criteria
+
+1. **Account hover**: Show account metadata, recent transactions, and balance information when available
+2. **Transaction hover**: Display transaction details, posting breakdowns, and validation status
+3. **Commodity hover**: Show commodity definitions, price information, and usage statistics
+4. **Performance target**: Hover responses delivered within 200ms for typical files
+5. **Graceful degradation**: Hover works with partial information when full computation unavailable
+
+#### Integration Verification
+
+**IV1**: Hover implementation doesn't interfere with existing LSP request handling
+**IV2**: Tree-sitter parsing provides adequate AST information for hover positioning
+**IV3**: Existing diagnostic and completion systems continue to operate normally
+
+### Story 1.4: Go-to-Definition and References Enhancement
+
+As a **beancount developer**,
+I want **precise navigation to symbol definitions and comprehensive reference finding**,
+so that **I can efficiently explore and understand large beancount codebases**.
+
+#### Acceptance Criteria
+
+1. **Go-to-definition**: Navigate to account opens, payee first usage, and commodity definitions
+2. **Enhanced references**: Find all references with context information and usage type
+3. **Cross-file navigation**: Seamless navigation across included files using forest management
+4. **Symbol disambiguation**: Handle cases where symbols might have multiple definitions
+5. **Performance optimization**: Leverage symbol database for fast lookups
+
+#### Integration Verification
+
+**IV1**: Existing reference finding functionality is enhanced, not replaced
+**IV2**: Document forest includes and dependencies are properly traversed
+**IV3**: Navigation works correctly with existing editor configurations
+
+### Story 1.5: Document Symbols and Structure
+
+As a **beancount developer**,
+I want **hierarchical document outline showing accounts, transactions, and file structure**,
+so that **I can quickly navigate and understand the organization of my beancount files**.
+
+#### Acceptance Criteria
+
+1. **Hierarchical symbols**: Account hierarchies displayed as nested structures
+2. **Transaction grouping**: Transactions grouped by date, type, or other logical criteria
+3. **Symbol metadata**: Include line numbers, types, and additional context information
+4. **Performance efficiency**: Symbol extraction integrated with existing parsing pipeline
+5. **Filtering support**: Support for filtering symbols by type or other criteria
+
+#### Integration Verification
+
+**IV1**: Document symbol extraction doesn't impact existing tree-sitter parsing performance
+**IV2**: Symbol information remains accurate as files are edited
+**IV3**: Multi-file projects show symbols appropriately across file boundaries
+
+### Story 1.6: Enhanced Completion System
+
+As a **beancount developer**,
+I want **smarter completions with better context awareness and additional completion types**,
+so that **I can write beancount code more efficiently with fewer errors**.
+
+#### Acceptance Criteria
+
+1. **Context-aware completions**: Account suggestions filtered by transaction context and patterns
+2. **Commodity completion**: Complete commodity names with proper formatting
+3. **Enhanced payee completion**: Payee suggestions with recent usage prioritization
+4. **Tag and link completion**: Complete hashtags and links with validation
+5. **Performance improvement**: Completion responses within 100ms using symbol database
+
+#### Integration Verification
+
+**IV1**: Existing completion functionality is enhanced while maintaining backward compatibility
+**IV2**: Completion accuracy improves without introducing incorrect suggestions
+**IV3**: VSCode extension and editor integrations receive improved completions seamlessly
+
+### Story 1.7: Folding Ranges and Semantic Highlighting
+
+As a **beancount developer**,
+I want **code folding for transactions and semantic highlighting for better code readability**,
+so that **I can manage large beancount files more effectively and understand code structure visually**.
+
+#### Acceptance Criteria
+
+1. **Transaction folding**: Fold individual transactions and transaction blocks
+2. **Account hierarchy folding**: Collapse account sections and related entries
+3. **Semantic highlighting**: Enhanced syntax coloring based on semantic analysis
+4. **Performance efficiency**: Folding and highlighting computed efficiently during parsing
+5. **Editor compatibility**: Features work across LSP-compatible editors
+
+#### Integration Verification
+
+**IV1**: Folding ranges don't interfere with existing text editing and formatting
+**IV2**: Semantic highlighting enhances but doesn't conflict with existing syntax highlighting
+**IV3**: Tree-sitter parsing pipeline accommodates additional analysis without performance degradation
\ No newline at end of file