feat(docs): update README and SKILL documentation for clarity and accuracy

Author: Akagi201

.gitignore

@@ -28,5 +28,4 @@ bun.lockb
 
 # Leptos
 /site/
-code2prompt/
-codebank/
+.rumdl_cache/

Cargo.toml

@@ -10,6 +10,10 @@ exclude = [".github/"]
 resolver = "3"
 
 [workspace.dependencies]
+# local crates
+ast-doc-core = { path = "crates/ast-doc-core", version = "0.1.0" }
+
+# external crates
 clap = "4.6.0"
 config = "0.15.22"
 cucumber = "0.22.1"

README.md

@@ -1,21 +1,128 @@
-# Rust Workspace Template
+# ast-doc
 
-Rust workspace template for `bin/` CLI crates and `crates/` shared libraries with Gherkin + `cucumber-rs` for BDD and `cargo test` for TDD.
+AST-powered code documentation tool for generating optimized `llms.txt` files from codebases.
+
+## Overview
+
+`ast-doc` is a Rust CLI tool that combines broad file traversal with deep AST-based semantic parsing to create optimized documentation. It uses a four-stage pipeline:
+
+1. **Ingestion** — File discovery, git metadata capture, directory tree generation
+2. **Parser** — tree-sitter AST extraction with pre-computed strategy variants
+3. **Scheduler** — Token budget optimization with intelligent degradation
+4. **Renderer** — Markdown assembly with anti-bloat rules
+
+## Supported Languages
+
+- Rust (`.rs`)
+- Python (`.py`)
+- TypeScript/JavaScript (`.ts`, `.tsx`, `.js`, `.jsx`)
+- Go (`.go`)
+- C (`.c`, `.h`)
+
+## Installation
+
+### As an Agent Skill
+
+Install this skill for use with AI coding agents:
+
+```bash
+npx skills add longcipher/ast-doc
+```
+
+### From Source
+
+```bash
+cargo install --path bin/ast-doc
+```
+
+### From crates.io
+
+```bash
+cargo install ast-doc
+```
 
 ## Features
 
-- `bin/` + `crates/` workspace layout
-- CLI example (`bin/cli-app`) using `clap`
-- Shared library example (`crates/common`)
-- BDD acceptance tests with Gherkin + `cucumber-rs`
-- TDD inner loop with `cargo test`
-- Property tests with `proptest` inside the normal `cargo test` flow
-- Optional fuzzing with `cargo-fuzz` for parser-like, protocol, or `unsafe`-heavy crates
-- Optional benchmarks with Criterion for performance-sensitive crates
-- Strict workspace lint configuration
-- `just` commands for format/lint/test/bdd/build
+- **Four-stage pipeline**: Ingestion → AST Parser → Token Scheduler → Renderer
+- **Output strategies**: Full, NoTests (strip tests), Summary (signatures only)
+- **Token budget management**: Configurable `--max-tokens` with automatic degradation
+- **Core file protection**: Mark files with `--core` patterns that never get degraded
+- **Git context**: Automatic branch, commit, and diff inclusion (disable with `--no-git`)
+- **Directory tree**: Visual project structure with language annotations (disable with `--no-tree`)
+- **Glob filtering**: Include/exclude patterns for fine-grained file selection
+- **Anti-bloat rules**: Compress blank lines, trim trailing whitespace
+- **BDD acceptance tests**: Gherkin scenarios with `cucumber-rs`
+- **TDD inner loop**: Unit tests with `cargo test`
+- **Property tests**: `proptest` in the standard test flow
+
+## Usage
+
+### Basic Usage
+
+```bash
+# Generate llms.txt to stdout
+ast-doc .
+
+# Write to a file
+ast-doc . --output llms.txt
+
+# Set token budget (default: 128,000)
+ast-doc . --max-tokens 64000
+```
+
+### Output Strategies
+
+```bash
+# Full source code (default)
+ast-doc . --strategy full
+
+# Strip test modules and functions
+ast-doc . --strategy no-tests
+
+# Signatures only, no implementations
+ast-doc . --strategy summary
+```
+
+### Core Files Protection
+
+```bash
+# Core files always use Full strategy, never degraded
+ast-doc . --core "src/main.rs" --core "src/lib.rs" --strategy summary
+```
+
+### File Filtering
+
+```bash
+# Include only Rust files
+ast-doc . --include "*.rs"
+
+# Exclude test files
+ast-doc . --exclude "*test*"
+
+# Combine include/exclude
+ast-doc . --include "*.rs" --exclude "target/**"
+```
+
+### Git and Tree Options
+
+```bash
+# Skip git context
+ast-doc . --no-git
+
+# Skip directory tree
+ast-doc . --no-tree
+
+# Copy to clipboard (not yet implemented)
+ast-doc . --copy
+```
+
+### Verbose Logging
+
+```bash
+ast-doc . --verbose
+```
 
-## Quick Start
+## Quick Start (Development)
 
 ```bash
 just setup
@@ -24,8 +131,9 @@ just test
 just bdd
 just test-all
 
-# Run the example CLI
-cargo run -p cli-app -- --name Rust
+# Run the CLI
+cargo run -p ast-doc -- --help
+cargo run -p ast-doc -- .
 ```
 
 ## Testing Matrix
@@ -93,4 +201,4 @@ That keeps the default template lean while still pointing parser-like, protocol,
 
 ## License
 
-MIT
+Apache-2.0

SKILL.md

@@ -0,0 +1,114 @@
+---
+name: ast-doc
+description: AST-powered code documentation tool for generating optimized llms.txt files from codebases. Use this skill when users need to generate LLM-optimized documentation, reduce token usage when feeding code to AI models, create structured Markdown summaries of code repositories, extract code architecture and public interfaces, or prepare code context for AI-assisted development.
+metadata:
+  author: longcipher
+  version: "0.1.0"
+---
+
+# AST Documentation Generator
+
+Generate optimized `llms.txt` files from codebases using AST-based semantic parsing.
+
+## Triggers
+
+Use this skill when users ask to:
+
+- Generate llms.txt or LLM-optimized documentation
+- Document a codebase for AI consumption
+- Create code summaries or architecture overviews
+- Optimize code context for AI models
+- Use code2prompt or similar tools
+
+## Installation
+
+Install the skill:
+
+```bash
+npx skills add longcipher/ast-doc
+```
+
+Install the CLI tool:
+
+```bash
+cargo install ast-doc
+```
+
+## Basic Usage
+
+```bash
+# Generate documentation from current directory
+ast-doc .
+
+# With token limit and output file
+ast-doc . --max-tokens 30000 --output llms.txt
+
+# Summary mode for quick overview
+ast-doc . --strategy summary --output summary.md
+
+# Protect core files from degradation
+ast-doc . --max-tokens 50000 --core "src/core/**" --output docs.txt
+
+# Copy to clipboard
+ast-doc . --copy
+```
+
+## Options
+
+| Option | Description |
+|--------|-------------|
+| `-o, --output <FILE>` | Output file path (default: stdout) |
+| `-m, --max-tokens <NUM>` | Maximum token budget (default: 100000) |
+| `-c, --core <PATTERN>` | Glob pattern for core files (never degrade) |
+| `-s, --strategy <STRATEGY>` | Default strategy: `full`, `no-tests`, or `summary` |
+| `--include <PATTERN>` | Include file patterns |
+| `--exclude <PATTERN>` | Exclude file patterns |
+| `--no-git` | Skip git context |
+| `--no-tree` | Skip directory tree |
+| `--copy` | Copy output to clipboard |
+| `-v, --verbose` | Verbose output |
+
+## Output Strategies
+
+1. **Full Mode**: Complete source code preservation
+2. **NoTests Mode**: Removes test modules and test functions
+3. **Summary Mode**: Extracts only public interfaces, signatures, and docstrings
+
+## Workflow
+
+1. Check if ast-doc is installed:
+
+   ```bash
+   which ast-doc
+   ```
+
+2. If not installed, install via cargo:
+
+   ```bash
+   cargo install ast-doc
+   ```
+
+3. Run ast-doc with appropriate options:
+
+   ```bash
+   ast-doc /path/to/project --max-tokens 50000 --output llms.txt
+   ```
+
+4. Review the generated report showing optimization statistics
+
+## Output Format
+
+The tool generates a standard `llms.txt` Markdown file containing:
+
+- Repository name and description
+- Directory tree with strategy annotations
+- Git context (branch, commits, changes)
+- Source files with their processing strategy and token counts
+
+## Tips
+
+- Use `--core` to protect critical files from being summarized
+- Start with higher `--max-tokens` values and decrease as needed
+- Use `--strategy summary` for quick architectural overviews
+- Combine with `--no-git` and `--no-tree` for pure code output
+- Check the optimization report to understand token savings

bin/ast-doc/Cargo.toml

@@ -5,7 +5,7 @@ edition.workspace = true
 description = "CLI for generating optimized llms.txt documentation from codebases"
 
 [dependencies]
-ast-doc-core = { path = "../../crates/ast-doc-core" }
+ast-doc-core = { workspace = true }
 clap = { workspace = true, features = ["derive"] }
 eyre = { workspace = true }
 tracing = { workspace = true }