Skip to content

Commit d987298

Browse files
1broseidon1broseidon
committed
Merge PR #5: Phase 1 - Core Library/API (v0.7.0-alpha)
2 parents 2ec329b + e15ead5 commit d987298

File tree

12 files changed

+2214
-1
lines changed

12 files changed

+2214
-1
lines changed

‎README.md‎

Lines changed: 125 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,8 @@ Perfect for:
121121
- 🐛 **Bug Investigation** — Let AI analyze related files together with proper context
122122
- 🔄 **Code Migration** — Give LLMs full legacy codebase context for refactoring
123123
- 🎯 **Prompt Engineering** — Create consistent, repeatable AI prompts from code
124-
- 🔌 **API Integration** — Generate structured code context for AI-powered dev tools
124+
- 🔌 **Library Integration** — Use the Go API to integrate code extraction into your AI/ML workflows
125+
- 🛠️ **Build Custom Tools** — Embed promptext capabilities in your own applications
125126

126127
---
127128

@@ -220,6 +221,129 @@ This helps you understand the trade-offs and adjust your filters or budget as ne
220221

221222
---
222223

224+
## Using as a Library
225+
226+
`promptext` can be used as a Go library in your own applications, allowing you to programmatically extract code context and integrate it into AI/ML workflows.
227+
228+
### Installation
229+
230+
```bash
231+
go get github.com/1broseidon/promptext/pkg/promptext
232+
```
233+
234+
### Quick Start
235+
236+
```go
237+
package main
238+
239+
import (
240+
"fmt"
241+
"log"
242+
243+
"github.com/1broseidon/promptext/pkg/promptext"
244+
)
245+
246+
func main() {
247+
// Simple extraction
248+
result, err := promptext.Extract(".")
249+
if err != nil {
250+
log.Fatal(err)
251+
}
252+
253+
fmt.Printf("Extracted %d files (%d tokens)\n",
254+
len(result.ProjectOutput.Files),
255+
result.TokenCount)
256+
257+
// Use the formatted output
258+
fmt.Println(result.FormattedOutput)
259+
}
260+
```
261+
262+
### Common Patterns
263+
264+
**Filter by extensions:**
265+
```go
266+
result, err := promptext.Extract(".",
267+
promptext.WithExtensions(".go", ".mod", ".sum"),
268+
promptext.WithExcludes("*_test.go", "vendor/"),
269+
)
270+
```
271+
272+
**AI-optimized extraction with token budget:**
273+
```go
274+
result, err := promptext.Extract(".",
275+
promptext.WithRelevance("auth", "login"),
276+
promptext.WithTokenBudget(8000),
277+
promptext.WithFormat(promptext.FormatPTX),
278+
)
279+
280+
// Send to AI API
281+
sendToAI(result.FormattedOutput)
282+
```
283+
284+
**Reusable extractor:**
285+
```go
286+
extractor := promptext.NewExtractor(
287+
promptext.WithExtensions(".go"),
288+
promptext.WithTokenBudget(5000),
289+
)
290+
291+
result1, _ := extractor.Extract("/project1")
292+
result2, _ := extractor.Extract("/project2")
293+
```
294+
295+
**Format conversion:**
296+
```go
297+
result, _ := promptext.Extract(".", promptext.WithFormat(promptext.FormatPTX))
298+
299+
// Convert to different formats
300+
markdown, _ := result.As(promptext.FormatMarkdown)
301+
jsonl, _ := result.As(promptext.FormatJSONL)
302+
```
303+
304+
### Available Options
305+
306+
- `WithExtensions(extensions ...string)` - Include specific file extensions
307+
- `WithExcludes(patterns ...string)` - Exclude files matching patterns
308+
- `WithGitIgnore(enabled bool)` - Respect .gitignore patterns (default: true)
309+
- `WithDefaultRules(enabled bool)` - Use built-in filtering rules (default: true)
310+
- `WithRelevance(keywords ...string)` - Filter by keyword relevance
311+
- `WithTokenBudget(maxTokens int)` - Limit output to token budget
312+
- `WithFormat(format Format)` - Set output format (PTX, JSONL, Markdown, XML)
313+
- `WithVerbose(enabled bool)` - Enable verbose logging
314+
- `WithDebug(enabled bool)` - Enable debug logging with timing
315+
316+
### Output Formats
317+
318+
- `FormatPTX` - PTX v2.0 (recommended for AI)
319+
- `FormatJSONL` - Machine-friendly JSONL
320+
- `FormatMarkdown` - Human-readable markdown
321+
- `FormatXML` - Machine-parseable XML
322+
323+
### Error Handling
324+
325+
```go
326+
result, err := promptext.Extract("/invalid/path")
327+
if err != nil {
328+
if errors.Is(err, promptext.ErrInvalidDirectory) {
329+
// Handle invalid directory
330+
}
331+
if errors.Is(err, promptext.ErrNoFilesMatched) {
332+
// Handle no matching files
333+
}
334+
}
335+
```
336+
337+
### Examples
338+
339+
See the [examples/](examples/) directory for complete working examples:
340+
- `examples/basic/` - Simple usage patterns
341+
- `examples/token-budget/` - AI-focused extraction with token limits
342+
343+
For full API documentation, see [pkg.go.dev/github.com/1broseidon/promptext/pkg/promptext](https://pkg.go.dev/github.com/1broseidon/promptext/pkg/promptext)
344+
345+
---
346+
223347
## Output Formats
224348

225349
`promptext` supports multiple output formats optimized for different use cases:

‎examples/README.md‎

Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
# Promptext Library Examples
2+
3+
This directory contains example programs demonstrating how to use the promptext library in your Go applications.
4+
5+
## Running the Examples
6+
7+
Each example can be run independently:
8+
9+
```bash
10+
# Basic usage examples
11+
cd basic
12+
go run main.go
13+
14+
# Token budget and relevance filtering
15+
cd token-budget
16+
go run main.go
17+
```
18+
19+
## Examples
20+
21+
### 1. Basic (`basic/`)
22+
23+
Demonstrates fundamental library usage:
24+
- Simple extraction with defaults
25+
- Filtering by file extensions
26+
- Excluding patterns
27+
- Different output formats
28+
- Format conversion
29+
- Saving to files
30+
- Reusable extractors
31+
- Builder pattern
32+
33+
**Run it:**
34+
```bash
35+
cd basic && go run main.go
36+
```
37+
38+
### 2. Token Budget (`token-budget/`)
39+
40+
Shows how to work with token budgets and relevance filtering:
41+
- Setting token budgets for AI model limits
42+
- Relevance-based file filtering
43+
- Combining relevance and token budgets
44+
- Optimizing for different AI models
45+
- Understanding token efficiency
46+
47+
**Run it:**
48+
```bash
49+
cd token-budget && go run main.go
50+
```
51+
52+
## Common Patterns
53+
54+
### Simple Extraction
55+
56+
```go
57+
result, err := promptext.Extract(".")
58+
if err != nil {
59+
log.Fatal(err)
60+
}
61+
fmt.Println(result.FormattedOutput)
62+
```
63+
64+
### With Options
65+
66+
```go
67+
result, err := promptext.Extract(".",
68+
promptext.WithExtensions(".go", ".mod"),
69+
promptext.WithExcludes("vendor/", "*_test.go"),
70+
promptext.WithFormat(promptext.FormatPTX),
71+
)
72+
```
73+
74+
### Relevance Filtering
75+
76+
```go
77+
result, err := promptext.Extract(".",
78+
promptext.WithRelevance("auth", "login"),
79+
promptext.WithTokenBudget(8000),
80+
)
81+
```
82+
83+
### Reusable Extractor
84+
85+
```go
86+
extractor := promptext.NewExtractor(
87+
promptext.WithExtensions(".go"),
88+
promptext.WithTokenBudget(5000),
89+
)
90+
91+
result1, _ := extractor.Extract("/project1")
92+
result2, _ := extractor.Extract("/project2")
93+
```
94+
95+
### Format Conversion
96+
97+
```go
98+
result, _ := promptext.Extract(".", promptext.WithFormat(promptext.FormatPTX))
99+
100+
// Convert to different formats
101+
markdown, _ := result.As(promptext.FormatMarkdown)
102+
jsonl, _ := result.As(promptext.FormatJSONL)
103+
```
104+
105+
## Use Cases
106+
107+
### 1. AI Context Generation
108+
109+
Generate optimized code context for AI assistants:
110+
111+
```go
112+
result, err := promptext.Extract(".",
113+
promptext.WithRelevance("authentication"),
114+
promptext.WithTokenBudget(8000),
115+
promptext.WithFormat(promptext.FormatPTX),
116+
)
117+
// Send result.FormattedOutput to AI
118+
```
119+
120+
### 2. Code Documentation
121+
122+
Extract code for documentation purposes:
123+
124+
```go
125+
result, err := promptext.Extract(".",
126+
promptext.WithExtensions(".go"),
127+
promptext.WithExcludes("*_test.go", "vendor/"),
128+
promptext.WithFormat(promptext.FormatMarkdown),
129+
)
130+
os.WriteFile("docs/codebase.md", []byte(result.FormattedOutput), 0644)
131+
```
132+
133+
### 3. CI/CD Integration
134+
135+
Analyze code in CI pipelines:
136+
137+
```go
138+
result, err := promptext.Extract(".",
139+
promptext.WithFormat(promptext.FormatJSONL),
140+
)
141+
// Parse result.FormattedOutput as JSONL for analysis
142+
```
143+
144+
### 4. Code Search and Analysis
145+
146+
Find relevant code across large codebases:
147+
148+
```go
149+
result, err := promptext.Extract(".",
150+
promptext.WithRelevance("database", "query", "migration"),
151+
)
152+
153+
for _, file := range result.ProjectOutput.Files {
154+
fmt.Printf("%s: %d tokens\n", file.Path, file.Tokens)
155+
}
156+
```
157+
158+
## Available Options
159+
160+
- `WithExtensions(extensions ...string)` - Filter by file extensions
161+
- `WithExcludes(patterns ...string)` - Exclude file patterns
162+
- `WithGitIgnore(enabled bool)` - Respect .gitignore (default: true)
163+
- `WithDefaultRules(enabled bool)` - Use built-in rules (default: true)
164+
- `WithRelevance(keywords ...string)` - Filter by keyword relevance
165+
- `WithTokenBudget(maxTokens int)` - Limit output tokens
166+
- `WithFormat(format Format)` - Set output format
167+
- `WithVerbose(enabled bool)` - Enable verbose logging
168+
- `WithDebug(enabled bool)` - Enable debug logging
169+
170+
## Output Formats
171+
172+
- `FormatPTX` - PTX v2.0 (recommended for AI, TOON-based)
173+
- `FormatTOON` - Alias for PTX (backward compatibility)
174+
- `FormatJSONL` - Machine-friendly JSONL
175+
- `FormatTOONStrict` - TOON v1.3 strict compliance
176+
- `FormatMarkdown` - Human-readable markdown
177+
- `FormatXML` - Machine-parseable XML
178+
179+
## Error Handling
180+
181+
```go
182+
result, err := promptext.Extract("/invalid/path")
183+
if err != nil {
184+
if errors.Is(err, promptext.ErrInvalidDirectory) {
185+
// Handle invalid directory
186+
}
187+
if errors.Is(err, promptext.ErrNoFilesMatched) {
188+
// Handle no matching files
189+
}
190+
// Handle other errors
191+
}
192+
```
193+
194+
## Further Reading
195+
196+
- [Library Documentation](../pkg/promptext/doc.go)
197+
- [Main README](../README.md)
198+
- [API Reference](https://pkg.go.dev/github.com/1broseidon/promptext/pkg/promptext)

0 commit comments

Comments
 (0)