Skip to content

Commit 09284ca

Browse files
committed
feat: Add initial documentation for CoAiAPy
- Created llms.txt with comprehensive details about CoAiAPy - Included key features, installation instructions, core CLI commands, and advanced workflows - Documented environment configuration and Python integration examples #35
1 parent 5670159 commit 09284ca

File tree

1 file changed

+261
-0
lines changed

1 file changed

+261
-0
lines changed

llms.txt

Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
# CoAiAPy
2+
3+
> Python package for AI workflow automation and observability with revolutionary pipeline templates and environment management
4+
5+
CoAiAPy is a comprehensive Python package that transforms complex AI workflow creation from 30+ minute manual processes into 30-second automated pipelines. It provides audio transcription, synthesis, Redis data stashing, and complete Langfuse integration for AI pipeline observability.
6+
7+
## Key Features
8+
9+
- **Pipeline Templates**: 5 built-in templates with Jinja2 variable substitution and conditional logic
10+
- **Environment Management**: Persistent cross-session variables with `.coaia-env` files
11+
- **Langfuse Integration**: Complete CLI for traces, observations, prompts, and datasets
12+
- **Audio Processing**: AWS Polly transcription and synthesis
13+
- **Redis Integration**: Key-value stashing with TTL support
14+
- **Python 3.6 Compatible**: Works with Pythonista iOS app
15+
16+
## Installation
17+
18+
```bash
19+
pip install coaiapy
20+
```
21+
22+
## Core CLI Commands
23+
24+
### Quick Start
25+
```bash
26+
coaia --help # Show all commands
27+
coaia init # Create sample config file
28+
```
29+
30+
### Pipeline Templates (Revolutionary Feature)
31+
```bash
32+
# List available templates
33+
coaia pipeline list
34+
coaia pipeline list --path --json
35+
36+
# Show template details
37+
coaia pipeline show simple-trace
38+
coaia pipeline show data-pipeline --preview
39+
40+
# Create automated workflows (30+ min → 30 sec)
41+
coaia pipeline create simple-trace --var trace_name="My Process" --var user_id="john"
42+
coaia pipeline create data-pipeline --var pipeline_name="ETL Process" --export-env
43+
44+
# Create custom templates
45+
coaia pipeline init my-template
46+
coaia pipeline init custom-workflow --from data-pipeline --location project
47+
```
48+
49+
### Environment Management
50+
```bash
51+
# Initialize persistent environment
52+
coaia env init # Project .coaia-env
53+
coaia env init --global # User ~/.coaia/global.env
54+
coaia env init --name dev # Named environment
55+
56+
# Manage variables
57+
coaia env set COAIA_USER_ID "john" # Persist to file
58+
coaia env get COAIA_TRACE_ID # Get variable
59+
coaia env list # Show all environments
60+
61+
# Shell integration
62+
eval $(coaia env source --export) # Load into shell
63+
coaia env save --name "my-context" # Save current state
64+
```
65+
66+
### Audio Processing
67+
```bash
68+
coaia transcribe audio.mp3 # Transcribe audio to text
69+
coaia transcribe audio.mp3 -O output.txt
70+
```
71+
72+
### Text Processing
73+
```bash
74+
coaia summarize "long text here" # Summarize text
75+
coaia summarize --f input.txt # Summarize from file
76+
coaia p dictkore "text to correct" # Custom process tags
77+
```
78+
79+
### Redis Operations
80+
```bash
81+
coaia tash mykey "value" # Stash key-value
82+
coaia tash mykey --file data.txt # Stash from file
83+
coaia fetch mykey # Retrieve value
84+
coaia fetch mykey --output save.txt # Save to file
85+
```
86+
87+
### Langfuse Integration (AI Observability)
88+
89+
#### Prompts Management
90+
```bash
91+
coaia fuse prompts list # List all prompts
92+
coaia fuse prompts get MyPrompt # Get latest prompt
93+
coaia fuse prompts get MyPrompt --prod # Production version
94+
coaia fuse prompts get MyPrompt -c # Content only
95+
coaia fuse prompts get MyPrompt -e # Escaped single line
96+
coaia fuse prompts create MyPrompt "content" --labels prod dev
97+
```
98+
99+
#### Datasets Management
100+
```bash
101+
coaia fuse datasets list # List all datasets
102+
coaia fuse datasets get MyDataset # Get dataset with items
103+
coaia fuse datasets get MyDataset --json
104+
coaia fuse datasets get MyDataset -oft > training.jsonl # OpenAI format
105+
coaia fuse datasets get MyDataset -gft > training.jsonl # Gemini format
106+
coaia fuse datasets create NewDataset --description "My dataset"
107+
```
108+
109+
#### Traces & Observations (Production AI Pipeline Support)
110+
```bash
111+
# Create trace with environment export
112+
eval $(coaia fuse traces create $(uuidgen) -s $(uuidgen) -u pipeline-user -n "AI Workflow" --export-env)
113+
114+
# Add observations with shortcuts
115+
coaia fuse traces add-observation $COAIA_TRACE_ID -te -n "Data Loaded" # EVENT
116+
coaia fuse traces add-observation $COAIA_TRACE_ID -ts -n "Processing" # SPAN
117+
coaia fuse traces add-observation $COAIA_TRACE_ID -tg -n "LLM Call" --model "gpt-4" # GENERATION
118+
119+
# Parent-child relationships
120+
eval $(coaia fuse traces add-observation $COAIA_TRACE_ID -ts -n "Main Workflow" --export-env)
121+
coaia fuse traces add-observation $COAIA_TRACE_ID -n "Step 1" --parent $COAIA_LAST_OBSERVATION_ID
122+
123+
# Batch operations
124+
coaia fuse traces add-observations $COAIA_TRACE_ID -f observations.json
125+
cat observations.yaml | coaia fuse traces add-observations $COAIA_TRACE_ID --format yaml
126+
```
127+
128+
#### Score Management
129+
```bash
130+
coaia fuse scores list # List scores
131+
coaia fuse score-configs list # List score configurations
132+
coaia fuse score-configs create "Quality" NUMERIC --min-value 0 --max-value 10
133+
coaia fuse score-configs presets list # Built-in presets
134+
```
135+
136+
## Built-in Pipeline Templates
137+
138+
1. **simple-trace**: Basic monitoring with single observation
139+
2. **data-pipeline**: Multi-step data processing with validation
140+
3. **llm-chain**: LLM interaction pipeline with input/output tracking
141+
4. **parallel-processing**: Concurrent task execution with synchronization
142+
5. **error-handling**: Robust error management with retry mechanisms
143+
144+
## Template Features
145+
146+
- **Jinja2 Templating**: Variable substitution with `{{ variable_name }}`
147+
- **Built-in Functions**: `uuid4()`, `now()`, `timestamp()` available
148+
- **Conditional Steps**: `{% if condition %}...{% endif %}`
149+
- **Parent-Child Hierarchies**: Automatic SPAN observation relationships
150+
- **Auto-Generation**: Trace IDs, observation IDs, timestamps
151+
- **Template Hierarchy**: Project → Global → Built-in discovery
152+
153+
## Environment Configuration
154+
155+
### Required Environment Variables
156+
```bash
157+
# AWS Services (audio transcription/synthesis)
158+
OPENAI_API_KEY
159+
AWS_KEY_ID
160+
AWS_SECRET_KEY
161+
AWS_REGION
162+
163+
# Redis (data stashing)
164+
REDIS_HOST
165+
REDIS_PORT
166+
REDIS_PASSWORD
167+
REDIS_SSL
168+
169+
# Langfuse (AI observability)
170+
LANGFUSE_SECRET_KEY
171+
LANGFUSE_PUBLIC_KEY
172+
LANGFUSE_HOST
173+
```
174+
175+
### Configuration Files
176+
- `~/coaia.json` - Main configuration with process tags and API settings
177+
- `.coaia-env` - Project environment variables (JSON or .env format)
178+
- `~/.coaia/global.env` - Global environment variables
179+
- `.coaia/templates/` - Custom pipeline templates
180+
181+
## Advanced Workflows
182+
183+
### One-Command Pipeline Creation
184+
```bash
185+
# Before: Complex 30+ minute manual setup
186+
export TRACE_ID=$(uuidgen)
187+
export SESSION_ID=$(uuidgen)
188+
coaia fuse traces create $TRACE_ID -u john -s $SESSION_ID
189+
export OBS_ID=$(uuidgen)
190+
coaia fuse traces add-observation $OBS_ID $TRACE_ID -ts -n "Step 1"
191+
# ... repeat for each step ...
192+
193+
# After: 30-second automation
194+
coaia pipeline create data-pipeline \
195+
--var user_id="john" \
196+
--var pipeline_name="ETL Process" \
197+
--export-env
198+
```
199+
200+
### Cross-Session Workflow Persistence
201+
```bash
202+
# Session 1: Start and persist
203+
coaia pipeline create llm-chain --var model="gpt-4" --export-env
204+
coaia env save --name "llm-session"
205+
206+
# Session 2: Resume seamlessly (hours/days later)
207+
eval $(coaia env source --name llm-session --export)
208+
coaia fuse traces add-observation $COAIA_TRACE_ID -n "Continued processing"
209+
```
210+
211+
### Custom Template Creation
212+
```bash
213+
# Create project-specific template
214+
coaia pipeline init company-etl --from data-pipeline --location project
215+
216+
# Template saved to ./.coaia/templates/company-etl.json
217+
# Edit with custom variables and steps
218+
219+
# Use custom template
220+
coaia pipeline create company-etl --var data_source="production_db"
221+
```
222+
223+
## Python Integration
224+
225+
```python
226+
import coaiapy
227+
from coaiapy import EnvironmentManager, TemplateLoader
228+
229+
# Environment management
230+
env_manager = EnvironmentManager()
231+
env_vars = env_manager.load_environment()
232+
233+
# Template loading
234+
loader = TemplateLoader()
235+
template = loader.load_template("data-pipeline")
236+
```
237+
238+
## Development & Building
239+
240+
```bash
241+
make clean # Remove build artifacts
242+
make bump # Auto-increment version
243+
make build # Create wheel and source distributions
244+
make test-release # Full automation: bump + clean + build + upload to TestPyPI
245+
make upload # Upload to production PyPI
246+
```
247+
248+
## Links
249+
250+
- [GitHub Repository](https://github.com/jgwill/coaiapy)
251+
- [PyPI Package](https://pypi.org/project/coaiapy/)
252+
- [Documentation Wiki](https://github.com/jgwill/coaiapy/wiki)
253+
- [Pipeline Templates Guide](https://github.com/jgwill/coaiapy/wiki/Pipeline-Templates)
254+
- [Environment Management Guide](https://github.com/jgwill/coaiapy/wiki/Environment-Management)
255+
- [Langfuse Integration Guide](https://github.com/jgwill/coaiapy/wiki/Langfuse-Integration)
256+
257+
## Python Compatibility
258+
259+
- **Minimum**: Python >=3.6 (Pythonista iOS compatible)
260+
- **Dependencies**: boto3<=1.26.137, mutagen<=1.45.1, redis==5.1.1, jinja2>=2.10
261+
- **Platforms**: Linux, macOS, Windows, iOS (Pythonista)

0 commit comments

Comments
 (0)