Skip to content

Commit 0503713

Browse files
AlmostBald-TRADINGAlmostBald-TRADING
committed
feat: align CI configuration and document environment variables
CI Configuration Alignment: - Consolidate CI workflows into single ci.yml with 4 jobs (lint, test, security, pre-commit) - Standardize Python version to 3.12 across all CI jobs - Add full pytest suite to CI with proper environment variables - Enforce strict security policy (fail on security issues) - Align linting scope to match local development (src, clients/python, tests) - Fix CF_API_KEY requirement with testing mode detection - Add asyncio configuration to pytest.ini Environment Variables Documentation: - Create comprehensive .env.example with all 17 environment variables - Document variables by category (Core, Embeddings, Backfill, Search, Client) - Update configuration guide with .env usage instructions - Add environment setup to development quick start guide - Update README with environment configuration section - Fix .gitignore to allow .env.example while excluding personal .env files Testing Improvements: - Add session-scoped test environment setup in conftest.py - Update Makefile smoke test to use test environment variables - All tests now pass without manual CF_API_KEY setup Breaking Changes: None - Maintains backward compatibility - All existing functionality preserved - Enhanced developer experience with better documentation
1 parent 54aa760 commit 0503713

25 files changed

+896
-263
lines changed

‎.env.example‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# ================================
2+
# ContextForge Memory Configuration
3+
# ================================
4+
# Copy this file to .env and customize for your environment
5+
# .env files are gitignored and will not be committed
6+
7+
# ====================
8+
# Core Configuration
9+
# ====================
10+
11+
# API Key (REQUIRED for production, auto-set for testing)
12+
# Set this to a secure random string for production use
13+
CF_API_KEY=your-api-key-here
14+
15+
# Data directory for persistent storage
16+
# Relative to the application root directory
17+
CF_MEMORY_DATA=./data
18+
19+
# Enable v1 embedding-based features
20+
# Set to "false" to disable v1 endpoints and use v0 only
21+
CF_ENABLE_V1=true
22+
23+
# Testing mode (automatically detected when running pytest)
24+
# Only set manually for testing outside of pytest
25+
# TESTING=true
26+
27+
# Session checkpoint TTL in seconds (1 hour default)
28+
# How long session data is kept in memory
29+
CF_EPHEMERAL_TTL_SECONDS=3600
30+
31+
# ====================
32+
# Embeddings Configuration
33+
# ====================
34+
35+
# Embeddings provider: "hash", "st", "openai"
36+
# - "hash": Fast hash-based embeddings (default, no external dependencies)
37+
# - "st": SentenceTransformers embeddings (requires sentence-transformers)
38+
# - "openai": OpenAI embeddings (requires OpenAI API key)
39+
CF_EMBEDDINGS_PROVIDER=hash
40+
41+
# SentenceTransformers model name
42+
# Only used when CF_EMBEDDINGS_PROVIDER=st
43+
CF_ST_MODEL=sentence-transformers/all-MiniLM-L6-v2
44+
45+
# OpenAI embedding model
46+
# Only used when CF_EMBEDDINGS_PROVIDER=openai
47+
CF_OPENAI_EMBED_MODEL=text-embedding-3-small
48+
49+
# OpenAI API key for embeddings and summarization
50+
# Required for OpenAI-based embeddings and summarization
51+
CF_OPENAI_API_KEY=sk-your-openai-api-key-here
52+
53+
# Standard OpenAI API key (alternative variable name)
54+
# Used in example files and client code
55+
OPENAI_API_KEY=sk-your-openai-api-key-here
56+
57+
# ====================
58+
# Backfill Configuration
59+
# ====================
60+
61+
# Maximum number of items to backfill during startup
62+
# Prevents excessive memory usage on large datasets
63+
CF_MAX_BACKFILL_ITEMS=1000
64+
65+
# Backfill operation timeout in seconds
66+
# How long to wait for backfill operations to complete
67+
CF_BACKFILL_TIMEOUT=30
68+
69+
# Batch size for backfill operations
70+
# Number of items processed in each batch
71+
CF_BACKFILL_BATCH_SIZE=100
72+
73+
# Maximum retry attempts for backfill operations
74+
# Number of times to retry failed backfill operations
75+
CF_BACKFILL_MAX_RETRIES=3
76+
77+
# Base retry delay in seconds for backfill operations
78+
# Delay between retry attempts (with exponential backoff)
79+
CF_BACKFILL_RETRY_DELAY=1
80+
81+
# ====================
82+
# Search Optimization
83+
# ====================
84+
85+
# File size warning threshold in bytes (50MB default)
86+
# Warns when data file exceeds this size
87+
CF_FILE_SIZE_THRESHOLD=52428800
88+
89+
# Embedding cache size
90+
# Number of embeddings to keep in memory cache
91+
CF_EMBEDDING_CACHE_SIZE=10000
92+
93+
# ====================
94+
# Client Configuration
95+
# ====================
96+
97+
# Base URL for ContextForge API client
98+
# Used by Python and TypeScript clients
99+
CF_BASE_URL=http://localhost:8085
100+
101+
# Client timeout in seconds
102+
# HTTP client timeout for API requests
103+
CF_TIMEOUT=30.0
104+
105+
# Maximum retry attempts for client requests
106+
# Number of times to retry failed API requests
107+
CF_MAX_RETRIES=3

‎.github/workflows/ci.yml‎

Lines changed: 65 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -11,24 +11,53 @@ permissions:
1111
pull-requests: write
1212

1313
jobs:
14-
test:
14+
lint:
1515
runs-on: ubuntu-latest
1616
steps:
1717
- uses: actions/checkout@v4
1818
- uses: actions/setup-python@v5
1919
with:
20-
python-version: '3.11'
20+
python-version: '3.12'
2121
- name: Install dependencies
2222
run: |
2323
python -m pip install --upgrade pip
2424
pip install -r requirements.txt
25+
pip install -r requirements-dev.txt
2526
- name: Lint
2627
run: |
27-
ruff check src
28-
black --check src
28+
ruff check src clients/python tests
29+
black --check src clients/python tests
30+
31+
test:
32+
runs-on: ubuntu-latest
33+
steps:
34+
- uses: actions/checkout@v4
35+
- uses: actions/setup-python@v5
36+
with:
37+
python-version: '3.12'
38+
- name: Install dependencies
39+
run: |
40+
python -m pip install --upgrade pip
41+
pip install -r requirements.txt
42+
pip install -r requirements-dev.txt
43+
- name: Run tests
44+
env:
45+
CF_API_KEY: test-key
46+
TESTING: true
47+
run: pytest tests/ -v
48+
49+
security:
50+
runs-on: ubuntu-latest
51+
steps:
52+
- uses: actions/checkout@v4
53+
- uses: actions/setup-python@v5
54+
with:
55+
python-version: '3.12'
56+
- name: Install dependencies
57+
run: |
58+
python -m pip install --upgrade pip
59+
pip install -r requirements.txt
2960
- name: Security audit
30-
id: security-audit
31-
continue-on-error: true
3261
run: |
3362
safety scan --json > safety-results.json
3463
pip-audit --desc --format=json --output=audit-results.json
@@ -40,45 +69,34 @@ jobs:
4069
path: |
4170
safety-results.json
4271
audit-results.json
43-
- name: Security audit failure notification
44-
if: failure() && steps.security-audit.outcome == 'failure'
45-
run: |
46-
echo "::warning::Security audit failed! Check the security-audit-results artifact for details."
47-
echo "::group::Safety scan results"
48-
if [ -f safety-results.json ]; then
49-
cat safety-results.json
50-
else
51-
echo "safety-results.json not found"
52-
fi
53-
echo "::endgroup::"
54-
echo "::group::Pip audit results"
55-
if [ -f audit-results.json ]; then
56-
cat audit-results.json
57-
else
58-
echo "audit-results.json not found"
59-
fi
60-
echo "::endgroup::"
61-
- name: Import smoke test
62-
run: |
63-
python - << 'PY'
64-
import importlib
65-
import sys
66-
sys.path.append('src')
67-
import contextforge_memory
68-
from contextforge_memory.main import app
69-
print('Import smoke test: OK')
70-
PY
71-
- name: Basic functionality test
72-
run: |
73-
python - << 'PY'
74-
import sys
75-
sys.path.append('src')
76-
from contextforge_memory.main import app, _embed_text
7772
78-
# Test embedding function
79-
test_text = "test embedding"
80-
embedding = _embed_text(test_text)
81-
assert len(embedding) == 32, f"Expected 32 dimensions, got {len(embedding)}"
82-
assert all(0 <= x <= 1 for x in embedding), "Embeddings should be normalized to [0,1]"
83-
print('Basic functionality test: OK')
84-
PY
73+
pre-commit:
74+
runs-on: ubuntu-latest
75+
steps:
76+
- uses: actions/checkout@v4
77+
- uses: actions/setup-python@v5
78+
with:
79+
python-version: '3.12'
80+
- name: Enable pip cache
81+
uses: actions/cache@v4
82+
with:
83+
path: |
84+
~/.cache/pip
85+
~/.cache/pip-audit
86+
key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
87+
restore-keys: |
88+
${{ runner.os }}-pip-
89+
- name: Cache pre-commit
90+
uses: actions/cache@v4
91+
with:
92+
path: ~/.cache/pre-commit
93+
key: ${{ runner.os }}-precommit-${{ hashFiles('.pre-commit-config.yaml') }}
94+
restore-keys: |
95+
${{ runner.os }}-precommit-
96+
- name: Install dependencies
97+
run: |
98+
python -m pip install --upgrade pip
99+
pip install -r requirements.txt
100+
pip install pre-commit pip-audit
101+
- name: Run pre-commit
102+
run: pre-commit run --all-files --show-diff-on-failure

‎.github/workflows/pre-commit.yml‎

Lines changed: 0 additions & 51 deletions
This file was deleted.

‎.gitignore‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
__pycache__/
22
.venv/
33
.env
4-
.env.*
4+
.env.local
5+
.env.development
6+
.env.production
7+
.env.test
58
.DS_Store
69
.pytest_cache/
710
.mypy_cache/

0 commit comments

Comments
 (0)