Skip to content

Commit 0816c6f

Browse files
thushanthushan
authored
Merge branch 'main' into dependabot/go_modules/github.com/expr-lang/expr-1.17.8
2 parents 0761384 + 2d763f1 commit 0816c6f

File tree

4 files changed

+1473
-1
lines changed

4 files changed

+1473
-1
lines changed

‎makefile‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ LDFLAGS := -ldflags "\
1818
-X '$(PKG).Tool=$(TOOL)' \
1919
-X '$(PKG).User=$(USER)'"
2020

21-
.PHONY: run clean build test test-verbose test-short test-race test-cover bench version install-deps check-deps vet
21+
.PHONY: run clean build test test-verbose test-short test-race test-cover bench version install-deps check-deps vet test-script-integration
2222

2323
# Build the application with version info
2424
build:
@@ -325,6 +325,11 @@ docker-compose:
325325
@echo "Starting with docker-compose..."
326326
@docker-compose up
327327

328+
# Run integration test scripts (requires running Olla instance)
329+
test-script-integration:
330+
@echo "Running integration test scripts..."
331+
@cd test/scripts && python integration/test-integration.py $(ARGS)
332+
328333
# Show help
329334
help:
330335
@echo "Available targets:"
@@ -367,4 +372,5 @@ help:
367372
@echo " install-deps - Install dependencies at pinned versions"
368373
@echo " check-deps - Check installed tool versions against requirements"
369374
@echo " ci - Run full CI pipeline locally"
375+
@echo " test-script-integration - Run integration test scripts (requires running Olla)"
370376
@echo " help - Show this help"

‎test/scripts/README.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,15 @@ Comprehensive test suites for automated validation.
5757
- Request size limits
5858
- Authentication and authorization
5959

60+
### `/integration` - Comprehensive Integration Tests
61+
End-to-end validation of all major API endpoints against a running Olla instance.
62+
- Health checks and internal monitoring endpoints
63+
- Unified model endpoints and proxy (OpenAI format) endpoints
64+
- Anthropic translator endpoints with passthrough mode validation
65+
- Provider-specific route discovery and testing
66+
- Response header validation and error handling
67+
- Auto-discovers backends and models for dynamic test matrix
68+
6069
### `/passthrough` - Anthropic Passthrough Tests
6170
Validates that the Anthropic Messages API translator correctly selects passthrough or translation mode based on backend capability.
6271
- Auto-discovers available backends and models

‎test/scripts/integration/README.md‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Comprehensive Integration Test Script
2+
3+
Validates all major API endpoints and functionality against a running Olla instance. Gives confidence there are no major regressions before release by auto-discovering backends and models to dynamically build the test matrix.
4+
5+
## What It Tests
6+
7+
The script runs nine phases covering every endpoint category:
8+
9+
| Phase | Area | Tests |
10+
|-------|------|-------|
11+
| 1 | **Health Check** | Olla reachability gate |
12+
| 2 | **Internal/Monitoring** | `/internal/health`, `/internal/status`, endpoints, models, stats, process, `/version` |
13+
| 3 | **Unified Models** | `/olla/models` listing, `/olla/models/{id}` lookup |
14+
| 4 | **Proxy (OpenAI)** | Model list, non-streaming chat, streaming SSE, response headers |
15+
| 5 | **Anthropic Translator** | Model list, non-streaming/streaming messages, token counting, passthrough mode |
16+
| 6 | **Passthrough/Translation** | Mode validation per backend type, SSE event types, response structure, translator stats |
17+
| 7 | **Provider Routes** | Per-discovered-backend `/olla/{provider}/v1/models` |
18+
| 8 | **Response Headers** | Version structure, `X-Olla-Request-ID`, `X-Olla-Response-Time` |
19+
| 9 | **Error Handling** | Non-existent model, invalid body, missing model field |
20+
| 10 | **Summary** | Pass/fail totals grouped by phase |
21+
22+
## Prerequisites
23+
24+
- Python 3.8+
25+
- Olla running with at least one healthy backend
26+
- Dependencies installed (`pip install -r ../requirements.txt`)
27+
28+
## Usage
29+
30+
```bash
31+
# Run full test suite
32+
python test-integration.py
33+
34+
# Custom Olla URL
35+
python test-integration.py --url http://localhost:8080
36+
37+
# Skip streaming tests (faster)
38+
python test-integration.py --skip-streaming
39+
40+
# Skip Anthropic translator tests
41+
python test-integration.py --skip-anthropic
42+
43+
# Skip provider-specific route tests
44+
python test-integration.py --skip-providers
45+
46+
# Show response bodies
47+
python test-integration.py --verbose
48+
49+
# Combine flags
50+
python test-integration.py --url http://myhost:40114 --timeout 60 --skip-streaming
51+
```
52+
53+
## CLI Arguments
54+
55+
| Argument | Default | Description |
56+
|----------|---------|-------------|
57+
| `--url` | `http://localhost:40114` | Olla base URL |
58+
| `--timeout` | `30` | Request timeout in seconds |
59+
| `--skip-streaming` | off | Skip streaming tests |
60+
| `--skip-anthropic` | off | Skip Anthropic translator tests |
61+
| `--skip-providers` | off | Skip provider-specific route tests |
62+
| `--verbose` | off | Show response bodies |
63+
64+
## Example Output
65+
66+
```
67+
========================================================================
68+
Olla Comprehensive Integration Test
69+
Validates all major API endpoints and functionality
70+
========================================================================
71+
72+
Phase 1: Health Check
73+
------------------------------------------------------------------------
74+
[OK] Olla is reachable
75+
76+
Discovering endpoints and models...
77+
[OK] Found 3 endpoint(s)
78+
[OK] Found 5 model(s)
79+
80+
Phase 2: Internal/Monitoring Endpoints
81+
------------------------------------------------------------------------
82+
[PASS] Health endpoint (/internal/health)
83+
[PASS] Status endpoint (/internal/status)
84+
[PASS] Endpoints status (/internal/status/endpoints)
85+
...
86+
87+
========================================================================
88+
Results Summary
89+
========================================================================
90+
8/8 Internal/Monitoring
91+
2/2 Unified Models
92+
4/4 Proxy (OpenAI)
93+
5/5 Anthropic Translator
94+
3/3 Provider Routes
95+
2/2 Response Headers
96+
3/3 Error Handling
97+
98+
Total: 27 | Passed: 27 | Failed: 0
99+
100+
All tests passed.
101+
```
102+
103+
## Exit Codes
104+
105+
| Code | Meaning |
106+
|------|---------|
107+
| `0` | All tests passed |
108+
| `1` | One or more tests failed, or Olla unreachable |
109+
| `130` | User interrupted (Ctrl+C) |

0 commit comments

Comments
 (0)