Skip to content

Commit 3454761

Browse files
python packaged server tests (FF-3494) (#88)
* python packaged server tests (FF-3494) * python does not support bandits or dynamic types * run package test on PR
1 parent 84b4e18 commit 3454761

File tree

11 files changed

+263
-9
lines changed

11 files changed

+263
-9
lines changed

.github/workflows/test-sdk-packages.yml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ on:
44
push:
55
branches:
66
- main
7+
pull_request:
78
workflow_dispatch:
89

910
jobs:
@@ -18,3 +19,15 @@ jobs:
1819
sdkName: 'eppo/php-sdk'
1920
sdkRelayDir: 'php-sdk-relay'
2021
secrets: inherit
22+
23+
test-python-sdk:
24+
strategy:
25+
fail-fast: false
26+
matrix:
27+
platform: ['linux']
28+
uses: ./.github/workflows/test-server-sdk.yml
29+
with:
30+
platform: ${{ matrix.platform }}
31+
sdkName: 'eppo/python-sdk'
32+
sdkRelayDir: 'python-sdk-relay'
33+
secrets: inherit

.github/workflows/test-server-sdk.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,9 @@ jobs:
4949
5050
5151
- name: "Checkout"
52-
uses: actions/checkout@v3
52+
uses: actions/checkout@v4
53+
with:
54+
ref: ${{ github.ref }}
5355

5456
# Set up docker (macos runners)
5557
- id: setup-docker

package-testing/php-sdk-relay/docker-run.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ docker remove php-relay
1212

1313
docker build . -t Eppo-exp/php-sdk-relay:$VERSION
1414

15-
docker run -p $SDK_RELAY_PORT:$SDK_RELAY_PORT \
15+
docker run -p $SDK_RELAY_PORT:$SDK_RELAY_PORT \
1616
--add-host host.docker.internal:host-gateway \
1717
-e SDK_REF \
1818
-e EPPO_BASE_URL \
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
bin
2+
lib
3+
venv
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
FROM python:3.12
2+
3+
WORKDIR /app
4+
5+
COPY requirements.txt .
6+
RUN pip install -r requirements.txt
7+
8+
# Copy the source code
9+
COPY src/ ./src/
10+
11+
CMD ["python", "/app/src/server.py"]
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Python Testing Server
2+
3+
Post test case files to this server and check the results against what's expected.
4+
5+
## Running locally with Docker
6+
7+
Build the docker image:
8+
9+
```shell
10+
docker build -t Eppo-exp/python-sdk-relay .
11+
```
12+
13+
Run the docker container:
14+
15+
```shell
16+
./docker-run.sh
17+
```
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
#!/usr/bin/env bash
2+
3+
# Default is to use the latest build
4+
VERSION="${1:-latest}"
5+
6+
echo "Starting deployment with version: $VERSION"
7+
8+
if [ -e .env ]; then
9+
echo "Loading environment variables from .env file"
10+
source .env
11+
fi
12+
13+
echo "Stopping existing container..."
14+
docker stop python-relay
15+
echo "Removing existing container..."
16+
docker remove python-relay
17+
18+
echo "Building new image..."
19+
docker build . -t Eppo-exp/python-sdk-relay:$VERSION
20+
21+
echo "Starting new container..."
22+
docker run -p $SDK_RELAY_PORT:$SDK_RELAY_PORT \
23+
--add-host host.docker.internal:host-gateway \
24+
-e SDK_REF \
25+
-e EPPO_BASE_URL \
26+
-e SDK_RELAY_PORT \
27+
--name python-relay \
28+
--rm \
29+
-t Eppo-exp/python-sdk-relay:$VERSION;
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
flask
2+
eppo-server-sdk==4.1.0
Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
import eppo_client
2+
import eppo_client.bandit
3+
4+
from flask import Flask, request, jsonify
5+
from os import environ
6+
from dataclasses import dataclass
7+
from eppo_client.config import Config, AssignmentLogger
8+
9+
app = Flask(__name__)
10+
11+
12+
class LocalAssignmentLogger(AssignmentLogger):
13+
def log_assignment(self, assignment):
14+
print(f"Assignment: {assignment}")
15+
16+
17+
@dataclass
18+
class AssignmentRequest:
19+
flag: str
20+
subject_key: str
21+
subject_attributes: dict
22+
assignment_type: str
23+
default_value: any
24+
25+
26+
@app.route('/', methods=['GET'])
27+
def health_check():
28+
return "OK"
29+
30+
@app.route('/sdk/reset', methods=['POST'])
31+
def reset_sdk():
32+
initialize_client_and_wait()
33+
34+
return "Reset complete"
35+
36+
@app.route('/sdk/details', methods=['GET'])
37+
def get_sdk_details():
38+
return jsonify({
39+
"sdkName": "python-sdk",
40+
"sdkVersion": "4.1.0",
41+
"supportsBandits": False,
42+
"supportsDynamicTyping": False
43+
})
44+
45+
@app.route('/flags/v1/assignment', methods=['POST'])
46+
def handle_assignment():
47+
data = request.json
48+
request_obj = AssignmentRequest(
49+
flag=data['flag'],
50+
subject_key=data['subjectKey'],
51+
subject_attributes=data['subjectAttributes'],
52+
assignment_type=data['assignmentType'],
53+
default_value=data['defaultValue']
54+
)
55+
print(f"Request object: {request_obj}")
56+
57+
client = eppo_client.get_instance()
58+
59+
try:
60+
match request_obj.assignment_type:
61+
case 'BOOLEAN':
62+
result = client.get_boolean_assignment(
63+
request_obj.flag,
64+
request_obj.subject_key,
65+
request_obj.subject_attributes,
66+
bool(request_obj.default_value)
67+
)
68+
case 'INTEGER':
69+
result = client.get_integer_assignment(
70+
request_obj.flag,
71+
request_obj.subject_key,
72+
request_obj.subject_attributes,
73+
int(request_obj.default_value)
74+
)
75+
case 'STRING':
76+
result = client.get_string_assignment(
77+
request_obj.flag,
78+
request_obj.subject_key,
79+
request_obj.subject_attributes,
80+
request_obj.default_value
81+
)
82+
case 'NUMERIC':
83+
result = client.get_numeric_assignment(
84+
request_obj.flag,
85+
request_obj.subject_key,
86+
request_obj.subject_attributes,
87+
float(request_obj.default_value)
88+
)
89+
case 'JSON':
90+
result = client.get_json_assignment(
91+
request_obj.flag,
92+
request_obj.subject_key,
93+
request_obj.subject_attributes,
94+
request_obj.default_value
95+
)
96+
97+
response = {
98+
"result": result,
99+
"assignmentLog": [],
100+
"banditLog": [],
101+
"error": None
102+
}
103+
print(f"response: {response}")
104+
return jsonify(response)
105+
except Exception as e:
106+
print(f"Error processing assignment: {str(e)}")
107+
response = {
108+
"result": None,
109+
"assignmentLog": [],
110+
"banditLog": [],
111+
"error": str(e)
112+
}
113+
return jsonify(response)
114+
115+
@dataclass
116+
class BanditActionRequest:
117+
flag: str
118+
subject_key: str
119+
subject_attributes: dict
120+
actions: list
121+
default_value: any
122+
123+
124+
@app.route('/bandits/v1/action', methods=['POST'])
125+
def handle_bandit():
126+
data = request.json
127+
request_obj = BanditActionRequest(
128+
flag=data['flag'],
129+
subject_key=data['subjectKey'],
130+
subject_attributes=data['subjectAttributes'],
131+
default_value=data['defaultValue'],
132+
actions=data['actions']
133+
)
134+
print(f"Request object: {request_obj}")
135+
136+
# TODO: Implement bandit logic
137+
return jsonify({
138+
"result": "action",
139+
"assignmentLog": [],
140+
"banditLog": [],
141+
"error": None
142+
})
143+
144+
def initialize_client_and_wait():
145+
print("Initializing client")
146+
api_key = environ.get('EPPO_API_KEY', 'NOKEYSPECIFIED')
147+
base_url = environ.get('EPPO_BASE_URL', 'http://localhost:5000/api')
148+
149+
client_config = Config(
150+
api_key=api_key,
151+
base_url=base_url,
152+
assignment_logger=LocalAssignmentLogger()
153+
)
154+
eppo_client.init(client_config)
155+
client = eppo_client.get_instance()
156+
client.wait_for_initialization()
157+
print("Client initialized")
158+
159+
if __name__ == "__main__":
160+
initialize_client_and_wait()
161+
162+
port = int(environ.get('SDK_RELAY_PORT', 7001))
163+
host = environ.get('SDK_RELAY_HOST', '0.0.0.0')
164+
print(f"Starting server on {host}:{port}")
165+
app.run(
166+
host=host,
167+
port=port,
168+
debug=True # Add debug mode
169+
)

package-testing/sdk-test-runner/README.md

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ The following env variable can be set when running the `test-sdk.sh` script
9191

9292
The following components are required to use the the package test runner with a new SDK
9393

94-
1. An **SDK relay server**. This is a REST server running at `localhost:4000` resonding to the [Asssignment and Bandit Request API](#sdk-relay-server)
94+
1. An **SDK relay server**. This is a REST server running at `localhost:4000` responding to the [Asssignment and Bandit Request API](#sdk-relay-server)
9595
1. OR, an **SDK relay client**. This is a client application that connects to the SDK test runner via `socket.io` and responses to [Assignment requests](#sdk-relay-client)
9696
2. Launch Script:
9797
1. A `build-and-run-<platform>.sh` file which fully configures the environment then initiates a [build and run of the relay server application](#build-and-runsh) **using the specified version of the SDK package**. <platform> is one of `linux`, `macos`, or `windows`.
@@ -201,13 +201,11 @@ Any non-empty response
201201

202202
##### SDK Details
203203

204-
`POST /sdk/details`
204+
`GET /sdk/details`
205205

206206
If possible, the SDK relay server should respond with the `sdkName` and `sdkVersion` in use. This may not be directly possible with all SDKs.
207207
If the SDK does not support Bandits or dynamic typing, the test runner will skip the related test cases if the corresponding values are `false`.
208208

209-
`GET /sdk/details`
210-
211209
```ts
212210
// Expected response data:
213211
type SDKDetailsResponse = {

0 commit comments

Comments
 (0)