Skip to content

Commit b032668

Browse files
carlpaulz
andauthored
Run example unit tests on CI (#65)
Run example unit tests on CI This pull request includes several changes to the test suite and minor code improvements. The most important changes include adding new test cases, updating existing test cases to handle edge cases, and improving logging for better debugging. ### Test Suite Enhancements: * [`.github/workflows/python-tests.yml`](diffhunk://#diff-207f09dfa0bc604a684631a1c78be7d9aa00f9a6c778eec30afa287990be3da1R41-R55): Added steps to run unit tests for examples and specific tests for example 1. * `examples/team_recommender/tests/example_1_text_response/test_compute_cosine_similarity.py`: - Marked `test_reproducing_the_same_text_embedding` as expected to fail due to snapshot mismatch. - Updated `test_cosine_similarity_generated_responses` to use a relative tolerance in the assertion. - Adjusted tolerance levels in `test_embedding_equivalence` to better reflect expected values. * [`examples/team_recommender/tests/example_1_text_response/test_good_fit_for_project.py`](diffhunk://#diff-db487fce77691c75d0084c3052d3e7dac4b1c16e5cc5b77c2d6a7e517a6f539aL115-R124): Updated `test_llm_will_hallucinate_given_no_data` to include an assertion for embedding length and improve stability. [[1]](diffhunk://#diff-db487fce77691c75d0084c3052d3e7dac4b1c16e5cc5b77c2d6a7e517a6f539aL115-R124) [[2]](diffhunk://#diff-db487fce77691c75d0084c3052d3e7dac4b1c16e5cc5b77c2d6a7e517a6f539aR137) ### Code Improvements: * [`examples/team_recommender/tests/example_1_text_response/openai_embeddings.py`](diffhunk://#diff-7e124963dbad8becc0d3cf8af970ffcdb8d3a15f08ec530ac66648503fe787c1L42-R42): Added type annotations to the `create_embedding_object` function. * [`examples/team_recommender/tests/helpers.py`](diffhunk://#diff-b54fa0ba78b35d1e06f5b7ca59c90ca4a88ce1676c58e0dc9f9e89f915b31dbaR87-R91): Replaced `print` statements with `logger.info` for better logging and debugging. [[1]](diffhunk://#diff-b54fa0ba78b35d1e06f5b7ca59c90ca4a88ce1676c58e0dc9f9e89f915b31dbaR87-R91) [[2]](diffhunk://#diff-b54fa0ba78b35d1e06f5b7ca59c90ca4a88ce1676c58e0dc9f9e89f915b31dbaL97-R111) * [`examples/team_recommender/tests/example_3_loop/test_allocations_loop.py`](diffhunk://#diff-a4c3d4f6f1052fc7c10a3a00e82a7d0c06a2192edcb5507fbe3409b9613dd072L46-R49): Reformatted a long assertion message for better readability. ### Minor Changes: * [`examples/team_recommender/tests/helpers.py`](diffhunk://#diff-b54fa0ba78b35d1e06f5b7ca59c90ca4a88ce1676c58e0dc9f9e89f915b31dbaR2): Imported the `logging` module. * [`tests/test_statistical_analysis.py`](diffhunk://#diff-1b63b741bad5a1599ef03397d53a764337af1e7591c991d4a28fba6e9665a1b1L26-R26): Reformatted parameterized test cases for consistency. [[1]](diffhunk://#diff-1b63b741bad5a1599ef03397d53a764337af1e7591c991d4a28fba6e9665a1b1L26-R26) [[2]](diffhunk://#diff-1b63b741bad5a1599ef03397d53a764337af1e7591c991d4a28fba6e9665a1b1L123-R120) --------- Signed-off-by: Paul Zabelin <paulzabelin@artium.ai> Co-authored-by: Paul Zabelin <59230+paulz@users.noreply.github.com> Co-authored-by: Paul Zabelin <paulz@users.noreply.github.com>
1 parent 635e55e commit b032668

File tree

12 files changed

+12356
-6174
lines changed

12 files changed

+12356
-6174
lines changed

.github/workflows/python-tests.yml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,21 @@ jobs:
3838
--color=yes
3939
--verbose --verbosity=10 --capture=no --tb=native --showlocals
4040
41+
- name: Run unit tests for examples
42+
run: >
43+
uv run pytest examples/team_recommender/tests/*.py
44+
--timeout=10
45+
--color=yes
46+
--verbose --verbosity=10 --capture=no --tb=native --showlocals
47+
48+
- name: Run all tests for example 1
49+
run: >
50+
uv run pytest examples/team_recommender/tests/example_1_text_response
51+
--timeout=10
52+
--color=yes
53+
--verbose --verbosity=10 --capture=no --tb=native --showlocals
54+
-k "not test_compute_alignment and not test_reproducing_the_same_text_embedding and not test_response_shows_developer_names and not test_llm_will_hallucinate_given_no_data and not test_cosine_similarity_generated_responses"
55+
4156
- name: Type check Python code
4257
run: uv run mypy src
4358

examples/team_recommender/tests/example_1_text_response/cosine_similarity.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
def compute_cosine_similarity(a: list, b: list) -> float:
55
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
66

7+
78
def compute_alignment(a: list, b: list) -> list:
89
# Calculate the difference vector
910
difference_vector = np.subtract(b, a)

examples/team_recommender/tests/example_1_text_response/openai_embeddings.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ def stabilize_float(x: float) -> float:
3939
return struct.unpack("f", struct.pack("f", x))[0]
4040

4141

42-
def create_embedding_object(text, model="text-embedding-3-small"):
42+
def create_embedding_object(text: str, model="text-embedding-3-small") -> dict:
4343
"""
4444
Create an embedding object with metadata
4545

0 commit comments

Comments
 (0)