Skip to content

[Fix] Partners integration tests #644

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dot-agi wants to merge 27 commits into from
Closed

[Fix] Partners integration tests #644

dot-agi wants to merge 27 commits into from

Conversation

@dot-agi
Copy link
Member

@dot-agi dot-agi commented Jan 16, 2025 •
edited
Loading

📥 Partners integration tests

Motivation

  • Adds integration tests for the partners in agentops
  • Fixes tests for the llm providers if they were skipped or flaky

Challenges

  • TaskWeaver has dependency issues with litellm over the jsonschema library version. litellm uses 4.20.00 while TaskWeaver requires 4.22.0
  • TaskWeaver has both a partner and provider integration so we need to add tests for both

Caveats

  • crewAI has bugs that need to be resolved in a separate PR
  • Llama Stack was not part of the original test suite hence is not addressed here

@dot-agi dot-agi marked this pull request as draft January 16, 2025 18:10
@teocns
Copy link
Contributor

teocns commented Jan 16, 2025

this drags commits from another branch that is not yet merged into main?

@dot-agi
Copy link
Member Author

dot-agi commented Jan 16, 2025 •
edited
Loading

this drags commits from another branch that is not yet merged into main?

Except feat/v0.4 - nope

@dot-agi dot-agi force-pushed the fix/partners-integration-tests branch from 9f7e992 to 9f72715 Compare January 16, 2025 18:30
@codecov
Copy link

codecov bot commented Jan 16, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
agentops/llms/providers/ollama.py 0.00% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

dot-agi added 18 commits January 17, 2025 02:13
@dot-agi
add auotgen integration tests
88d59f5
@dot-agi
change vcr config to not check body
b5165d1
@dot-agi
consolidate autogen integration tests into a single test
649500e
@dot-agi
update vcr config to handle dynamic body content in autogen integrati…
f98497f 
…on test
@dot-agi
re-record autogen cassette after new vcr config
1b09285
@dot-agi
Merge branch 'main' into fix/partners-integration-tests
fefb160
@dot-agi
Add code to redact jwt in response body
53129be
@dot-agi
re-record autogen cassette
f1e2932
@dot-agi
update vcr config to handle body response more robustly
4c81469
@dot-agi
re-record autogen cassette (again)
b7f761e
@dot-agi
Merge branch 'main' into fix/partners-integration-tests
403059d
@dot-agi
add taskweaver in provider integration tests and fix litellm integrat…
dd5ed08 
…ion test
@dot-agi
add taskweaver fixture to conftest
b5fc7cc
@dot-agi
fix ai21 version
86d5c6e
@dot-agi
ignore host huggingface.co
ef3a4d3
@dot-agi
add gemini provider integration test
42d9087
@dot-agi
modify model in groq provider integration test
7e8aecd
@dot-agi
re-record groq cassette
1c5f06a
@dot-agi dot-agi requested a review from teocns January 21, 2025 08:26
@dot-agi dot-agi marked this pull request as ready for review January 21, 2025 08:26
@dot-agi
Copy link
Member Author

dot-agi commented Jan 24, 2025

Closed in favor of small PRs.

@dot-agi dot-agi closed this Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@teocns teocns Awaiting requested review from teocns

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@dot-agi @teocns