Skip to content

Commit cbe04e3

Browse files
jxnlclaudecursoragent
authored
Standardize provider factory methods in codebase (#1898)
Co-authored-by: Claude <[email protected]> Co-authored-by: Cursor Agent <[email protected]>
1 parent 04ff85e commit cbe04e3

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

75 files changed

+1503
-4138
lines changed

‎.github/workflows/evals.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,6 @@ jobs:
2828
run: uv sync --all-extras --dev
2929

3030
- name: Run all tests
31-
run: uv run pytest tests/
31+
run: uv run pytest tests/ --asyncio-mode=auto
3232
env:
3333
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

‎.github/workflows/test.yml‎

Lines changed: 110 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,11 @@ jobs:
2222
- name: Install the project
2323
run: uv sync --all-extras
2424
- name: Run core tests
25-
run: uv run pytest tests/ -k 'not llm and not openai and not gemini and not anthropic and not cohere and not vertexai and not mistral and not xai and not docs'
25+
run: >-
26+
uv run pytest tests/ --asyncio-mode=auto -n auto
27+
-k 'not test_core_providers and not test_openai and not test_anthropic
28+
and not test_gemini and not test_genai and not test_writer and not
29+
test_vertexai and not docs'
2630
env:
2731
INSTRUCTOR_ENV: CI
2832
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
@@ -31,15 +35,110 @@ jobs:
3135
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
3236
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
3337

38+
# Core provider tests for OpenAI
39+
core-openai:
40+
name: Core Provider Tests (OpenAI)
41+
runs-on: ubuntu-latest
42+
needs: core-tests
43+
44+
steps:
45+
- uses: actions/checkout@v2
46+
- name: Install uv
47+
uses: astral-sh/setup-uv@v4
48+
with:
49+
enable-cache: true
50+
- name: Set up Python
51+
run: uv python install 3.11
52+
- name: Install the project
53+
run: uv sync --all-extras
54+
- name: Run core provider tests (OpenAI)
55+
run: uv run pytest tests/llm/test_core_providers -v --asyncio-mode=auto -n auto -k "openai"
56+
env:
57+
INSTRUCTOR_ENV: CI
58+
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
59+
60+
# Core provider tests for Anthropic
61+
core-anthropic:
62+
name: Core Provider Tests (Anthropic)
63+
runs-on: ubuntu-latest
64+
needs: core-tests
65+
66+
steps:
67+
- uses: actions/checkout@v2
68+
- name: Install uv
69+
uses: astral-sh/setup-uv@v4
70+
with:
71+
enable-cache: true
72+
- name: Set up Python
73+
run: uv python install 3.11
74+
- name: Install the project
75+
run: uv sync --all-extras
76+
- name: Run core provider tests (Anthropic)
77+
run: uv run pytest tests/llm/test_core_providers -v --asyncio-mode=auto -n auto -k "anthropic"
78+
env:
79+
INSTRUCTOR_ENV: CI
80+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
81+
82+
# Core provider tests for Google
83+
core-google:
84+
name: Core Provider Tests (Google)
85+
runs-on: ubuntu-latest
86+
needs: core-tests
87+
88+
steps:
89+
- uses: actions/checkout@v2
90+
- name: Install uv
91+
uses: astral-sh/setup-uv@v4
92+
with:
93+
enable-cache: true
94+
- name: Set up Python
95+
run: uv python install 3.11
96+
- name: Install the project
97+
run: uv sync --all-extras
98+
- name: Run core provider tests (Google)
99+
run: uv run pytest tests/llm/test_core_providers -v --asyncio-mode=auto -n auto -k "google"
100+
env:
101+
INSTRUCTOR_ENV: CI
102+
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
103+
104+
# Core provider tests for other providers
105+
core-other:
106+
name: Core Provider Tests (Other)
107+
runs-on: ubuntu-latest
108+
needs: core-tests
109+
110+
steps:
111+
- uses: actions/checkout@v2
112+
- name: Install uv
113+
uses: astral-sh/setup-uv@v4
114+
with:
115+
enable-cache: true
116+
- name: Set up Python
117+
run: uv python install 3.11
118+
- name: Install the project
119+
run: uv sync --all-extras
120+
- name: Run core provider tests (Cohere, xAI, Mistral, etc)
121+
run: uv run pytest tests/llm/test_core_providers -v --asyncio-mode=auto -n auto -k "cohere or xai or mistral or cerebras or fireworks or writer or perplexity"
122+
env:
123+
INSTRUCTOR_ENV: CI
124+
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
125+
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
126+
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
127+
CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }}
128+
FIREWORKS_API_KEY: ${{ secrets.FIREWORKS_API_KEY }}
129+
WRITER_API_KEY: ${{ secrets.WRITER_API_KEY }}
130+
PERPLEXITY_API_KEY: ${{ secrets.PERPLEXITY_API_KEY }}
131+
34132
# Provider tests run in parallel
35133
provider-tests:
36134
name: ${{ matrix.provider.name }} Tests
37135
runs-on: ubuntu-latest
136+
needs: [core-openai, core-anthropic, core-google, core-other]
38137
strategy:
39138
fail-fast: false
40139
matrix:
41140
provider:
42-
- name: Openai
141+
- name: OpenAI
43142
env_key: OPENAI_API_KEY
44143
test_path: tests/llm/test_openai
45144
- name: Anthropic
@@ -51,12 +150,12 @@ jobs:
51150
- name: Google GenAI
52151
env_key: GOOGLE_API_KEY
53152
test_path: tests/llm/test_genai
54-
- name: Cohere
55-
env_key: COHERE_API_KEY
56-
test_path: tests/llm/test_cohere
57-
- name: XAI
58-
env_key: XAI_API_KEY
59-
test_path: tests/llm/test_xai
153+
- name: Vertex AI
154+
env_key: GOOGLE_API_KEY
155+
test_path: tests/llm/test_vertexai
156+
- name: Writer
157+
env_key: WRITER_API_KEY
158+
test_path: tests/llm/test_writer
60159

61160
steps:
62161
- uses: actions/checkout@v2
@@ -69,7 +168,7 @@ jobs:
69168
- name: Install the project
70169
run: uv sync --all-extras
71170
- name: Run ${{ matrix.provider.name }} tests
72-
run: uv run pytest ${{ matrix.provider.test_path }} -n auto
171+
run: uv run pytest ${{ matrix.provider.test_path }} --asyncio-mode=auto -n auto
73172
env:
74173
INSTRUCTOR_ENV: CI
75174
${{ matrix.provider.env_key }}: ${{ secrets[matrix.provider.env_key] }}
@@ -78,6 +177,7 @@ jobs:
78177
auto-client-test:
79178
name: Auto Client Tests
80179
runs-on: ubuntu-latest
180+
needs: [core-openai, core-anthropic, core-google, core-other]
81181

82182
steps:
83183
- uses: actions/checkout@v2
@@ -90,7 +190,7 @@ jobs:
90190
- name: Install the project
91191
run: uv sync --all-extras
92192
- name: Run Auto Client tests
93-
run: uv run pytest tests/test_auto_client.py
193+
run: uv run pytest tests/test_auto_client.py --asyncio-mode=auto -n auto
94194
env:
95195
INSTRUCTOR_ENV: CI
96196
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

‎.github/workflows/test_docs.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,6 @@ jobs:
3131
- name: Install the project
3232
run: uv sync --all-extras
3333
- name: Run tests
34-
run: uv run pytest tests/docs
34+
run: uv run pytest tests/docs --asyncio-mode=auto
3535
env:
3636
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

‎CLAUDE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
66

77
## Commands
88
- Install deps: `uv pip install -e ".[dev,anthropic]"` or `poetry install --with dev,anthropic`
9-
- Run tests: `uv run pytest tests/`
9+
- Run tests: `uv run pytest tests/ -n auto`
1010
- Run specific test: `uv run pytest tests/path_to_test.py::test_name`
1111
- Skip LLM tests: `uv run pytest tests/ -k 'not llm and not openai'`
1212
- Type check: `uv run ty check`

‎docs/integrations/google.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -275,6 +275,16 @@ for user in users:
275275
#> name='Mike' age=28
276276
```
277277

278+
## Known Limitations (as of Nov 12, 2024)
279+
280+
Google Gemini has the following known limitations when used with Instructor:
281+
282+
1. **Union Types**: Gemini does not support Union types (except for Optional). Use separate response models or Literal types instead.
283+
2. **Enum Types**: Gemini returns string values instead of properly typed Enum instances. You may need to manually convert strings to enums after extraction.
284+
3. **Union Streaming**: Streaming is not supported for Union types with Iterable.
285+
286+
These limitations are specific to Google Gemini and do not affect other providers like OpenAI or Anthropic. Tests automatically skip these features for Google to prevent failures.
287+
278288
## Instructor Modes
279289

280290
We provide several modes to make it easy to work with the different response models that Gemini supports:

‎instructor/processing/function_calls.py‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -399,10 +399,7 @@ def parse_anthropic_json(
399399
# read: https://docs.anthropic.com/en/docs/build-with-claude/tool-use/web-search-tool#response
400400
text_blocks = [c for c in completion.content if c.type == "text"]
401401
last_block = text_blocks[-1]
402-
# Strip raw control characters (0x00-0x1F) that would cause json.loads to fail
403-
# Note: This preserves escaped sequences like \n in JSON strings, which are handled
404-
# correctly by the JSON parser. Only raw, unescaped control bytes are removed.
405-
text = re.sub(r"[\u0000-\u001F]", "", last_block.text)
402+
text = last_block.text
406403

407404
extra_text = extract_json_from_codeblock(text)
408405

@@ -411,7 +408,7 @@ def parse_anthropic_json(
411408
extra_text, context=validation_context, strict=True
412409
)
413410
else:
414-
# Allow control characters.
411+
# Allow control characters to pass through by using the non-strict JSON parser.
415412
parsed = json.loads(extra_text, strict=False)
416413
# Pydantic non-strict: https://docs.pydantic.dev/latest/concepts/strict_mode/
417414
model = cls.model_validate(parsed, context=validation_context, strict=False)

0 commit comments

Comments
 (0)