Skip to content

Commit 744f6a8

Browse files
authored
fix: Fix another compatibility issue for unstructured-client==0.26.0 (#79)
* fix: Fix another compatibility issue for unstructured-client==0.26.0 The internals of the client have changed, and the `PartitionParameters` class is now a Pydantic model. We were using `dataclass.fields` to get a list of available params, which will break against the new client. For compatibility, let's try to access `PartitionParameters.__fields`, and fall back to the dataclass approach for clients < 0.26.0. Testing: Try a simple ingest command with the old and new clients: ``` pip install unstructured-client==0.25.5 unstructured-ingest local --input-path /path/to/some/files --output-dir local-ingest-output --num-processes 2 --recursive --partition-by-api --partition-endpoint https://api.unstructuredapp.io/general/v0/general --api-key $MY_SERVERLESS_KEY pip install git+https://github.com/Unstructured-IO/unstructured-python-client Traceback (most recent call last): File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/v2/pipeline/interfaces.py", line 158, in run_async return await self._run_async(fn=fn, **kwargs) File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/v2/pipeline/steps/partition.py", line 66, in _run_async partitioned_content = await fn(**fn_kwargs) File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/v2/processes/partitioner.py", line 215, in run_async return await self.partition_via_api(filename, metadata=metadata, **kwargs) File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/utils/dep_check.py", line 50, in wrapper_async return await func(*args, **kwargs) File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/v2/processes/partitioner.py", line 200, in partition_via_api partition_params = self.create_partition_parameters(filename=filename) File "/Users/austin/.pyenv/versions/3.9.16/envs/unstructured-ingest/lib/python3.9/site-packages/unstructured_ingest/v2/processes/partitioner.py", line 167, in create_partition_parameters possible_fields = [f.name for f in fields(PartitionParameters)] File "/Users/austin/.pyenv/versions/3.9.16/lib/python3.9/dataclasses.py", line 1035, in fields raise TypeError('must be called with a dataclass type or instance') pip install -e . ``` * Cut the 0.0.7 release
1 parent 6dcdae8 commit 744f6a8

File tree

3 files changed

+11
-3
lines changed

3 files changed

+11
-3
lines changed

CHANGELOG.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
## 0.0.7-dev3
1+
## 0.0.7
22

33
### Enhancements
44

@@ -8,6 +8,7 @@
88
### Fixes
99

1010
* **Weaviate access configs access** Weaviate access config uses pydantic Secret and it needs to be resolved to the secret value when being used. This was fixed.
11+
* **unstructured-client compatibility fix** Fix an error when accessing the fields on `PartitionParameters` in the new 0.26.0 Python client.
1112

1213
## 0.0.6
1314

unstructured_ingest/__version__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "0.0.7-dev3" # pragma: no cover
1+
__version__ = "0.0.7" # pragma: no cover

unstructured_ingest/v2/processes/partitioner.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,14 @@ def create_partition_parameters(self, filename: Path) -> "PartitionParameters":
164164
from unstructured_client.models.shared import Files, PartitionParameters
165165

166166
partition_request = self.config.to_partition_kwargs()
167-
possible_fields = [f.name for f in fields(PartitionParameters)]
167+
168+
# Note(austin): PartitionParameters is a Pydantic model in v0.26.0
169+
# Prior to this it was a dataclass which doesn't have .__fields
170+
try:
171+
possible_fields = PartitionParameters.__fields__
172+
except AttributeError:
173+
possible_fields = [f.name for f in fields(PartitionParameters)]
174+
168175
filtered_partition_request = {
169176
k: v for k, v in partition_request.items() if k in possible_fields
170177
}

0 commit comments

Comments
 (0)