fix: raise httpx timeout value for split pdf requests (#168)

We're still seeing some ReadTimeout errors when a pdf is split and sent
for `hi_res` processing. For now, let's raise the default timeout to 30
minutes, and allow for tweaking via the
UNSTRUCTURED_CLIENT_TIMEOUT_MINUTES. Users should not generally need to
adjust this, but it may help us debug their environment. When our split
pdf hook is able to reuse the SDK logic, the client timeout will be
exposed as a parameter, and we can remove this variable.

Other changes:
* Update CI workflow to run against release branches. This 0.25.x branch
should be running tests as long as it sticks around.
* Bump the gen.yaml version to 0.25.7. The next generate/publish job on
this branch will use this.

Author: awalker4

.github/workflows/ci.yaml

@@ -2,9 +2,9 @@ name: CI
 
 on:
   push:
-    branches: [ main ]
+    branches: [ main, release/* ]
   pull_request:
-    branches: [ main ]
+    branches: [ main, release/* ]
   merge_group:
     branches: [ main ]
 

gen.yaml

@@ -10,7 +10,7 @@ generation:
   auth:
     oAuth2ClientCredentialsEnabled: false
 python:
-  version: 0.25.6
+  version: 0.25.7
   additionalDependencies:
     dependencies:
       deepdiff: '>=6.0'

src/unstructured_client/_hooks/custom/split_pdf_hook.py

@@ -4,6 +4,7 @@
 import io
 import logging
 import math
+import os
 from collections.abc import Awaitable
 from typing import Any, Coroutine, Optional, Tuple, Union
 
@@ -252,9 +253,15 @@ def before_request(
                 page_count % split_size,
             )
 
+        # Use a variable to adjust the httpx client timeout, or default to 30 minutes
+        # When we're able to reuse the SDK to make these calls, we can remove this var
+        # The SDK timeout will be controlled by parameter
+        client_timeout_minutes = 30
+        if timeout_var := os.getenv("UNSTRUCTURED_CLIENT_TIMEOUT_MINUTES"):
+            client_timeout_minutes = int(timeout_var)
+
         async def call_api_partial(page):
-            # Individual calls should return within 10 minutes
-            client_timeout = httpx.Timeout(60 * 10)
+            client_timeout = httpx.Timeout(60 * client_timeout_minutes)
             async with httpx.AsyncClient(timeout=client_timeout) as client:
                 try:
                     httpx_response = await request_utils.call_api_async(