Skip to content

Commit ba37ca2

Browse files
committed
fix(ci): pre-download iscc-sct model to prevent race conditions
Add model caching and pre-download steps to prevent parallel test failures caused by concurrent model downloads. Model download race conditions were corrupting the ONNX file, causing INVALID_PROTOBUF errors in GitHub Actions tests. CI Changes: - Cache ~/.local/share/iscc-sct between GitHub Actions runs - Pre-download model before running parallel tests (pytest -n auto) Docker Changes: - Pre-download model during Docker build in builder stage - Copy model from builder to runtime stage - Eliminates model download on container startup (~200-300 MB) Fixes test failures where multiple pytest workers simultaneously attempted to download the model to the same location. Related upstream issues filed on iscc/iscc-sct: - #18: Add file locking during model download - #19: Add configurable model storage path - #20: Add CLI command to pre-download models
1 parent e6c3e5f commit ba37ca2

File tree

2 files changed

+20
-0
lines changed

2 files changed

+20
-0
lines changed

.github/workflows/docker-publish.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,18 @@ jobs:
2121
with:
2222
python-version: "3.12"
2323

24+
- name: Cache iscc-sct model
25+
uses: actions/cache@v4
26+
with:
27+
path: ~/.local/share/iscc-sct
28+
key: iscc-sct-model-${{ hashFiles('**/pyproject.toml') }}
29+
restore-keys: |
30+
iscc-sct-model-
31+
32+
- name: Pre-download iscc-sct model
33+
run: |
34+
uv run python -c "import iscc_sct.code_semantic_text as sct; sct.model()"
35+
2436
- name: Run tests
2537
run: uv run poe test
2638

Dockerfile

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,11 @@ COPY . .
3030
# This creates compiled bytecode and optimized install
3131
RUN uv sync --locked --no-editable --no-dev
3232

33+
# Pre-download iscc-sct model weights during build
34+
# This ensures the model is available immediately on container start
35+
# Model is downloaded to ~/.local/share/iscc-sct/ during build
36+
RUN .venv/bin/python -c "import iscc_sct.code_semantic_text as sct; sct.model()"
37+
3338

3439
# Stage 2: Runtime - minimal production image
3540
FROM python:3.12-slim
@@ -47,6 +52,9 @@ COPY --from=builder /app/iscc_search /app/iscc_search
4752
# Copy OpenAPI schema files (needed by the app)
4853
COPY --from=builder /app/iscc_search/openapi /app/iscc_search/openapi
4954

55+
# Copy pre-downloaded iscc-sct model from builder
56+
COPY --from=builder /root/.local/share/iscc-sct /root/.local/share/iscc-sct
57+
5058
# Add virtual environment to PATH
5159
ENV PATH="/app/.venv/bin:$PATH"
5260

0 commit comments

Comments
 (0)