Skip to content

Enable pytest-asyncio tests in CI #92

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
May 6, 2025
Merged

Enable pytest-asyncio tests in CI #92

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
bfce9ac
Set asyncio_mode="auto" in pytest.ini config
weiji14 May 1, 2025
695bc5f
Refactor contents of test_cog.py into a proper unit test
weiji14 May 1, 2025
bfb3a43
Uncomment `uv run pytest` in .github/workflows/test-python.yml
weiji14 May 1, 2025
d0f98ac
Add `--color=yes`flag to force GitHub Actions logs to have color
weiji14 May 1, 2025
b08fe51
Ruff format
weiji14 May 1, 2025
28973b9
Remove timing assertions
weiji14 May 1, 2025
12b230f
Bump astral-sh/setup-uv from 3 to 6
weiji14 May 1, 2025
638d602
Remove `uv sync` part
weiji14 May 2, 2025
3be22b7
Bump pytest-asyncio from 0.24.0 to 0.26.0
weiji14 May 2, 2025
ca31048
Revert "Remove `uv sync` part"
weiji14 May 4, 2025
773fad8
Typo geoindex-rs -> async-tiff
weiji14 May 4, 2025
aa21368
Add back `--no-project` flag to `uv run` commands
weiji14 May 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/test-python.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ jobs:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like the line above this is incorrect

Copy link
Member Author

@weiji14 weiji14 May 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean uv sync --no-install-package geoindex-rs should be just uv sync?

Edit: Just removed the entire uv sync ... line at commit 638d602, and it seems to run fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, it needs to be uv sync --no-install-package async-tiff. And then the lines after that still need the --no-project.

You see this line: https://github.com/developmentseed/async-tiff/actions/runs/14786441598/job/41515604193#step:7:19

When you run uv sync it builds the rust code in release mode as part of the current project. For fastest CI we want to build and test in debug mode, not release mode. That means we want to tell uv not to automatically build async-tiff, because we'll manually build it after with maturin develop

Copy link
Member Author

@weiji14 weiji14 May 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think you're right. Checking the timings:

So let's do uv sync --no-install-package async-tiff && uv run --no-project maturin develop.

Copy link
Member Author

@weiji14 weiji14 May 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final timings at commit aa21368, job https://github.com/developmentseed/async-tiff/actions/runs/14825301405/job/41617636468:

  • uv sync --no-install-package async-tiff = 3s
  • uv run --no-project maturin develop = 46s
  • Total: 49s 🚀

- name: maturin venv Build
working-directory: python
run: uv run --no-project maturin develop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --no-project was here because I thought whenever we ran uv sync or uv run, uv would automatically build the current project, which would compile the rust code and take a while.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I though if the working directory is set to python/ already, then it should only compile what's inside the python/ folder? Don't we need to compile the rust code anyway before compiling the pyo3 code? A little confused here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not the cargo project that's the issue. It's the uv project. We don't want uv automatically building the rust parts of async-tiff in release mode.

run: uv run maturin develop

# - name: Run pytest
# working-directory: python
# run: uv run --no-project pytest
- name: Run pytest
working-directory: python
run: uv run pytest --verbose
4 changes: 4 additions & 0 deletions python/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,7 @@ dev-dependencies = [
"pytest>=8.3.3",
"ruff>=0.8.4",
]

[tool.pytest.ini_options]
addopts = "--color=yes"
asyncio_mode = "auto"
44 changes: 24 additions & 20 deletions python/tests/test_cog.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,27 +1,31 @@
import async_tiff
from time import time
from async_tiff import TIFF

from async_tiff import TIFF, enums
from async_tiff.store import S3Store

store = S3Store("sentinel-cogs", region="us-west-2", skip_signature=True)
path = "sentinel-s2-l2a-cogs/12/S/UF/2022/6/S2B_12SUF_20220609_0_L2A/B04.tif"

tiff = await TIFF.open(path, store=store, prefetch=32768)
async def test_cog_s3():
"""
Ensure that TIFF.open can open a Sentinel-2 Cloud-Optimized GeoTIFF file from an
s3 bucket, read IFDs and GeoKeyDirectory metadata.
"""
store = S3Store("sentinel-cogs", region="us-west-2", skip_signature=True)
path = "sentinel-s2-l2a-cogs/12/S/UF/2022/6/S2B_12SUF_20220609_0_L2A/B04.tif"

start = time()
tiff = await TIFF.open(path=path, store=store, prefetch=32768)
end = time()
assert end - start < 5 # take less than 5 seconds to open

start = time()
tiff = await TIFF.open(path, store=store, prefetch=32768)
end = time()
end - start
ifds = tiff.ifds
assert len(ifds) == 5

ifds = tiff.ifds
ifd = ifds[0]
ifd.compression
ifd.tile_height
ifd.tile_width
ifd.photometric_interpretation
gkd = ifd.geo_key_directory
gkd.citation
gkd.projected_type
gkd.citation
ifd = ifds[0]
assert ifd.compression == enums.CompressionMethod.Deflate
assert ifd.tile_height == 1024
assert ifd.tile_width == 1024
assert ifd.photometric_interpretation == enums.PhotometricInterpretation.BlackIsZero

dir(gkd)
gkd = ifd.geo_key_directory
assert gkd.citation == "WGS 84 / UTM zone 12N"
assert gkd.projected_type == 32612
Loading