Skip to content

Conversation

@aaronsteers
Copy link
Contributor

Replace MyPy with ty type checker (do not merge)

Summary

This PR replaces MyPy with ty (the new Rust-based Python type checker from astral-sh) across both PyAirbyte and airbyte-python-cdk repositories for performance and usability testing. This is an experimental change to evaluate ty as an alternative to MyPy.

Key changes:

  • Replace mypy dependency with ty ^0.0.1a16 in pyproject.toml
  • Remove MyPy configuration sections and add [tool.ty.src] configuration
  • Update CI workflows from mypy-check to ty-check
  • Create/update generic type-check Poe tasks using ty commands
  • Configure appropriate file exclusions for ty checking

⚠️ Important Notes:

  • ty is significantly stricter than MyPy (found 108 errors in PyAirbyte, 205 in airbyte-python-cdk)
  • This is marked "do not merge" as it's for testing/evaluation only
  • ty lacks CLI auto-noqa features (unlike pyrefly), requiring manual error suppression

Review & Testing Checklist for Human

  • Run type checking locally: Execute poetry run poe type-check in both repos to verify ty works correctly
  • Test CI workflow changes: Verify that the updated GitHub Actions workflows execute ty-check properly
  • Verify exclusion patterns: Confirm that [tool.ty.src] exclude settings properly skip tests/docs/fixtures directories
  • Test poethepoet integration: Run poetry run poe check to ensure ty integrates correctly with existing task chains
  • Check development workflow: Ensure the stricter type checking doesn't break normal development processes

Recommended test plan: Install dependencies in both repos, run type-check tasks locally, verify output shows expected ty diagnostics, then check that CI workflows complete (failures expected due to existing type violations).


Diagram

%%{ init : { "theme" : "default" }}%%
graph TD
    PyAirbyte_pyproject["PyAirbyte/<br/>pyproject.toml"]:::major-edit
    PyAirbyte_workflow["PyAirbyte/<br/>.github/workflows/<br/>python_lint.yml"]:::major-edit
    PyAirbyte_lock["PyAirbyte/<br/>poetry.lock"]:::minor-edit
    
    CDK_pyproject["airbyte-python-cdk/<br/>pyproject.toml"]:::major-edit
    CDK_workflow["airbyte-python-cdk/<br/>.github/workflows/<br/>python_lint.yml"]:::major-edit
    CDK_mypy_ini["airbyte-python-cdk/<br/>mypy.ini"]:::major-edit
    
    
    ty_config["[tool.ty.src]<br/>configuration"]:::context
    poe_tasks["poethepoet<br/>type-check tasks"]:::context
    
    PyAirbyte_pyproject --> ty_config
    CDK_pyproject --> ty_config
    PyAirbyte_pyproject --> poe_tasks
    CDK_pyproject --> poe_tasks
    
    PyAirbyte_workflow --> |"replaces mypy-check<br/>with ty-check"| PyAirbyte_pyproject
    CDK_workflow --> |"replaces mypy-check<br/>with ty-check"| CDK_pyproject
    CDK_mypy_ini --> |"deleted"| CDK_pyproject
    
    subgraph Legend
        L1["Major Edit"]:::major-edit
        L2["Minor Edit"]:::minor-edit
        L3["Context/No Edit"]:::context
    end

classDef major-edit fill:#90EE90
classDef minor-edit fill:#87CEEB
classDef context fill:#FFFFFF
Loading

Notes

  • This work was requested by @aaronsteers as part of evaluating new Rust-based Python type checkers
  • Session URL: https://app.devin.ai/sessions/89f33f9cbd9f48bda7165514bc3c1005
  • ty version 0.0.1a16 is very experimental - expect potential issues
  • A follow-up PR will test pyrefly (Meta's Rust-based type checker) for comparison
  • The stricter type checking may reveal legitimate type issues that were missed by MyPy

- Replace mypy dependency with ty in pyproject.toml
- Remove mypy.ini configuration file
- Add ty.src configuration with exclude patterns
- Update CI workflow from mypy-check to ty-check
- Update type-check Poe task to use ty check command
- Exclude bin, docs, and unit_tests directories from ty checking

This change enables testing of the new Rust-based ty type checker
as an alternative to mypy for performance and usability evaluation.

Co-Authored-By: AJ Steers <[email protected]>
@devin-ai-integration
Copy link
Contributor

Original prompt from AJ Steers:

@Devin - I'm wanting to put the two new rust Python typing libraries to the test. Create one PR to use ty and one PR to use pyre-something from meta.

@devin-ai-integration
Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions github-actions bot added the enhancement New feature or request label Jul 30, 2025
@github-actions
Copy link

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1738195530-test-ty-type-checker#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1738195530-test-ty-type-checker

Helpful Resources

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /poe <command> - Runs any poe command in the CDK environment

📝 Edit this welcome message.

@github-actions
Copy link

github-actions bot commented Jul 30, 2025

PyTest Results (Fast)

3 700 tests  ±0   3 689 ✅ ±0   6m 31s ⏱️ +5s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit df07ecb. ± Comparison against base commit a59d25f.

♻️ This comment has been updated with latest results.

Comment on lines 57 to 78

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 5 months ago

To fix the issue, we will add a permissions block at the root of the workflow file. This block will apply to all jobs in the workflow unless overridden by job-specific permissions blocks. Since the workflow only performs read-only operations (e.g., checking out code, installing dependencies, and running checks), we will set contents: read as the minimal required permission.


Suggested changeset 1
.github/workflows/python_lint.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/python_lint.yml b/.github/workflows/python_lint.yml
--- a/.github/workflows/python_lint.yml
+++ b/.github/workflows/python_lint.yml
@@ -8,2 +8,5 @@
 
+permissions:
+  contents: read
+
 jobs:
EOF
@@ -8,2 +8,5 @@

permissions:
contents: read

jobs:
Copilot is powered by AI and may make mistakes. Always verify output.
@github-actions
Copy link

github-actions bot commented Jul 30, 2025

PyTest Results (Full)

3 703 tests  ±0   3 692 ✅ ±0   18m 3s ⏱️ -48s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit df07ecb. ± Comparison against base commit a59d25f.

♻️ This comment has been updated with latest results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants