Skip to content

Conversation

@ricardoV94
Copy link
Member

@ricardoV94 ricardoV94 commented Jun 7, 2024

What is broken / not supported by Numba:

The plan now

  • Numba becomes a default dependency
  • FAST_RUN becomes an abstract mode that can use either C or Numba
    • For now it will remain C but we'll switch to Numba ASAP
    • Why Numba over C?
      • Faster
      • Easier to maintain/ extend
      • Easier to install (pip install should be safe)
  • New config flag determines the backend of FAST_RUN.
    • This way downstream libraries can test if anything would break with the switch, and user can switch back if it breaks stuff for them
    • CI will run with both configs. To avoid runtime explosion it may be time to restrict floatX and fast_compile tests to a few files instead of being an orthogonal parametrization like it is now

Immediate action

  • I'll be cherry picking the countless fixes into their own PRs. I need help with review as both me and @jessegrabowski are pretty much at capacity
  • This PR will end up with just the test tweaks and new config flags
  • We need to implement more Ops in numba mode. We'll follow up with a PR that fails on fallback to obj mode. Help needed!

@ricardoV94 ricardoV94 force-pushed the test_on_numba branch 2 times, most recently from 2eb7fe7 to 1f093cd Compare June 7, 2024 17:24
@ricardoV94
Copy link
Member Author

Locally I ran all the tests in tests/tensor/rewriting/test_basic.py twice (to allow caching) before and after making Numba default.
Before: 6s
After: 1m34s

Numba caching is baaaaaaaaaaaaaaaaaaaaaaaaaaaaaaad

@ricardoV94

This comment was marked as outdated.

@ricardoV94 ricardoV94 changed the title Test on numba Run whole test suite on numba backend Jun 7, 2024
@aseyboldt

This comment was marked as outdated.

@ricardoV94

This comment was marked as outdated.

@ricardoV94

This comment was marked as outdated.

@ricardoV94

This comment was marked as outdated.

@ricardoV94 ricardoV94 force-pushed the test_on_numba branch 4 times, most recently from 6f6e8bb to 53d9a26 Compare June 13, 2024 14:43
@ricardoV94 ricardoV94 added this to the 3.0 release milestone Feb 14, 2025
@ricardoV94
Copy link
Member Author

After caching the test file now runs in 50s after caching vs 6s before the PR, so only 8x slower now :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants