Skip to content

Optimize database interactions and migrate to SQLAlchemy 2#799

Open
jack89roberts wants to merge 15 commits intodevelopfrom
feat/db-opt
Open

Optimize database interactions and migrate to SQLAlchemy 2#799
jack89roberts wants to merge 15 commits intodevelopfrom
feat/db-opt

Conversation

@jack89roberts
Copy link
Copy Markdown
Contributor

@jack89roberts jack89roberts commented Feb 27, 2026

Vibe coded all the way... will do more comparisons with develop but it seems to be a decent speedup.

This pull request refactors many database queries throughout the codebase to use SQLAlchemy 2.x's select() and scalars() APIs instead of the legacy query() API. The changes improve performance, reliability, and future compatibility, and in several places also optimize data access by prefetching related objects or batching queries. Additionally, some relationships in the schema are updated for more efficient loading and ordering.

Database Query Refactoring and Optimization

  • Replaced legacy .query() API with .scalars(select(...)) and .execute(select(...)) in most database access patterns across the codebase, including in airsenal/conftest.py, airsenal/framework/api_utils.py, airsenal/framework/aws_utils.py, airsenal/framework/bpl_interface.py, airsenal/framework/optimization_utils.py, and airsenal/framework/prediction_utils.py. This modernizes the codebase for SQLAlchemy 2.x and improves query performance.
  • Updated several query patterns to batch-load related objects (e.g., using selectinload) and avoid N+1 query issues, especially in player and fixture data access.

Performance and Data Access Improvements

  • Optimized player and transfer suggestion lookups by batching queries and building in-memory maps, reducing the number of database roundtrips (notably in build_suggestion_string, get_session_players, and get_player_history_df).
  • Improved absence and player score checks by prefetching all relevant data and processing in memory, which enhances efficiency for large datasets.

Schema and Relationship Enhancements

  • Updated the Player model's attributes relationship to use lazy="selectin" and to order by season and gameweek descending, improving both query efficiency and result ordering when accessing player attributes.
  • Added imports for Index, UniqueConstraint, and select in schema.py to support new schema and query patterns.

These changes collectively modernize the codebase, improve performance, and set the foundation for future SQLAlchemy compatibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant