Migrate to SQLModel and improve relationship loading#4547
Merged
Conversation
Changes the default relationship loading strategy to raise in order to avoid generating N+1 queries. We also override this default on many relationships based on how we use those relationships. You can currently load and save the attendee form and load the attendee shifts page.
A lot of paginated pages would break if they had 0 rows to display. This fixes that.
Noticed this while testing the API reference page -- several services didn't have prefilled suggestions.
We still want the old behavior for repr as that makes the Tracking table much easier to read. Also fixes issues with passing fields to to_dict.
Adds required loading to pages accessible from the site map. Not all reports were checked, and forms are not working right now so testing was quite basic (e.g. most pages had no data to display). Also adds relationship hints to each model because I got tired of jumping around different relationship definitions to remember which relationships were set to be joined/selected in by default.
Adds the SQLModel library and updates all the model definitions to define their type. However, indexes seem to be broken still.
We had to remove backrefs as they broke SQLModel's ability to assign the backref columns. Also recasts all the foreign keys into SQLModel's Field(), and partially converts the regular columns (for Attendee and Department, specifically). Thanks to all this and a couple other fixes, forms will load and save now.
Adds a new version of the relationship and column overrides to account for the use of Field() and Relationship() on their own, and updates some types of fields to use sa_type instead of sa_column in order to get to server to stop generating the wrong kind of column in migrations. Also removes datetime.utcnow() as it's deprecated.
Fixes FK key name generation and the corresponding migration, and finishes converting all columns to SQLModel.
A bit of a clumsy wrap-up, but this rolls back the raise relationship loading strategy while also fixing bugs introduced by the SQLAlchemy upgrade/SQLModel implementation.
Takes care of a few bugs when you edit a badge during prereg.
bitbyt3r
reviewed
Mar 8, 2026
docs/script_example.py
Outdated
| with Session() as session: | ||
| initialize_db() | ||
| session = Session().session No newline at end of file | ||
| session = Session() No newline at end of file |
Member
There was a problem hiding this comment.
Does this need to be called? Shouldn't session already be defined and set to the session by the context manager?
Member
Author
There was a problem hiding this comment.
Huh, yea, this must have been an old copy-paste error.
Anytime you set a novel attribute that started with 'is_', we were checking to see if the object's model class matched what came after. I didn't even know our code did this and we only use it in one place inside the magprime plugin, so we're removing it now. Also removes a 'return' on __setattr__.
Fixes a find-and-replace issue that was breaking emails.
In cases where we were defining the ribbon as part of the object, it was ending up as an integer and not a string. Hopefully ribbons are the only Choice column that we sometimes set manually to an integer (which we probably shouldn't, but do due to legacy code where you could only have one ribbon at a time).
We weren't adding columns as columns anymore. This is only a partial solution as it's not 1:1 -- you have to define all columns in plugins as Field() variables -- but it's working at least. Also makes our migrations use sa.Unicode instead of sqlmodel's unnecessary AutoString class.
This breaks our ability to correctly generate migrations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It turned out that upgrading SQLAlchemy, which made all our model properties
Nonebefore saving to the database, broke our form instantiating logic. Rather than fight with WTForms to accommodate that, I chose to move us to SQLModel to let us set Pydantic default types on models defined for SQLAlchemy. This involved:backrefrelationship definitions, which were deprecated in SQLAlchemy and unsupported in SQLModel, with explicit relationshipspassive_deleteoption to support those DB-level cascade deletesuselistproperty was removed as it's no longer required -- SQLAlchemy is smart enough to see whether we type the relationship property aslist[Model]or just a singularModelsingle_parentwith auniqueconstraint to enforce one-to-one relationships. Thesingle_parentproperty has also been removed from any relationship that isn't one-to-oneSeparately, an attempt was made to default our relationships to raise loading to prevent excessive lazy loads/N+1 queries. This ultimately didn't pan out because of the enormous scope involved, but I did review all current relationships for places where it was appropriate to do joined loading or selectin loading by default, so we should still see some improvements.
Old description below since it has some to-dos that are worth keeping: