Fix: Restore Meta.indexes when altering fields (#486, #491) by robberwick · Pull Request #498 · microsoft/mssql-django

robberwick · 2026-02-11T13:11:58Z

Fixes #486 — Indexes defined via Meta.indexes are dropped and not recreated when altering an indexed column (e.g., changing max_length or nullability).

Fixes #491 — Duplicate index error (ProgrammingError) when changing AutoField to BigAutoField in a combined migration where another field has db_index=True.

Issue #486: Meta.indexes not restored after field alteration

When SQL Server needs to alter a column's type or nullability, _alter_field in schema.py drops all indexes on the affected column first (SQL Server requires this). However, the restoration logic that runs afterward only handled:

db_index=True single-field indexes
index_together indexes
Unique constraints

Indexes defined via Meta.indexes were never restored. This meant any index defined in Meta.indexes would be silently dropped whenever a participating column was altered.

Issue #491: Duplicate index on AutoField → BigAutoField change

When AutoField is changed to BigAutoField, the restoration logic unconditionally recreated all db_index=True indexes via self.execute(). In a combined migration (where CreateModel and AlterField are in the same migration file), the index already existed in deferred_sql (queued during CreateModel but not yet executed). This caused the same index to be created twice, resulting in a ProgrammingError.

Fix

The index restoration phase in _alter_field has been refactored to:

Restore Meta.indexes: After column alteration, indexes from model._meta.indexes involving the altered field are now restored using index.create_sql().
Handle AutoField/BigAutoField comprehensively: When an AutoField/BigAutoField is altered, ALL indexes are restored (db_index, index_together, and Meta.indexes), since SQL Server drops all indexes on the table for IDENTITY column changes.
Deduplicate index creation: Before executing any index creation, the SQL is checked against both deferred_sql and post_actions. This prevents the duplicate creation that caused Duplicate index error when changing AutoField to BigAutoField in combined migration with db_index=True field #491.
Remove the broken nullability-path restoration: The old nullability change path had ad-hoc index restoration logic. This has been removed in favor of the unified restoration block that handles all cases.

Changes

mssql/schema.py: Refactored _alter_field index restoration to handle Meta.indexes, AutoField/BigAutoField changes, and deduplication in a unified block.

Testing

testapp/tests/test_indexes.py: Added new test methods in TestMetaIndexesRetained covering:

Type changes, nullability changes, and combined type+nullability changes
Field renames
ForeignKey alterations
Multiple indexes on the same field
db_index=True combined with Meta.indexes
unique_together coexistence with Meta.indexes
AutoField → BigAutoField changes (both split and combined migrations)
index_together retention (Django < 5.1)
Three-column indexes

Each test runs in both "split migration" and "combined migration" modes because Django's schema_editor uses a deferred_sql queue that changes the timing of index creation, and each mode exercises different code paths:

Split mode (two separate schema_editor contexts): Simulates two separate migration files. The first context creates the table and its indexes, then exits, at which point deferred_sql is flushed and all indexes physically exist in the database. The second context then runs the AlterField operation, which queries the database, finds and drops the indexes, alters the column, and must restore them.
Combined mode (single schema_editor context): Simulates a single migration file containing both CreateModel and AlterField. Here, CreateModel adds index creation SQL to deferred_sql but it has not been executed yet when AlterField runs. The indexes don't physically exist in the database, so the DROP phase finds nothing. However, the RESTORE phase must not blindly create indexes that are already queued in deferred_sql, or they will be created twice when the context exits and flushes the queue.

Known existing bugs not fixed by this PR (documented with `@expectedFailure` tests)

Rename + alter in same migration: When a field is renamed via RenameField AND altered in the same migration, _delete_indexes() fails because it looks up the field by the old name which no longer exists in model state.
Indexes from Meta.indexes not restored when RenameField and AlterField (type or nullability change) occur in the same migration #499
unique_together + unique=True on same field: When a field has both unique=True and participates in unique_together, only the single-field unique constraint is restored (the unique_together restoration is in an else branch). unique_together constraint not restored after field alteration when field has unique=True #494

Copilot

Pull request overview

This PR fixes SQL Server migration behavior in mssql/schema.py so that indexes dropped during ALTER COLUMN operations are properly restored, including indexes defined via Meta.indexes, and avoids duplicate index creation in combined migrations (CreateModel + AlterField).

Changes:

Refactors _alter_field() index/constraint restoration to include Meta.indexes and to restore all indexes for AutoField/BigAutoField alterations.
Adds deduplication against deferred_sql and post_actions to prevent creating the same index twice in combined migrations.
Adds an extensive regression test suite covering type/nullability changes, renames, FK alterations, multi-column indexes, and combined vs split migration contexts.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
`mssql/schema.py`	Refactors index restoration in `_alter_field()` to restore `Meta.indexes` and deduplicate creation, including special handling for AutoField/BigAutoField changes.
`testapp/tests/test_indexes.py`	Adds `TransactionTestCase` regression coverage for index retention/restoration across many migration scenarios and execution contexts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-11T13:17:39Z

testapp/tests/test_indexes.py

+        Runs with both split and combined migrations
+        """
+        for use_single_migration in [False, True]:
+            context_desc = "combined context" if use_single_migration else "split contexts"


context_desc is assigned but never used. This will be flagged by flake8/pyflakes (F841) and can break CI. Remove the assignment or use it in the subTest message/error output.

Suggested change

context_desc = "combined context" if use_single_migration else "split contexts"

superfluous assignment removed and rebased into 4e5e36a

Copilot · 2026-02-11T13:17:39Z

testapp/tests/test_indexes.py

+            context_desc = "combined context" if use_single_migration else "split contexts"
+            with self.subTest(single_migration=use_single_migration):


context_desc is assigned but never used. This will be flagged by flake8/pyflakes (F841) and can break CI. Remove the assignment or use it in the subTest message/error output.

superfluous assignment removed and rebased into 4e5e36a

Copilot · 2026-02-11T13:17:40Z

mssql/schema.py

+            # --------------------------------------------------------------------------------
+            for index in model._meta.indexes:
+                # Get the field objects for this index
+                index_fields = [model._meta.get_field(field_name) for field_name in index.fields]


Index restoration uses model._meta.get_field(field_name) for each entry in index.fields. In Django, Index(fields=...) can include ordering prefixes like '-created_at', which would raise FieldDoesNotExist here. Consider normalizing field names (strip leading '-' / '+') or using index.fields_orders (when available) to resolve actual model fields before computing index_columns_list.

Suggested change

index_fields = [model._meta.get_field(field_name) for field_name in index.fields]

# Index.fields may include ordering prefixes like '-created_at'.

# Prefer fields_orders when available to resolve actual field names.

if hasattr(index, "fields_orders") and index.fields_orders:

index_field_names = [field_name for field_name, _order in index.fields_orders]

else:

index_field_names = list(index.fields)

index_fields = [

model._meta.get_field(field_name.lstrip("+-"))

for field_name in index_field_names

]

This may be an issue, but if it is, I think it already exists elsewhere e.g. in the [delete_indexes method](https://github.com/microsoft/mssql-django/blob/b1b5f3cc125038ba6b4d56b7acff93e8a278faf2/mssql/schema.py#L961C1-L964C46).

Given that this is an established pattern for resolving field names, I'd suggest punting this out as a separate issue instead of attempting to address it here.

robberwick · 2026-02-11T13:18:18Z

mssql/schema.py


-                if (
-                    new_field.get_internal_type() not in ("JSONField", "TextField") and
-                    (old_field.db_index or not new_field.db_index) and
-                    new_field.db_index or
-                    ((indexes_dropped and sorted(indexes_dropped) == sorted([index.name for index in model._meta.indexes])) or
-                     (indexes_dropped and sorted(indexes_dropped) == sorted(auto_index_names)))
-                ):
-                    create_index_sql_statement = self._create_index_sql(model, [new_field])
-                    if create_index_sql_statement.__str__() not in [sql.__str__() for sql in self.deferred_sql]:


Note: the restoration step here is deleted, as it was not correctly restoring Meta.indexes. The restoration is now handled in the column alteration cleanup phase below.

robberwick · 2026-02-11T14:25:51Z

@microsoft-github-policy-service agree company="HP"

…oft#491)

bewithgaurav · 2026-02-12T17:24:03Z

/azp run

azure-pipelines · 2026-02-12T17:24:14Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot AI review requested due to automatic review settings February 11, 2026 13:11

Copilot started reviewing on behalf of robberwick February 11, 2026 13:12 View session

Copilot AI reviewed Feb 11, 2026

View reviewed changes

robberwick commented Feb 11, 2026

View reviewed changes

robberwick mentioned this pull request Feb 11, 2026

Indexes from Meta.indexes not restored when RenameField and AlterField (type or nullability change) occur in the same migration #499

Open

robberwick force-pushed the meta-indexes-restoration branch from 232164c to 6eb4b68 Compare February 11, 2026 15:09

robberwick added 2 commits February 11, 2026 15:33

TEST: Test restoration of Meta.indexes

4e5e36a

FIX: Restore Meta.indexes when altering fields (microsoft#486, micros…

6953139

…oft#491)

robberwick force-pushed the meta-indexes-restoration branch from 6eb4b68 to 6953139 Compare February 11, 2026 15:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Fix: Restore Meta.indexes when altering fields (#486, #491)#498

Fix: Restore Meta.indexes when altering fields (#486, #491)#498
robberwick wants to merge 2 commits intomicrosoft:devfrom
robberwick:meta-indexes-restoration

robberwick commented Feb 11, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

robberwick Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

robberwick Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

robberwick Feb 11, 2026

Uh oh!

robberwick Feb 11, 2026

Uh oh!

robberwick commented Feb 11, 2026

Uh oh!

bewithgaurav commented Feb 12, 2026

Uh oh!

azure-pipelines bot commented Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		context_desc = "combined context" if use_single_migration else "split contexts"
		with self.subTest(single_migration=use_single_migration):

-                index_fields = [model._meta.get_field(field_name) for field_name in index.fields]
+                # Index.fields may include ordering prefixes like '-created_at'.
+                # Prefer fields_orders when available to resolve actual field names.
+                if hasattr(index, "fields_orders") and index.fields_orders:
+                    index_field_names = [field_name for field_name, _order in index.fields_orders]
+                else:
+                    index_field_names = list(index.fields)
+                index_fields = [
+                    model._meta.get_field(field_name.lstrip("+-"))
+                    for field_name in index_field_names
+                ]

Comments

Conversation

robberwick commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue #486: Meta.indexes not restored after field alteration

Issue #491: Duplicate index on AutoField → BigAutoField change

Fix

Changes

Testing

Known existing bugs not fixed by this PR (documented with @expectedFailure tests)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

robberwick Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

robberwick Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

robberwick Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

robberwick Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

robberwick commented Feb 11, 2026

Uh oh!

bewithgaurav commented Feb 12, 2026

Uh oh!

azure-pipelines bot commented Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

robberwick commented Feb 11, 2026 •

edited

Loading

Known existing bugs not fixed by this PR (documented with `@expectedFailure` tests)