Skip to content

Add ColumnPrefixIndex for MySQL column prefix key parts #1151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

JaeHyuckSa
Copy link
Contributor

Description

This PR implements ColumnPrefixIndex to support MySQL's Column Prefix Key Parts feature. This feature can significantly improve performance for istartswith queries on TEXT or long VARCHAR fields while being space-efficient.

Related issue: #1114

Example Usage

class Article(models.Model):
    title = models.CharField(max_length=200)
    content = models.TextField()

    class Meta:
        indexes = [
            ColumnPrefixIndex(
                fields=['title', 'content'],
                prefix_lengths=(10, 50),
                name='title_content_prefix_idx'
            ),
        ]

@JaeHyuckSa JaeHyuckSa force-pushed the feature/column-prefix-index branch from 3cb6ea1 to 8dfdde4 Compare June 23, 2025 08:45
@adamchainz
Copy link
Owner

Hi

On a second reading, I don't think this should be a separate index class, but instead an expression/function.

MySQL allows indexes to mix prefixes as well as full column indexes, like: CREATE INDEX example_idx ON example (title(10), state). To support this, an expression should allow us to do something like:

Index(
   Prefix("title", 10),
   "state",
   ...,
)

Can you investigate that path instead?

@JaeHyuckSa JaeHyuckSa force-pushed the feature/column-prefix-index branch from 8dfdde4 to 77acf2d Compare June 23, 2025 08:49
@JaeHyuckSa
Copy link
Contributor Author

Hi

On a second reading, I don't think this should be a separate index class, but instead an expression/function.

MySQL allows indexes to mix prefixes as well as full column indexes, like: CREATE INDEX example_idx ON example (title(10), state). To support this, an expression should allow us to do something like:

Index(
   Prefix("title", 10),
   "state",
   ...,
)

Can you investigate that path instead?

Hi @adamchainz —I’ll implement the Prefix Func and include it in the PR!

@adamchainz
Copy link
Owner

Ah, wait a second, I forgot that slicing an F() is possible since Django 5.1. That should allow F("title")[:10] to work in indexes, in theory, although it won't return the exact right syntax right now, I think. Maybe this should be a Django change, instead of a Django-MySQL one? Indexing the substr expression will already work on all databases, it would just be an optimization on MySQL to use the column(n) syntax instead.

@JaeHyuckSa
Copy link
Contributor Author

JaeHyuckSa commented Jun 23, 2025

@adamchainz You mean that in MySQL’s _create_index_sql, whenever it encounters a Slice expression for an index, it should emit the column(n)) syntax instead of SUBSTRING(col, 1, n), correct?

@adamchainz
Copy link
Owner

Yes, that could be all it needs! It would need to be for slices with start=None and step=None, specifically.

@JaeHyuckSa
Copy link
Contributor Author

JaeHyuckSa commented Jun 23, 2025

Yes, that could be all it needs! It would need to be for slices with start=None and step=None, specifically.

Thanks a lot for your feedback. I’ve opened a ticket for it here: https://code.djangoproject.com/ticket/36475. Once it’s approved, I’ll proceed with the implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants