fix: re-create index so that it actually works

[omBratteng]

Previous index did not actually do anything, so new migration to make it actually work.

Jira ticket

AS-1172

[pulumi]

🍹 The Update (preview) for dailydotdev/api/prod (at 38d5c17) was successful.

Resource Changes

    Name                                            Type                           Operation
~   vpc-native-hourly-notification-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-validate-active-users-cron           kubernetes:batch/v1:CronJob    update
~   vpc-native-private-deployment                   kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-zombie-images-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-bg-deployment                        kubernetes:apps/v1:Deployment  update
~   vpc-native-update-tag-recommendations-cron      kubernetes:batch/v1:CronJob    update
~   vpc-native-generate-search-invites-cron         kubernetes:batch/v1:CronJob    update
~   vpc-native-generic-referral-reminder-cron       kubernetes:batch/v1:CronJob    update
~   vpc-native-temporal-deployment                  kubernetes:apps/v1:Deployment  update
~   vpc-native-update-views-cron                    kubernetes:batch/v1:CronJob    update
+   vpc-native-api-migration-bce75232               kubernetes:batch/v1:Job        create
~   vpc-native-update-highlighted-views-cron        kubernetes:batch/v1:CronJob    update
~   vpc-native-update-trending-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-update-source-tag-view-cron          kubernetes:batch/v1:CronJob    update
~   vpc-native-update-tags-str-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-stale-user-transactions-cron   kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-users-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-calculate-top-readers-cron           kubernetes:batch/v1:CronJob    update
~   vpc-native-update-current-streak-cron           kubernetes:batch/v1:CronJob    update
~   vpc-native-deployment                           kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-gifted-plus-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-daily-digest-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-check-analytics-report-cron          kubernetes:batch/v1:CronJob    update
~   vpc-native-sync-subscription-with-cio-cron      kubernetes:batch/v1:CronJob    update
-   vpc-native-api-migration-627807ab               kubernetes:batch/v1:Job        delete
~   vpc-native-clean-zombie-user-companies-cron     kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-deployment       kubernetes:apps/v1:Deployment  update
~   vpc-native-update-source-public-threshold-cron  kubernetes:batch/v1:CronJob    update
~   vpc-native-ws-deployment                        kubernetes:apps/v1:Deployment  update

[omBratteng]

@claude compare the new index I created, to the query I intend to run. "sourceId" is dynamic based on the query.

SELECT
	*
FROM
	source_member
WHERE
	"sourceId" = '4c4a674b-c7d4-45ab-9333-1c79ef6b1d99'
	AND COALESCE((flags ->> 'hasUnreadPosts')::boolean, FALSE) != TRUE;

[claude]

Claude finished @omBratteng's task —— View job

---

Index Analysis

I'll compare your new index with the SQL query to ensure optimal performance.

Todo List:

• Read the migration file to understand the new index structure
• Analyze the SQL query pattern and filtering conditions
• Compare index design with query requirements
• Provide optimization recommendations

Analysis

Your New Index:

CREATE INDEX "IDX_source_member_flags_hasUnreadPosts" 
ON public.source_member ("sourceId", COALESCE((flags ->> 'hasUnreadPosts')::boolean, FALSE));

Your Query:

SELECT * FROM source_member 
WHERE "sourceId" = '4c4a674b-c7d4-45ab-9333-1c79ef6b1d99'
  AND COALESCE((flags ->> 'hasUnreadPosts')::boolean, FALSE) != TRUE;

Assessment: ✅ Excellent Match

Your index is perfectly designed for this query:

1. Column Order: sourceId first is optimal since it's an equality condition that will dramatically reduce the result set
2. Expression Match: The index expression COALESCE((flags ->> 'hasUnreadPosts')::boolean, FALSE) exactly matches what's used in your WHERE clause
3. Condition Coverage: Your query condition != TRUE will efficiently use the index since PostgreSQL can leverage the pre-computed boolean values

Performance Benefits

• Fast Lookup: The equality filter on sourceId will quickly narrow to the relevant partition
• Index-Only Scans: For queries selecting just these columns, PostgreSQL can satisfy them entirely from the index
• Expression Pre-computation: The COALESCE logic is pre-calculated in the index, avoiding runtime computation

Previous vs New Index

Looking at the down() migration, the old index only covered (("flags"->>'hasUnreadPosts')) - missing sourceId entirely and not handling the COALESCE logic. Your new composite index is a significant improvement!

Verdict: This index will provide excellent performance for your query pattern. Well done! 🎯

---

[omBratteng]

@capJavert thoughts on using a new migration, vs just updating the migration and running it in production?

[capJavert]

don't see why new migration hurts, you can call it "Fix" or something to better show what you did.

[capJavert]

don't see why new migration hurts, you can call it "Fix" or something to better show what you did.