Improve title resolution; make it a background process

[code-geek]

Description

Right now title resolution is a foreground process that is long-running and can possibly time out. We want to defer this to the background and have a mechanism to track when it succeeds/fails.

Implementation Considerations

• Some way of storing success/failure status of title resolution in the database
• If two title resolution tasks are kicked off at the same time, which one takes precedence, especially given they run in the background and we can't predict their order.

Deliverable

• When a user adds a title pattern on the frontend, they are shown a message that their title pattern is being resolved
• The user can then track this title pattern and see the status: "pending", "resolved", "failed"
• If some title resolution fails, we need some logs
• A logging model with an admin seems sufficient

Dependencies

No response

[CarsonDavis]

For now, it is probably fine to put title resolution errors within the DeltaResolvedTitleError model. In the future, we may want to integrate it with the broader notifications/logs system that has been proposed.

[Kirandawadi]

Demo

• Background resolution of titles
• Can schedule resolution for multiple title patterns at once

Background_Title_Resolution.mov

[Kirandawadi]

Tests failing (Fixed Now)

• Currently, the title resolution task is running as expected when ran from the COSMOS but tests are failing.
• Here's how apply() method is modified.

    def apply(self) -> None:

        # Inserting here to avoid circular import issue
        from ..tasks import process_title_resolutions

        def queue_task():
            process_title_resolutions.delay(self.id)

        # Queue the background task only after the transaction commits (i.e, after apply() method)
        transaction.on_commit(queue_task)

• During tests, my guess is that the celery workers are not working properly and handling the background tasks as they should.

Methods Tried

• Set CELERY_ALWAYS_EAGER = True and CELERY_TASK_ALWAYS_EAGER = True on config/test.py to make async workflow to synchronous.
• Inherit TransactionTestCase for TestDeltaTitlePattern because we are dealing with transaction-related tests.

@pytest.mark.django_db(transaction=True)
class TestDeltaTitlePattern(TransactionTestCase):

• Marking test functions with decorators (transaction=True)

    @pytest.mark.django_db(transaction=True)
    def test_apply_generates_delta_url_if_title_differs(self):

• This method is working but it would be troublesome to write this everytime just to apply the pattern

        # Call the `apply()` method and capture/execute `on_commit` callbacks
        with self.captureOnCommitCallbacks(execute=True):  # Ensure `on_commit` callbacks are executed
             pattern.apply()