v1.1.2 - Improvements to Stripe object ingestion

Improvements to POST /stripe_from_pubsub, for better behavior with production-level traffic.

API

• POST /stripe_from_pubsub and POST /stripe return a 409 Conflict if the changes fail due to a database error, such as an IntegrityError due to duplicate IDs, or a deadlock. Previously, these returned 500 Server Error.
• If a Stripe customer is submitted that has the same Firefox Account ID (FxA ID) as an existing Stripe customer, the existing Stripe customer is deleted. This was seen on stage, but not production, and may be due to a bug or direct interaction with Stripe. The deletion most closely matches what we believe happens in the FxA Stripe cache, which is indexed by FxA ID.
• Stripe ingest now correctly updates invoice.default_source_id and invoice_line_item.stripe_subscription_item_id.

Deployments

• Database changes that may impact request timing and success:
  • Stripe ingest endpoints now use SELECT ... FOR UPDATE. This will hopefully reduce write conflicts, but may lead to increased deadlocks.
  • Duplicate FxA ID detection adds an extra database SELECT to customer creation and some customer updates (when the FxA ID changes), potentially slowing requests.
  • Stripe ingest updates now correctly parse Stripe timestamps as UTC timestamps rather than timezone-naive timestamps, which avoids database writes that do not change the data.
• Log changes:
  • Structured request logs for /stripe_from_pubsub and /stripe have context changes:
    • Added fxa_id_conflict, listing the FxA ID that was found on a new and existing Stripe customer.
    • Added ingest_actions to detail the contents of the Stripe payload. The keys are the action taken: created, updated, no_change, deleted, and skipped), and the values are a list of objects, represented as object_type:object_id, such as ["subscription:sub_abc123"].
    • Removed stripe_unknown. Unknown objects are now found in ingest_actions["skipped"].
  • An error-level log ("Severity": 3) is emitted with message "IntegrityError converted to 409", or "OperationalError converted to 409", for database exceptions handled by the Stripe endpoints. The log contains the context, and they are no longer sent to Sentry. This is followed by the standard request log for the 409 Conflict returned to the caller.
  • In the Acoustic sync service log message "sync_service cycle complete", the context retry_backlog now has the correct value. Previously, it was a duplicate of the sync_backlog value. The associated metric gauge ctms_background_acoustic_sync_retries had the correct value.

Other

• Updated from Python 3.9.7 to 3.9.9.
• Updated fastapi from 0.65.3 to 0.70.0, starlette from 0.14.2 to 0.16.0, and lxml from 4.6.4 to 4.7.1.
• Updated several documents:
  • Updated overview architecture diagram for Cinchy interaction and Stripe ingestion from FxA.
  • Added a Stripe entity diagram.
  • Synced docs/configuration.md with the current environment configuration, including adding the Acoustic Sync configuration variables.
  • Updated docs/deployment_guide.md, syncing with the current deployment methods, updating the logging section, and adding overview, metrics, and dashboards sections.
  • Updated docs/developer_setup.md with an "Updating Dependencies" section.