[HNT-2149] Switch curated recommendations to v2 GCS bucket

[mmiermans]

References

JIRA: HNT-2149
Related: SVCSE-4367 (decommission Merino GCP V1 project)
Related: mozilla/bigquery-etl#9105 (stop writing to v1 bucket)

Description

Switch curated_recommendations.gcs in production config from the v1 GCP project to v2:

	v1 (before)	v2 (after)
bucket	merino-airflow-data-prodpy	merino-airflow-data-prod
project	moz-fx-merino-prod-1c2f	moz-fx-merino-prod-5de4

This is required before the v1 project can be decommissioned. The v1 bucket is listed first in the bigquery-etl write loop, so if v1 were decommissioned without this change, the write would fail before reaching the v2 bucket, breaking curated recommendations data.

QA

Add merino/configs/development.local.toml to point at the v2 production bucket:

[development]
dynaconf_merge = true

[development.runtime]
skip_gcp_client_auth = true

[development.curated_recommendations.gcs]
bucket_name = "merino-airflow-data-prod"
gcp_project = "moz-fx-merino-prod-5de4"

Start Merino locally with MERINO_LOGGING__FORMAT=mozlog make dev, then:

1. Verify engagement data loads from v2 bucket:

grep "fetch_recommendation_engagement" /dev/stderr
# → "msg": "Cron: successfully ran task fetch_recommendation_engagement"

2. Verify engagement and prior metrics are non-zero:

grep -E "recommendation\.(engagement|prior)\.(size|count)" /dev/stderr
# → merino.recommendation.engagement.size:3007427
# → merino.recommendation.prior.size:832
# → merino.recommendation.engagement.count:20000
# → merino.recommendation.engagement.global.count:10068
# → merino.recommendation.engagement.us.count:4072

3. Verify Thompson Sampling uses the loaded engagement data:

curl -s -X POST localhost:8000/api/v1/curated-recommendations \
  -H 'Content-Type: application/json' \
  -d '{"locale":"en-US","region":"US","feeds":["sections"]}' \
  | jq '[.feeds | to_entries[:3][] | {key, items: (.value.recommendations | length)}]'
# → sections with recommendations, confirming TS is ranking with engagement data

4. Verify no GCS errors for curated recommendations:

grep -E "(ERROR|WARNING).*(recommendation\.(engagement|prior)|curated_rec)" /dev/stderr
# → (no output)

🤖 Generated with Claude Code

[jpetto]

short and sweet. do you think it would be beneficial to add a comment stating these values are for GCP V2? idk how convoluted those references are throughout the code base.