Imported Feeds Not Appearing

[ianktc]

Describe the bug

Imported feeds from PR #1191 in the mobility-database-catalogs don't seem to appear in the database.

Steps/Code to Reproduce

Can verify these absent feeds in the mobility database by searching up the following:

• HK Just Use Wheels (mdb-2933)
• Fortaleza ETUFOR (mdb-2934)
• Fortaleza ARCE (mdb-2935)
• Alexandria DOT (mdb-2936)
• Marin (mdb-2937)
• Mandai (mdb-2938)

The accompanying realtime feeds for some of these schedule feeds are also not updated. It appears that none of the changes from this PR have been included in the database.

Expected Results

We should see these new feeds appear in the database.

Actual Results

The feeds are absent from the database.

Screenshots

No response

Files used

No response

Additional notes

The feeds appear in sources.csv, but do not appear in feeds_v2.csv.

I checked the catalog-sources-update workflow and #33 and #34 both failed on the first attempts but successfully executed after retries. This should imply that the prod database contains these feeds?

[jcpitre]

A bit of investigation:

• feat: gbfs error message for no versions #1191 was merged 2025-12-15 15:35
• The export-to-csv action ran at 15:41. The resulting csv contains the change.
• The uploaded https://bit.ly/catalogs-csv (a.k.a this address) contains the change.
• The time on https://bit.ly/catalogs-csv is 15:51:53.
• The catalog-update.yml ran at 15:51 (here)
• The first run failed because of a tunnel problem
• It was rerun at 15:59 with no error.
• So the data should be in the DB (It's not)

The logs here

• One of the changes in feat: gbfs error message for no versions #1191 is for realtime feed 1579, where the static reference was changed from 71 to 2937.
• The log says:

INFO:GTFSDatabasePopulateHelper:Set feed references for mdb-1579: ['mdb-71'] -> ['mdb-71']

• It looks like the old value is used.

[jcpitre]

Ran locally. I get the proper result and the logs show this:

INFO:GTFSDatabasePopulateHelper:Set feed references for mdb-1579: ['mdb-71'] -> ['mdb-2937']

So I suspect the wrong source.csv is used.

[jcpitre]

Reran the catalog-update.yml and it worked. Spot checked that the changes were in the DB.

I can only surmise that the wrong sources.csv was used.
It's strange. In this run the sequence of events is:

• create the csv from the json files
• upload the csv as an artifact of the run (This lets us examine it and see that the file contains the right data)
• Download the same csv artifact of the run
• upload to the goole bucket (I examined the uploaded file and it contains the right data)
• Notify the mobility-feed-api/catalog-update.yml workflow
• This workflow downloads the file with wget -O sources.csv https://bit.ly/catalogs-csv for prod, qa and dev.
  • So it's downloaded 3 times, but with the same destination! i.e. it's written 3 times with the same name on the local disk (local to the workflow)
  • The populate is run on dev, qa and prod concurrently. That means the file can be written by one thread while another is trying to read it.
  • Not sure if this is the problem, but it's wrong and should be corrected.