feat: JBDA import #1407

cka-y · 2025-10-20T13:26:44Z

Summary:
Closes https://github.com/MobilityData/product-tasks/issues/154
Closes https://github.com/MobilityData/product-tasks/issues/132
This PR support for "related links" on feeds, adds a new JBDA import task, and includes associated model, API, and test updates. The most significant changes are the addition of the FeedRelatedLink model and its integration into feed objects, updates to the API schema and database handling, and the implementation and testing of the JBDA import workflow.
Schedule feed:

Realtime feed:

Please make sure these boxes are checked before submitting your pull request - thanks!

Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
Add or update any needed documentation to the repo
Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
Linked all relevant issues
Include screenshot(s) showing how this pull request works and fixes the issue(s)

github-actions · 2025-10-20T13:40:10Z

*Lighthouse ran on https://mobility-feeds-dev--pr-1407-3zugjph7.web.app/ * (Desktop)
⚡️ HTML Report Lighthouse report for the changes in this PR:

Performance	Accessibility	Best Practices	SEO
🟠 57	🟢 100	🟢 100	🟢 100

*Lighthouse ran on https://mobility-feeds-dev--pr-1407-3zugjph7.web.app/feeds * (Desktop)
⚡️ HTML Report Lighthouse report for the changes in this PR:

Performance	Accessibility	Best Practices	SEO
🟠 75	🟢 91	🟢 100	🟢 100

*Lighthouse ran on https://mobility-feeds-dev--pr-1407-3zugjph7.web.app/feeds/gtfs/mdb-2126 * (Desktop)
⚡️ HTML Report Lighthouse report for the changes in this PR:

Performance	Accessibility	Best Practices	SEO
🟠 52	🟢 95	🟢 100	🟢 100

*Lighthouse ran on https://mobility-feeds-dev--pr-1407-3zugjph7.web.app/feeds/gtfs_rt/mdb-2585 * (Desktop)
⚡️ HTML Report Lighthouse report for the changes in this PR:

Performance	Accessibility	Best Practices	SEO
🟢 90	🟠 88	🟢 100	🟢 100

*Lighthouse ran on https://mobility-feeds-dev--pr-1407-3zugjph7.web.app/gbfs/gbfs-flamingo_porirua * (Desktop)
⚡️ HTML Report Lighthouse report for the changes in this PR:

Performance	Accessibility	Best Practices	SEO
🟢 100	🟢 100	🟢 100	🟢 100

github-actions · 2025-10-20T13:40:20Z

Preview Firebase Hosting URL: https://mobility-feeds-dev--pr-1407-3zugjph7.web.app

Alessandro100 · 2025-10-20T18:14:35Z

[Question] For the Related Links section in the UI, is this solely reserved for future gtfs feed url, or can it be anything related to the feed?

web-app/src/app/screens/Feed/components/FeedSummary.tsx

cka-y · 2025-10-20T18:18:01Z

[Question] For the Related Links section in the UI, is this solely reserved for future gtfs feed url, or can it be anything related to the feed?

It can be any related link. From the JBDA api i was only able to retrieve next_1 links, but their api supports next_1, next_2, prev_1 and prev_2. I also made the type generic enough so that it can be used for any feeds (including rt and gbfs)

Alessandro100 · 2025-10-20T18:32:25Z

[Question] For the Related Links section in the UI, is this solely reserved for future gtfs feed url, or can it be anything related to the feed?

It can be any related link. From the JBDA api i was only able to retrieve next_1 links, but their api supports next_1, next_2, prev_1 and prev_2. I also made the type generic enough so that it can be used for any feeds (including rt and gbfs)

When I see 'Related links' I assume it could mean anything ex: social media urls, feeds, news about them, etc. If this is about feeds, I'd rename it to Related Feeds. And rename "next_1" -> "Future Feed", next_2 -> "Future Feed (2)", "prev_1" -> "Previous Feed" etc.

cka-y · 2025-10-20T18:34:08Z

[Question] For the Related Links section in the UI, is this solely reserved for future gtfs feed url, or can it be anything related to the feed?

It can be any related link. From the JBDA api i was only able to retrieve next_1 links, but their api supports next_1, next_2, prev_1 and prev_2. I also made the type generic enough so that it can be used for any feeds (including rt and gbfs)

When I see 'Related links' I assume it could mean anything ex: social media urls, feeds, news about them, etc. If this is about feeds, I'd rename it to Related Feeds. And rename "next_1" -> "Future Feed", next_2 -> "Future Feed (2)", "prev_1" -> "Previous Feed" etc.

It could be social media. I think the goal was to keep it generic enough.

functions-python/tasks_executor/src/tasks/data_import/data_import_utils.py

functions-python/tasks_executor/src/tasks/data_import/import_jbda_feeds.py

infra/functions-python/vars.tf

liquibase/changes/feat_pt_154.sql

Alessandro100 · 2025-10-20T18:49:40Z

[Question] For the Related Links section in the UI, is this solely reserved for future gtfs feed url, or can it be anything related to the feed?

It can be any related link. From the JBDA api i was only able to retrieve next_1 links, but their api supports next_1, next_2, prev_1 and prev_2. I also made the type generic enough so that it can be used for any feeds (including rt and gbfs)

When I see 'Related links' I assume it could mean anything ex: social media urls, feeds, news about them, etc. If this is about feeds, I'd rename it to Related Feeds. And rename "next_1" -> "Future Feed", next_2 -> "Future Feed (2)", "prev_1" -> "Previous Feed" etc.

It could be social media. I think the goal was to keep it generic enough.

Ah ok if it can be anything (even outside of feed related stuff) then related links as a title makes sense. I still think that "Future / Previous Feed" would be a better more descriptive title vs "next_1" "prev_1"

cka-y · 2025-10-20T18:50:57Z

think that "Future / Previous Feed

next_1 is actually the code taken straight from the JBDA api. We could introduce mapping from code to better description. This could be outside of the scope of this PR

functions-python/tasks_executor/src/tasks/data_import/import_jbda_feeds.py

emmambd

@cka-y Discussed changes written here. @Alessandro100 let me know if you have any additional thoughts.

docs/DatabaseCatalogAPI.yaml

web-app/src/app/screens/Feed/components/FeedSummary.tsx

functions-python/tasks_executor/src/tasks/data_import/import_jbda_feeds.py

davidgamez

Amazing work! 🥇

cka-y added 2 commits October 20, 2025 09:24

feat: jbda import

3d93645

merge: main

b1dc6a7

merge: main

9124cc5

cka-y changed the title ~~Feat/pt 154~~ feat: JBDA import Oct 20, 2025

fix: tests

8d331bf

cka-y marked this pull request as ready for review October 20, 2025 16:27

cka-y added 5 commits October 20, 2025 12:42

fix: trigger dataset download

d9f02c5

fix: var name

bb321ee

fix: downstream tasks tested

44d2ded

fix: missing schedule

c576fe8

fix scheduler payload

6fc3950

Alessandro100 reviewed Oct 20, 2025

View reviewed changes

web-app/src/app/screens/Feed/components/FeedSummary.tsx Outdated Show resolved Hide resolved

pr comments

dd0c372

davidgamez reviewed Oct 20, 2025

View reviewed changes

jcpitre reviewed Oct 21, 2025

View reviewed changes

functions-python/tasks_executor/src/tasks/data_import/import_jbda_feeds.py Outdated Show resolved Hide resolved

emmambd requested changes Oct 21, 2025

View reviewed changes

docs/DatabaseCatalogAPI.yaml Outdated Show resolved Hide resolved

docs/DatabaseCatalogAPI.yaml Show resolved Hide resolved

web-app/src/app/screens/Feed/components/FeedSummary.tsx Show resolved Hide resolved

cka-y commented Oct 21, 2025

View reviewed changes

functions-python/tasks_executor/src/tasks/data_import/import_jbda_feeds.py Show resolved Hide resolved

cka-y added 5 commits October 27, 2025 14:43

merge: main

d1e9afe

functionnal code

c1e4bbd

fix: pr comment

49f6c9d

fix: pr comments

a5db774

fix: ts types

e6bd3c0

cka-y added 3 commits October 28, 2025 13:13

Merge branch 'main' into feat/pt-154

f0c970e

fix: operation api model

1236542

fix: missing timestamp

a100ffc

davidgamez approved these changes Oct 28, 2025

View reviewed changes

cka-y requested a review from emmambd October 28, 2025 18:09

emmambd approved these changes Oct 28, 2025

View reviewed changes

cka-y merged commit 5a109be into main Oct 28, 2025
6 of 7 checks passed

cka-y deleted the feat/pt-154 branch October 28, 2025 18:32

feat: JBDA import #1407

feat: JBDA import #1407

Uh oh!

Conversation

cka-y commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 20, 2025

Uh oh!

Alessandro100 commented Oct 20, 2025

Uh oh!

Uh oh!

cka-y commented Oct 20, 2025

Uh oh!

Alessandro100 commented Oct 20, 2025

Uh oh!

cka-y commented Oct 20, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Alessandro100 commented Oct 20, 2025

Uh oh!

cka-y commented Oct 20, 2025

Uh oh!

Uh oh!

emmambd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davidgamez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

cka-y commented Oct 20, 2025 •

edited

Loading

github-actions bot commented Oct 20, 2025 •

edited

Loading