Skip to content

Conversation

@cka-y
Copy link
Contributor

@cka-y cka-y commented Feb 28, 2025

Summary:
Closes #1975

  • Updated the duplicate_key notice to include only defined field names and values for composite keys.
  • Fixed mix up between oldCsvRowNumber and newCsvRowNummber

Expected behavior:
Screenshot 2025-02-28 at 9 35 27 AM

Please make sure these boxes are checked before submitting your pull request - thanks!

  • Run the unit tests with gradle test to make sure you didn't break anything
  • Add or update any needed documentation to the repo
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)

@github-actions
Copy link
Contributor

📝 Acceptance Test Report

📋 Summary

✅ The rule acceptance has passed for commit 46ce307
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Errors (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

New Warnings (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Warnings (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1824 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.80 3.92 ⬆️+0.12
Median -- 1.34 1.44 ⬆️+0.10
Standard Deviation -- 11.18 11.26 ⬆️+0.08
Minimum in References Reports us-california-city-of-wasco-gtfs-1788 0.48 0.57 ⬆️+0.09
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 286.14 288.76 ⬆️+2.61
Minimum in Latest Reports us-california-san-juan-capistrano-free-weekend-trolley-gtfs-2235 0.48 0.48 ⬇️-0.00
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 286.14 288.76 ⬆️+2.61
📜 Memory Consumption
Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 468.55 MiB 470.88 MiB ⬆️+2.32 MiB
Median -- 335.92 MiB 335.92 MiB ⬇️0 bytes
Standard Deviation -- 774.86 MiB 783.31 MiB ⬆️+8.44 MiB
Minimum in References Reports us-south-dakota-sioux-area-metro-sam-gtfs-192 38.51 MiB 59.21 MiB ⬆️+20.71 MiB
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 11.01 GiB 11.29 GiB ⬆️+287.23 MiB
Minimum in Latest Reports us-california-yubasutter-transit-gtfs-79 53.07 MiB 39.57 MiB ⬇️-13.50 MiB
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 11.01 GiB 11.29 GiB ⬆️+287.23 MiB

@emmambd emmambd requested a review from skalexch February 28, 2025 16:35
Copy link
Contributor

@skalexch skalexch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good on the spec side. Thanks for working on this!

@github-actions
Copy link
Contributor

📝 Acceptance Test Report

📋 Summary

✅ The rule acceptance has passed for commit c99e359
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (1 out of 1824 datasets, ~0%) ✅

Details of new errors due to code change, which is less than the provided threshold of 1%.

Dataset Notice Code
us-california-san-mateo-county-transit-district-samtrans-gtfs-49 missing_required_column
Dropped Errors (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

New Warnings (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Warnings (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1824 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.74 3.79 ⬆️+0.05
Median -- 1.36 1.41 ⬆️+0.06
Standard Deviation -- 10.96 10.99 ⬆️+0.04
Minimum in References Reports us-california-catalina-express-gtfs-299 0.47 0.50 ⬆️+0.04
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 291.48 295.17 ⬆️+3.70
Minimum in Latest Reports us-indiana-south-shore-line-gtfs-585 0.48 0.48 ⬇️-0.01
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 291.48 295.17 ⬆️+3.70
📜 Memory Consumption
Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 471.54 MiB 469.02 MiB ⬇️-2.52 MiB
Median -- 335.92 MiB 335.92 MiB ⬇️0 bytes
Standard Deviation -- 798.67 MiB 784.07 MiB ⬇️-14.60 MiB
Minimum in References Reports us-ohio-allen-county-regional-transit-authority-gtfs-210 39.26 MiB 407.92 MiB ⬆️+368.66 MiB
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 11.10 GiB 11.10 GiB ⬆️+601.54 KiB
Minimum in Latest Reports us-california-kings-area-rural-transit-gtfs-42 403.92 MiB 38.88 MiB ⬇️-365.04 MiB
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 11.10 GiB 11.10 GiB ⬆️+601.54 KiB

Copy link
Member

@davidgamez davidgamez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@cka-y cka-y merged commit 0ca8740 into master Mar 3, 2025
132 checks passed
@cka-y cka-y deleted the feat/1975 branch March 3, 2025 17:20
@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2025

📝 Acceptance Test Report

📋 Summary

✅ The rule acceptance has passed for commit 164a860
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Errors (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

New Warnings (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Warnings (0 out of 1824 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1824 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.73 3.85 ⬆️+0.12
Median -- 1.35 1.44 ⬆️+0.09
Standard Deviation -- 11.07 11.20 ⬆️+0.13
Minimum in References Reports us-florida-citrus-county-transit-gtfs-630 0.47 0.57 ⬆️+0.10
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 289.52 295.86 ⬆️+6.33
Minimum in Latest Reports ml-bamako-somatra-gtfs-1807 0.53 0.54 ⬆️+0.01
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 289.52 295.86 ⬆️+6.33
📜 Memory Consumption
Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 470.06 MiB 458.99 MiB ⬇️-11.07 MiB
Median -- 335.92 MiB 335.92 MiB ⬇️0 bytes
Standard Deviation -- 787.25 MiB 731.63 MiB ⬇️-55.62 MiB
Minimum in References Reports us-iowa-citibus-gtfs-2304 39.70 MiB 411.92 MiB ⬆️+372.22 MiB
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 10.84 GiB 11.00 GiB ⬆️+164.65 MiB
Minimum in Latest Reports us-mississipi-jtran-gtfs-155 407.92 MiB 38.91 MiB ⬇️-369.01 MiB
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 10.84 GiB 11.00 GiB ⬆️+164.65 MiB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

duplicate_key: dynamically changing what's included in fieldValue1 column for fare_transfer_rules.txt

4 participants