OMF Transportation Serialization logic#66
Conversation
There was a problem hiding this comment.
Pull request overview
This PR implements a serialization pipeline for Overture Maps Foundation (OMF) transportation data. It introduces a TransportationCollection struct that downloads connectors and segments, transforms them into an OmfGraphVectorized structure, and writes vertices and edges to compressed CSV files in a format compatible with RouteE Compass.
Key changes:
- Adds graph module with splitting logic to convert transportation segments into graph edges
- Creates
TransportationCollectionto aggregate downloaded connectors and segments - Implements CSV serialization with gzip compression for vertices and edges
- Adds CLI application scaffolding with a Download operation
Reviewed changes
Copilot reviewed 16 out of 17 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| rust/bambam-omf/src/lib.rs | Exports new app and graph modules |
| rust/bambam-omf/src/graph/omf_graph.rs | Core graph structure with CSV writing logic |
| rust/bambam-omf/src/graph/segment_split.rs | Defines split logic for creating edges from segments |
| rust/bambam-omf/src/graph/serialize_ops.rs | Helper functions for connector mapping and segment splitting |
| rust/bambam-omf/src/graph/vertex_serializable.rs | Serialization format for vertices |
| rust/bambam-omf/src/graph/mod.rs | Module exports for graph components |
| rust/bambam-omf/src/collection/record/transportation_segment.rs | Adds distance calculation and makes fields public |
| rust/bambam-omf/src/collection/record/transportation_connector.rs | Adds vertex conversion logic and makes id public |
| rust/bambam-omf/src/collection/record/transportation_collection.rs | New struct to aggregate connectors and segments |
| rust/bambam-omf/src/collection/record/mod.rs | Exports TransportationCollection |
| rust/bambam-omf/src/collection/mod.rs | Re-exports transportation types |
| rust/bambam-omf/src/collection/error.rs | Adds new error variants for serialization |
| rust/bambam-omf/src/bin/bambam_omf.rs | CLI binary entry point |
| rust/bambam-omf/src/app/serialize_options.rs | Unused configuration structs (not integrated) |
| rust/bambam-omf/src/app/omf_app.rs | CLI app structure with hardcoded test values |
| rust/bambam-omf/src/app/mod.rs | Module exports for app |
| rust/bambam-omf/Cargo.toml | Adds dependencies for CLI, compression, and itertools |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
rust/bambam-omf/src/collection/record/transportation_collection.rs
Outdated
Show resolved
Hide resolved
| if filepath.exists() && !overwrite { | ||
| return None; | ||
| } | ||
| let file = File::create(filepath).unwrap(); |
There was a problem hiding this comment.
File::create can fail if the directory doesn't exist or permissions are insufficient. Using unwrap() here will cause a panic instead of returning a proper error. This should return a Result and propagate the error properly using map_err, similar to how other errors are handled in write_compass.
| let row_filter_config = RowFilterConfig::Bbox { | ||
| xmin: -105.254, | ||
| xmax: -105.197, | ||
| ymin: 39.733, | ||
| ymax: 39.784, | ||
| }; | ||
|
|
||
| let collection = TransportationCollection::try_from_collector( | ||
| collector, | ||
| release, | ||
| Some(row_filter_config), | ||
| )?; | ||
| let vectorized_graph = OmfGraphVectorized::try_from_collection(collection, 0)?; | ||
| vectorized_graph.write_compass(Path::new("./"), true)?; |
There was a problem hiding this comment.
Hardcoded test/development values for production code. The bounding box coordinates (Golden, CO area) and output path ("./") are hardcoded in what appears to be production CLI code. These should be command-line arguments or configuration parameters to make the tool usable for different areas and output locations.
robfitzgerald
left a comment
There was a problem hiding this comment.
LGTM! have a great holiday! 🎄
rust/bambam-omf/src/app/omf_app.rs
Outdated
|
|
||
| #[derive(Debug, Clone, Serialize, Deserialize, Subcommand)] | ||
| pub enum OmfOperation { | ||
| /// download all of the GTFS archives |
There was a problem hiding this comment.
incorrect doc comment
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…n.rs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
In this PR, I implement the pipeline for serializing vertices and edges into compressed csv files. Crucially, I define
TransportationCollectionto be the downloaded set of connectors and segments that then gets transformed into aOmfGraphVectorized, which knows how to split segments into vertices and edges and how to write them into a csv.This setup also allows for future implementations of splitting logic that go beyond just connectors.