Can metrics be a flat data file, or must it be a dynamic API?

[schnuerle]

Based on the data returned in the Metrics methodology, can we provide calculated metrics data at a granular enough level to be useful for consumers to filter and use what they need, thus reducing the producer's software burden? Or does it need to be a dynamic endpoint with multiple parameters to be useful, though increasing the technical burden for the producer?

Flat File

A flat Metrics file could be created that has available information for each defined Curb API geography (areas, zones, places) for every hour historically, and calculations could be left up to the consumer.

For example (not final format, just to illustrate an example with hourly measurements):

Areas
- h3j4k5
  -  total_events
    - 2021-10-13
      - 9,8,7,6,3,7,3,5,3,4,7,5,4,3,1,3,4,65,9,8,5,4,34,3
    - 2021-10-12
      - 5,3,4,7,5,4,3,1,3,4,65,9,8,5,4,34,3,9,8,7,6,3,7,3

Curbs
- k2l3d8
  - total_events
    - 2021-10-13
      - 7,5,4,3,1,3,4,65,9,8,5,4,34,3,9,8,7,6,3,7,3,5,3,4
  - turnover
    - 2021-10-13
      - 1,3,4,65,9,8,5,4,34,3,9,8,7,6,3,7,3,5,3,4,7,5,4,3
   - avg_dwell_time
    - 2021-10-13
      - 65,9,8,5,4,34,3,9,8,7,6,3,7,3,5,3,4,7,5,4,3,1,3,4

Spaces
- l2s9g3
  - occupancy_percent
    - 2021-10-13
      - 34,3,9,8,7,6,3,7,3,5,3,4,7,5,4,3,1,3,4,65,9,8,5,4
- s0m3t6
  - occupancy_percent
    - 2021-10-13
      - 6,3,7,3,5,3,4,7,5,4,3,1,3,4,65,9,8,5,4,34,3,9,8,7

Dynamic Endpoint

A queryable endpoint with parameters passed in to make it easy for the consumer to get only the metrics they want, reduce file size, and pass the work onto the producer.

The returned data would look like the above flat file with hourly metrics, but would be filtered down by combinations of things like:

• geographic bounding box
• start and end date(s)
• start and end hour(s)
• day(s) of the week
• list of relevant areas/zones/spaces by ID
• list of metrics

[jacobmalleau]

I think this relates to discussions on other APIs on how customizable calls should be compared to providing a consistent, larger set of information.
I personally lean towards the latter as it ultimately lets the end user choose what they want to do with the data.

[bhamlinSDOT]

In theory, if a city or third party is already hosting the event data, wouldn't a dynamic API using the agreed upon methodology be pretty straightforward to achieve?

Speaking from a city perspective, I would really like to see our Metric results via an easy to use and shareable dashboard or web map. If we have a folder on a hard drive that holds monthly flat files worth of data, they will likely be far less useful to the agency. It's not to say that they won't be used but they will be much less effective as a planning tool.

That said, having the capability to export your selections from the dynamic API as a flat file would be useful for smaller analysis that require manipulation in excel or some other software.

[schnuerle]

Making a dynamic API is not a trivial thing vs a static file. A static file could be created with a database query/script that is run monthly and then saved on a web/cloud drive/location. But an API would require programming, a scripting language, support for parameter combinations, and maybe rate limiting or account creation/keys to keep it stable.

Either way, the resulting data file could be used to create a shareable dashboard or web map. Being dynamic/static doesn't really impact the ability to create this, and the resulting file format would be the same either way (just with less data if dynamic).

A third party could handle a dynamic API, but it may be a barrier for some cities with less budget/internal tech.

[bhamlinSDOT]

After our Metrics meeting on 10/19, we determined that a flat file would be sufficient but an "option for dynamic API if city wants/needs" could be pursued.

Should that be interpreted as, if a city wants a dynamic API, they are on their own to develop one for now? (with or w/o a consultant) Or will CDS include some basic specifications that would allow dynamic API to exist?

[sfmta0kw]

My vote would be for providing basic specifications for a Dynamic API in CDS. This is something we would like to do in San Francisco to match our APIs used for metered parking. —————————————————————- Kenya Wheeler, AICP Principal Transportation Planner Parking & Curb Management Streets Division Pronouns: he/him Office 415.646.2717 ***@***.*** San Francisco Municipal Transportation Agency 1 South Van Ness Avenue, 8th floor San Francisco, CA 94103 SFMTA COVID-19 Transportation Resources (Transit, Parking, Biking/Walking/Slow Streets): https://www.sfmta.com/projects/covid-19-developments-response

…

________________________________ From: Brian Hamlin ***@***.***> Sent: Wednesday, November 10, 2021 5:42:46 PM To: openmobilityfoundation/curb-data-specification ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [openmobilityfoundation/curb-data-specification] Can metrics be a flat data file, or must it be a dynamic API? (Discussion #31) EXT After our Metrics meeting on 10/19, we determined that a flat file would be sufficient but an "option for dynamic API if city wants/needs" could be pursued. Should that be interpreted as, if a city wants a dynamic API, they are on their own to develop one for now? (with or w/o a consultant) Or will CDS include some basic specifications that would allow dynamic API to exist? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#31 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIF4CJK7HE2BKXKQXG7MZF3ULMNRNANCNFSM5F6DHRKA>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. This message is from outside of the SFMTA email system. Please review the email carefully before responding, clicking links, or opening attachments.

[schnuerle]

I've updated both endpoints in the draft to be optionally dynamic:

https://github.com/openmobilityfoundation/curb-data-specification/blob/feature-release-work-1/metrics/README.md#query-parameters

An agency may choose to make this endpoint static (and return all the available data at once in a single file) or dynamic (and allow the use of query parameters to filter the data).