Integrate Opencivicdata identifiers

[showerst]

Understanding this is a massive lift, it would be really cool if you could add OpenCivicData identifiers to the metadata where possible.

I know openelections uses them, so maybe it can be integrated into your workflow somehow.

[nvkelso]

Maybe? It would help to have some examples. We have data in a couple states, counties, and cities so far in the data dir. What would their concordances with OpenCivicData look like? Right now we've been relying on FIPS codes.

…

On Tue, Mar 7, 2017 at 2:41 PM, showerst ***@***.***> wrote: Understanding this is a massive lift, it would be really cool if you could add OpenCivicData <https://github.com/opencivicdata/ocd-division-ids> identifiers to the metadata where possible. I know openelections uses them, so maybe it can be integrated into your workflow somehow. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#56>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA0EO3EJ-8sX1z6LAByOe6wSXvvMrYQTks5rjd0vgaJpZM4MWIEs> .

[migurski]

I'm also interesting to know more about OCD identifiers. Who issues them, and how are they used?

[showerst]

OCD's division IDs are an attempt at a common standard for identifying political divisions.

The nice thing about it is that there's some uptake from a variety of projects; Google's Civic information API, Open Elections, and the OpenCivicData Legislative Scrapers.

The value here would be being able to cross reference these with all their data without writing translation layers. I'm not actually involved in the OCD Division IDs other than as a user, but the repo is being run by a bunch of known characters in the civic data world.

If everything in this project will likely have a FIPS code it's less of an issue since those are well standardized, but if you're looking at other levels of division it would be a good standard to pick. I'll ask @jamesturk if there's an easy FIPS->OCD mapping somewhere, and either PR or at least link it so you can consider it.

[jamesturk]

In addition to the uses mentioned above (Google clearly being the biggest force driving adoption) I'm told they are now used in some company's voterfiles as well, driven by their use in the Voting Information Project: https://votinginfoproject.org/

At yesterday's Google-hosted gathering for people working on this data I was actually surprised to see how ubiquitous they'd become among people working w/ election geographies.

FIPS codes are definitely good for state & county, but if you want to have identifiers to link to other data at a level beneath that (SLDL, SLDU, precinct, school board, etc.) OCD Division IDs are probably the way to go. Other IDs you'll find (Census geoid, etc.) aren't available across all data sets and aren't stable.

If it'll be helpful, a FIPS mapping would be a good idea and a pretty easy lift from OCD's end.

[migurski]

Thanks James! Do they exist for precincts, and is such a list gettable someplace?

[jamesturk]

https://github.com/opencivicdata/ocd-division-ids/tree/master/identifiers/country-us there are precinct IDs but they're at ~2 years old right now. it looks like Aaron Strauss and Mike Jensen were maintaining them

[migurski]

Thanks! I don’t see a place in the identifiers for a year. Since districts and precincts may change from year to year, how do OCD identifiers track that kind of change?

[jamesturk]

The identifiers stay the same but can be associated w/ different geographies over time. There are also associated attributes in the ocd-division-id repository (validThrough, etc.) that let a division be retired.

…

On Mon, Mar 13, 2017 at 2:15 AM, Michal Migurski ***@***.***> wrote: Thanks! I don’t see a place in the identifiers for a year. Since districts and precincts may change from year to year, how do OCD identifiers track that kind of change? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#56 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAfYu3SnfLpBaOfuhLIxDi49FkZ35loks5rlN8TgaJpZM4MWIEs> .

[migurski]

Thanks for the clarification, James. I’d like to push on that a little: in the case of legislative districts and voting precincts, the meanings can change over time. Today’s “5th District” is only the same as yesterday’s in the sense that it’s the fifth item on a list. OCD identifiers would be helpful here if they offered some promise of uniqueness and stability, rather than restating other parts of a unit’s name. Is this a situation you’ve encountered with OCD before?

[jamesturk]

Could you give an example of what you're referring to? not sure I'm following

…

On Thu, Mar 16, 2017 at 2:02 AM, Michal Migurski ***@***.***> wrote: Thanks for the clarification, James. I’d like to push on that a little: in the case of legislative districts and voting precincts, the meanings can change over time. Today’s “5th District” is only the same as yesterday’s in the sense that it’s the fifth item on a list. OCD identifiers would be helpful here if they offered some promise of uniqueness and stability, rather than restating other parts of a unit’s name. Is this a situation you’ve encountered with OCD before? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#56 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAfYiXNycKlaM9rBo7yl50LqF4h5Kajks5rmNBYgaJpZM4MWIEs> .

[migurski]

Sure, yeah. If you look at Florida’s 5th, it’s changed substantially in the past year. The pre-2016 and post-2016 areas it covers are almost disjoint, so there’s not really a meaningful way for a single OCD ID to point to them both.

Florida 5 now looks like this:

Previously, it looked like this:

Conversely, Barbara Lee has been my representative for almost 20 years, but during that time her district has been the 9th and later the 13th district of California.

[jamesturk]

Ah I see, ocd-ids do not deal w/ geography directly, as you correctly note there's a sort of existential question if there is any meaning between FL-5(1990) and FL-5(2010). In fact, many OCD IDs cannot even correctly be represented by geography (precincts, zip codes, etc.) ocd-ids solve a different problem of providing a common name across systems, not the problem of boundaries/etc. Systems that deal w/ them wind up storing geometries/etc. separately: (ocd-division/country:us/state:fl/cd:5) Geometries: (ocd-division/country:us/state:fl/cd:5, 1990-01-01, 1999-12-31, {....}) Voter File: (ocd-division/country:us/state:fl/cd:5, 1990-01-01, 1999-12-31, abcdef-0000-0000-123456) Voter File Address Range would then be tied to the above guid, not to the ID directly

…

On Thu, Mar 16, 2017 at 9:54 PM, Michal Migurski ***@***.***> wrote: Sure, yeah. If you look at Florida’s 5th, it’s changed substantially in the past year. The pre-2016 and post-2016 areas it covers are almost disjoint, so there’s not really a meaningful way for a single OCD ID to point to them both. Florida 5 now looks like this: <https://camo.githubusercontent.com/403f728de17a8554267c4b8d92c0e2203db165ca/687474703a2f2f6d63696d6170732e636f6d2f77702d636f6e74656e742f75706c6f6164732f323031352f31322f4e65772d44697374726963742d31303234783739312e706e67> Previously, it looked like this: <https://camo.githubusercontent.com/77aac299125d7db1c5e6d52a85776d83460a23ed/68747470733a2f2f75706c6f61642e77696b696d656469612e6f72672f77696b6970656469612f636f6d6d6f6e732f7468756d622f662f66372f466c6f726964615f55535f436f6e6772657373696f6e616c5f44697374726963745f355f25323873696e63655f323031332532392e7469662f6c6f73736c6573732d70616765312d3137333870782d466c6f726964615f55535f436f6e6772657373696f6e616c5f44697374726963745f355f25323873696e63655f323031332532392e7469662e706e67> Conversely, Barbara Lee has been my representative for almost 20 years, but during that time her district has been the 9th and later the 13th district of California. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#56 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAfYjJ6JRAdFAgzGVP8OLWlrMbkp7W0ks5rmefRgaJpZM4MWIEs> .

[migurski]

That’s helpful clarification, thank you. OCD seems a bit less helpful if it doesn't deal with uniqueness, but it sounds like it might still help with external concordances as a long as a user understands that the same OCD ID may or may not identify the same entity across time.

[migurski]

Okay, followup question. http://editor.opencivicdata.org/geo/select/ is down, so what would be a correct pattern for a precinct like we're storing here?

ocd-division/country:us/state:{postal-abbrev}/county:{name}/precinct:{?}

They’re also variously called "wards" and confusingly "districts" in other contexts, and the names can be complex, e.g. "Town Of Brighton Ward 1" for an example in Wisconsin:

ocd-division/country:us/state:wi/county:kenosha/precinct:Town Of Brighton Ward 1