[WIP] Reconcilliation Service

[fgregg]

This OCDEP proposes a design spec and governance model for an opencivicdata entity resolution service. This will allow publishers of civic data use the same ocd ids while staying loosely coupled.

[fgregg]

@jamesturk @jpmckinney @aepton @djbridges @gordonje

I'm starting on this work here. Before I go further, I'd like two clarifications from this group

• I envision that the matching API will be supported, long term by our dedupe.io service. While we can make legal provisions to secure that support for OCD for some time period, DataMade will not make the source code of the dedupe.io service open. Therefore, this would have the reconciliation service depend upon a proprietary service.

Is this acceptable?

• Is a proposal for a design spec and governance model for an opencivicdata entity resolution service in scope for an OCDEP.

Thanks for your timely input.

[jpmckinney]

I don't have a clear idea of what the API design will be, but presumably we can design it such that whatever dedupe.io is doing can be a black box, and the OCD entity reconciliation service communicates with it in a standard, open, documented way? (or through a proxy if you don't want to change dedupe.io's API)

[fgregg]

@jpmckinney that's what I would like too.

[jamesturk]

I think my concerns are the same, but in general I'm 👍 and we can probably hash those concerns out more (if needed) once there's a proposal

…

On Sun, Jun 11, 2017 at 6:58 PM, Forest Gregg ***@***.***> wrote: @jpmckinney <https://github.com/jpmckinney> that's what I would like to. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#91 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAfYm-ac8pc389wC2VPos7P0DtfcMlrks5sDHELgaJpZM4N2Z0Q> .

[fgregg]

Okay, good. I just wanted to raise this early. On Mon, Jun 12, 2017 at 8:25 AM, James Turk <notifications@github.com> wrote:

…

I think my concerns are the same, but in general I'm 👍 and we can probably hash those concerns out more (if needed) once there's a proposal On Sun, Jun 11, 2017 at 6:58 PM, Forest Gregg ***@***.***> wrote: > @jpmckinney <https://github.com/jpmckinney> that's what I would like to. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#91# issuecomment-307662678>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAAfYm- ac8pc389wC2VPos7P0DtfcMlrks5sDHELgaJpZM4N2Z0Q> > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#91 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAgxbc89CzeSRwVOVtgYb88WqP1dSF_Jks5sDTw7gaJpZM4N2Z0Q> .

-- 773.888.2718

[aepton]

Would it be worth supporting both dedupe.io and https://github.com/newsdev/nyt-entity-service (the latter of which seems much less robust than dedupe, but Let A Thousand Flowers Bloom and all that - might help resolve some concerns around proprietariness if multiple (incl. some open-source) backends are supported?

[fgregg]

closing in favor of #92