Switch to sssom-java instead of sssom-py as the "reference implementation"

[matentzn]

sssom-java is maintained much more diligently and completely than sssom-py is. A reference implementation of a specification is supposed to (1) demonstrate the correct spec-conformant behaviour for processing SSSOM files and (2) serve as a reliable tool for users seeking to process (merge, canonicalise, etc) their SSSOM files according to the latest schema.

While Open Source ninjas like @cthoyt keep modernising sssom-py for our benefit, its massively behind in terms of spec support. I cant lift the update, and I dont tink anyone else can right now (tell me if I am wrong).

Therefore, I am motioning to declare that sssom-java is the new reference implementation. If I dont here complains in the next weeks, I will move to updating the documentation pages to that end.

Note: practically for most people, nothing much changes - the main consequence is that users will be pointed to sssom-java first when requesting access to a SSSOM CLI processor. sssom-py will still continue to be maintained, just limping behind.

[cthoyt]

I agree with this in light of most users being through the Java-based ontology stack and Damien's unique insight into how it fits into OWLAPI/ROBOT

I think some of the main difficulties with maintaining the Python implementation are:

1. Overcoming technical debt / weak testing
2. Dependence on LinkML stack (both in terms of performance, developer experience, and user experience)
3. Governance structure (too much burden is on Nico to keep up with external contributions)
4. Lack of documentation / examples / language-agnostic tests in the spec and lack of stricter standards for documentation when adding new parts to the spec (most important)

However, I still think it's worth maintaining the Python package, since this is how average programmers will interface with it. If we don't do this, then I fear SSSOM will only survive as an internal OBO community standard, instead of making bigger impact.

I'll offer time and expertise to this.

[sierra-moxon]

@matentzn - do you have someone willing to do the hard work of breaking down "massively behind in terms of spec support" into issues/tasks/concrete work assignments? I can find folks to help with LinkML, but I need specific tasks—no opinion on reference implementation.

[gouttegd]

For obvious reasons I will abstain from commenting on the main issue here – I neither approve nor disapprove the proposed change.

I simply want to highlight (as I have already done on Slack) that the ultimate goal should be not to have a reference implementation at all.

For a specification to have an anointed “reference implementation” is an admission that the spec is incomplete or imprecise. If the spec was sufficiently complete and precise, you wouldn’t need to look up to a reference implementation to check what the “spec-conformant behaviour” is!

The only situation where a reference implementation is actually needed is when the format or protocol is so complex that it is deemed impossible to fully and unambiguously describe it in plain natural language (I know of only one such case: the Opus audio codec, defined in RFC 6716). SSSOM hardly qualifies here, the format is certainly not complex (if it were, it wouldn’t deserve to be called the “Simple Standard…” :p ).

Now, it is of course quite common for a specification to be in fact incomplete and/or imprecise, and to need a reference implementation. Writing a fully complete, sufficient specification is hard, and by not being complete, SSSOM is in good company with many other widely-used-yet-incomplete specifications!

But that doesn’t mean that we should resign to the idea of having an incomplete spec. Declaring an implementation to be the “reference implementation” (or changing which implementation is considered to be the reference) is simply a stop-gap measure until we reach the ideal situation where the spec is truly self-sufficient.