Proposal: Lexicon schema resolution lexicon

[tom-sherman]

This lexicon is based off of the discussion in https://github.com/orgs/lexicon-community/discussions/42 and my experience with building lpm.

This lexicon (just a single procedure right now) defines an interface for an appview that would be used in lpm to form a package manager for lexicons. The goal is for it to be generic though so it could drive other package managers, registries, or schema-related tooling. (This also justifies me adding it to community.lexicon and not some lpm NSID 😅)

The AppView fills the problem space described in the lexicon resolution RFC:

For services which do dynamic resolution and validation, we think that making requests to aggregation/index services could be a better option instead of doing live resolution. Such services can act as CDNs, archives, and mediating services to resolve disputes or security issues in Lexicons.

License and Assignment

• I assign all rights of this contribution to Lexicon Community as an open source contribution under the MIT license.

[tom-sherman]

It would be nice to encode this type of "at uri with optional cid" as a shared type as I think it's probably valuable in many use cases. Naming is hard tho, community.lexicon.meta.optionallyStrongRef? lol

[verdverm]

weakRef?

[tom-sherman]

It might be strong tho if the CID is included! Sometimes it's weak, sometimes it's strong

[tom-sherman]

Dependency strong refs are returned so that a package manager could resolve a point-in-time snapshot of the lexicon tree. Discussion about that here.

The actual CID that the appview returns would be implementation specific and so could either return the CIDs at the time the parent was indexed, the latest CIDs regardless of parent CID that was requested, or something else.

[verdverm]

Should this be a query + parameters since it would be a non-modifying GET operation?

[tom-sherman]

I meant to leave a comment about this. It's not really possible to express the input as parameters without inventing some new string format that's like at://identity/nsid/rkey#cid or something.

I'd prefer not to do that invention here (it should be an atproto core spec) but also stuffing those kinds of strings into the query string would reduce the amount of lexicons this endpoint could return I think (because of URL length limits of many systems and tooling).

[verdverm]

I'm not sure there needs to be any inventing here. Lexicon retrieval is more or less a getRecord which includes all 4 fields in the parameters

https://github.com/bluesky-social/atproto/blob/main/lexicons/com/atproto/repo/getRecord.json

[tom-sherman]

This endpoint returns multiple lexicons, not a single one. It's closer in spirit to app.bsky.feed.getPosts than it is to getRecord.

[tom-sherman]

There's literally no way to encode a "list of at URIs with optional CIDs" without inventing a new string format.

[verdverm]

It can be helpful to note direct vs indirect dependencies.

I also wonder about what "modules" mean in ATProto, maybe beyond the scope here...? As things work today, each lexicon is an independent module, however we typically think of modules as a collection of related types versioned together. (Using "module" like Go (as a collection of packages), where as JS/TS use "package" for the same conceptual grouping)

[tom-sherman]

It can be helpful to note direct vs indirect dependencies.

Clarified that, hopefully!

Not sure what you mean about modules, I'm not looking to add a new concept with this schema. It's really just a thin layer on top of the protocol (with caching and backlinks from an identity in the form of listLexicons)

[verdverm]

my comment about "modules" may be beyond the scope here. To clarify a bit, consider the situation where someone updates the record lexicon and the query lexicon associated with it. This is a single logical change, which we cannot group logically. It's like if today, every type and function in a TypeScript package was an independent dependency in the npm world.

w.r.t. this work, how do we know that related lexicon are aligned in the dependency list? Is that something that is not being worried about at this point?