A pithy primer of `Polykey`

[tegefaulkes]

Polykey is a kind of a complex program. It is made up groups of functionality we call domains These are a collection of classes, utilities and types, some are made up of a single class.

For example, we have a domain called keys. This handles the base level password, keys and certificate information that the rest of Polykey makes use of. It contains 2 main classes, the CertManager and the KeyRing. What they each do is self explanatory and I'll leave it to you to dig deeper.

Keep in mind that for the most part each of these classes are decorated with an async-init giving the class a life-cycle. So the KeyRing is a CreateDestroyStartStop structure. Which just means that it has implemented a CreateKeyRing static factory function for creating the class. And contains a start, stop and destroy method. Any method inside KeyRing that is decorated with @ready can only be called when the KeyRing is in the running state. Calling stop will cause it to stop and active state and calling destroy will clear any persistent state. That's all you really need to know for now. You will see this pattern everywhere in Polykey and even other projects.

Polykey can be subdivided into a few aspects.

• Cryptography features for handling keys, passwords, certificates and claims. Handled by the keys, claims and sigChain domains
• Networking code for making and handling connections between nodes, Handled by the nodes and discovery domains.
• Tracking relationships and permissions, handled by the gestalts, sigchain and ACL domains.
• Storing encrypted secrets, handled by the vaults domain.
• Client level control and communication, handled via the client domain.

There are some others but those are the main ones. If you look into

Polykey/src/PolykeyAgent.ts

Line 244 in 2a88416

try {

to see how they are all created and depend on each other. Look inside the CreatePolykeyAgent method inside the main try {} catch block.

Arguably the most important domain for Polykey as a product is the vaults domain. It's where all of the secret handling is and maybe one of the easier domains to read through. For that you'll want to start at the VaultManager https://github.com/MatrixAI/Polykey/blob/staging/src/vaults/VaultManager.ts. Get a feel for that and the dig down into VaultInternal and how that handles the data. Keep an eye out for the withF pattern. You'll see that all over the place.

After that the nodes domain is the next important. It handles all of the logic for managing connections in the NodeConnectionManager. Notably tracking data about other nodes in the NodeGraph and handling all of the logic for finding nodes you want to connect to and establishing that connection in the NodeManager. All of the connection related stuff is pretty complex so just skim over this. Dig into how the NodeConnectionManager https://github.com/MatrixAI/Polykey/blob/staging/src/nodes/NodeConnectionManager.ts handles creating connections and how the NodeConnection https://github.com/MatrixAI/Polykey/blob/staging/src/nodes/NodeConnection.ts wraps them. It uses an object map and locking to ensure that we don't create duplicate NodeConnections. Skim over what NodeManager handles, especially the logic for finding nodes and NodeConnectionQueue, its pretty complex. Besides that, some good related reading is the kademlia spec https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf. Get an idea of how closeness works as a concept in Kademlia.

Polykey uses a lot of persistent state, this will be stored in the DB as encrypted data. So to get a good feel of what domains need data persistence just trace what depends on the DB. Along side this the keys domain maintains most of the information used for they cryptography functions. It stores the private and public keys. But also manages encryption keys for the DB and vaults domain.

Polykey needs to track and maintain relationships and permissions between nodes. There are a few parts to this. One of the main ones are claims. These in essences are a claim that are signed by one or more nodes. usually to state that two nodes own each other in a way that forms a gestalt. Other claims will be made and they can come later. These are stored in the SigChain which functions similar to a block chain where immutability of the chain is enforced by a claim including a hash of it's parent within it. So the history can't be modified without breaking the chain.

The ACL tracks permissions we give to other nodes. The main example of this is the permission to see and clone vaults from our node. It is a simple access control list that maps a permission to a NodeID

The Gestalts domain manages keeping track of claims between nodes within a gestalt. A Gestalt is a graph formed of all nodes that hold claims between each other. So a collection of nodes can be considered a group of a whole. given how the structure works, you can't really reference a gestalt as a whole, you can only really reference it by a member or check if two nodes are part of a gestalt. The GestaltGraph tracks our own gestalt but also other nodes gestalts. For the most part we only care about our own, or gestalts we have a first order relationship with via social or permission links.

Following on from gestalts we have the discovery domain. This handles the logic for exploring and mapping these claims and links between nodes. It fills in the GestaltGraph with this information and makes sure it's maintained.

Have a read over this @aryanjassal @brynblack if you have any questions just ask them in the comments here. They'll be a great reference for later.

[linear]

ENG-369 A pithy primer of `Polykey`

[tegefaulkes]

This might function better as a discussion. Maybe I'll convert it.