Skip to content

perf(ServiceProviderRegistry): efficient productData #307

@wjmelements

Description

@wjmelements

@wjmelements we've just had a sync discussion about this and we're both open to changing this even to the point of nuking PDPOffering entirely and just having a k/v list with an external schema (i.e. that the SDK and Curio agree on, everything not within that schema is essentially ignored). In the SDK we can filter out SPs that don't have the keys that we need. Listen to the recording of the daily sync for more but my thoughts are something like this: we mainly care about this where we might need information cross-contract and it shouldn't be too expensive to find it. e.g. a hypothetical service contract for a PDP service that only deals in SPs that have ipniIndexing turned on, the contract would have to go and figure that out from the registry. But to your point - currently having to decode the blob is a cost as well, so nothing is free here. Possibly also we just convert this one to a string=bytes too for efficiency. Right now the main one I care about is serviceUrl, but if I have a good way to get the whole set of keys/values in a single call then we can deal with it in the SDK just fine.
So, permission to radically pull this apart if you want. Maybe an "offering" is simply just a key/value list, with nothing fancy.

Originally posted by @rvagg in #297

The current design has two kinds of product attributes:

  1. productData, which contained the fields mandated by the productType
  2. capabilityKeys, which allowed products to surface additional information extending the standard schema

However, looking up fields from the bytes storage productData is inefficient because on-chain consumers must decode the entire bytestring to read one field from it. There are two ways to fix this.

  1. An extsload pattern could provide a better interface for reading the fields individually without needing to decode them all. We would have one view library per product type.
  2. Make every attribute into a capability key. Then we could use bloom filters to enforce the schema for specific product types.

The extsload approach works decoding typed data because it is ABI-encoded. We can also make some assumptions about the shape and size of the fields that we cannot make if everything is bytes or string. However this requires a view contract per product type. Also, the optional fields would still be typed as bytes or string.

The remove productData approach has some nice simplicity to it. Instead of managing two kinds of attributes, there would only be one type. If fields are all bytes, some space is wasted for fields such as uint256 that are not bytes, string, or []. This freedom could be chaotic onchain if there was some important field that was supposed to be uint256 but was not. Products not encoded properly could be filtered-out off-chain, but valid products can be updated to become invalid. This is a problem with all of the capability keys but I raise it here because this approach would make everything into capability keys.

I am currently preferring to remove productData because it simplifies the codebase and does not create more contracts. The benefit of less storage for scalar fields for the extsload approach is limited to the ones in productData, and I believe there must be at least two of them to break-even.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    📌 Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions