Simplifications

[blythed]

Saving components in one table

Add a class attribute base_schema to Component/ Base. This attribute will determine which table the sub-components are saved into, and also, which fields get their own column in that table. This will avoid having too many tables in the databackend.

For example:

class Component(Base):
    ...
    base_schema: t.ClassVar[t.Type] = Component


class Other(Component):
    ...
    my_field: str = 'bla'


db = ...

# inserts to the component table
db.insert([Other('my_other')])

# also inserts to the component table
db.insert([Component('my_component')])

Adding additional fields to Component table for filtering

Name	Description	Why?
context	UUID of ancestral root	Make it easy to filter for components
branch	Class name of class below Component	Make it easier to "group" components

Use REGISTRY to load schemas quickly

We already have a registry in superduper.base.base.REGISTRY. We should use this in order to avoid loading schemas from a different place.

This function in metadata.py looks up from a table. In order to avoid this, the classes need to be defined in advance. This could be a configuration or otherwise.

def _get_component_class_info(self, component):
    cache = _component_cache.get()

    if cache is not None and component in cache:
        return cache[component].copy()

    info = self.get_component('Table', component, version=0)

    if cache is not None and info:
        cache[component] = info.copy()

    return info

[jieguangzhou]

1. For REGISTRY, it doesn’t seem necessary. Once we also save _path into the database, when we load a component it will be fetched together. The extra time cost will just be the import, and thanks to Python’s reimport caching mechanism, that overhead is negligible.
2. Could the concept of a branch be replaced with an explicit attribute instead? For example, when initializing all components, we could add a group attribute. This would be more meaningful at the application layer.

WDYT?

[blythed]

I think that _path is a security vulnerability and will make refactoring difficult (changing the location of the class). The branch needs to be determined programmatically, for example to load all "CDC" components. We could add additional user features if necessary.