optional/required fields behaviour

[nikgraf]

We should decide how the schema in the framework should behave. In GRC-20 every field except for id, Created at, Updated At, Created by, Versions are optional.

Here our options. For simplicity I would focus on createEntity and useQuery. With this example schema:

{
  Todo: {
    name: type.Text,
    completed: type.Checkbox,
  },
}

createEntity({ name: "Setup call", completed: true }); // valid call with all attributes

const result = useQuery({ types: ['Todo']});
/*
[
  { id: "abc", name: "Setup call", completed: true },
  { id: "abc", name: "Draft post", completed: false }
]
*/

Everything always optional

// would be valid and `completed` would just not be set
createEntity({ name: "Setup call" });

const result = useQuery({ types: ['Todo']});
/* returns all entities with the types regardless if they comply to the schema
[
  { id: "abc", name: "Setup call" },
  { id: "abc", completed: false }
]
*/

Everything always required

// valid
createEntity({ name: "Setup call", completed: true });
// would NOT be valid and fail to create the todo
createEntity({ name: "Draft post" });

const result = useQuery({ types: ['Todo']});
/* returns only entities where the entry complies to the schema. Entires like `{ id: "abc", completed: false }` are filtered out
[
  { id: "abc", name: "Setup call", completed: true },
]
*/

Developers can decided

Schema:

{
  Todo: {
    name: type.Text,
    completed: type.Checkbox,
    dueUntil: type.Date({ required: false }),
  },
}

createEntity({ name: "Setup call", completed: false }); // valid
createEntity({ name: "Draft post", completed: false, dueUntil: "2025-02-10" }); // valid
createEntity({ name: "Do laundry" }); // would NOT be valid and fail to create the todo

const result = useQuery({ types: ['Todo']});
/* returns only entities where the entry complies to the schema. Entires like `{ id: "abc", completed: false }` are filtered out
[
  { id: "abc", name: "Setup call", completed: false },
  { id: "abc", name: "Draft post", completed: false, dueUntil: "2025-02-10" },
]
*/

Other suggestions

Feel free to add your options here …

[nikgraf]

First impression:

Everything always optional makes me nervous when I need to think about building an application :D Because in every view where I use a property I need to check if it actually exists. I probably would write myself a helper to filter out optional entries to have a better experience when building UIs.

Everything always required feels too strict.

Developers can decided seems about the write balance for UI development.

[baiirun]

I'm in favor of the "Developers decide" approach.

IMO this should be a property of the framework’s relationship with the codegen/query layer. If the framework user wants strict enforcement of their schema then the codegen/query layer should filter out any entities that don’t strictly match the schema and optionality.

[cmwhited]

I agree with. @baiirun/the last approach. The developer should decide. That way we get type safety, which I think is key.
I like the effect approach of letting developers decide:

const name = Schema.optionalWith(Schema.NonEmptyString)

[nikgraf]

@baiirun can you elaborate what you mean with

codegen/query layer should filter out any entities that don’t strictly match the schema and optionality

Is it just this approach above or more to it?

cases we need to consider:

1. data can come from the public graph or over end-to-end encrypted sync
2. I can have existing data locally and can change the schema (App update)

[baiirun]

@baiirun can you elaborate what you mean with

codegen/query layer should filter out any entities that don’t strictly match the schema and optionality

Is it just this approach above or more to it?

cases we need to consider:

1. data can come from the public graph or over end-to-end encrypted sync
2. I can have existing data locally and can change the schema (App update)

It's the same as the above approach. The problem we have is that the actual data in the public knowledge graph is fully optional, but developers want more more guarantees over the strictness of their schema/data model. We should let developers write their schemas as strictly or as loosely as they want.

This schema interfaces with two remote data sources 1) the sync server and 2) the public graph. The sync server likely adheres to the developer's schema already, but the public knowledge graph might not.

For #2 there should be a separate query layer that takes the developer's schema and queries it from the public knowledge graph. This query layer should take into consideration the developer's schema. At query time this layer should take the result of queries for entities in the public graph and validate that they contain the fields the schema expects, if not they should be filtered out.

[nikgraf]

got it, fully agree with the approach you described 👍

Note: since the data of the sync server is e2e encrypted the sync server can't validated the content. It's coming from trusted peers and most likely is valid, but there is no guarantee.