[Use Case]: Fetching

[gavinking]

As a ...

• Application user/user of the configuration itself
• API user (application developer)
• SPI user (container or runtime developer)
• Specification implementer

I need to be able to ...

... specify that certain fields of an entity should be fetched by a repository method.

Which enables me to ...

... control the number of queries executed against the datastore and the amount of data which is fetched.

Additional information

By analogy with @OrderBy, I propose a @Fetch or @Fetching annotation:

@Find 
@Fetching(_Book.AUTHORS)
@Fetching(_Book.PUBLISHER)
Book bookWithAuthors(String isbn);

For implementations based on Jakarta Persistence, this might be backed by construction of an EntityGraph.

[gavinking]

Relevant: jakartaee/persistence#735

[beikov]

I understand the desire to have this, but it seems like Jakarta Data does not specify lazy loading, so it feels a bit awkward to have this annotation, don't you think?
Here is a crazy idea about representing the intent without referring to fetching or lazy loading. Let the user define interfaces that represent the subset of data they want to expose and fetch eagerly at the same time. When the entity type implements that interface, we can derive load graphs automatically from the methods of the interface.

That way, people can ensure they don't access data through lazy loading, and we allow them to express the data that they want available. Thoughts?

[gavinking]

I understand the desire to have this, but it seems like Jakarta Data does not specify lazy loading, so it feels a bit awkward to have this annotation, don't you think?

It's quite easy to provide well-defined semantics for this operation without needing to talk explicitly about lazy fetching.

Let the user define interfaces that represent the subset of data they want to expose and fetch eagerly at the same time. When the entity type implements that interface, we can derive load graphs automatically from the methods of the interface.

I'm not quite sure I have understood what you're envisioning here, but perhaps something like the following.

I think you're suggesting the user would write something like:

@FetchProfile
interface BookWithAuthors {
    String getIsbn();
    String getTitle();
    Set<Author> getAuthors();
}

(Not sure if the annotation is really necessary, so ignore if for now.)

And then the entity class would actually implement this interface, as a witness that the entity really has the required fields:

@Entity
class Book implements BookWithAuthors {
    @Id String isbn;
    String title;
    @OneToMany Set<Author> authors;

    // getters, setters
    ...
}

And then the repository would indicate this fetch profile in its return type:

@Find
BookWithAuthors bookWithAuthors(String isbn);

This is a nice idea and frankly quite elegant. Definitely appealing in some ways.

I believe it can even in principle be extended to handle "deeper" fetch graphs by writing stuff like:

Set<? extends AuthorWithThing> getAuthors();

But it does have some distinct downsides:

1. There's a need to declare not only associations, but even basic attributes of Book on the interface.
2. If you're planning on mutating Book, you even need to declare setters. If you're planning on calling its methods, you even need to declare them.
3. The entity class might grow quite a lot of interfaces if it has many associations and is used in many different scenarios. Arguably, tho presence of those interfaces in the schema of Book breaks the layering of the program.
4. It sorta starts to lose the whole advantage of working with entity classes (compared to having record projections which represent the shape of query result sets) in that you can no longer write reusable functionality that operates generically over Books. Instead, code has to be written to work with BookWithAuthors, etc.

Issues 1+2+4 together would push you toward a diamond-like inheritance structure with Book implements BookWithAuthors extends BookOperations. Yes it's more type-safe, but it's also a lot more verbose.

On the other hand, perhaps what you mean is that the fetch profile would be declared like this:

@FetchProfile
interface BookWithAuthors {
    Set<Author> getAuthors();
}

@Entity
class Book implements BookWithAuthors {
    //same as before
}

where basic fields are implicitly included in the fetch profile.

This streamlined profile could not be used as the return type of the repository.

```java
@Find @Fetch(BookWithAuthors.class)
Book bookWithAuthors(String isbn);

This loses type safety again (we're back to LazyInitializationExceptions).

My knee-jerk is to say that this second possibility is in almost every respect worse than using the static metamodel.

[beikov]

I understand the desire to have this, but it seems like Jakarta Data does not specify lazy loading, so it feels a bit awkward to have this annotation, don't you think?

It's quite easy to provide well-defined semantics for this operation without needing to talk explicitly about lazy fetching.

If you say so, I guess that's ok then. Like I wrote, it just felt awkward to talk about fetching since Jakarta Data has no lazy loading and only with that it makes sense to me to even be talking about fetching.

I'm not quite sure I have understood what you're envisioning here, but perhaps something like the following.

Yeah, my suggestion is close to your first thought experiment.

1. There's a need to declare not only associations, but even basic attributes of Book on the interface.

True, but if you think of lazy loading of basic attributes, this could make sense.

3. If you're planning on mutating Book, you even need to declare setters. If you're planning on calling its methods, you even need to declare them.

You could downcast it to Book if you want to change the state which you'd have to do in stateless world anyhow since you have to pass that book to an @Update method then.

3. The entity class might grow quite a lot of interfaces if it has many associations and is used in many different scenarios. Arguably, tho presence of those interfaces in the schema of Book breaks the layering of the program.

I'm not sure what you mean by "layering" in this context. This is just as much part of the domain as the definition of entity graphs IMO, but sure, the amount of such interfaces could grow if you have use cases with varying needs. For simple CRUD applications though, you would probably only have 1-2 interfaces.

4. It sorta starts to lose the whole advantage of working with entity classes (compared to having record projections which represent the shape of query result sets) in that you can no longer write reusable functionality that operates generically over Books. Instead, code has to be written to work with BookWithAuthors, etc.

That's true, which is why I usually suggest folks that use such an approach to have a base type e.g. BookWithId that has only the id. In past projects it often turned out to be enough to refer to such a type for most use cases. I'm not sure what kind of other code you're imagining to operate on domain data, but I agree that this might change the way to program a bit.

[gavinking]

it just felt awkward to talk about fetching since Jakarta Data has no lazy loading

I mean, Jakarta Data implementations are certainly going to have lazy loading, it's just that we have not (yet) really specified what associations are and how they work.

But it's OK, we can just say something like: "@Fetch specifies that the client will access the given field of the entity returned by the repository method". And then in a non-prescriptive NOTE give some explanation of the purpose.

True, but if you think of lazy loading of basic attributes, this could make sense.

Sure, sure, it's conceptually perfectly reasonable.

But in the overwhelmingly common case that these basic attributes are fetched by default, it's just extremely verbose and starts to somewhat undermine DRY, smearing out the schema of the entity.

You could downcast it to Book if you want to change the state which you'd have to do in stateless world anyhow since you have to pass that book to an @Update method then.

Right, so at that point you're starting to undermine the type-safety which was gained via the introduction of the interfaces.

I'm not sure what you mean by "layering" in this context. This is just as much part of the domain as the definition of entity graphs IMO

Well, sure, that's why I said it's "arguable"; it depends on what you personally view the layering to be.

In my usual mental model, a fetch strategy is not part of the "problem domain", so it's not part of the entity model. If you draw boxes and sticks on a whiteboard (a class or ER diagram, or a description logic, or whatever), you would not draw in the fetch profile. Instead, it's something that exists to service an interaction with a certain client. So, in my mind, it belongs to an upper layer.

OTOH, I agree that this is a relatively weak argument, not based in any fundamental principle.

If you happen to think that fetch profiles are quite stable objects in your program, then it might be reasonable to view them as belonging to the model of the problem domain.

Anyway, we can agree it's not the principal objection to this idea.

[njr-11]

But it's OK, we can just say something like: "@Fetch specifies that the client will access the given field of the entity returned by the repository method". And then in a non-prescriptive NOTE give some explanation of the purpose.

I think it's a valid concern that some users will read that, overlook or misunderstand the note, and go on to make their repository find methods list every entity attribute they intend to use. I'd rather just directly state it's an override of the lazy fetch capability that is available in some entity models. Giving the annotation a more specific name might help, too, but at the tradeoff of being less concise. I'm curious what Jakarta NoSQL currently does (I didn't see anything in the API) or plans to do in this space.

I suppose an alternative approach might be having the annotation value be a single String pointing at a NamedEntityGraph name. This would offer the most flexibility and delegates definition of how it works to Jakarta Persistence (and possibly Jakarta NoSQL). The downside is that it's a lot more complex for users, and they need to learn about that complexity from another spec. It's certainly simpler just to list the entity attribute names and let the Jakarta Data provider translate to an entity graph, so I can see the appeal of that.

[gavinking]

I'd rather just directly state it's an override of the lazy fetch capability that is available in some entity models.

That's also fine.

I suppose an alternative approach might be having the annotation value be a single String pointing at a NamedEntityGraph name.

Yeah it's possible but:

1. you have to introduce a name with global scope for something that's most-likely specific to just one repository method (that's why I was looking into Consider allowing @NamedEntityGraph at the method level persistence#735 as a possible solution), and
2. @NamedEntityGraph has quite poor usability, and a bad name.

And then it's simply not the case that lazy fetching is specific to Jakarta Persistence. While you maybe don't need it for a document database, or for a key-value store, I think it's clear you need it for a graph database.

[pipinet]

how to dynamic fetching?

@Find 
Book bookWithAuthors(String isbn, Fetchings fetchings);

[gavinking]

how to dynamic fetching?

My knee-jerk is that this is much, much less common than programmatic restrictions, orders, or .even projections.

[pipinet]

how to dynamic fetching?

My knee-jerk is that this is much, much less common than programmatic restrictions, orders, or .even projections.

Yes, I completely agree. Since the purpose of fetching data must be to support subsequent business logic, all such fetching operations can be fully enumerated. Examples include bookWithA, bookWithB, bookWithC.

[graemerocher]

Micronaut Data JDBC does not support lazy loading, which is by design. This case is handled in one of two ways:

1. Using @Join (roughly equivalent to this proposal). See https://micronaut-projects.github.io/micronaut-data/latest/guide/#dbcJoinQueries
2. Use a DTO. For example defining BookDTO(String title, Author author) will fetch book and author

[gavinking]

@graemerocher if by "lazy loading" you mean transparent lazy fetching of an association on access, then neither does our implementation.

[hantsy]

It looks like the JPA EntityGraph, I wish to unify the usage and concept here. In JPA EntityGraph itself defines the load/fetch nodes in the query result.