Local API - Select which fields to return

[johannesschaffer]

While the local API doesn't have the api/http overhead, it still needs to download all the data from the database. For large documents (long text) this becomes quite the overhead when only needing a part of it (e.g. some metadata like title & description to display a card on the homepage).
The workaround I'm using right now is React's dedupe feature, e.g. const getProject = cache(async (id) => payload.find()), but a Prisma-like feature to have more fine-grained control over what's returned would be great (especially with next-payload, which will probably result in far greater use of the Local API 😜)

[ssyberg]

👍🏼 on this, came here to add this feature request. I'd add a few points to the above:

1. I don't totally grok the find code in core payload, but it looks to me like exposing the select functionality in mongoose would be fairly straightforward.
2. This is crucial for avoiding infinite loops on computed (virtual) fields without implementing hacky workarounds.

I'm working on an enterprise level payload project now and the lack of more granular mongo support in the local api means we end up bypassing it all over the place and hitting mongoose directly via payload.collections[collectionSlug].Model. It's awesome that payload exposes this for flexibility, but when that becomes a common way of interacting with data I'd suggest you're in anti-pattern territory for a CMS/Framework.

[ssyberg]

One more thing, I think this is adjacent to this PR and may even have some implementation implications on it #1288

[emirotin]

+1 on this, I have a massive relation field in one doc and just fetching it seems to crash the Payload instance while I don't need it for every usage scenario

[r1tsuu]

PR that adds select and populate options #5942 , works with both Postgres and MongoDB

Copy paste my message from Discord:

https://discord.com/channels/967097582721572934/967097582721572937/1257435094377955368

If we wouldn't do it on the DB level, then does it make that much sense? I wrote a plugin that does it, but It's really only reliable if you as well apply outside of this caching / Static Site Generation, because not needed data still bloats the server memory.

Yet, outside of a given problem with afterRead, it's hard to solve Type-Safety problem with select as types can be nested. It could be easy if we'd just allow selecting only top-level fields though, but it doesn't sound good. if we compare it with how for example Prisma does it, in fact they have a simpler case to be type-safe because all the nested data is a relational data.

I would also add about populate option importance part. While depth is easy to use, it's unreliable when you need to populate only needed fields. Though it's not hard to write your own populateDoc function that could look like this and accept fields option.

I'm currently building a medium sized app that should be well performant. For that, I went with my own backend layer with trpc + Kysely (type-safe SQL query builder,).
I'm only using Payload for handling the db schema / migrations, "admins" authentication and content. Of course i need to write more code, but i feel like i just can't accept Payload abstractions cost, especially in the select and population part. I have a table with 50 columns, and in my opinion it's not right to fetch all the columns if i need only id.