Allow plugin to extend payload config

[teunmooij]

It would be nice to be able to extend the config with extra information for a plugin.

Why

I'm working on a payload-swagger plugin and would like to have some extra information in the config. For instance on a custom endpoint I would have description and response that I would use in the openapi document.

Current situation

When I build the config with some extra information, I get an error saying there's a forbidden field in my config.
Since the plugin runs on the raw config it is already possible to do this, but there's 2 issues with this:

• I need the sanitized config for my plugin (because I use a lot of the default values and don't want to code them into my plugin and I need express), so I hook into the onInit function to start my plugin;
• I would need to remove all the fields I add in order for the sanitize to accept the config, which I don't think is a good idea

Possible solution

Since the sanitize function uses joy for schema validation, simply adding .unknown(true) to the schemas would solve my issue.

If the idea is accepted, I'd be happy to create a PR for this.

[teunmooij]

Is this something that can be done, or should I start exploring other options?

[teunmooij]

@jacobsfletch , would you perhaps be able to answer this question? I'd be happy to do the work.

[jmikrut]

Hey @teunmooij — I can answer this one for ya. The problem lies in the fact that the Payload config is strongly typed, so we'd also have to allow the config itself to accept unknown properties as well as opening up the joi schema validation.

Because of these logical issues, I think the right way is to make it so your plugin is what accepts the additional properties, and at that point, you can strongly type / validate the incoming options accordingly.

You can see an example of this in our own officially supported plugins like the cloud storage plugin.

For example, that plugin is formatted as a function that returns a plugin. With this pattern, we can allow the function to accept properties, build the plugin accordingly, and then return the plugin itself:

export default buildConfig({
  plugins: [
    cloudStorage({
      collections: {
        'my-collection-slug': {
          adapter: theAdapterToUse,
        },
      },
    }),
  ],
});

See what I mean? This is the way to go. We are unlikely to expose unknown keys because of the TypeScript implications, and for the best dev experience, you should strongly type arguments to your plugins anyway. With this pattern I think you get the best of all worlds!

[teunmooij]

Thanks @jmikrut for answering my question. I understand the implications and have also implemented my plugin as a chained function that accepts strongly typed options. This does however not cover all use cases in a way that's easy to use for the users of the plugin.
I'll explain my use case in a bit more details:

In my plugin, payload-swagger, I'm generating a openAPI document based on the payload config. This includes the custom endpoints. There's however no way for me to determine query parameters, description or return type of the custom endpoint.

So let's say given the demo project I'd create a custom endpoint that returns all the posts for a given category. I'd add that endpoint to the posts collection and it would look something like this:

  endpoints: [
    {
      path: '/category/:category',
      method: 'get',
      handler: async (req, res) => {
        // implementation
      },
    },
  ],

So if I follow your suggested solution, if I wanted to document this endpoint, I'd do it like this

export default buildConfig({
  plugins: [
    swagger({
      collections: {
        posts: {
           '/category/:category': {
             get: {
                description: 'Get the posts for a given category',
                returnType: 'posts'
             }
           }
        }
      }
  })
  ]
});

This would not be a very good dev experience and quite error prone.

What I'd like to do is to allow the user to write this documentation where it makes sense: in the custom endpoint definition:

endpoints: [
    {
      path: '/category/:category',
      method: 'get',
      description: 'Get the posts for a given category',
      returnType: 'posts'
      handler: async (req, res) => {
        // implementation
      },
    },
  ],

With that, the collection config would still be a valid CollectionConfig. It would not break the payload config.
There's multiple ways for me to provide strong typing for the extension of the CollectionConfig:

• I can re-export the a CollectionConfig from my package that extends the payload CollectionConfig;
• I can provide a function that takes the extended config as an input and returns a config;
• etc.

None of these would cause any issues or compromise the overall dev experience of payload.
I hope you might reconsider.

[jmikrut]

Ah I see. Hmm. Maybe what we could do is create some type of plugins key on the collection, global, and field schemas. Just a way to make it predictable. I'm also trying to avoid naming conflicts. For example, if we decided to use the description or returnType keys on an endpoint. Not saying that is likely, but if we open this up as a supported paradigm, I could see ways that this could get complicated quickly.

Then, we could also create some pattern where buildConfig, CollectionConfig, GlobalConfig, and Field are generics that take plugin extension types.

I don't think we'd be opposed to this and I think your arguments are 100% valid.

Are you interested in doing some digging and maybe make a PR here?

[teunmooij]

Here's the PR: GH-2436

[teunmooij]

Yes, that approach sounds reasonable. Happy to look into it and see if I can make a PR for it. Thanks

[grmkris]

maybe we can use this as inspiration https://ez.robintail.cz/ for custom endpoint builder.... it is pretty awesome

[teunmooij]

Not sure I understand how this relates to this conversation and how you think this could be used.

[grmkris]

oh. sorry for confusion. currently you define custom endpoints as handlers

my understanding is that this way it is hard to do inference for types / outputs to build the openapi schema..

if we modify how the endpoints to work like in the screenshot below

we can use zod for runtime validation + use the existing plugin to give us fully typesafe openapi schema.

maybe this doesn't really directly relates to the convo, but I think it does atleast a little bit

[teunmooij]

ah, I see. sounds nice, but I'm not sure that would work since the handlers are not plain express handlers, but payload handlers.
And I don't know how you use custom endpoints, but a lot of mine actually return payload types that have already been defined, and I do not have a zod schema for. Of course you could write a zod-to-payload plugin, but I think that will not be that straightforward, because you'd need to accommodate also for custom fields.

[grmkris]

yeah that is right.. but i suppose I could just hook that library i showed on the same express server that payload is running on.. I suppose I can use localAPI and all.. this way I can completely ignore the payload endpoints if I want and I build more complex endpoint using this library. Do you see any downsides of this approach? if you ignore the bloat of having express + payload + zod-api

…

On Fri, 31 Mar 2023 at 12:12, Teun Mooij ***@***.***> wrote: ah, I see. sounds nice, but I'm not sure that would work since the handlers are not plain express handlers, but payload handlers. And I don't know how you use custom endpoints, but a lot of mine actually return payload types that have already been defined, and I do not have a zod schema for. Of course you could write a zod-to-payload plugin, but I think that will not be that straightforward, because you'd need to accommodate also for custom fields. — Reply to this email directly, view it on GitHub <#2369 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADWTG26CPDSUGKODUCZZUL3W62UZTANCNFSM6AAAAAAWCCDTNA> . You are receiving this because you commented.Message ID: ***@***.***>

[teunmooij]

I do want to build some other tools and plugins and i want to build them in a way that their metadata can be used in my openapi package. That and the custom endpoints was the reason i started this thread. I'll be writing about that here: https://github.com/teunmooij/payload-tools/discussions
I won't support zod-api in my payload-openapi tool, but it does support merging in external openapi documents.