Application progress review

[drognisep]

I wanted to try out Apicurio to see if it can meet my requirements. There are a few things that make it a deal breaker for me, but I wanted to add my notes here in case it's helpful. I went through the OpenAPI, protobuf, and JSON Schema feature sets - in that order - and this is what I found.

---

General UI

The UI seems very small. I'm testing on 1.5x UI zoom at 1920x1080 resolution.

• This wouldn't be as much of an issue if the add/remove controls weren't so small and close together. As it is, this exacerbates that issue.

Decent performance with no glaring UI bugs encountered.

I didn't expect UI links to be hard set to ":8888". This caused some initial confusion when running in Docker. I'm not sure why this is a necessary default.

OpenAPI

The delete and add operations are too close to each other with no delete confirmation. It's way too easy to accidentally delete something that was just created when what was intended was creating another entry. This is likely even more perilous for those with certain disabilities using the UI.

Paths don't seem to be aware of servers that have already been defined.

When defining a query parameter and a query parameter is not set as "Required," it looks like the type specification is disabled when it actually isn't.

Expected to see Object or a previously defined type as an option when specifying "application/json" response body type.

• It doesn't look like I'm able to select a previously defined response body for a path response.

I'm not very familiar with JsonSchema, so it would be great to be able to build a schema with a UI builder rather than a text editor.

• I tried just pasting in a JSON schema that I defined separately, but I got an error that "required" was not expected here. I think this is just me not knowing what I'm doing here.
• A nice change would be a UI builder that allows me to build up a response body without knowing a lot about OpenAPI or JSON Schema.
• If I were an OpenAPI expert, I'd probably skip the UI and just write the YAML/JSON to define the spec. So it's probably worth thinking through what value the UI is expected to provide to what kinds of users. Understanding who this application is for would help a lot.

I would prefer that completing high-level operations saved the model. As it is, I often forget to hit the save button, which creates some UX friction.

Protobuf

Started with OpenAPI, so I expected protobuf to support multiple files for related types in a singular "model"

• This isn't really useful for me without this feature.
• It seems like this would be a less-than-ideal UX to try to look through many protobuf elements in the UI to try to find multiple, related types at scale.

The protobuf editor should support Ctrl/Cmd+S for saving. This is an intuitive and fairly ubiquitous pattern for code editor experiences.

[EricWittmann]

Thanks for taking the time to do this, especially since you have found the tool to not meet your needs. I have some replies inline below. :)

General UI

The UI seems very small. I'm testing on 1.5x UI zoom at 1920x1080 resolution.

I would be interested in a screenshot of this. I'm using a 1920x1080 monitor at 1.0x, and it looks OK for me. However, I'm using a 32" monitor....

* This wouldn't be as much of an issue if the add/remove controls weren't so small and close together. As it is, this exacerbates that issue.

Fair enough!

Decent performance with no glaring UI bugs encountered.

👍

I didn't expect UI links to be hard set to ":8888". This caused some initial confusion when running in Docker. I'm not sure why this is a necessary default.

I'm not sure what this means. Can you expand? When running using docker compose, it runs the UI on port 8888 because other services use the more common ports (e.g. 8080). But I don't think there should be any hard-coded settings - so moving the UI to another port should be easy enough. I also have an example that uses nginx as a reverse proxy so that all services run on the same port but different context-paths. Would be interested to learn what your expectation was.

OpenAPI

The delete and add operations are too close to each other with no delete confirmation. It's way too easy to accidentally delete something that was just created when what was intended was creating another entry. This is likely even more perilous for those with certain disabilities using the UI.

This is fair. However the intention is that users would use the "undo" functionality if they accidentally deleted something (or accidentally made any unintended change). Perhaps that doesn't come through.

Paths don't seem to be aware of servers that have already been defined.

Do you have an example? I admit that most of my OpenAPI specs are written without any of the concrete service details (like servers), so I may be missing an obvious thing here.

When defining a query parameter and a query parameter is not set as "Required," it looks like the type specification is disabled when it actually isn't.

I see - I guess because the text is grey? I think we were going for a "not set" visual rather than a "disabled" visual on this.

Expected to see Object or a previously defined type as an option when specifying "application/json" response body type.

* It doesn't look like I'm able to select a previously defined response body for a path response.

Interesting. Adding something to the "Add Media Type" dialog to select a previously defined type sounds like a nice improvement:

As for the previously defined response, that happens when adding a Response, not when adding a Response Body:

I'm not very familiar with JsonSchema, so it would be great to be able to build a schema with a UI builder rather than a text editor.

Agreed. There is a lot of room for improvement here. We have some basic form builder support for schemas, but definitely not enough. The problem is that JSON Schema is a complex specification, and difficult to model effectively in a UI. Here's what we have now:

You can define basic info about the type, some simple inheritance options, and then the list of properties the type has. But it's admittedly rather simple.

* I tried just pasting in a JSON schema that I defined separately, but I got an error that "required" was not expected here. I think this is just me not knowing what I'm doing here.

Perhaps. I'd need to see the flow and the resulting error. :)

* A nice change would be a UI builder that allows me to build up a response body without knowing a lot about OpenAPI or JSON Schema.

Agreed.

* If I were an OpenAPI expert, I'd probably skip the UI and just write the YAML/JSON to define the spec. So it's probably worth thinking through what value the UI is expected to provide to what kinds of users. Understanding who this application is for would help a lot.

In my opinion this is also a core problem with OpenAPI + JSON Schema. What are these specs trying to accomplish? OpenAPI is a allegedly a way to describe an API - its shape, functionality, etc. But JSON Schema is a technology to validate data, not to describe a data type. Those two things may sound the same, but I would argue they are not. So I think the combination of the two is problematic.

That said, you are right that there is a question about who this tool helps. My intention was always to have a tool that could help users who want to describe an API but don't know the OpenAPI spec well enough to do it effectively using a text editor (even one with intellisense completions). So the editor should provide some discoverability/guidance.

I would prefer that completing high-level operations saved the model. As it is, I often forget to hit the save button, which creates some UX friction.

Interesting. The original version of Studio operated more like Google Docs - it would automatically save every change you made as you made them. Undo/redo would still work. There was no explicit Save button. I liked that version better for sure. But there are workflow implications to that model as well, which tended to move the friction to other parts of that workflow.

Protobuf

Started with OpenAPI, so I expected protobuf to support multiple files for related types in a singular "model"

* This isn't really useful for me without this feature.

* It seems like this would be a less-than-ideal UX to try to look through many protobuf elements in the UI to try to find multiple, related types at scale.

Yes agreed - it's a terrible Protobuf editor solution. Any types other than OpenAPI/AsyncAPI are essentially just relegated to a dumb text editor experience.

The protobuf editor should support Ctrl/Cmd+S for saving. This is an intuitive and fairly ubiquitous pattern for code editor experiences.

Yeah that's fair. 👍

[drognisep]

The general takeaway is that there's a lot of potential here, but it's not what I'm looking for yet.

UI scaling is one of those things that, the more you look at it, the more normal it seems. From an unexposed perspective, it just looks small with small controls. Of course, take that with a huge grain of salt, because my opinion doesn't actually matter that much. 😆
I'm also using Brave on Linux. I don't know if that means much, but figured it might be a relevant detail to include.

Addressing some specific points:

I would be interested in a screenshot of this.

Eh, I already removed everything, and my HDMI port has become super buggy, so I probably can't reproduce this any time soon. Sorry. 🤷‍♂️

I'm not sure what this means. Can you expand?

I have conventions for port assignments, and the UI didn't exactly work as expected without using the host ports in the docs example. That indicates to me that the UI is expecting a particular host port. I didn't look too close at it, I just noted it and moved on.

However the intention is that users would use the "undo" functionality...

Honestly, I didn't notice that was an option. Either I was focusing too much on what I was seeing to see the undo option, or maybe there's a UI improvement there to make that option more clear. I'm leaning toward the former, but the latter might be worth a check.

I admit that most of my OpenAPI specs are written without any of the concrete service details...

I tend to define a "local dev" server config so a UI render of a spec has that for easy pre-push testing. When defining a path, it didn't seem to be aware of servers that have been defined. I tried saving and backing out of the view, and refreshing too, but I couldn't get the option to populate the server. I went back to the top level servers view to make sure it wasn't just a memory persistence limitation, and the UI was aware of what was already defined in the servers view.

I see - I guess because the text is grey?

Exactly. That's conventionally coded as "disabled" in a lot of other UIs, so your users will likely interpret it that way. I'd recommend sticking with placeholder text to convey the intended context.

But JSON Schema is a technology to validate data, not to describe a data type. Those two things may sound the same, but I would argue they are not.

This is a really interesting perspective! IMHO, shape and constraints should be intrinsically linked as much as possible, and if you think about it they are unless we declare that a field can be anything. Even just assigning the shape of string brings in the constraint of textual data. So it's true that the shape of a data structure does not completely define what is valid for that structure, but validity is inexorably linked to shape. How do you describe what is valid for a field without naming the field? With that in mind, JSON Schema can define both, but it's not always easy to fully specify both. It's certainly a complicated spec to implement well, and like I said, I'm certainly no expert.

Speaking of...

My intention was always to have a tool that could help users who want to describe an API but don't know the OpenAPI spec well enough to do it effectively using a text editor...

I think this will be a tool for me. :)
That will be especially true with expanded protobuf and JSON Schema support, and some more refinement on the front end state management and UX. I really like the vision for this tool, so I hope you take this as encouragement to improve it rather than just some rando criticizing your hard work. Keep up the good work, boss. 👍

[EricWittmann]

I really like the vision for this tool, so I hope you take this as encouragement to improve it rather than just some rando criticizing your hard work. Keep up the good work, boss. 👍

This made me actually LOL. :) I am always happy for feedback, critical or not, as long as it is well intentioned, reasoned, and delivered! (as yours was) All good.

As for the future of the project, that's harder to predict. Honestly I worry about the usefulness of editor tooling like this, given the increasing prevalence of AI workflows. I wonder if an AI based version of Apicurio Studio would make more sense, with the focus being more on analysis type tools (e.g. visual diff, docs rendering, validation, etc) rather than editing.

[drognisep]

Well, I'm not an ML/LLM expert, but I can say that I'm looking for what you're working toward despite the existence of AI tools. AI is inherently probabilistic, so there's always a non-zero chance that it spits out garbage. What AI seems to really excel at is either A) situations where there is a massive corpus of easily obtainable, high quality, specific results for training, or B) problems where there are many correct results for the same input. An OA specification has to be both syntactically valid and semantically correct. There are also many different tools in use in the domain, with a wide range of output quality and spec version support. So I think the niche a tool like this fills is the guided experience. Like a good tour guide helping you see everything you really care about while answering every question about what's in front of you, without making you worry about all the logistical details. If we can agree that AI will not be 100% accurate, then we can agree that setting AI up to be the tour guide will inevitably fail, and in non-obvious ways. You don't get a stack trace in your server log when an LLM hallucinates. This would require the user to dive into the complexity for themselves, but that's not a bad thing for all users. I'd wager the proportion that wouldn't feel great about it would be significant.

If I were an OpenAPI expert, I'd probably skip the UI and just write the YAML/JSON to define the spec.

The questions I would be asking myself in your shoes would be:

• Given what I know about the OpenAPI domain and the risks of a probabilistic approach, would it be better to contend with bugs I can fix or an AI I can't?
• Is there an opportunity where the strengths of both probabilistic and deterministic approaches can be combined?