API/Integration into CMS?

[skurfuerst]

Hey,

I am one of the core developers of the neos.io open source CMS. Right now we have a rudimentary image editing component in the system, which can only zoom and crop images.

I'd like to provide imagorstudio as an option for more powerful image editing - and this would also offload us from handling imagemagick/vips ourselves.

Our base application is a React application, so the basics would fit - but I'd like to completely use Neos as imagorstudio storage backend, authentication layer etc; so that the image listing etc would be provided from Neos api wise.

Curious about your thoughts here. Would this be something of interest to you, or would you rather say this is out of scope for imagorstudio (which of course is also totally fine :))

Thanks for your feedback,
Sebastian

[cshum]

This would be a very interesting collaboration!
Imagor Studio currently consists of two parts: the image gallery/file browser/admin and the image editor.

The image gallery/file browser/admin is closely tied to a graphql backend for all the CRUD it takes.

Image editor, however, the only dependencies tied to the backend API is signed URL generation (for security), then the rest is offloaded to imagor itself.

There are multiple phases/approaches we may facilitate the integration. It sounds like the first step can be decoupling image editor (React + tailwind + shadcn UI) with URL generation API + imagor itself.

What do you think?

[skurfuerst]

Image editor, however, the only dependencies tied to the backend API is signed URL generation (for security), then the rest is offloaded to imagor itself.

Yeah for me/us, I would like to keep browsing etc in Neos; and only use the image editor component (maybe wrapped in an iFrame, for easier embedding) :)

Ideally for me would be if we could standardize a postMessage based JSON api for

• "these image modifications should be done, generate an image for it" or sth like this - so then Neos could handle the signing and returning an Imagor URL.
• "load an image in the editor", with a JSON of modifications to apply.

While a bit un-orthodox, I found the draw.io embed API pretty easy to work with: https://www.drawio.com/doc/faq/embed-mode (for inspiration).

All the best 🙂
Sebastian

[skurfuerst]

BTW do you want to use imagorpath/params.go as internal json structure or is there another API you have in mind? :)

[cshum]

BTW do you want to use imagorpath/params.go as internal json structure or is there another API you have in mind? :)

Yes internally it uses imagorpath to generate the signed URL path.

Currently the API is wrapped in such schema: https://github.com/cshum/imagor-studio/blob/main/graphql/imagor.graphql#L20
Even though it is graphql, wrapping it under a REST endpoint with json payload should be no problem.

However the main blocker is regarding how the decoupling for the embedded mode should be. If the URL generation endpoint has to depend on upstream application, it means such verion of imagor-studio needs to be customerized for the CMS, instead of a fully decoupled one like draw.io.

In order for imge editor to be fully decoupled, the dependency should be frontend + imagor only.

If we sacarfise security by passing in imagor secret to the image editor, then there is no need for backend to generate the signed URL, which makes the decoupling works. If we assume image editor is authenicated, would that make such trade-off tolerable?

[cshum]

Another possiblilty can be imagor to accept auth token with expiry time, such as JWT token. Then even with the frontend + imagor setup there is no concern the imagor secret is exposed.

However such setup means the generated image URL is now with expiry time. This means "Copy URL" would not work because all the generated image URLs are not permanent.

[skurfuerst]

Hey @cshum ,

also thought about this a bit... My 2 cents:

• How do we store the "edited image"? I.e. we need a way to re-initialize the editor with the "edited image" -> so that means the backend would anyways store such an image modification somehow. This could be by storing the internal JSON, or by storing the Imagor URL (which encodes every modification; but would need to be parsed again by calling the params endpoint).
  • Thus, the edited image must NOT have any lifetime associated with it, so that it is displayable a long time after its initial creation.
• if possible, I'd like to avoid leaking the Imagor secret to the end-user (as it is effectively a key for a DoS attack).
• I also thought about JWT etc, so that a 3rd party application (i.e. a CMS like Neos) could pass information to imagor or imagorStudio via URL/postMessage/... through the untrusted browser while still being secure. However I dislike that JWT etc are kind of complex to set up. If we go the JWT route, this could act as some kind of time-limited credentials towards Imagor; and Imagor would need to be adjusted as well for this...

My current favourite would be, if you want to integrate with Imagor Studio Editor: The Editor will call you (i.e. the CMS) via postMessage, and send an imagor string without HASH. The CMS integration would the need to sign the string server-side (after checking user authentication etc), and return the Image URL with hash, which can then be displayed by the Imagor Studio UI.

If being in a completely trusted environment, one could simply prepend "UNSAFE" as usual. Drawback of this solution would be that there is a "middle-man" between the Imagor Studio UI and Imagor, which needs to take care of signing.

Curious about your further thoughts :)
Sebastian

[cshum]

My current favourite would be, if you want to integrate with Imagor Studio Editor: The Editor will call you (i.e. the CMS) via postMessage, and send an imagor string without HASH. The CMS integration would the need to sign the string server-side (after checking user authentication etc), and return the Image URL with hash, which can then be displayed by the Imagor Studio UI.

Having thought about that, I am not familiar with having postMessage as API communication protocol. I am also curious how similar applications define API protocol around postMessage.

[cshum]

The Editor will call you (i.e. the CMS) via postMessage, and send an imagor string without HASH. The CMS integration would the need to sign the string server-side (after checking user authentication etc), and return the Image URL with hash, which can then be displayed by the Imagor Studio UI.

Having look at how iframe based applications communicate, it seems postMessa (ge is more commonly used than I originally thought.
But before I can make postMesssage communication simple enough for integration, I need to first migrate the backend URL generation logic (imagorpath) to frontend, except the signature HASH part.

[skurfuerst]

I would be fine either with the JWT or the PostMessage API - and if wanted/needed I can help implementing some parts of it, but of course needs some coordination which parts you want to do and which parts you would want to offload :) .

For reference, the postMessage API from Draw.io is explained at https://www.drawio.com/doc/faq/embed-mode at Description of the JSON protocol.

So is your preference now the JWT idea (potentially in imagor itself) or the PostMessage/"Proxy" idea? :)

All the best,
Sebastian

[cshum]

A more consolidated thought so far:

I am looking to create a docker build say imagor-studio-embedded where there is no database CRUD dependencies and the homepage is basically the image editor, with imagor included.

The imagor studio docker instance is expected to have secret, storage path etc all configured under docker env.

Phase 1: To access imagor studio, CMS would need to generate a JWT auth token, (secret key defined under docker env) and pass into editor iframe via url param ?token=....&path=... for stateless authentication with image path. Then the rest is expected to be handled by imagor studio itself.

Phase 2: if the embedded mode is fully working, I am planning to expose a postMessage API for all UI related interactions from/to the iframe. That will require your input on how/what to be incorporated.

Phase 3: the load/save editor state - this shall be done when postMessage API is ready

[skurfuerst]

Awesome, this sounds really good 😊 😎 thanks for pushing this forward and let me know if I can support you any way! All the best!

[cshum]

Hi @skurfuerst ,

Phase 1: To access imagor studio, CMS would need to generate a JWT auth token, (secret key defined under docker env) and pass into editor iframe via url param ?token=....&path=... for stateless authentication with image path. Then the rest is expected to be handled by imagor studio itself.

Phase 1 is now available!

The initial embedded build is available at imagor-studio-embedded docker build:

ghcr.io/cshum/imagor-studio-embedded:0.7.7

Documentation: https://docs.studio.imagor.net/deployment/embedded-mode/

The implementation includes:

• JWT-based stateless authentication via URL parameters ?token=...&path=...
• Theme control via URL parameter &theme=light|dark
• Path prefix scoping
• No database dependencies

I'd love to have you try integrating the ?token=...&path=... approach. Once you have it up and running, let's discuss the postMessage API details for Phase 2.

If you encounter any bugs or feedback during integration testing please let me know. Looking forward to hearing about your experience with CMS integration!

Best,
Adrian

[skurfuerst]

really cool ❤️❤️❤️ will try this out in the upcoming days and report back!

[skurfuerst]

Hey @cshum ,

I finally got around to build a minimal integration from the Neos side of things, and things are working nicely so far.

Imagor Studio is triggered via the Inspector on the right, when clicking the "crop" icon of the image:

When I click this, Imagor Studio shows nicely in the middle, with the image selected being active:

Great work so far ❤️ ❤️

By playing around with the editor within Neos, I got the following ideas for improvement:

Base URL

Base URL of the editor should be configurable; so that it does not live on / but on a sub-path. This is useful to be able to host Neos (or any other system) + imagor studio / imagor under a single domain.

visual cropping

It would be great to display + adjust the crop area visually, f.e. by using https://www.npmjs.com/package/react-image-crop or sth. like it - as cropping is certainly (for us) the most important operation to be done.

Thumbnail support

This is kind of a hard one: In Neos, people do NOT choose an image size in the image editor, but the image size + output format is chosen at rendering time. So that means:

• people choose an image and edit it ("crop / adjust / ..."); and then, when rendering this image, this is automatically output in an Image Source-Set with multiple image sizes and multiple formats.
• I am not sure if it is enough if Imagor Studio would return the full image URL to the image (with all processing); because then, I'd need to parse this URL, modify it (resize + strip the HMAC), sign it again, and use this. Thus, the format exposed by Imagor Studio IMHO should be something like https://github.com/cshum/imagor-studio/blob/main/graphql/imagor.graphql#L20 or even https://github.com/cshum/imagor/blob/master/imagorpath/params.go#L22 ? -> then the parameters could still be transformed by the backend before the final URL at usage site is generated. Should this URL generation then use generateImagorUrl() of GraphQL API, or is this considered internal and another endpoint should be exposed for this?

Configurable Editor Ordering + Hiding

It would be great if the editors from Imagor Studio could be re-ordered on call and partially hidden. In my example, I'd put "crop" at the top, and hide Dimensions/Resize + Output/Compression.

Feedback via postMessage when parameters are changed

This is for "Phase 2". In Neos, we would like to reflect every image configuration change directly in the inspector on the right. If we get informed (and get the Image Parameters back) via PostMessage, we could directly refresh the small thumbnail on the right, in the inspector - which is the UX we are aiming for.

Configurable default theme

As Neos is dark-themed, I'd love to switch to dark theme by default (maybe via URL parameter?). It would be perfect if we could also specify a background color or a style.css override -> this way we could make the UI blend visually into Neos perfectly.

Again, THANKS SO MUCH for your continued support and your great work. I'm really curious on your ideas on the topics above; please take them all as positive suggestions :-)

Thanks again and all the best,
Sebastian
PS: as already stated, in case I can help with any kind of pull request, please let me know :)

[cshum]

Hi @skurfuerst, thank you so much for the detailed feedback and for taking the time to build the integration!

I want to take a bit of time to digest all these points properly and come back with thoughts on implementation. All of them make a lot of sense. I had wanted visual cropping from the start but the implementation is harder than I initially thought - but I totally agree this has to be done if we want to push it for use cases like this.

Quick one - dark theme config should be supported viatheme=dark URL param (https://docs.studio.imagor.net/deployment/embedded-mode#theme-configuration), would you lije to try it out?

Will get back to you on the rest soon!

Best,
Adrian

[skurfuerst]

Quick one - dark theme config should be supported viatheme=dark URL param (https://docs.studio.imagor.net/deployment/embedded-mode#theme-configuration), would you lije to try it out?

you are right, works like a charm :)

[cshum]

Hi @skurfuerst PR would be very welcome too! To kick start with local development setup, you may try to get server and web up and running with the following config:

Server config server/.env

JWT_SECRET={your jwt secret}
FILE_STORAGE_BASE_DIR={file storage path}
EMBEDDED_MODE=true

Server requires libvips and ffmpeg install, same as imagorvideo.

Start server

cd server 
make run

Web config web/.env.local

VITE_API_BASE_URL=http://localhost:8080
VITE_EMBEDDED_MODE=true

Start web

cd web 
npm run dev

If you're interested in working on the postMessage API part (Phase 2), that would be amazing! Let me know once you have the dev environment running.

[skurfuerst]

works all, I have the dev setup running! Thinking about a postMessage API now, and will push a draft PR as basis for discussion in the upcoming 1-2 weeks!

[cshum]

@skurfuerst the latest version of Imagor Studio introduces visual cropping with aspect ratio control and unified crop/resize workflow! The visual cropping allows drag-and-drop resize handles with visual overlay showing the crop area. It is also equipped with locked aspect ratio control while cropping.

[skurfuerst]

really cool ❤️ I am sorry I did not continue on the integration yet; the last two weeks were really busy at work (and it seems the next week will also be...). I am still wanting to push it further, but just need a bit more time :)

[cshum]

Hi @skurfuerst,

No worries at all - I completely understand! I'm also squeezing time for this project alongside work, so I totally get it. It will also take time for me to make this a full-fledged editor with all the features needed for production use.

I wanted to let you know about a new feature that should help unblock the editor state part you mentioned earlier. The latest version now includes Edit History and URL State Persistence.

Take your time with the integration - looking forward to seeing how it works with Neos when you get a chance!

Best,
Adrian