Skip to content

Add ability to edit existing images in the media library#305

Open
dkotter wants to merge 23 commits intoWordPress:developfrom
dkotter:feature/media-library-image-edits
Open

Add ability to edit existing images in the media library#305
dkotter wants to merge 23 commits intoWordPress:developfrom
dkotter:feature/media-library-image-edits

Conversation

@dkotter
Copy link
Collaborator

@dkotter dkotter commented Mar 13, 2026

What?

Closes #236, #237

Adds the ability to edit existing images in the media library views.

Why?

We already have the ability to generate images and we're adding the ability to refine those images in #292 but we don't have a way to refine/edit existing images. This PR adds that in via a prompt approach and with a few preset options (Remove background, Expand background).

How?

  • Inject our needed component into the WordPress Media Library edit screen when that is loaded (there are no hooks here to use unfortunately)
  • This component will render three preset buttons, Remove background, Expand background, and Refine Image, which will add a prompt textarea when clicked that allows you to make conversational edits to the image
  • When using this prompt, set the flow up to match our image refinement process used elsewhere. Allows you to refine the image and then import the image once done
  • When using a preset option, add built-in prompts that power those. For expanding background, we first build a new canvas that is 1.5x larger than the image. We add the image to the center of this and we send this to the LLM, allowing it to fill in the new blank space
  • Add better state management, both in this new flow and in the existing image refinement flows, so we keep track of each image generation and allow a user to paginate back and forth. This allows them to "go back in time" and import an older generated image or start refinement over from that point.
  • Keep state in place after importing so a user can import another image or continue to refine

Use of AI Tools

Used both Claude Code (running Opus 4.6) and Cursor (running GPT-5.3 Codex) to plan and execute, with final refinement and clean up done by me.

Testing Instructions

  1. Install the AI Provider for Google plugin
  2. Pull this PR down and run npm install && npm run build
  3. Add valid Google credentials
  4. Turn on the Image Generation experiment
  5. Ensure you have an existing image in the Media Library
  6. In the Media Library grid view, click on an image and then in the modal that shows, click the Edit Image button
  7. Ensure you see the new buttons. Click on Refine Image
  8. Add a prompt and click Generate
  9. Ensure an image is generated and that it shows side-by-side with the original image
  10. From here, test out the various buttons below the images, like refining further, running the generation again, starting over and saving the image
  11. Go back to the main edit screen and try out the other buttons
  12. Ensure those work as expected (background is removed and replaced with white or image is expanded)
  13. Run these same tests again but in the block editor. Add an image block and choose Media Library. Choose an image in the modal and click Edit image
  14. If desired, can also test the refinement flow after generating an image (either in the media library or in the editor) and ensure you can refine the image multiple times and paginate back and forth through those results

Screenshots

Image edit screen showing the new Expand Background, Remove Background and Refine Image buttons Showing the refine image textarea after the button is clicked Showing the generate loading state after refining an image Showing the first image refinement Showing the second image refinement Open WordPress Playground Preview

dkotter added 15 commits March 11, 2026 09:19
… as well as a prompt area for specific edits. Set this up to match the flow we use in our image generation components. Add necessary CSS and a helper function to turn an image URL into a base64 encoded image
…n flow. This allows you to refine an image multiple times, refining the new refined image each time. Ensure we keep track of all prompts used to store with the image. Update text to match our other image generation flows
…wn. Ensure we don't add extra spacing when our tools are hidden
…n to remove a certain color and replace with transparency that we can try out if we want transparent background instead of white background
…he requested image and then sending that along, so the LLM has known boundaries to expand to. Update our remove background prompt to work better with modern LLMs
…t happens, allowing users to paginate back and forth to those images, allowing them to import previously generated images or go back and start the refinment process from a different state
@dkotter dkotter added this to the 0.6.0 milestone Mar 13, 2026
@dkotter dkotter self-assigned this Mar 13, 2026
@codecov
Copy link

codecov bot commented Mar 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.43%. Comparing base (1f2f7a8) to head (aeb65be).

Additional details and impacted files
@@            Coverage Diff             @@
##             develop     #305   +/-   ##
==========================================
  Coverage      57.43%   57.43%           
  Complexity       596      596           
==========================================
  Files             42       42           
  Lines           3101     3101           
==========================================
  Hits            1781     1781           
  Misses          1320     1320           
Flag Coverage Δ
unit 57.43% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jeffpaul
Copy link
Member

For some reason my Google credentials aren't working, so I can't fully test this (will continue to see what's going on there so I can test more fully, but have some thoughts for now). I would advocate for just removing the AI Exit button press requirements and just add the Expand Background (capital B) and Remove Background buttons on the row below the current Crop and other buttons.

Given that the prompt textarea and Generate button appear to be tied to a "refine" flow, let's perhaps add a Refine Image button that adds the helper text, textarea, and button (perhaps those appear below the image so it doesn't jump the image downward when those are added in. Let's also make the DESCRIBE... text sentence case.

Also for the Expand and Remove and Refine buttons let's ensure the base image is still present, as for now clicking AI Edit replaces the image and renders buttons but would be ideal to keep the image and add in the buttons.

dkotter added 3 commits March 18, 2026 15:40
…. Hide the prompt textarea and add a new Refine Image button that when clicked will show that. Minor text and style updates
@dkotter
Copy link
Collaborator Author

dkotter commented Mar 18, 2026

I would advocate for just removing the AI Exit button press requirements and just add the Expand Background (capital B) and Remove Background buttons on the row below the current Crop and other buttons.

This is done now. Removed the AI Edit button and instead we show the other buttons by default (and capitalized the second word). I've updated screenshots in the PR description showing the new UI.

Given that the prompt textarea and Generate button appear to be tied to a "refine" flow, let's perhaps add a Refine Image button that adds the helper text, textarea, and button (perhaps those appear below the image so it doesn't jump the image downward when those are added in). Let's also make the DESCRIBE... text sentence case.

Added a Refine Image button and clicking that now shows the prompt flow. The text above the textarea is now sentence case instead of ALL UPPERCASE, though noting that style was added by Core so we are overriding that here now.

For now I haven't added the textarea below the image as it's pretty common for an image to be larger than the screen height and so you'd have to scroll down to see those options. So easy to miss those were added in that scenario.

Personally I think this conversational image editing is the most useful thing here (comparing to removing background or expanding the image). My make sense to have that be the first button, at the moment I've left that as the last.

Also for the Expand and Remove and Refine buttons let's ensure the base image is still present, as for now clicking AI Edit replaces the image and renders buttons but would be ideal to keep the image and add in the buttons.

I've left the original image while generation is in place. Once the image is generated, the original image hides as we show that side-by-side with the generated image.

@jeffpaul
Copy link
Member

The text above the textarea is now sentence case instead of ALL UPPERCASE, though noting that style was added by Core so we are overriding that here now.

Oh, didn't realize that was core default, huh ok then I guess ignore me and go back to YELLING AT PEOPLE?

For now I haven't added the textarea below the image as it's pretty common for an image to be larger than the screen height and so you'd have to scroll down to see those options. So easy to miss those were added in that scenario.

Ok, that makes sense, I concur.

Personally I think this conversational image editing is the most useful thing here (comparing to removing background or expanding the image). My make sense to have that be the first button, at the moment I've left that as the last.

Totally fine moving that to be the first button.

@dkotter dkotter marked this pull request as ready for review March 19, 2026 16:36
@github-actions
Copy link

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@jeffpaul jeffpaul self-requested a review March 19, 2026 16:39
@jeffpaul
Copy link
Member

Once you get into a Refine, Expand, or Remove button flow the button options tend to disappear. It would be good to ensure those options are present as image iterations are created.

@jeffpaul
Copy link
Member

Here's getting into the Refine flow where ideally we still have Expand and Remove options present:

Screenshot 2026-03-19 at 11 55 00 AM

Here's getting into the Expand flow where ideally we still have Expand and Remove options present:

Screenshot 2026-03-19 at 11 56 05 AM

Here's getting into the Remove flow where ideally we still have Expand and Remove options present:

Screenshot 2026-03-19 at 11 57 26 AM

@dkotter
Copy link
Collaborator Author

dkotter commented Mar 19, 2026

Once you get into a Refine, Expand, or Remove button flow the button options tend to disappear. It would be good to ensure those options are present as image iterations are created.

Would you expect those buttons to stay where they are at (above the image, below the default buttons)? Or should those show with the other buttons (Save, Refine, Generate, Start over) we render in that preview state, below the images?

Pros and cons to both in my mind. I like keeping them where they are at in all states (so above the image) but does also make sense to have them with the other buttons we render, noting we already have 4 buttons in that row so adding 2 more does get overwhelming

@jeffpaul
Copy link
Member

If we go with the top, we're effectively duplicating the Refine option in both places. So perhaps in this flow we keep all three buttons at the top and remove the second Refine one from below the image?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add generative image edit actions (inpaint and outpaint)

2 participants