Non-destructive Metadata Editing API

[jubilant-enigma]

I created this discussion because I can't seem to find any other serious talk about it, only a few comments on issues etc. There is #1784, but it's slightly different and I didn't want to derail that thread.

I am very interested in the ability to edit metadata for images without having to re-save them. Depending on the image format, this would either prevent generational losses, or avoid having to spend CPU re-encoding the image. It's possible in the next few weeks that I might have time to create a PR with this functionality.

I'm wondering if the ImageSharp maintainers would have any suggestions on how that API would look on the front end? A naive method would be to add a new UpdateMetadata static function on Image, that would accept an input stream/path, output stream/path, and an ImageMetadata object.

I have a feeling that the ImageSharp maintainers have probably thought a bit more about this than I have though, so I'd appreciate any suggestions or links to existing discussion that I might have missed.

[antonfirsov]

Personally, I'm fine with adding such a feature as a community contribution in general, but the core team definitely cannot prioritize it right now, which makes it challenging, since even the API/PR review is something that needs a lot of attention from us.

Regardless, I can try to support such an effort, but I'm sure it's going to be an enormous amount of work for you.

First thoughts:
Upon IImageEncoder and IImageDecoder and IImageFormatDetector we need to introduce a new interface IUpdateImageMetadata:

public interface IUpdateImageMetadata
{
    void UpdateMetadata(Stream stream, Action<ImageMetadata> updateAction);
    Task UpdateMetadataAsync(Stream stream, Action<ImageMetadata> updateAction, CancellationToken cancellationToken);
}

Channeling this up to Image.* static methods is the easier part, the real challenge is to implement it for particular codecs with code sharing between IImageEncoder/IImageDecoder/IUpdateImageMetadata implementations.

IMO the first step would be to prototype IUpdateImageMetadata for Jpeg, if you can do it on a branch we can make further conclusions based on the code changes.

Again: this is going to be a loooong and costly process.

[br3aker]

Frontend API is the least challenging problem here IMO :)
I'll try to help on Jpeg part though!

[JimBobSquarePants]

V3 maybe.... It's massive amount of work.

[jubilant-enigma]

Just wondering, is there a particular reason why you think Jpeg should be the format to prototype with? I'm asking because I already made a prototype for Png in about ~100 LOC. Is Png unrealistically simple compared to other formats?

Also, I'm thinking about the IUpdateImageMetadata interface you suggested above. Generally, I would imagine that you'd want the image metadata to be at the beginning (before the image data) of the file (as a matter of convention and for potentially improved performance for applications that only want to read the metadata). However, it sounds like with the suggested IUpdateImageMetadata you'd only be able to put it at the end of the file.

With Png for example, there might be multiple chunks that would contribute data to an ImageMetadata object. So the only way you'd know that you had finished reading all metadata out of a Png would be to read it to the end. If you already have to parse the whole Png to read the metadata, do you think it would make more sense to have something like this:

public interface IImageDecoder
{
    // ...
    // other existing IImageDecoder methods
    // ...

    void UpdateMetadata(Stream src, Stream dst, ImageMetadata newMetadata);
    Task UpdateMetadataAsync(Stream src, Stream dst, ImageMetadata newMetadata, CancellationToken cancellationToken);
}

(the Action<ImageMetadata> parameters have been converted to just ImageMetadata objects)

With this interface, you'd just use the existing Identify() function to load the metadata, then call UpdateMetadata(). At least for my use case, this would be more efficient, since I'd probably want to be calling Identify() anyway before trying to update the metadata.

It feels a bit ugly to put this functionality in the image decoder class, but I'm just suggesting that because 95% of the functionality UpdateMetadata() would need is in the decoder, and the only encoder logic that would need to be shared is the WriteChunk() (or whatever the format equivalent is).

[JimBobSquarePants]

Also, I'm thinking about the IUpdateImageMetadata interface you suggested above. Generally, I would imagine that you'd want the image metadata to be at the beginning (before the image data) of the file (as a matter of convention and for potentially improved performance for applications that only want to read the metadata). However, it sounds like with the suggested IUpdateImageMetadata you'd only be able to put it at the end of the file.

Image metadata could be anywhere within a file that the image specification allows. I'm not quite sure why you think @antonfirsov suggested API limits you to the end of a file. I'm assuming an Action is being used to prevent users supplying their own ImageMetadata instance which is certainly something we want to avoid.

With this interface, you'd just use the existing Identify() function to load the metadata, then call UpdateMetadata(). At least for my use case, this would be more efficient, since I'd probably want to be calling Identify() anyway before trying to update the metadata.

To successfully implement lossless metadata management we would have to capture the position of each metadata chunk within the input image so that we can directly copy the input stream across to our output. Identify cannot do that for us and we would have to implement the functionality in specialized implementations.

I think we should put together a formal API proposal before anyone goes further. It's incredibly important for us to keep our APIs as simple as possible and the way metadata editing would need to work adds complexity in a lot of areas due to it's cross boundary concerns.