Bring Geeqie up to professional workflow standards

[obsidianeagle076]

Setup (please complete the following information):

• Geeqie version 2.0.1 (Debian 12)

Describe your feature request
Geeqie is very close to being usable for professional-level photography. All that stands in the way are a few UI/UX improvements and performance optimizations.

The current standard for culling large numbers of images prior to import into a database is Photo Mechanic. Photo Mechanic is used as a high-speed light-weight "front end" image sorting tool, to quickly do the initial sort before moving to "heavier" software that can't move though images quickly enough.

Geeqie already has the most important feature from Photo Mechanic: treating RAW+JPEG image pairs as identical images, and using the JPEG to speed up rendering. But Geeqie is still missing a few details.

To bring Geeqie "up to standard," consider the following "user story":

"It's late. I just finished a 12-hour day. I have 10,000 60-megapixel images to go through before I go to bed.

I'm going to run through all 10,000 images, pick the initial selects (i.e the good ones) and give them one star. Then I'm going to go through the one-star images and give the best of those images two stars. Then I'm going to pick out some specific images that need to be sent to the client immediately, and label them red.

Then I'm going to have my RAW editor (Rawtherapee, Adobe Camera Raw, Capture One) or photo database program (Lightroom, Darktable, etc) import the folder, so I can do some RAW post-processing.

My RAW editor or database needs to seamlessly pick up all the marks (star ratings, color labels, etc) that I made on the images. Therefore, everything I did needs to be written to IPTC/XMP in a standards-compliant way."

From this, we can say that Geeqie ought to do the following:

• be able to go through images INCREDIBLY FAST. If the user has an image open, and then presses page down to get the next image, the next image needs to load instantanously. Milliseconds count when you have 10,000 images to go through! This may mean that Geeqie generates and caches image previews for the entire folder when that folder is open, rather than generating previews when the image is loaded. At the very least, read-ahead needs to extend for many images, and loading cached previews must be imediate.

It's been a long day, you've been up since 5AM, but you've got to get through these pictures before you can call it a night. Any delay in loading images is keeping you awake unnecessarily.

• Seamlessly filter images by IPTC/XMP star rating, so that the user can go through e.g "all one-star images" again. Right now, you have to manually select all images with a star rating, and then give them a "label" value, and then filter by "label=1."

Geeqie currently supports "Sort by rating," but this is not enough. If you sort by rating, you will lose your place in the "image stack" as soon as you change the rating of the current image. (e.g setting a 1-star image to 2 stars will move it to the top)

• Apply IPTC/XMP changes to both the JPEG and RAW file, when operating on grouped/stacked JPEG/RAW pairs. Right now it looks like Geeqie is only adding IPTC/XMP star ratings to the JPEG file. This makes it impossible to e.g "Select all 3-star RAW files" when opening the folder in RawTherapee.

Obviously, the RAW file is what gets operated on in later stages of the workflow. Therefore, all metadata changes should be reflected in the RAW file (if necessary using an XMP sidecar file) -- not just the JPEG file.

• Use standard IPTC/XMP fields (e.g IPTC/XMP rating) natively, instead of relying on proprietary and non-standard "labels." No external software knows about "labels" -- why use them at all, when standard equivalents are available?

• Support color classes/color labels, and store them in IPTC/XMP. This post from 2007 suggests it's the XMP Label field. This AI-generated page suggests the xmp.xmp.label is the one to use. This forum post references both the Xmp.photomechanic.ColorClass and Xmp.xmp.Label fields. Probably Xmp.xmp.label is the most standard field.

[caclark]

Support color classes/color labels, and store them in IPTC/XMP.

It seems to me that rating/star is value:
0 - > 5 == bad -> best

and color rating is status e.g.
0 -> n == started work -> sent to customer
i.e. color refers only to some way of indicating a selected value to the user.

Geeqie has a Marks facility.
An image can be given a Marks tag of 1 to 10.
The default value of each is the text string "Mark <n>". Each of these text strings can be modified by the user.
Marks values are ORed together for each image.

Originally, Marks persisted only for the current run, but now the Marks values are preserved - in a text file under .config.

Maybe this facility could be expanded to provide a form of color class, which would require:

1. Better way for the user to set Mark text (label) names
2. User option to store Marks text as user-definable XMP field
3. Option to attach a color indicator to each mark (if the values are ORed, that will lead to some pretty colors)
4. If color rating is status, remove the OR operation and use a clock hand symbol instead of color

Marks can also be connected to Keywords, which complicates things.
Finding some way to use Marks as color class would be my preference.

[xsdg]

Hi, thanks for the thoughtful and detailed suggestions!

First and foremost, I would offer that Geeqie is not a Photo Mechanic clone. And in particular, Geeqie needs to operate in a way that works for media that doesn't come from cameras, and that may not support things like embedded EXIF or XMP or IPTC metadata (for example, an animated gif). It needs to support operating on read-only files or filesystems. And it should be able to operate uniformly even when a single directory (or collection) contains images that both do and don't come from cameras. This isn't to take anything away from your recommendations, which by and large make a lot of sense. But hopefully that context helps explain some of the difference in how Geeqie works vs how Photo Mechanic works.

Beyond that, I also wanted to point out that there are geeqie workflows that can already support professional photography usecases, even when they aren't the same workflows that a person would use with Photo Mechanic. I'm also a professional photographer (IG @dopplerfto), and I've been relying on Geeqie to support that for the last 15 to 20 years (depending on whether you count Geeqie's predecessor gqview).

I will describe my workflow here. Again, not to take away from your recommendations. But rather, to show how a Geeqie workflow might differ from a Photo Mechanic workflow.

First and foremost, "marks" are Geeqie's main strength in terms of filtering images. The more you can operate using marks, the more efficient you'll be (in Geeqie).

Next, my personal preference is that I never want my raw files to be modified. I don't want to have to think about whether a backed-up version of a file is the right or wrong version, and I don't want software bugs to be able to corrupt images. Also, I'm never in the position of delivering raw files to clients. So for me personally, I bias towards safety at the expense of some convenience.

That said, I shoot Nikon on assignment. One standard way of tagging images in the field on Nikon is by marking them read-only ("protected"). Every pro Nikon body has that button. I haven't been able to find any way that the protected state shows up in the embedded image metadata — it definitely marks the file as read-only on the sd card, and that might be it. So I have an import script that looks for the read-only files, and selects them within Geeqie so that I can mark them (3) as part of the import process. That mark is backed by a label ("decent") so that it's persistent.

After that, when I'm on a deadline, I'll only go through the images at or around the mark 3 images, and tag them with mark 4 ("top100"), then mark 1 ("top20") and finally mark 2 ("top 10") to get down to the image set that I need to deliver, which will usually be 5-20 images, depending on the client. I do them that way so that visually, the most important marks are all the way at the left, which makes them easier to scan in the image list (hit "m" to make sure that marks are shown visually in the list).

I have mark 6 set as "remember", for images that are important regardless of whether or not they're good. For example, maybe I shot a score sheet to be able to write a caption. And then I'll use marks 5, 7, 8, 9, and 0 in an ad-hoc manner to support whatever I'm doing at the time.

After filtering, I have a plugin set up to write all of the selected images out to a file, and I use that to create a new subdirectory (with symlinks back to the original files) where I'll run darktable from. I have zero interest in running darktable with thousands of images. So darktable only ever sees the images that I'm intending to retouch and deliver.

For the curious, the core of that plugin is just this:

Exec=ruby -e 'File.open(File.expand_path("~/gq_images"), "a+") {|f| f.puts ARGV.join("\n")}' %F

And then I'll manually

$ cd /path/to/photos
$ mkdir dt-client1
$ cd dt-client1
$ mv ~/gq_images ./
$ ln -s $(cat gq_images) ./  # or for RAWs with sidecars, ln -s $(sed -e 's|\.jpg$|.raf|' gq_images) ./
$ darktable ./

[caclark]

Apply IPTC/XMP changes to both the JPEG and RAW file, when operating on grouped/stacked JPEG/RAW pairs. Right now it looks like Geeqie is only adding IPTC/XMP star ratings to the JPEG file. This makes it impossible to e.g "Select all 3-star RAW files" when opening the folder in RawTherapee.

Edit/Preferences/File Filters/File Types determines what can be written to.
exiv2 is used to write metadata, but I do not think there are many raw file types it can write to.

For a jpeg/xmp pair, if both are writeable the jpeg is used.

I suggest you set the file filter so that xmp files are used for metadata.
RawTherapee can read rating from xmp files.

[obsidianeagle076]

and color rating is status e.g. 0 -> n == started work -> sent to customer i.e. color refers only to some way of indicating a selected value to the user.

Geeqie has a Marks facility. An image can be given a Marks tag of 1 to 10. The default value of each is the text string "Mark ". Each of these text strings can be modified by the user. Marks values are ORed together for each image.

Originally, Marks persisted only for the current run, but now the Marks values are preserved - in a text file under .config.

Maybe this facility could be expanded to provide a form of color class, which would require:

1. Better way for the user to set Mark text (label) names

2. User option to store Marks text as user-definable XMP field

3. Option to attach a color indicator to each mark (if the values are ORed, that will lead to some pretty colors)

4. If color rating is status, remove the OR operation and  use a clock hand symbol instead of color

Marks can also be connected to Keywords, which complicates things. Finding some way to use Marks as color class would be my preference.

That sounds good. There are two current problems with Marks:

1. that they are proprietary and not visible to other applications that handle the image later on, and
2. that Geeqie relies on Marks and, as a result, does not provide basic functionality (filtering images) for other, standards-compliant image properties (like IPTC/XMP ratings)

If Geeqie Marks can (optionally) be translated into IPTC/XMP color classes, then that solves point 1.

Image filtering, for example, should also be generalised over different properties, though: ratings, write-protect status, etc.

[obsidianeagle076]

Hey Omari,

Hi, thanks for the thoughtful and detailed suggestions!

You're welcome.

First and foremost, I would offer that Geeqie is not a Photo Mechanic clone.

It doesn't have to be... it can also offer functionality that Photo Mechanic doesn't. But PM is such a standard tool that we can consider it to be the "minimum bar" for functionality.

And in particular, Geeqie needs to operate in a way that works for media that doesn't come from cameras, and that may not support things like embedded EXIF or XMP or IPTC metadata (for example, an animated gif).

That's fine. This issue is about handling images where, by definition, it is possible to write metadata -- either directly (JPEG files) or using an XMP sidecar file (RAW images). And, incidentally, while I have not checked the XMP sidecar spec, I am willing to bet you could use XMP sidecar files for any image type, even if it didn't come from a camera or is an animated GIF from the 1990s.

But if it is not possible or desirable to write metadata, there is no need to eliminate the ability for the software to keep working.

It needs to support operating on read-only files or filesystems. And it should be able to operate uniformly even when a single directory (or collection) contains images that both do and don't come from cameras. This isn't to take anything away from your recommendations, which by and large make a lot of sense. But hopefully that context helps explain some of the difference in how Geeqie works vs how Photo Mechanic works.

Sure, all that makes sense. The problem isn't that Geeqie can handle those cases. The problem is that Geeqie can't handle the "user story" described in the original issue post.

First and foremost, "marks" are Geeqie's main strength in terms of filtering images. The more you can operate using marks, the more efficient you'll be (in Geeqie).

Yeah, and this is a problem. The user should be able to get the same functionality (filtering, sorting, etc) and efficiency if they are operating using some other metadata field (ratings, protected status, color class).

If you only use Geeqie then marks are fine. If you need to integrate Geeqie into a larger software ecosystem, then marks, for lack of a better word, suck.

What if tomorrow you decided to start using some other software? Your ratings and selects for your entire archive would be gone!

On the other hand, using a standards-based (IPTC/XMP ratings, etc) approach, you'd have no problem. All your ratings would be visible to every standards-compliant software on the market, from now until eternity.

Next, my personal preference is that I never want my raw files to be modified. I don't want to have to think about whether a backed-up version of a file is the right or wrong version, and I don't want software bugs to be able to corrupt images. Also, I'm never in the position of delivering raw files to clients. So for me personally, I bias towards safety at the expense of some convenience.

Great. That's why software like Photo Mechanic usually writes metadata into XMP sidecar files (e.g DSC_31337.xmp for the raw image DSC_31337.NEF) instead of modifying the RAW file directly. All modern, standards-compliant software knows to look for metadata updates in the corresponding XMP sidecar file. (For DNG files, it may be safe to write metadata changes into the DNG file directly.)

That said, I shoot Nikon on assignment. One standard way of tagging images in the field on Nikon is by marking them read-only ("protected"). Every pro Nikon body has that button. I haven't been able to find any way that the protected state shows up in the embedded image metadata — it definitely marks the file as read-only on the sd card, and that might be it. So I have an import script that looks for the read-only files, and selects them within Geeqie so that I can mark them (3) as part of the import process. That mark is backed by a label ("decent") so that it's persistent.

That's cool. I assume you've tried using exiftool and diff to see if there's any difference in the EXIF between a field-tagged and non-field-tagged image?

But... again... if you were writing the field-tagged status as a standard IPTC/XMP field, then this information would be visible in any software, not just Geeqie.

After that, when I'm on a deadline, I'll only go through the images at or around the mark 3 images, and tag them with mark 4 ("top100"), then mark 1 ("top20") and finally mark 2 ("top 10") to get down to the image set that I need to deliver, which will usually be 5-20 images, depending on the client. I do them that way so that visually, the most important marks are all the way at the left, which makes them easier to scan in the image list (hit "m" to make sure that marks are shown visually in the list).

I have mark 6 set as "remember", for images that are important regardless of whether or not they're good. For example, maybe I shot a score sheet to be able to write a caption. And then I'll use marks 5, 7, 8, 9, and 0 in an ad-hoc manner to support whatever I'm doing at the time.

Yeah, this is pretty much what people do with ratings and color classes. See e.g Krogh's "The DAM Book"

After filtering, I have a plugin set up to write all of the selected images out to a file, and I use that to create a new subdirectory (with symlinks back to the original files) where I'll run darktable from. I have zero interest in running darktable with thousands of images. So darktable only ever sees the images that I'm intending to retouch and deliver.

Cool idea, but usually I just filter by rating=5 or color class=octarine. Then copy the files to the desired directory. No scripts required :)

[obsidianeagle076]

Edit/Preferences/File Filters/File Types determines what can be written to. exiv2 is used to write metadata, but I do not think there are many raw file types it can write to.

For a jpeg/xmp pair, if both are writeable the jpeg is used.

I suggest you set the file filter so that xmp files are used for metadata. RawTherapee can read rating from xmp files.

Great, that's a good interim solution. Of course, the best approach is to write to both the XMP sidecar file and the JPEG.

[obsidianeagle076]

@caclark I just checked Edit | Preferences | File Filters | File Types and .xmp is already checked. How do I "set the file filter so that xmp files are used for metadata" ...?

[caclark]

Of course, the best approach is to write to both the XMP sidecar file and the JPEG

How do I "set the file filter so that xmp files are used for metadata" ...?

If you do not enable write for jpegs, then an xmp file will be written to.

---

There is a comment about metadata writing in https://www.geeqie.org/help/GuideReferenceMetadata.html.

There is also this: https://s3.amazonaws.com/software.tagthatphoto.com/docs/mwg_guidance.pdf, page 22.

And this: https://archimedespalimpsest.net/Documents/External/XMP/XMPSpecificationPart3.pdf, page 55.

And this: https://dl.photoprism.app/pdf/specifications/20120101-Adobe_XMP_Specification_Part_3.pdf, page 51.

(There is a well-know phrase - The good thing about standards is that there are plenty of them.)

In my opinion, writing the same thing to different places is not a good idea, but there are statements such as "Existing forms must be updated or removed when a value changes. "

Maybe the code should be changed.

[caclark]

Seamlessly filter images by IPTC/XMP star rating, so that the user can go through e.g "all one-star images" again. Right now, you have to manually select all images with a star rating, and then give them a "label" value, and then filter by "label=1."

See commit 4524a90

[obsidianeagle076]

Of course, the best approach is to write to both the XMP sidecar file and the JPEG

How do I "set the file filter so that xmp files are used for metadata" ...?

If you do not enable write for jpegs, then an xmp file will be written to.

Ok, so I have to uncheck the .jpeg;.jpe;.jpg tickbox in File Filters | File types? Or do I need to check Sidecar is allowed in the File types box?

One note on UI/UX: until I read the documentation, I didn't know the File types box had columns Class, Writeable, Sidecar is allowed. The UI by default only showed me the first 3 columns, and there was nothing to suggest I could scroll off to the side.

There is a comment about metadata writing in https://www.geeqie.org/help/GuideReferenceMetadata.html.

That's useful, in particular the documentation on file filters options.

Still, there should be a checkbox in Preferences | Metadata under Step 1 Options to "Store metadata in both RAW and JPEG images of RAW+JPEG pairs," or something similar.

(There is a well-know phrase - The good thing about standards is that there are plenty of them.)

xkcd standards dot png

In my opinion, writing the same thing to different places is not a good idea, but there are statements such as "Existing forms must be updated or removed when a value changes. "

Maybe the code should be changed.

Making it optional is probably enough. Basically the RAW and JPEG files represent the same underlying data. But different consumers or applications may treat either file as unique and unrelated to the other. For very practical reasons we should therefore keep both as up to date as possible.

[obsidianeagle076]

Seamlessly filter images by IPTC/XMP star rating, so that the user can go through e.g "all one-star images" again. Right now, you have to manually select all images with a star rating, and then give them a "label" value, and then filter by "label=1."

See commit 4524a90

Great, thanks!