T2i Semantic Segmentation Color Reference Chart - v21

[AugmentedRealityCat]

EDIT: a new version with proper HEX codes has been posted over here.

This document presents the colors associated with the 182 classes of objects recognized by the T2i Semantic Segmentation model.

This version (v21) is complete and all data has been cross-checked against the official code, thanks to u/Doggettx .

Any previous version should be deleted as they were containing errors and there was also missing data for some classes.

pdf version
T2i_Segmentation_Color_Ref_v21.pdf

spreadsheet in LibreOffice ods format
T2i_Segmentation_Color_Ref_v21.ods

[if-ai]

Does exist a Sheet for the openpose bone colors like this one. I tried color picking the oficial documentation for my free rig here https://ko-fi.com/impactframes but then the model was coming up backwards. I notice the Color in the posex extension were different

[AugmentedRealityCat]

The right hand looks like it has a little issue as it is not guiding as it should, I might need to run more tests to be sure, but so far is great.

I would have to find more information about the hand model official colors to understand. I did not include hands at all in my study when I made the chart. Please run more tests and let me know if you find something wrong with the right wrist joint or the forearm bone link colors per se. Maybe by removing the hands and see if there is still a problem without them? I understand you want to use hands, but this is just to test the chart's accuracy before officially releasing it.

also the neck bone is missing from the chart, but it is not giving me trouble at all.
It's called head in bones list - neck is the name used for the joint (point).

Very nice examples by the way. I made many spiderman pictures for a project I was pitching to a client last fall and it was really hard to get anatomically-correct pictures of that character. ControlNet and T2i are really changing that !

Thanks for your feedback.

[if-ai]

Understood I will use the heads then, and yes, I meant the forearm but is not a problem I think that is why it looks more orange color in the openpose extension. Sorry I was so busy I could not answer you before but I needed to retexture the blender model and try it.

Again Thank you for working on this I see the characters guidance has already improve it.

[hafriedlander]

Openpose bones don't really have different colors to the points - instead both the bones and the points should be semi-opaque (at around 50-60%). The bones should also be ovals, not just lines.

(From reference image https://cmu-perceptual-computing-lab.github.io/openpose/web/html/.github/media/keypoints_pose_18.png from https://cmu-perceptual-computing-lab.github.io/openpose/web/html/doc/md_doc_02_output.html)

I've confirmed that the ControlNet annotator uses the same colors / opacity / bone shape as the above reference by looking at the code, and that's the way I've implemented in gyre.ai too.

(ControlNet reference - https://github.com/lllyasviel/ControlNet/blob/main/annotator/openpose/util.py#L36)

However both the ControlNet and T2I openpose adapter don't seem particularly sensitive to errors in the skeleton - the A1111 extension gets (got? haven't checked in a while) the right hand color completely wrong, and it mostly works fine.

[AugmentedRealityCat]

Thanks a lot for confirming that the color chart is accurate ! Your help is greatly appreciated.

About the transparency, I had discovered that, and the discrepancy between the transparency values from application to application. For this version, I used the transparency shown by the pre-processor, which is 60% , as indicated by the "60% color" label I've put at the top of the joint colors column. I also kept joints and bones that share the same basic color on the same row to help with that association.

Since this whole transparency and color sharing between joints and bones was not apparent to you when you first looked at the chart though, I have to conclude that I could do a better job explaining that. If you have any suggestion for a better version, I would love to hear about it.

To conclude about the transparency values, I've seen 70% used in an openpose extension for A1111 (I did use that for a previous version of the chart), but I also saw one extension where all the colors were opaque, and it was still working.

As for the hands, there are some examples of what you are saying in the spiderman pictures shown above in this thread, and it was my intention to create a chart for those colors as well.

Can you give me some clue about where and how to find more data about the openpose hand model colors ? And what the differences should be between the right and the left hand ?
I've seen this ref picture already but maybe there is a list with RGB values in a repo somewhere :

Thanks again for your help.

[if-ai]

Openpose bones don't really have different colors to the points - instead both the bones and the points should be semi-opaque (at around 50-60%). The bones should also be ovals, not just lines.

(From reference image https://cmu-perceptual-computing-lab.github.io/openpose/web/html/.github/media/keypoints_pose_18.png from https://cmu-perceptual-computing-lab.github.io/openpose/web/html/doc/md_doc_02_output.html)

I've confirmed that the ControlNet annotator uses the same colors / opacity / bone shape as the above reference by looking at the code, and that's the way I've implemented in gyre.ai too.

(ControlNet reference - https://github.com/lllyasviel/ControlNet/blob/main/annotator/openpose/util.py#L36)

However both the ControlNet and T2I openpose adapter don't seem particularly sensitive to errors in the skeleton - the A1111 extension gets (got? haven't checked in a while) the right hand color completely wrong, and it mostly works fine.

Thank you so much for clarifying all this. I am going to improve the rig to account for the transparency and oval shapes of the limbs.

[hafriedlander]

https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/include/openpose/hand/handParameters.hpp#L16 is the original handmodel colors. openpose is under a non-free license - my personal opinion is using just the colors is fair use, but IANAL.

[hafriedlander]

Hmm. Those don't seem to line up with the colors in the reference image though 😕

[monsieurpooh]

What are the actual hex colors? The ones in the image are 7 digits long. But hex colors are either 6 digits long if no Alpha, 8 digits long with Alpha. What is the 7th digit doing? I thought about ignoring the 7th digit. But clearly the magenta of "gravel" is wrong in that case. It looks more like E040C0. Then what does E0400C0 mean? What color is it really?

[AugmentedRealityCat]

Thanks for flagging this mistake. I've now fixed it.
Here is a new version (v22) with proper HEX color codes.
T2i_Segmentation_Color_Ref_v22.pdf
T2i_Segmentation_Color_Ref_v22.xls
T2i_Segmentation_Color_Ref_v22.ods

And here is a picture of it:

The wrong HEX values were induced by an error in the RGB to HEX code. It was using 3 digits for the B channel instead of 2. You can cross-check it if you load the XLS or ODS version. For the first item, the new code should be =DEC2HEX(D6,2)&DEC2HEX(E6,2)&DEC2HEX(F6,2) instead of the old (bad) =DEC2HEX(D6,2)&DEC2HEX(E6,2)&DEC2HEX(F6,3).