Drawn shape with color set to None is drawn and read as black

[jn-chrn]

Describe the bug

A drawing added to a shape for which no color is applied (shape.finish(color=None)) is then drawn as shape with stroke color set to (0, 0, 0). When read using Page.get_drawings(), the same occurs: the drawing is parsed as being black.

To Reproduce

The following script draws a Rect without color on a page. The drawings from the page are then read, and the path which contains the Rect object is now black.

import fitz

rect = fitz.Rect(10, 10, 30, 30)

# Write the PDF
out_pdf = fitz.open()
out_page = out_pdf.new_page(width=50, height=50)
shape = out_page.new_shape()

shape.draw_rect(rect)
shape.finish(color=None)
shape.commit()

# Read the rect from the PDF
drawings = out_page.get_drawings()
assert len(drawings) == 1
path = drawings[0]
assert path["color"] is not None
print(path["color"])
# Output:
# (0.0, 0.0, 0.0)

Expected behavior

Maybe the behavior shown by the script was actually expected. However, for me it seems inconsistent to draw an object without color and get a color out of it. In some of my use cases, PDF have no stroke color, but still have a fill color. Thus, I would expect here an error, because stroke color or fill color must be set. But I may be overlooking something.

Screenshots

Your configuration

Result of the command:

3.8.12 (default, Oct 12 2021, 13:49:34) 
[GCC 7.5.0] 
 linux 
 
PyMuPDF 1.19.5: Python bindings for the MuPDF 1.19.0 library.
Version date: 2022-02-01 00:00:01.
Built for Python 3.8 on linux (64-bit).

PyMuPDF is installed inside a conda environment with pip.

[JorjMcKie]

So what would you expect to happen if neither a fill nor a stroke color is given?

[jn-chrn]

I would say I would raise an error. I guess when no stroke nor fill are given, nothing should be drawn? Or is it possible to have paths in PDF which have absolutely no color (but for which opacity is 1)?

[JorjMcKie]

Raising an error definitely is an option - probably the most adequate one.
There also exists a PDF painting command which does nothing (n) - look at pages 643 of the PDF spec.
Setting opacity makes no sense at all.

[JorjMcKie]

The next version will react with an exception if color=None and fill=None.

[JorjMcKie]

After giving it more thought, I would like to discuss this in more detail with you guys.

[JorjMcKie]

@jn-chrn and @Luux - Please react to the following argument:

In all of PDF specification, the stroke color default is black. Which means, it will be taken, if an explicit stroke color is absent.

This is exactly what has been implemented in PyMuPDF: if both color arguments, color and fill have the default None then the color is set to black. If fill is not None, then no stroke color is generated if color == None.

If I do make the change you request, then PyMuPDF would

• violate the so-far principle of PDF compatibility
• potentially cause existing scripts to break. Imagine: every single page.draw_rect(rect) would now lead to an exception and need be repair with an additional color parameter.

So I have changed my mind: I think this update would be wrong.

[JorjMcKie]

In fact, I just reconfirmed:

• If color == fill == None then I do not output any color and let PDF take its default action.
• I however do output the PDF command for "stroke" in that case.

I could certainly output the PDF command for "no-op" instead of "stroke" in the case we are discussing. But then my argument above is a serious contradiction: an unknown number of working scripts may suddenly produce nothing anymore ...

I think I cannot do this!

[jn-chrn]

We agree you should not add a breaking change. This is even more true in this particular case, as this follows the PDF specifications. From our side, we should also be able to work around it. So we think not adding an exception is fine.

However the user needs then some insights that this may happen: potential solutions would be a warning, more details in the docstring and/or mentions of this issue in the documentation. What do you think?

[JorjMcKie]

Thanks for the feedback!
Sure, I will improve documentation to make very clear, that some drawing will always result - even if no colors at all are being specified.
One could turn around the argument and ask: what sense does it make at all to first issue draw commands and then negate the result of that effort by withholding all colors?

May be you tell me the background of your request - at the moment I cannot imagine a use case, which might motivate this.

[jn-chrn]

We have a use case where:

• we read information from a page (using Page.get_drawing() mostly)
• we import all drawing elements into an internal representation
• we filter/update elements
• we export the result to PyMuPDF representation
• we save the result

Our internal representation was however not complete, as we limited the amount of properties to import to the bare minimum for the sake of simplicity. To test that everything goes well, we also draw some intermediate steps to see what is going on.

Since we read and write again, we would have hoped that not much information is lost there. This seems now not to be straightforward at all, as you explained.

Our representation of paths has for the moment a stroke color but no fill color. There appeared the case where we would read an element with empty stroke color, but a fill color (which we do not store). Drawing such an element afterwards would result in setting only a stroke color to None, but not setting any fill color.

On unit tests, drawing a file with some elements without stroke color, then reading it again thus resulted in error. The elements given as input and the output were different.

[JorjMcKie]

Ok, thank you, I understand better now.

An additional hickup, that you may have noticed already:

When reading drawings from the page, all color specifications (if not none), are normalized to RGB values, i.e. 3 color components - no matter what the original colorspace was (like GRAY, CMYK, ICC, indexed, ...).
The reason for this is the plethora of different colorspaces out there in the wild (ICC, indexed, colorspaces with only two components, what not). PyMuPDF only supports standard DeviceGRAY, DeviceRGB, DeviceCMYK. Even if the color in a drawing has a compatible length of one of 1, 3, 4, it may happen that it is still none of our supported colorspaces.
So I am using a MuPDF conversion utility, which produces a PDF standard DeviceRGB color triple from any color value. That same utility is BTW also used behind the curtain for returning the text color (in sRGB format) appearing in page.get_text("dict"). I was actually tempted to also produce sRGB values instead of color triples in page.get_drawings() - but it was too late for that, once people started using the drawings method massively.

So bottom line for you is, that you will find (0, 0, 0) in a drawing, even if it was created with color=0.