Changing text color in PDF

[qwertynik]

Hello,

Is there a way the color of the font in the existing PDF can be changed? The script linked here might help.
However, wanted to know if there are direct ways of doing this.

Thanks!

[JorjMcKie]

There is no "direct" way to do this. To do it in the clean way requires rebuilding the file.
One nonetheless does have a few (dirtier) options:

• Use redactions to remove selected text, then re-insert same text with all the same font properties (except the color) in the same place - using TextWriter. In contrast to specifying the replacement text already in the redaction definition, will not work well, because of (inevitable) positioning inaccuracies. So extract the text with all relevant information, define & apply a redaction, then insert the recolored text.
• Dig you way through the /Contents object(s) and insert / replace color info directly there. This is hacky, but importantly difficult because you will have problems finding the text of your choice. There is usually no way to back-translate the code found between text object markers "BT" / "ET".

[qwertynik]

These approaches sound interesting @JorjMcKie
Would be helpful if you could point to some working samples for these approaches.

This is hacky, but importantly difficult because you will have problems finding the text of your choice.

At this moment, the goal is to replace white text with black or a different colored text. This is to be done because some text, in white color, in the original PDF has a colored background, and when all drawings/images are removed, the text which was originally visible is no longer visible.

[JorjMcKie]

which was originally visible is no longer visible.

Well in this case you simply could make some new background - at more or less no effort.

[qwertynik]

Yes certainly. But in this case, the goal is to change the text's color.

[JorjMcKie]

The (preferred) redaction approach would go along these lines:

• take rect of white text
• take its font, fontsize, etc
• make a redaction annotation with the rect and apply it
• make a TextWriter, tw, append the original text in desired color using old font, fontsize etc.
• the position (insertion point) must be the original span["origin"]. If the original text was no span, take the origin of the first span of the text.
• write the tw.
• done

[JorjMcKie]

So you see that clean contents changes stuff a lot. Among that it gets rid of unnecessary hex specification of text.
Only works in this case because of the simple text font Helvetica.
In general the latter does not happen.

[JorjMcKie]

If text was colored using CMYK, you would see coloring codes k, K and color quadrupel.

[qwertynik]

After some tinkering with the code you shared, came up with the following code that does the color replacement.

xref_lst = page.get_contents() 

for xref in xref_lst:
    stream = doc.xref_stream(xref).decode()

    print(stream)

    colorReplaced = stream \
        .replace('1 1 1 rg', '1 0 0 rg') \
        .replace('1 1 1 RG', '1 0 0 RG') \
        .replace('1 g', '0 g') \
        .replace('1 G', '0 G')
    ncont = bytes(colorReplaced, "utf-8")
    doc.update_stream(xref, ncont)

doc.save('output/sample.pdf')

Adding color using different color codes, helped in realizing that there would be different color codes that needs to be handled.
@JorjMcKie Do you see any problems with the way the color is being replaced? One problem that stands out is that, if there is text 1 g in the pdf that will also be incorrectly replaced.

[JorjMcKie]

Yes that's the way it works.
You could of course make a sophisticated search for the "right" color specs - instead of radically changing all.
You may have seen that MuPDF's cleaning algorithm puts text color specs at a certain place - immediately before BT. You could use this fact (reliable?).
Also, every clean contents command will appear on its own line - makes things a lot easier.

contlines = doc.xref_stream(xref).splitlines()
for i, line in enumerate(contlines):
    if line == b"BT":
        # look if up to 2 preceeding lines .endswith() a color command
        # and if so change them - or replace them with b"" => black text!
doc.update_stream(xref, b"\n".join(contlines))

[qwertynik]

You could of course make a sophisticated search for the "right" color specs - instead of radically changing all.
Yes, from my brief experience working with PDFs noticed that there are 3 color specs, YUV, CMYK and RGB. Any master list to refer to? Did not find the same info on PDF spec

You may have seen that MuPDF's cleaning algorithm
The cleanup is done using page.clean_contents(), correct?

With so many considerations when using the Contents, looking to try out the redaction approach. One of the main contentions for not trying this first was the text becoming misaligned. But now I assume that should not be the case. Is there any code sample that I can refer to @JorjMcKie?

[JorjMcKie]

Look at this snippet:

import fitz
from pprint import pprint
text = "Kleine Schwertwale bilden Schulen"
doc = fitz.open("text-oc.pdf")
page = doc[0]
rl = page.search_for(text)
assert len(rl) == 1
clip = rl[0]
# extract text info now - before the redacting removes it.
blocks = page.get_text("dict", clip=clip)["blocks"]
span = blocks[0]["lines"][0]["spans"][0]
assert span["text"] == text

# remove text
page.add_redact_annot(clip)
page.apply_redactions()

# re-insert same text - different color
font = fitz.Font("figo")  # this must be known somehow - or simply try some font else
tw = fitz.TextWriter(page.rect, color=(1, 0, 0))
# text insertion must use the original insertion poin and font size.
# if not original font, then some fontsize adjustments will probably be required:
# check bbox.width against textlength computed with new font
tw.append(span["origin"], text, font=font, fontsize=span["size"])
tw.write_text(page)
doc.ez_save("x.pdf")

Before:

After:

[JorjMcKie]

Obviously, this TextWriter approach is both, safer and more flexibel: you could also make selected text bold or italic instead of just changing color obviously.

If you use a different font and hence adjust the fontsize, be aware that things may look awkward ...

[qwertynik]

Given the end of the winter solstice was away for a longer duration. Tinkered with the script a little and it works as expected @JorjMcKie.
Was uncertain about the color representation of white as 16777215. After some searching realized that this method fitz.sRGB_to_rgb(16777215) converts to RGB.

[JorjMcKie]

As a courtesy 😁, here an example for a font changer:

import fitz
from pprint import pprint

text = "Kleine Schwertwale bilden Schulen"
doc = fitz.open("text-oc.pdf")
page = doc[0]
rl = page.search_for(text)
assert len(rl) == 1
clip = rl[0]
blocks = page.get_text("dict", clip=clip)["blocks"]
span = blocks[0]["lines"][0]["spans"][0]
assert span["text"] == text
textbox = fitz.Rect(span["bbox"])
page.add_redact_annot(clip)
page.apply_redactions()

font = fitz.Font("cobo")  # be awkward: courier bold
textlength = font.text_length(text, span["size"])
if textlength != textbox.width:  # adjust fontsize where necessary
    fontsize = textbox.width / textlength * span["size"]
else:
    fontsize = span["size"]
tw = fitz.TextWriter(page.rect, color=(1, 0, 0))
tw.append(span["origin"], text, font=font, fontsize=fontsize)
tw.write_text(page)
doc.ez_save("x.pdf")

This is how this would look:

[qwertynik]

{'courier': 'Courier',
 'courier-oblique': 'Courier-Oblique',
 'courier-bold': 'Courier-Bold',
 'courier-boldoblique': 'Courier-BoldOblique',
 'helvetica': 'Helvetica',
 'helvetica-oblique': 'Helvetica-Oblique',
 'helvetica-bold': 'Helvetica-Bold',
 'helvetica-boldoblique': 'Helvetica-BoldOblique',
 'times-roman': 'Times-Roman',
 'times-italic': 'Times-Italic',
 'times-bold': 'Times-Bold',
 'times-bolditalic': 'Times-BoldItalic',
 'symbol': 'Symbol',
 'zapfdingbats': 'ZapfDingbats',
 'helv': 'Helvetica',
 'heit': 'Helvetica-Oblique',
 'hebo': 'Helvetica-Bold',
 'hebi': 'Helvetica-BoldOblique',
 'cour': 'Courier',
 'coit': 'Courier-Oblique',
 'cobo': 'Courier-Bold',
 'cobi': 'Courier-BoldOblique',
 'tiro': 'Times-Roman',
 'tibo': 'Times-Bold',
 'tiit': 'Times-Italic',
 'tibi': 'Times-BoldItalic',
 'symb': 'Symbol',
 'zadb': 'ZapfDingbats'}

@JorjMcKie Found the above fonts being supported out of the box. Are there any fonts that can be flattened? Some services extracting images and other data from a PDF cannot parse a PDF that uses a non-CID font - most likely because they cannot flatten it. Not certain why font flattening is required though.

Thanks!

[JorjMcKie]

For cases other than your current concerns:
Keep in mind, that inserting new text always will store it separate from where the old text was coded in the /Contents. Specifically, it will not be subject to conditional display logic that may exist for old text, or be covered by other objects like that one.
Also, any later text extraction will yield the replacement text separately: additional app logic will be required to let it appear as if positioned as before.

[qwertynik]

Specifically, it will not be subject to conditional display logic that may exist for old text, or be covered by other objects like that one.

That's interesting to know @JorjMcKie. Wasn't aware that PDF supports conditional display logic. Makes the PDF format all the more powerful

Also, any later text extraction will yield the replacement text separately: additional app logic will be required to let it appear as if positioned as before.

Yes realized the same. After changing the font color using the above script, could not parse the same text using Ghostscript. Any ideas on how to insert the text in a way that it is extractable?

[JorjMcKie]

to insert the text in a way that it is extractable?

Extractability as such is warranted! Only the sequence will be changed - and it is your responsibility to reestablish readability:

>>> doc=fitz.open("x.pdf")
>>> page=doc[0]
>>> print(page.get_text())
Der Kleine Schwertwal (Pseudorca crassidens),
auch bekannt als Unechter oder Schwarzer
Schwertwal, ist eine Art der Delfine (Delphinidae)
und der einzige rezente Vertreter der Gattung
Pseudorca.
Er ähnelt dem Orca in Form und Proportionen, ist
aber einfarbig schwarz und mit einer
Maximallänge von etwa sechs Metern deutlich
kleiner. 
 von
durchschnittlich zehn bis fünfzig Tieren, wobei sie
sich auch mit anderen Delfinen vergesellschaften
und sich meistens abseits der Küsten aufhalten.
Sie sind in allen Ozeanen gemäßigter,
subtropischer und tropischer Breiten beheimatet,
sind jedoch vor allem in wärmeren Jahreszeiten
auch bis in die gemäßigte bis subpolare Zone
südlich der Südspitze Südamerikas, vor
Nordeuropa und bis vor Kanada anzutreffen.
Kleine Schwertwale bilden Schulen

The last line is the corrected one and logicaly belong to the one in front of the lonely "von".

[JorjMcKie]

Found the above fonts being supported out of the box. Are there any fonts that can be flattened?

What does "flattened" mean in this context?
TextWriter should support all fonts that use Identity encoding. This includes TTF, OTF and those Base 14 files, but also all the Google NOTO fonts. Plus Droid Sans Fallback Regular via the font code "cjk" (use it in place of codes like "helv"). CJK support all extended Latin plus Korean, Japanese, Chinese.
If you install pymupdf-fonts you will get additional fonts accessible via some "code" shortcuts (FiraGO, Fira Mono, Space Mono, MS Cascadia and more).

[qwertynik]

What does "flattened" mean in this context?

This was mentioned in the error of an open-source library. Presumably flattening is a processing step - not sure what it is though. Mentioned it here thinking that, given your experience with PDFs, it might be something that you would be aware of.

Will attempt using pymupdf-fonts and see if it helps resolving the error 'Cannot flatten a non-CID font'

[JorjMcKie]

Hm, still no clue what flatten might mean.
Anyway, I am afraid that the other fonts then all are neither usable by that obscure software: they are no CID fonts, but TTF and OTF.
The only way out I currently see is that you go by and use something like fontforge to convert a font of your liking from eg TTF to CID ...

[qwertynik]

Yes. Or, probably explore more on what CID and flattening means and get the pdf working.