Wrong text output

[PeterStaar-IBM]

Stafford.County.-.VA.Zoning.Ordinance.pdf

[dhdaines]

I've bisected this on a document that is giving me the same problem and it seems to be due to a change that happened between 3.1.2 and 3.2.0. My document and the one above seem to have in common that their text objects consist of individual characters positioned absolutely, like this (you can use playa --content-streams to see this easily 😉)

BT
/F4 10.6599998 Tf
1 0 0 -1 34.015625 29 Tm
<0016> Tj
5.9285736 0 Td <0012> Tj
2.9616852 0 Td <0014> Tj
5.9285736 0 Td <001B> Tj
5.9285736 0 Td <0012> Tj
2.9616852 0 Td <0015> Tj
5.9285736 0 Td <0017> Tj
5.9285736 0 Td <000F> Tj
2.9616852 0 Td <0003> Tj
2.9616852 0 Td <0014> Tj
5.9285736 0 Td <0015> Tj
5.9285736 0 Td <001D> Tj
2.9616852 0 Td <0015> Tj
5.9285736 0 Td <0019> Tj
5.9285736 0 Td <0003> Tj
2.9616852 0 Td <0033> Tj
7.1101227 0 Td <0030> Tj
ET

Probably there is some bug in the way docling-parse is handling the text matrix that is causing the characters to end up in the wrong place when they are positioned on the page. They are nonetheless in the correct reading order in the content stream.

[dhdaines]

Without actually building docling-parse here, if I had to guess, I'd bet that #91 is somehow responsible (but GitHub won't let me see the diff for some reason...)

[dhdaines]

Confirmed that it is #91, when comparing parse_v2.exe output between 760b932..v3.2.0 I get this sort of difference:

@@ -35424,7 +35512,139 @@
               648.32,
               37.074,
               648.32,
-              "Consult the following schematics for information about the various ports used by TrueSight Infrastructure Management. The port numbers",
+              "C os u t   t h  f o o i n  shm t i c  f o   i n o m t i o  aot   t h  vr os prs u e  b  T uS gt   I n r at r u t u e M ngm n .   Te pr t   nm e s",
+              -1,
+              4.149,
+              "",
+              "STANDARD",
+              "/F4",
+              "/AAAAAA+OpenSans",
+              false
+            ],
+            [

[dhdaines]

Interestingly enough there are no differences in the output for the individual glyphs, just the merged cells. So there isn't any issue in the actual positioning of the text, just the merging.

This makes sense since it looks like that's what changed in #91 ...

[dhdaines]

The glyphs are also in the right order in ["pages"][0]["original"]["cells"]["data"], and the x-positions are correct. So it is just the "sanitizing" that is not working correctly for some reason.

[dhdaines]

As I suspected it's this specific commit which caused the problem: 75a28e5

[dhdaines]

If I had to guess (my allergies to C++ are really flaring up here) then I would say that the problem is the function pdf_sanitator<PAGE_CELLS>::contract_cells_into_lines_v2

[dhdaines]

Can further confirm that switching back to contract_cells_into_lines_v1 in https://github.com/DS4SD/docling-parse/blob/main/src/v2/pdf_sanitators/cells.h#L153 solves the problem, for instance:

@@ -63350,30 +63304,7 @@
               740.399,
               27.75,
               740.399,
-              "C hp t e   2  -   Z N G RI NNE",
-              -1,
-              4.445,
-              "",
-              "STANDARD",
-              "/F5",
-              "/BAAAAA+Open-Sans-Light",
-              false,
-              true
-            ],
-            [
-              38.137,
-              740.399,
-              160.864,
-              757.627,
-              38.137,
-              757.627,
-              160.864,
-              757.627,
-              160.864,
-              740.399,
-              38.137,
-              740.399,
-              "a r 8 OIN ODAC",
+              "Chapter 28 - ZONING ORDINANCE",
               -1,
               4.445,
               "",

Can you explain the difference between the two contract_cells_into_lines methods? They are not quite obfuscated C++ but they are not exactly clearly written either.

[dhdaines]

I've gone through the differences between the two methods - clearly the v2 has been refactored to be much clearer and more general with respect to RTL and rotated text.

Unfortunately the documents in question have a peculiarity which is that the bounding boxes of their text cells overlap. This is probably a bug in docling-parse and not inherent to the actual PDF content, because if I plot the character cells corresponding to the Tj operators with PAVÉS we can see that (a) they are in the correct reading order, and (b) their actual bounding boxes, as defined by the font's character widths, do not overlap:

But if we look at the output for the first two characters in "Chapter 28", we see this in docling-parse's output (some lines snipped) - note how the x1 value for the first character (38.528) is actually greater than the x0 value for the second character (33.013);

            [
              27.75,
              740.399,
              38.528,
              "C",
            ],
            [
              33.013,
              740.399,
              43.507,
              "h",
            ],

The correct bounding boxes (from playa --text-objects, also trimmed) should be:

{
  "chars": "C",
  "bbox": [
    27.74999884375,
    753.5077582758436,
    33.01271874446999,
    761.8628360717153
  ]
},
{
  "chars": "h",
  "bbox": [
    33.01271875938978,
    753.5077582758436,
    38.13673131388924,
    761.8628360717153
  ]
},

What this means in practice is that pdf_resource<PAGE_CELL>.is_adjacent_to and pdf_resource<PAGE_CELL>.merge_with do not do the right thing, because they assume that cells do not overlap. This means that spaces get inserted everywhere, and also, presumably, that cells get put in wacky orders (actually I am not totally sure why that happens).

This means that the effect of "sanitization" is to partition the text into non-overlapping subsets of cells, with spaces in between them, as you can see if you look closely the results a little ways above:

"C hp t e   2  -   Z N G RI NNE",
   "a r 8 OIN ODAC",

This is clearly not what you want if you want to actually produce readable text ;-)

PR forthcoming here: #105