Output of difflib.HtmlDiff.make_table on the same input is not constant between multiple runs

[stefan6419846]

Bug report

Bug description:

difflib.HtmlDiff.make_table (and make_file as well) generate non-deterministic results without this being documented anywhere.

Background: I am using this functionality and run some integration/unit tests for my own code which checks the HTML output as well. Doing so I discovered that the results would differ, depending on the execution order of the tests itself.

Let's consider this simple example:

import shutil
from difflib import HtmlDiff
from tempfile import NamedTemporaryFile


with NamedTemporaryFile() as file1, NamedTemporaryFile() as file2:
    file1.write(b'Hello World!\n')
    file2.write(b'Foo Bar\n')
    file1.seek(0)
    file2.seek(0)

    html1 = HtmlDiff().make_table(fromlines=['Line 1\n', 'Line 2\n'], tolines=['Line 1\n', 'Line 3\n'])
    html2 = HtmlDiff().make_table(fromlines=['Line 1\n', 'Line 2\n'], tolines=['Line 1\n', 'Line 3\n'])

print(html1)
print('=' * shutil.get_terminal_size().columns)
print(html2)

assert html1 == html2

This will fail with an assertion error due to different HTML:

    <table class="diff" id="difflib_chg_to0__top"
           cellspacing="0" cellpadding="0" rules="groups" >
        <colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>
        <colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>
        
        <tbody>
            <tr><td class="diff_next" id="difflib_chg_to0__0"><a href="#difflib_chg_to0__0">f</a></td><td class="diff_header" id="from0_1">1</td><td nowrap="nowrap">Line&nbsp;1</td><td class="diff_next"><a href="#difflib_chg_to0__0">f</a></td><td class="diff_header" id="to0_1">1</td><td nowrap="nowrap">Line&nbsp;1</td></tr>
            <tr><td class="diff_next"><a href="#difflib_chg_to0__top">t</a></td><td class="diff_header" id="from0_2">2</td><td nowrap="nowrap">Line&nbsp;<span class="diff_chg">2</span></td><td class="diff_next"><a href="#difflib_chg_to0__top">t</a></td><td class="diff_header" id="to0_2">2</td><td nowrap="nowrap">Line&nbsp;<span class="diff_chg">3</span></td></tr>
        </tbody>
    </table>
================================================================================

    <table class="diff" id="difflib_chg_to1__top"
           cellspacing="0" cellpadding="0" rules="groups" >
        <colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>
        <colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>
        
        <tbody>
            <tr><td class="diff_next" id="difflib_chg_to1__0"><a href="#difflib_chg_to1__0">f</a></td><td class="diff_header" id="from1_1">1</td><td nowrap="nowrap">Line&nbsp;1</td><td class="diff_next"><a href="#difflib_chg_to1__0">f</a></td><td class="diff_header" id="to1_1">1</td><td nowrap="nowrap">Line&nbsp;1</td></tr>
            <tr><td class="diff_next"><a href="#difflib_chg_to1__top">t</a></td><td class="diff_header" id="from1_2">2</td><td nowrap="nowrap">Line&nbsp;<span class="diff_chg">2</span></td><td class="diff_next"><a href="#difflib_chg_to1__top">t</a></td><td class="diff_header" id="to1_2">2</td><td nowrap="nowrap">Line&nbsp;<span class="diff_chg">3</span></td></tr>
        </tbody>
    </table>

The specific issue is that both tables have a different index. Digging through the code, this is due to all instances using the same counter _default_prefix, referenced through the class instead of self:

cpython/Lib/difflib.py

Lines 1886 to 1895 in 78aeb38

	def _make_prefix(self):
	"""Create unique anchor prefixes"""
	# Generate a unique anchor prefix so multiple tables
	# can exist on the same HTML page without conflicts.
	fromprefix = "from%d_" % HtmlDiff._default_prefix
	toprefix = "to%d_" % HtmlDiff._default_prefix
	HtmlDiff._default_prefix += 1
	# store prefixes so line format method has access
	self._prefix = [fromprefix,toprefix]

The corresponding code has not been touched since its introduction in 2004 (e064b41). I understand that it is required to somehow maintain a global state when there are multiple tables on a page, but this should be documented and the user should have a public interface to change/reset the counter accordingly.

CPython versions tested on:

3.9, 3.11, CPython main branch

Operating systems tested on:

Linux

[terryjreedy]

You should have pointed out the difference(s). The only one I see is that in"#difflib_chg_to0__top">, '0' changes to '1'. This change is quite deterministic. I presume that if I reran the program, the total output would be the same. I am not persuaded that there is any bug.

@tim-one Any comment?

[stefan6419846]

I have indeed pointed out the difference:

The specific issue is that both tables have a different index.

To be more specific: The numbers inside the ID attributes are not reset between runs and/or cannot be reset with a public interface. This leads to issue with (integration) tests in randomized order for example, while my code snippet just showed the basic difference.

[tim-one]

@tim-one Any comment?

Nope, sorry. I had nothing to do with the HTML-diff'ing code. Heck, I've never even used it 😉.

[terryjreedy]

(I meant that you did not quote the actual text that changed, as I did. So I had to spend a few minutes finding it so I could understand the issue. But anyway...)

This is a feature request. It is not a bug that the output in not specified in every detail.

But making a change appears unnecessary in that adding HtmlDiff._default_prefix = 0 before an HtmlDiff(...) call would completely solve your issue. Or subclass HtmlDiff with an __init__ that does the same. Or copy and edit the code. The latter would guarantee consistency across versions, which is not guaranteed now, unless you decided to change.

Absent a core dev ever agreeing to the change and being willing to do all the work needed just from this report, it would require evidence that multiple users need it, which likely means a discussion on discuss.python.org.

Without such, I personally would be inclined to close this as not planned, because a) not worth the effort, given the ease of a workaround, and b) unlikely to ever be a high enough priority to act on it.

[stefan6419846]

My intent never has been to forcefully replace the default behavior. I have already been referring to an alternative solution:

1. Document the behavior accordingly, id est that for one Python session, running/generating the same HtmlDiff multiple times will always lead to a slightly different output.
2. Provide a public interface for the user to reset the prefix.

But making a change appears unnecessary in that adding HtmlDiff._default_prefix = 0 before an HtmlDiff(...) call would completely solve your issue. Or subclass HtmlDiff with an __init__ that does the same. Or copy and edit the code. The latter would guarantee consistency across versions, which is not guaranteed now, unless you decided to change.

I indeed currently work around this by resetting the prefix variable whenever I need to. My issue with this is that _default_prefix and _make_prefix are internal/private API members and thus referencing them in my own code has not been intended initially. (And for consistency across Python versions: The code only runs on a defined set of Python versions and I am okay with adapting the expected output if required between versions. Maintaining a complete copy of the code is overkill and an additional maintenance overhead IMHO.)