unit test FileFormatCTF errors in test_cpu* with 'difficult' XML unknown elements

[JGoldstone]

MacOS Sequoia 15.7.1
Most of the component libraries are external (and installed with MacPorts, but that shouldn't matter).

/usr/local/repos/git/aswf/opencolorio/tests/cpu/fileformats/FileFormatCTF_tests.cpp:1474:
FAILED: std::string::npos != StringUtils::Find(parts[i], ErrorOutputs[I])
values were '18446744073709551615' and '18446744073709551615'
[ 229/1083] [FileFormatCTF / difficult_xml_unknown_elements ] - FAILED

For what it's worth, here's those two values, stacked vertically
18446744073709551615
18446744073709551615

Those aren't equal, you say? Hmm.

This shows up in the outputs of test_cpu, test_cpu_no_accel and test_cpu_sse2.

Suggestions as to how to investigate further?

[doug-walker]

The test is that they should not be equal. That large number is npos, so it means the desired error string was not found.

Are you sure you're using OCIO 2.5.0 and Expat >= 2.6? See PR #2192.

This is a file format parsing test, so it is not surprising that it would show up in all the CPU test runs for various SSE options.

[JGoldstone]

The output of port echo installed, filtered for matches against the chart in the PR, looks like this:

expat     @2.7.3_0 
yaml-cpp  @0.8.0_0 
zlib      @1.3.1_0 
imath     @3.2.2_0 
lcms2     @2.17_0 
pybind11  @3.0.1_0 
openexr   @3.4.2_0 

OpenImageIO isn't present, nor OSL, nor openfx.

pystring and minizip-ng were installed from the OCIO-embedded copies and are at 1.1.4 and 4.0.10, respectively.

The expat referenced in the test binary is the one in /opt/local/lib

 1069 % otool -L ./test_cpu_exec
./test_cpu_exec:
	/System/Library/Frameworks/ColorSync.framework/Versions/A/ColorSync (compatibility version 1.0.0, current version 3808.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 4040.1.255)
	/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1957.0.0)
	/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
	@rpath/libOpenColorIO.2.5.dylib (compatibility version 2.5.0, current version 2.5.0)
	/opt/local/lib/libexpat.1.dylib (compatibility version 13.0.0, current version 13.1.0)
	/opt/local/lib/libImath-3_2.30.dylib (compatibility version 30.0.0, current version 30.3.2)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1356.0.0)
	/opt/local/lib/libyaml-cpp.0.8.dylib (compatibility version 0.8.0, current version 0.8.0)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.3.1)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 2000.63.0)
 1070 %  

which as noted above in the port echo installed output is 2.7.3. Note that the shared library names are not required to match the release name, as shown below.

1062 % ls -l /opt/local/lib/*expat*
-rwxr-xr-x  1 root  wheel  149936 Oct 14 20:25 /opt/local/lib/libexpat.1.11.1.dylib
lrwxr-xr-x  1 root  wheel      21 Oct 14 20:25 /opt/local/lib/libexpat.1.dylib -> libexpat.1.11.1.dylib
-rw-r--r--  1 root  wheel  178800 Oct 14 20:25 /opt/local/lib/libexpat.a
lrwxr-xr-x  1 root  wheel      21 Oct 14 20:25 /opt/local/lib/libexpat.dylib -> libexpat.1.11.1.dylib
 1063 %      

An added line of code:

        ...
        for (size_t i = 0; i < parts.size(); ++i)
        {
            std::cout << "testing errror output " << i << "(\"" << ErrorOutputs[i] << "\"" << std::endl;
            OCIO_CHECK_NE(std::string::npos, StringUtils::Find(parts[i], ErrorOutputs[i]));
        }
        ...

allowed me to see which error message was causing the failure:

 1061 % ./test_cpu_exec --run_only FileFormatCTF/difficult_xml_unknown_elements

 OpenColorIO_Core_Unit_Tests 

ARM +sse2

testing errror output 0("(10): Unrecognized element 'Ignore' where its parent is 'ProcessList' (8): Unknown element"
testing errror output 1("(22): Unrecognized attribute 'id' of 'Array'"
testing errror output 2("(22): Unrecognized attribute 'foo' of 'Array'"
testing errror output 3("(27): Unrecognized element 'ProcessList' where its parent is 'ProcessList' (8): The Transform already exists"
testing errror output 4("(30): Unrecognized element 'Array' where its parent is 'Matrix' (16): Only one Array allowed per op"
testing errror output 5("(37): Unrecognized element 'just_ignore' where its parent is 'ProcessList' (8): Unknown element"
testing errror output 6("(69): Unrecognized element 'just_ignore' where its parent is 'Description' (66)"
testing errror output 7("(70): Unrecognized element 'just_ignore' where its parent is 'just_ignore' (69)"
testing errror output 8("(75): Unrecognized element 'Matrix' where its parent is 'LUT1D' (43): 'Matrix' not allowed in this element"
/usr/local/repos/git/aswf/opencolorio/tests/cpu/fileformats/FileFormatCTF_tests.cpp:1475:
FAILED: std::string::npos != StringUtils::Find(parts[i], ErrorOutputs[i])
	values were '18446744073709551615' and '18446744073709551615'
testing errror output 9("(76): Unrecognized element 'Description' where its parent is 'Matrix' (75)"
testing errror output 10("(77): Unrecognized element 'Array' where its parent is 'Matrix' (75)"
[ 229/1083] [FileFormatCTF / difficult_xml_unknown_elements              ] - FAILED


1 tests failed with 1 errors 0 skips.

(returned status 1) 1062 %

Interestingly / disturbingly this failure does not occur when I am running the tests as part of a MacPorts install:

1059 % sudo ctest --verbose
UpdateCTestConfiguration  from :/opt/local/var/macports/build/opencolorio-4e042554/work/build/tests/cpu/DartConfiguration.tcl
UpdateCTestConfiguration  from :/opt/local/var/macports/build/opencolorio-4e042554/work/build/tests/cpu/DartConfiguration.tcl
Test project /opt/local/var/macports/build/opencolorio-4e042554/work/build/tests/cpu
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: test_cpu

1: Test command: /opt/local/var/macports/build/opencolorio-4e042554/work/build/tests/cpu/test_cpu_exec
1: Working Directory: /opt/local/var/macports/build/opencolorio-4e042554/work/build/tests/cpu
1: Test timeout computed to be: 10000000
1: 
1:  OpenColorIO_Core_Unit_Tests 
1: 
1: ARM +sse2
1: 
1: [   1/1146] [CategoryHelpers / categories                                ] - PASSED
...
1: [ 261/1146] [FileFormatCTF / info_example                                ] - PASSED
1: [ 262/1146] [FileFormatCTF / difficult_syntax                            ] - PASSED
1: [ 263/1146] [FileFormatCTF / difficult_xml_unknown_elements              ] - PASSED
1: [ 264/1146] [FileFormatCTF / unknown_elements                            ] - PASSED
1: [ 265/1146] [FileFormatCTF / wrong_format                                ] - PASSED
...
1: [1146/1146] [ViewTransform / basic                                       ] - PASSED
1: 
1: 
1: 0 tests failed with 0 errors 0 skips.
1: 
1/3 Test #1: test_cpu .........................   Passed    1.42 sec

This issue is not a blocker per se: in getting MacPorts' OpenColorIO 3.5.0 to pass unit tests, but it doesn't leave me with feelings of confidence that a client of the installed library won't get bitten by linking to that library.

Does the failure on that particular error output suggest anything to you?

[doug-walker]

Hi Joseph, you don't seem to be using OCIO 2.5.0. In the current source code for that test, the line number is removed from the "Matrix where its parent is LUT1D" test, because it is expat library version specific.

[JGoldstone]

@doug-walker thanks, I found my MacPorts configuration error where I was still pulling down the earlier version, though I thought I was asking for 2.5.0. Closing this.