Debugging

• Debugging a Mismatch

---

Debugging a Mismatch

So you've found that validation-test.py or plaidbench-validation-test.py reports a mismatch between two of your runs. How do we approach debugging this mismatch?

The first thing to verify is whether we should expect the runs to be mismatch-free. Mismatch checking only works if the costs between the runs are comparable. It's expected to have mismatches if you change the cost function, e.g. from PERP to SLIL.

Minimize the test case

If you discovered the mismatch from compiling all of SPEC CPU2006, choose a single benchmark which has a mismatch and only run that. Ideally, you'd choose a smaller benchmark rather than a larger one.

To determine which benchmark a block comes from, spills.dat from the output of runspec-wrapper-optsched.py shows the function names, making it possible to search that file for the given block (minus the :##; block names are functionName:#blkNumber#, e.g. foo:23).

Here's a script which does that search for you:

findblock.py

#!/usr/bin/env python3
import sys
import argparse

parser = argparse.ArgumentParser(description='Search spills.dat to find the benchmark for a block')
parser.add_argument('spills', help='The spills.dat file to search in. - for stdin')
parser.add_argument('blocks', help='The blocks to search for. This may include the `:##` part, or it may just be the mangled function name', nargs='*')

result = parser.parse_args()

with open(result.spills, 'r') as f:
    file = f.read()

fns = (block.split(':')[0] for block in result.blocks)

fn_locs = [file.find(fn) for fn in fns]
fn_benchmarks = [file.rfind(':', 0, fnindex) for fnindex in fn_locs]
fn_benchmark_spans = [(file.rfind('\n', 0, e), e) for e in fn_benchmarks]
fn_benchmarks = [file[b + 1:e] for (b, e) in fn_benchmark_spans]

print('\n'.join(fn_benchmarks))

Visualize the DDG

It can be very helpful to visualize the DDG for a small mismatch case, presuming there is a small mismatch case.

Dumping a DDG

DataDepGraph has a function WriteToFile which can be used to dump the DDG to a file. Inside sched_region.cpp's FindOptimalSchedule function, check if (dataDepGraph_->GetDagID() == std::string{"MyBlockOfConcern:32"}), writing the DDG to a file with the aforementioned function if it is. Then, re-run the single benchmark.

You may want to make a hotfuncs.ini file which only lists the function with the mismatched block, speeding up the run to dump the DDG:

hotfuncs.ini

MyBlockOfConcern YES

Rough code for dumping DDG:

if (dataDepGraph_->GetDagID() == std::string{"MyBlockofConcern:123"}) {
  Logger::Info("Writing DDG."); // Help to verify that we actually did write the DDG
  auto f = fopen("/home/<me>/path/to/the.ddg", "w");
  dataDepGraph_->WriteToFile(f, RES_SUCCESS, 1, 0);
  fclose(f);
}

Visualizing the DDG

Convert the DDG to a graphical format. Graphviz has dot, which may be desirable.

% TODO: A script which converts the DDG WriteToFile output into a .dot file.