pythonizing regression

Pythonizing: Regression

There exist two workflows for regression testing, interactive mode and global mode.

Interactive Workflow

1. git clone two repositories
2. Go to tag mpi-3.1 on one of them, go to the one you are trying to verify in the other repository
3. Run "./binding_tool/regress.py --interactive ../mpi-repo-reference ../mpi-repo-new-shiny

The interactive mode will iterate through all common bindings between the two repositories and show them to the user for approval.

When a non-approved binding is found a diff is shown to the user with a likeness ratio and locations in files of both repositories with line of file, respectively.

This is an example of a difference:

################################################################################
################################################################################

likeness ratio 0.9048843187660668
original ../mpis/mpi3/chap-coll/coll.tex, 1154
modified chap-coll/coll-rendered.tex, 1103


- \mpifnewbind{MPI\_Scatter(sendbuf, sendcount, sendtype, recvbuf, recvcount, re
?                                                                               
+ \mpifnewbind{MPI\_Scatter(sendbuf, sendcount, sendtype, recvbuf, recvcount, re
?                                                                               

cvtype, root, comm, ierror) \fargs TYPE(*), DIMENSION(..), INTENT(IN) :: sendbuf
                                                                                
cvtype, root, comm, ierror) \fargs TYPE(*), DIMENSION(..), INTENT(IN) :: sendbuf
                                                                                

 \\ TYPE(*), DIMENSION(..) :: recvbuf \\ INTEGER, INTENT(IN) :: sendcount, recvc
   -------------------------------------

 \\ INTEGER, INTENT(IN) :: sendcount, recvcount, root \\ TYPE(MPI\_Datatype), IN
                                                                                

ount, root \\ TYPE(MPI\_Datatype), INTENT(IN) :: sendtype, recvtype \\ TYPE(MPI\

TENT(IN) :: sendtype, recvtype \\ TYPE(*), DIMENSION(..) :: recvbuf \\ TYPE(MPI\
                                 +++++++++++++++++++++++++++++++++++++


_Comm), INTENT(IN) :: comm \\ INTEGER, OPTIONAL, INTENT(OUT) :: ierror}

_Comm), INTENT(IN) :: comm \\ INTEGER, OPTIONAL, INTENT(OUT) :: ierror}

approve(a), skip(s), exit(x): 

A "######" marker is used for difference below 95% matching ratio. A "<><><>" is shown for less severe mismatches.

The prompt asks for approval for this binding. The user can exit or skip the binding instead of approving. When approved a binding, when encountered is automatically skipped.

Approvals serve as a mechanism to allow for changes between MPI 3.1 version bindings and MPI 4.x version bindings. For example spacing changes, reordering of tildas (non-linebreaking spacing in latex), reordered bindings, etc.

Global Workflow

1. git clone two repositories
2. Go to tag mpi-3.1 on one of them, go to the one you are trying to verify in the other repository
3. Run "./binding_tool/regress.py ../mpi-repo-reference ../mpi-repo-new-shiny

TODO