-
Notifications
You must be signed in to change notification settings - Fork 0
pythonizing regression
There exist two workflows for regression testing, interactive mode and global mode.
- git clone two repositories
- Go to tag mpi-3.1 on one of them, go to the one you are trying to verify in the other repository
- Run "./binding_tool/regress.py --interactive ../mpi-repo-reference ../mpi-repo-new-shiny
The interactive mode will iterate through all common bindings between the two repositories and show them to the user for approval.
When a non-approved binding is found a diff is shown to the user with a likeness ratio and locations in files of both repositories with line of file, respectively.
This is an example of a difference: ` ################################################################################ ################################################################################
likeness ratio 0.9048843187660668
original ../mpis/mpi3/chap-coll/coll.tex, 1154
modified chap-coll/coll-rendered.tex, 1103
- \mpifnewbind{MPI_Scatter(sendbuf, sendcount, sendtype, recvbuf, recvcount, re ?
- \mpifnewbind{MPI_Scatter(sendbuf, sendcount, sendtype, recvbuf, recvcount, re ?
cvtype, root, comm, ierror) \fargs TYPE(*), DIMENSION(..), INTENT(IN) :: sendbuf
cvtype, root, comm, ierror) \fargs TYPE(*), DIMENSION(..), INTENT(IN) :: sendbuf
\ INTEGER, INTENT(IN) :: sendcount, recvcount, root \ TYPE(MPI_Datatype), IN
ount, root \ TYPE(MPI_Datatype), INTENT(IN) :: sendtype, recvtype \ TYPE(MPI\
TENT(IN) :: sendtype, recvtype \ TYPE(*), DIMENSION(..) :: recvbuf \ TYPE(MPI
+++++++++++++++++++++++++++++++++++++
_Comm), INTENT(IN) :: comm \ INTEGER, OPTIONAL, INTENT(OUT) :: ierror}
_Comm), INTENT(IN) :: comm \ INTEGER, OPTIONAL, INTENT(OUT) :: ierror}
approve(a), skip(s), exit(x): `
A "######" marker is used for difference below 95% matching ratio. A "<><><>" is shown for less severe mismatches.
The prompt asks for approval for this binding. The user can exit or skip the binding instead of approving. When approved a binding, when encountered is automatically skipped.
Approvals serve as a mechanism to allow for changes between MPI 3.1 version bindings and MPI 4.x version bindings. For example spacing changes, reordering of tildas (non-linebreaking spacing in latex), reordered bindings, etc.
- git clone two repositories
- Go to tag mpi-3.1 on one of them, go to the one you are trying to verify in the other repository
- Run "./binding_tool/regress.py ../mpi-repo-reference ../mpi-repo-new-shiny