diff --git a/auto-run-nondex/README.md b/auto-run-nondex/README.md index bd5e21717..ce0bf512c 100644 --- a/auto-run-nondex/README.md +++ b/auto-run-nondex/README.md @@ -1,22 +1,20 @@ -# `runNondex` +# runNondex -This script runs individual modules in the specified project with the `Nondex` tool and shows results automatically in the `markdown` format. -Output file is located inside the project folder and called `report_md.md` The output would only contain tests that are detected by the nondex tool as flaky, and the result includes the seeds used for the test and the status of the test result. +What repo we want to run Nondex on? -All the log information would be stored inside the `.runNondex` folder inside the project. +1. This repo has not appeared in the latest `pr-data.csv` +2. This repo contains pom.xml file. -Inside the `.runNondex` folder: -- `./LOGSSS` contains nondex logs for all submodules. -- `LOGresult` contains a list of file paths that contain test errors. -- `htmlOutput` contains a list of file paths for the HTML files generated by the nondex tool. + + +# Dependencies It is recommended to use a virtual python environment for the required modules: ```bash python3 -m venv source /bin/activate ``` - Make sure to first install packages in `requirements.txt`. ```bash @@ -24,27 +22,27 @@ Make sure to first install packages in `requirements.txt`. cd auto-run-nondex pip install -r requirements.txt ``` +# runNondexAll -# `runNondexUnderAuthor` - -This script runs all the valid repos under a given author using the `runNondex` script above. +This is simplified script for quick start +Go to folder of .pom.xml files, and check all the result in result.log -The valid repo is defined as: +# runNondex module: +```bash +./runNondex.sh +``` -1. This repo has not appeared in the latest `pr-data.csv` -2. This repo contains pom.xml file. +With the second parameter in `./runNondex.sh`, you can only run Nondex on modules in a certain directory instead of the entire project. +# runNondexUnderAuthor -### Usage: ```bash -./runNondex.sh ./runNondexUnderAuthor.sh ``` -With the second parameter in `./runNondex.sh`, you can only run Nondex on modules in a certain directory instead of the entire project. - +How to find the Author? > To find the author_repository_url, find the author that you want to run Nondex with, and get this author's repository overview url (e.g., https://github.com/orgs/spotify/repositories). Note that you don't need to provide the url with other parameters, as the script will automatically add the parameters needed. > @@ -65,3 +63,10 @@ With the second parameter in `./runNondex.sh`, you can only run Nondex on module > `find . -maxdepth 1 -type d \( ! -name . \) -exec bash -c "cd '{}' && git rev-parse HEAD && git config --get remote.origin.url | rev | cut -c5- | rev && cat .runNondex/htmlOutput && cat report_md.md" \; &> progress_stats.md` +# Folder structure + +All the log information would be stored inside the `.runNondex` folder inside the project. + +- `./modulelog` contains nondex logs for all submodules. +- `result` contains a list of file paths that contain test errors. +- `htmlOutput` contains a list of file paths for the HTML files generated by the nondex tool. diff --git a/auto-run-nondex/runNondex.sh b/auto-run-nondex/runNondex.sh index 62d4a6c93..c66ffe797 100755 --- a/auto-run-nondex/runNondex.sh +++ b/auto-run-nondex/runNondex.sh @@ -1,25 +1,33 @@ #!/bin/bash DIR="${PWD}" -nondex_version="2.1.1" +nondex_version="2.1.1" #LST version + runNondex () { + #get the modules cd $1 mvn install -DskipTests mvn -Dexec.executable='echo' -Dexec.args='${project.artifactId}' exec:exec -q -fn | tee modnames if grep -q "[ERROR]" modnames; then - echo !!!!! + echo "ERROR: There are errors in the project" exit 1 else - echo OK + echo "No errors detected" fi + mkdir .runNondex - mkdir ./.runNondex/LOGSSS + mkdir ./.runNondex/modulelog + + + # run nondex on each module input="modnames" while IFS= read -r line do - mvn edu.illinois:nondex-maven-plugin:$nondex_version:nondex -pl :$line -Dlicense.skip=true | tee ./.runNondex/LOGSSS/$line.log + mvn edu.illinois:nondex-maven-plugin:$nondex_version:nondex -pl :$line -Dlicense.skip -Drat.skip --fail-at-end | tee ./.runNondex/modulelog/$line.log done < "$input" - grep -rnil "There are test failures" ./.runNondex/LOGSSS/* | tee ./.runNondex/LOGresult - input=".runNondex/LOGresult" + grep -rnil "There are test failures" ./.runNondex/modulelog/* | tee ./.runNondex/result + + # format the result + input=".runNondex/result" while IFS= read -r line do grep "test_results.html" $line | tee ./.runNondex/htmlOutput diff --git a/auto-run-nondex/runNondexAll.sh b/auto-run-nondex/runNondexAll.sh new file mode 100644 index 000000000..cf6d4de5f --- /dev/null +++ b/auto-run-nondex/runNondexAll.sh @@ -0,0 +1,5 @@ +# Assume this script is run from the folder of .pom.xml files + +mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dlicense.skip -Drat.skip --fail-at-end | tee ./nondex.log + +awk '/\[INFO\] Across all seeds:/{flag=1; next} /\[INFO\] Test results can be found at:/{flag=0} flag' "./nondex.log" > "./result" \ No newline at end of file