Deliverable 3

Deliverable 3

###Team Progress Experience

Our team is working well since we have a better grasp of the project and each other’s skills. We decided as a team that, since we all knew python, it would be best to capitalize on that skill. This made it easier to understand what our code was doing. Additionally, our team has emphasized each other’s strengths by dividing some of the work. For example, we each wrote test cases. So far, this project has definitely helped in learning how to collaborate with others on a project and how important it is to have a team leader to keep everyone on task. As planned out in our Deliverable #2, our detailed week-by-week schedule has helped us not fall behind on what is required with each deliverable.

Architectural Description of Framework

Our full script plan is to read the file, located in /testCases, and populate a list for each test case. The script then parses the file, meaning the script partitions the lines in the test case file to locate the code to be tested, the driver to be used, and the information needed for the report. It also handles importing the driver and the method to be tested. After that, it is sent to the driver and calls the functions from parse. The script executes the function specified inputs retrieved by parse function according to the test case. A compare helper method compares the results with the expected results, the oracle, and saves results in the test report. Then, the script uses the information and outputs to the report function and generates a Html document table in a web browser. The testing framework will be called by a single script using ./scripts/runAllTests.sh and will access the test case folder of test case specifications, which has a single test case specification file for each test case. These specification files follow a strict specification format by following the template. Each test case specification file has the data to start and execute the test case and to collect the results of the test case execution of whether it passed or failed.

Framework Directory Structure

TestAutomation:
/project
/scripts runAllTests.sh
tests.py

/testCases → contains 25 test cases
testCase1.txt
testCase2.txt
...
/testCasesExecutables
testCase1to25

/temp (for output from running tests ... cleans at the start of runAllTests)
testCase1results (might be folder or file)
...
/oracles → (contains expected output)
testCase1Oracle (might be folder or file)
...
/docs → (deliverables and readme files)
README.txt

/reports → (html report document)
testReport.html

How-To Documentation

Prerequisites: Python 2.7+

The first step is to clone the TeamIsntThisFun repository. Afterwards, clone the repository located at Beets Github into the /project/src folder in the cloned TeamIsntThisFun repository.

To run the automated testing framework navigate to the folder IsntThisFun/scripts.

With these commands, “chmod +x runAllTests.sh” and “./runAllTests.sh”, the testing framework should execute automatically and output a test report in a web browser.