Deliverable #4
Silver Bullets
Dr. Bowring
CS 362
12 November 2015
Deliverable #4
Team Silver Bullet was able to successfully design, implement, and test 25 test cases to check the limitations and abilities of the Calculator activity within Sugar. For the most part, everything went easily. The subtraction and division methods went off without a hitch. We ran into a small snag when working with multiplication, however. In Python, from what we found, when one multiplies a string and a number, the result will be a string that is simply the original string, but repeated X times, where X is the number in the equation. As an example, we originally expected a ValueError to result from multiplying the following:
“silver”
4
However, our outcome file contained “silversilversilversilver”. Truth be told we initially had 373 as our number, so the file was quite long, so we shortened it to 3. Going from here, we had an interesting problem; when we read the test result strings from the expected output text files, we saw through the terminal that both the expected outputs and oracles were the same, however they failed when we attempted to compare them. We tried many different ways to fix the issue, and eventually found out that there was a line break in the files that we had to remove before testing would continue.
We had a small snag with our mod function testing as well. On one of our tests, we attempted to run a mod function with a decimal, however what we found was that the Calculator activity wasn’t designed to handle decimals in that function! After some brief discussion, we decided that it was a limitation of the program, and the test ended up failing.
Testing Framework:
/scripts/ - contains main python running script, runAllTests.py
/testCases/ - contains text files detailing each test case. Holds information to be passed
/project/src/ - location of the python Calculator activity that is being tested
/oracles/ - contains text files with expected outcomes of each test case
/testCaseExecutables/ - contains the python executable files to run each test case
/reports/ - contains the HTML file containing the aggregated result information
/Outputs/ - contains text files of test case executable results after the program is run
Running Instructions:
-
Clone the project from the TestAutomationProject branch to any desired filepath
-
Navigate to /TestAutomation/scripts/
-
Run 'runAllTests.py' with python2
-
An HTML file will open, the location of this file is at /TestAutomation/reports/
Test Case Overview
