Deliverable 2

CSCI 362 Deliverable #2 - Test Plan H/FOSS Project: Beets

Assignment: Produce detailed test plan Report on your experiences and provide the details produced in the 5 test cases, including detailed specifications of your test plan and at least 5 test cases.

Introduction

Our team has decided to switch from our original choice, Open Trip Planner, to our other choice, the project Beets. Open Trip Planner contained too many bugs and problems and would be a problem to actually test. Beets is a python program and runs straight from command line. Beets also provides a fair number of testable classes. Our goal for the project is to create an automatic testing framework for the open source project Beets through collaboration. Ultimately, the test results, success or failed, will be displayed on a HTML page. In this deliverable, we will outline five of our tests cases and detailed specifications of our test plan.

The Testing Process

Team IsntThisFun will create a script to run 5 automated tests on 5 methods within the project Beets, for a total of 25 tests. These tests will compare the output of each method with an oracle for that method. The script will then display the results in a web browser. The way our team will do this is running executable files which create output files to be compared to the corresponding oracle files. The results of this comparison between the output file and oracle file will determine whether the test passes, and then the results will be recorded and then opened and displayed on an HTML page within a web browser.

Requirements Traceability
Users are most interested in the system meeting its requirements and individually testing all requirements. Requirements of methods:

1. human_bytes( size ) - The method should take in an integer corresponding to file size and output a human readable string displaying the file size. 2. human_seconds( ) - The method should take in an integer corresponding to a number of seconds and output a human readable string displaying the time interval using English words. 3. human_seconds_short( ) - The method should take in an integer corresponding to a number of seconds and output a short human-readable M:SS string displaying the time. 4. _sc_decode( ) - The method should... 5. _string_dist_basic( str1, str2 ) - The method should...

Tested Items The methods of the project, Beets, that are to be tested are as follows. Method Location human_bytes( size ) → beets/ui/init.py human_seconds( ) → beets/ui/init.py human_seconds_short () → beets/ui/init.py _sc_decode( soundcheck ) → beets/mediafile.py _string_dist_basic( str1, str2) → beets/autotag/hooks.py

Testing Schedule Overall Project Testing Schedule Based on Syllabus 9/29: Deliverable #2 - Create Test Plan with 5 of 25 test cases 10/22: Deliverable #3 - Design and build an automated testing framework including architectural description (with at least 5 test cases) 11/12: Deliverable #4 - Complete design and implementation of testing framework and create 25 test cases 11/24: Deliverable #5 - Design and inject five faults into code 12/1,3: Team Project Presentations and Final Report Detailed by Week Schedule 9/28 - 10/4: Revise Test Plan if necessary 10/5 - 10/11: Designing automated testing framework /scripting / 5 Test Cases 10/12 - 10/18: Designing automated testing framework / scripting / 5 Test Cases 10/17-20: Fall Break / 10/21- 25: 5 Test Cases 10/26 - 11/1: Scripting framework / 5 Test Cases 11/2 -11/8: Scripting framework 11/9 - 11/15: Scripting framework / Finish Deliv #4 / Injecting faults into code 11/16 - 11/22: Inject 5 faults into code & working on Final Presentation 11/23 - 11/29 - Finish Deliverable #5 / Thanksgiving! 12/1, 3 - Final Presentations

Test Recording Procedures

Test cases will be recorded on a table with columns, test ID, method name, inputs, expected outcomes, actual outputs, and results(success or fail). Once the tests are completed, they will be outputted html text file and reviewed and checked by all team members.

Hardware and Software Requirements Linux Python 2.7 Levenshtein (optional) Python IDE

Constraints Scheduling is a major hindrance affecting the testing process. Since the team members have different schedules that usually do not align, unfortunately only half of the team members were able to meet and work on the project in person. The rest of the collaboration had to be done virtually.

Test Cases Test ID: 1a
Requirements being tested: The method should format file size in a human readable way. Component being tested: beets/ui/init Method being tested: human_bytes( size ) Test inputs containing command-line arguments: human_bytes( 1023 ) Expected Outcome: 1023.0 B

Test ID: 1b
Requirements being tested: The method should format file size in a human readable way. Component being tested: beets/ui/init Method being tested: human_bytes( size ) Test inputs containing command-line arguments: human_bytes( -1 ) Expected Outcome: -1.0 B

Test ID: 1c
Requirements being tested: The method should format file size in a human readable way. Component being tested: beets/ui/init Method being tested: human_bytes( size ) Test inputs containing command-line arguments: human_bytes( 0 ) Expected Outcome: 0.0 B

Test ID: 1d Requirements being tested: The method should format file size in a human readable way. Component being tested: beets/ui/init Method being tested: human_bytes( size ) Test inputs containing command-line arguments: human_bytes( 2000000000 ) Expected Outcome: 1.9 GB

Test ID: 1e
Requirements being tested: The method should format file size in a human readable way. Component being tested: beets/ui/ init Method being tested: human_bytes( size ) Test inputs containing command-line arguments: human_bytes( None ) Expected Outcome: TypeError

Experiences

Beets was extremely well documented and had great instructions for cloning and building the source code into Ubuntu. All that was necessary was cloning the github repository and installing pip. All team members were able to build beets correctly. Beets became frustrating once we started looking deeply at the code because a lot of the methods modify and use databases. After analyzing each method and deciding we should make sure there were several testable ones, we finally found five methods, three fairly easy and two more complex. The challenging part of this deliverable was to decide that we were definitely changing projects. Initially, we were unsure of where to begin this deliverable; however, in researching and looking through the website provided on the syllabus, it became clear how to format correctly a test plan. This was a definitely a learning experience. It has taught us that, when pressed for time to meet a deadline, making good, fast decisions and changing from OTP to beets was necessary in order to complete the deliverable and move on in our team project.

References Resources used to complete the test plan: http://iansommerville.com/software-engineering-book/web/test-planning/

... Test ID: 2a Requirements being tested: The method should compute the basic edit distance between two strings. Component being tested: beets/autotag/hooks.py ? Method being tested: _string_dist_basic(str1, str2) Test inputs containing command-line arguments: _string_dist_basic(format, doormen) Expected Outcome: 4.0 / (7.0)