-
Notifications
You must be signed in to change notification settings - Fork 46
ONOS TESTPLAN
Test-ON for ONOS: Paxterra’s Test-ON is made out of a set of drivers that connect to various components. In our case, these Components will be ZK, Cassandra, ONOS, and mininet and can be pointed to either the same or different IP addresses. Test-ON manipulates these components through a set of ssh handles it creates using python’s pexpect. The test structure is comprised of a set of different tests, each with a number of test-cases inside. These test-cases each return a pass/fail result and can stand alone in theory. However more complete tests can be created by running multiple test-cases in series, which will return a percentage pass/fail result of all the containing test-cases. There will be certain test-case combinations that make more sense then others, and they will be placed in a sensible order, but in the end it is up to the user to determine logical sequences that produce meaningful data.
File Structure: There are three main files that define a specific test in Test-ON:
-
Topology file ( .topo ) This file defines all the components and options that Test-ON will use to execute its test. For example the topo file would include the login info and mininet parameters for the machine that was going to host the mininet for the test. At the beginning of the test, Test-ON will automatically connect all of these components.
-
Parameter file ( .params ) This file stores all of the variables that the test will call upon to run the test such as which test-cases to run. This lets you tweak wait times and other values to tune the test to your needs all in one file.
-
Python file ( .py ) This is where all of the test-cases are written. It calls upon functions implemented in the respective drivers to create a progression of events that produces some sort of pass/fail outcome.
###TEST 1: OnosScale
This test tests the time it takes for an ONOS node to converge. It uses rest calls to ONOS to compare the number of switches and links ONOS sees to the actual number. Once these two numbers are the same, it assumes that ONOS is converged.
Pre-requisites:
As long as the components are pointed to the correct addresses, this test will start and verify everything by itself.
Hardware requirements:
One VM that is capable of running ONOS and mininet.
Execution:
Testcase 1.1. Verify Startup.
1.1.1 Check if Zookeeper has ‘Mode’ assigned to it
1.1.2 Check if Cassandra status returns ‘Up’ and ‘Normal
1.1.3 Check if ONOS process is running and log is moving with no error
1.1.4 if step 1.3 returns false, restarts ONOS once
1.1.5 Check if ONOS process is running and log is moving with no errors
Pass: Step 1.1.5 returns that ONOS is running
Fail: Step 1.1.5 returns that ONOS is not running or has errors
Testcase 2. Measure Convergence Time.
2.1 Point all mininet switches to no controller
2.2 Shut down ONOS, and drop keyspace
2.3 Start ONOS, and point all switches at it
2.4 Record time until ONOS rest call returns the correct number of switches and links
Pass: If ONOS converges, and the convergence time returned by 2.4 is less then the supplied target time.
Fail: If ONOS does not converge, or the convergence time returned by 2.4 is greater then target time.
###Test2: OnosPerf
This test is designed to retrieve ONOS black box performance numbers. It will mainly focus on flow re-route times in a multi-node ONOS cluster.
Pre-requisites:
As it stands, this test assumes that ONOS nodes are already running in the cluster and the topology is already created.
Hardware requirements:
Four to Eight interconnected VMs that are capable of running ONOS and mininet. This test defaults to having 6 physical switches in the core of the topology. For all software switches, the DPIDs must be changed in the parameter file.
Execution:
Testcase 1. Verify Startup.
1.1 Check if Zookeeper has ‘Mode’ assigned to it
1.2 Check if Cassandra status returns ‘Up’ and ‘Normal’
1.3 Check if ONOS process is running and log is moving with no errors
1.4 if step 1.3 returns false, restarts ONOS once
1.5 Check if ONOS process is running and log is moving with no errors
Pass: Step 1.5 returns that ONOS is running
Fail: Step 1.5 returns that ONOS is not running or has errors
Testcase 2. Verify Topology.
2.1 Bring up the links defined in the topology file.
2.2 Check that MN switches are running
Pass: All MN switches are running
Fail: All MN switches are not running
Testcase 3. Add Flows.
3.1 Clear out all residual flows in ONOS
3.2 Add flows as defined in topology file
3.3 Check up to 10 times if all the flows have been both calculated, and flow tables updated.
Pass: Previous flow check passes
Fail: Not all flow tables are calculated or flows not pushed
Testcase 4. Ping Test.
4.1 Ping along one flow
Pass: There are no packets lost
**Fail: ** There are packets lost
Testcase 5. Start Continuous Pings.
5.1 Start continuous pings on mininet hosts
5.2 Check if there are ping processes running on hosts
**Pass: ** There are ping processes running
Fail: There are no ping processes running
Testcase 6. Loop Link down/up.
6.1 Bring link down and wait a pre-defined amount of time
6.2 Bring link up and wait a pre-defined amount of time
6.3 Repeat steps 6.1 and 6.2 for a defined number of iterations
6.4 Stop continuous pings on the mininet hosts
Pass: Ping processes are stopped
Fail: Ping processes are not stopped
Testcase 7. Process Ping Data. 7.1 Copy ping logs to machine running the test 7.2 Calculate re-reroute times for each ping from timestamps 7.3 Calculate overall average re-route time from all the pings Pass: All the pings re-routed and the overall average re-route time is less then target time Fail: Either some pings did not re-route, or the overall average re-route time is greater then the target time.
This test is designed to quickly test the basic functionality of ONOS in a 4 node cluster
Pre-requisites: FLOWDEF file placed on TestON VM (This file will be copied over to the /tmp directory on the machine the ONOS instance which you add flows on is running)
Hardware requirements: 4 ONOS VMs properly configured to run as a cluster and a single VM capable of running mininet (Mininet machine may also be an ONOS machine).
Execution:
Testcase 1. Verify Startup.
1.1 Check if Zookeeper has ‘Mode’ assigned to it
1.2 Check if Cassandra status returns ‘Up’ and ‘Normal’
1.3 Check if ONOS process is running and log is moving with no errors
1.4 if step 1.3 returns false, restarts ONOS once
1.5 Check if ONOS process is running and log is moving with no errors
Pass: Step 1.5 returns that ONOS is running
Fail: Step 1.5 returns that ONOS is not running or has errors
Testcase 2. Ping Test. 2.1 Ping a single MN host to be certain MN was created 2.2 Add ONOS controllers to MN switches. A single controller is added first to be the master, then rest are assigned. Pass: MN host successfully pinged Fail: MN host unreachable
Testcase 3. Add Flows. 3.1 Clear out all residual flows in ONOS 3.2 Add flows as defined in topology file 3.3 Check up to 6 times if all the flows have been both calculated, and flow tables updated. Pass: Previous flow check passes; return time taken for flows to add and pass flow check Fail: Not all flow tables are calculated or flows not pushed
Testcase 4. Node Failure. 4.1 Disconnect ONOS 2, 3, and 4 from all switches 4.2 Ping hosts on opposite sides of the topology 4.3 If step 4.2 fails, wait 10 seconds, then restart step 4.2 Pass: All 10 pings successfully complete sequentially; return time taken to successfully ping Fail: A single ping fails 6 times
Testcase 5. Add Nodes. 5.1 Reconnect ONOS 2, 3, and 4 to all switches 5.2 Ping hosts on opposite sides of the topology 5.3 If step 5.2 fails, wait 10 seconds, then restart step 5.2 Pass: All 10 pings successfully complete sequentially; return time taken to successfully ping Fail: A single ping fails 6 times
Testcase 6. Link Down. 6.1 Bring link down 6.2 Ping hosts on opposite sides of the topology 6.3 If step 6.2 fails, wait 10 seconds, then restart step 6.2 Pass: All 10 pings successfully complete sequentially; return time taken to successfully ping Fail: A single ping fails 6 times
Testcase 7. Link Up. 7.1 Bring link up 7.2 Ping hosts on opposite sides of the topology 7.3 If step 7.2 fails, wait 10 seconds, then restart step 7.2 Pass: All 10 pings successfully complete sequentially; return time taken to successfully ping Fail: A single ping fails 6 times
This test will use the same modules as the OnosSanity4nodes test, but will loop through cases 4, 5, 6, and 7 as many times as they are listed in the Params file
Pre-requisites: FLOWDEF file placed on TestON VM (This file will be copied over to the /tmp directory on the machine the ONOS instance which you add flows on is running)
Hardware requirements: 4 ONOS VMs properly configured to run as a cluster and a single VM capable of running mininet (Mininet machine may also be an ONOS machine).
Execution:
Testcase 1. Verify Startup.
1.1 Check if Zookeeper has ‘Mode’ assigned to it
1.2 Check if Cassandra status returns ‘Up’ and ‘Normal’
1.3 Check if ONOS process is running and log is moving with no errors
1.4 if step 1.3 returns false, restarts ONOS once
1.5 Check if ONOS process is running and log is moving with no errors
Pass: Step 1.5 returns that ONOS is running
Fail: Step 1.5 returns that ONOS is not running or has errors
Testcase 2. Ping Test. 2.1 Ping a single MN host to be certain MN was created 2.2 Add ONOS controllers to MN switches. A single controller is added first to be the master, then rest are assigned. Pass: MN host successfully pinged Fail: MN host unreachable
Testcase 3. Add Flows. 3.1 Clear out all residual flows in ONOS 3.2 Add flows as defined in topology file 3.3 Check up to 6 times if all the flows have been both calculated, and flow tables updated. Pass: Previous flow check passes; return time taken for flows to add and pass flow check Fail: Not all flow tables are calculated or flows not pushed
Begin looping the following test cases by listing them multiple times in the Params file under test cases (i.e 1,2,3,4,5,6,7,4,5,6,7,4,5,6,7 … 4,5,6,7 The reason why it was chosen to loop in this manner is so each case can still return its own Pass/Fail result.
Testcase 4. Kill ONOS nodes. 4.1 Disconnect ONOS 2, 3, and 4 from all switches 4.2 Ping hosts on opposite sides of the topology 4.3 If step 4.2 fails, wait 10 seconds, then restart step 4.2 Pass: All 10 pings successfully complete sequentially; return time taken to successfully ping Fail: A single ping fails 6 times
Testcase 5. Add ONOS nodes. 5.1 Reconnect ONOS 2, 3, and 4 to all switches 5.2 Ping hosts on opposite sides of the topology 5.3 If step 5.2 fails, wait 10 seconds, then restart step 5.2 Pass: All 10 pings successfully complete sequentially; return time taken to successfully ping Fail: A single ping fails 6 times
Testcase 6. Link Down. 6.1 Bring link down 6.2 Ping hosts on opposite sides of the topology 6.3 If step 6.2 fails, wait 10 seconds, then restart step 6.2 Pass: All 10 pings successfully complete sequentially; return time taken to successfully ping Fail: A single ping fails 6 times
Testcase 7. Link Up. 7.1 Bring link up 7.2 Ping hosts on opposite sides of the topology 7.3 If step 7.2 fails, wait 10 seconds, then restart step 7.2 Pass: All 10 pings successfully complete sequentially; return time taken to successfully ping Fail: A single ping fails 6 times