Error on start nstat

[lucar91]

I would like to try nstat, but can I start it with a controller that is not on docker?
If I try to start nstat by setting up a remote controller, i have this error:

2018-05-14 09:47:21,083 INFO ] [open_ssh_connection] Initiating SSH session with ODL node.
[2018-05-14 09:47:21,083 INFO ] [utils.netutils.ssh_connect_or_return] Trying to connect to 155.185.49.81:6633 (1/10)
[2018-05-14 09:49:28,322 ERROR ] Fail to establish ssh connection with controller node. :::::::::: Exception :::::::::::
[2018-05-14 09:49:28,323 ERROR ] Fail to establish ssh connection with controller node.
[2018-05-14 09:49:28,324 ERROR ] Error number:1
[2018-05-14 09:49:28,324 ERROR ] Fail to establish ssh connection with controller node. - ODL Exception: <class 'stress_test.controller_exceptions.CtrlNodeConnectionError'>, 132
Traceback (most recent call last):
File "/opt/nstat/stress_test/controller.py", line 125, in init_ssh
self.ssh_pass, 10)
File "/opt/nstat/util/netutil.py", line 246, in ssh_connect_or_return
password=password)
File "/usr/lib/python3.4/site-packages/paramiko/client.py", line 251, in connect
retry_on_signal(lambda: sock.connect(addr))
File "/usr/lib/python3.4/site-packages/paramiko/util.py", line 270, in retry_on_signal
return function()
File "/usr/lib/python3.4/site-packages/paramiko/client.py", line 251, in
retry_on_signal(lambda: sock.connect(addr))
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/nstat/stress_test/controller.py", line 132, in init_ssh
raise(stress_test.controller_exceptions.
stress_test.controller_exceptions.CtrlNodeConnectionError: Fail to establish ssh connection with controller node.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/nstat/stress_test/nstat.py", line 104, in
main()
File "/opt/nstat/stress_test/nstat.py", line 101, in main
nstat_test.test_selector(args)
File "/opt/nstat/stress_test/test_type.py", line 99, in test_selector
nstat_test_type_run)
File "/opt/nstat/stress_test/test_run.py", line 44, in init
json_conf)
File "/opt/nstat/stress_test/controller.py", line 77, in new
return ODL(ctrl_base_dir, test_config)
File "/opt/nstat/stress_test/controller.py", line 554, in init
self.init_ssh()
File "/opt/nstat/stress_test/controller.py", line 135, in init_ssh
self._error_handling(e.err_msg, e.err_code)
File "/opt/nstat/stress_test/controller.py", line 105, in _error_handling
raise(stress_test.controller_exceptions.CtrlError)
stress_test.controller_exceptions.CtrlError: Controller generic exception
[2018-05-14 09:49:28,362 INFO ] Run controller stop.
[2018-05-14 09:51:35,682 ERROR ] [ssh_connection_open] error: check connection object
[2018-05-14 09:51:35,682 ERROR ] Controller stop failure. :::::::::: Exception :::::::::::
[2018-05-14 09:51:35,683 ERROR ] Controller stop failure.
[2018-05-14 09:51:35,683 ERROR ] Error number:1
[2018-05-14 09:51:35,683 ERROR ] Controller stop failure. - ODL Exception: <class 'stress_test.controller_exceptions.CtrlStopError'>, 353
[2018-05-14 09:51:35,683 INFO ] Fail stopping Controller during cleanup. Exception message: Controller generic exception
[2018-05-14 09:51:35,683 INFO ] Run controller cleanup.
[2018-05-14 09:51:35,683 INFO ] [Controller] Cleaning up

[konstantinos-papadopoulos]

Hello Luca,
can you please attach the test configuration *.json file you use as an input to nstat?

[lucar91]

Here is the json file, I copied the contents to a txt file.
The controller is pox. The address is correct, mininet connects.

boron_RPC_sb_active_scalability_mtcbench.txt

[lucar91]

Hi, I tried to change the name of the controller in the json file. From the error I found:

  File "/opt/nstat/stress_test/controller.py", line 81, in new
     raise NotImplementedError ('Not supported yet')

I arrived therefore to this:

  if (name == 'ODL'):
             return ODL (ctrl_base_dir, test_config)
         elif name == 'ONOS':
             raise NotImplementedError ('ONOS is not supported yet')
         else:
             raise NotImplementedError ('Not supported yet')

in the file stress_test / in controller py.
But then nstat only supports the ODL controller (I suppose it is opendaylight) ??

That's why it can not connect to the pox controller?
This is the mistake I have when I try to connect with pox:

[2018-05-16 08: 42: 33,866 INFO] Authentication (password) failed.
[2018-05-16 08: 42: 33,866 ERROR] [utils.netutils.ssh_connect_or_return] authentication failed when connecting to 155.185.49.81

[konstantinos-papadopoulos]

• NSTAT supports only OpenDaylight at the moment. However it is possible to use the existing nstat architecture to add testing of other controllers. I would start first with

1. how a new controller works ie POX, what are its interfaces (southbound, northbound),

2. manual deployment testing/connecting to a multinet/cbench topology and finally

3. adding all necessary adaptations to NSTAT (if you plan to do so we can support you from the NSTAT code point of view)

so better leave in you *.json configuration file "controller_name":"ODL",

• One more diff I also see is
  "controller_node_ip":"155.185.49.81", which means that you try to deploy odl on a node with ip which is on a different subnet with the rest of the NSTAT nodes. This will not work. Better

1. leave it as it is "controller_node_ip":"10.0.1.11" and better run your test using the three node architecture (container based test setup) 1) one nstat node 2) controller node 3) sb-emulator-node with ips
   "nstat_node_ip":"10.0.1.10",
   "controller_node_ip":"10.0.1.11",
   "sb_emulator_node_ip":"10.0.1.12",
   respectively.

Make sure to prepare your containers first. Check below for futher details
https://github.com/intracom-telecom-sdn/nstat/wiki/Installation
Be careful what docker files to chose if you sit behind a proxy or not.

[lucar91]

hello, I had put the different ip (155.185.49.81) to connect to pox started not on docker. I had already installed nstat on docker following the instructions, so I should be ready to start the tests. The problem is that even if you do not change the addresses of the .json file, the test does not start.
Before the operation:
[2018-05-17 12:23:57,910 DEBUG ] [controller.build] Cloning into '/opt/nstat/controllers/odl_boron_pb/nstat-sdn-controllers'...

i have this error:
File "/opt/nstat/stress_test/controller.py", line 426, in build
format(cmd_output)), exit_status)
TypeError: exceptions must derive from BaseException

I attach the file with the error

nstaterror.txt

[konstantinos-papadopoulos]

From the error you sent it is now the controller build handler that complains

destination path '/opt/nstat/controllers/odl_boron_pb/nstat-sdn-controllers' already exists and is not an empty directory.
Try removing the /opt/nstat/controllers/odl_boron_pb/nstat-sdn-controllers from the node you try to run the controller at, and shoot again.
Before doing this you can standalone check the controller handlers (build/clean) located under the controllers dir. The idea of this handlers is that

• build handler brings the necessary controller to the controller node and builds it (prepares the binary)
• clean handler, cleans the controller node from any controller relevant sw, binaries etc
  Br

[lucar91]

Thanks, I deleted the folder and it worked.
There remains a problem, starting the test, for example:

docker exec -i nstat /bin/bash -c "export PYTHONPATH=/opt/nstat; source /opt/venv_nstat/bin/activate;
python3.4 /opt/nstat/stress_test/nstat.py
--test=sb_idle_scalability
--ctrl-base-dir=/opt/nstat/controllers/odl_boron_pb/
--sb-emulator-base-dir=/opt/nstat/emulators/mt_cbench/
--json-config=/opt/nstat/stress_test/sample_test_confs/boron/boron_RPC_sb_idle_scalability_mtcbench.json
--json-output=/opt/nstat/results.json
--html-report=/opt/nstat/report.html
--output-dir=/opt/nstat/results_boron_RPC_sb_idle_scalability_mtcbench/"

i have an error " /opt/nstat/emulators/mt_cbench/ directory not exists". I solved by changing the folder from
"/opt/nstat/emulators/mt_cbench/ \ " to "/opt/nstat/emulators/sbemu/mtcbench/" (I hope it is correct).

I have another question:
the build.sh makes a git clone, but as it does not start. To start it correctly i have to edit the file and put sudo before git clone. Is there another way?
If i start the test putting sudo before docker does not work anyway...

[konstantinos-papadopoulos]

• Yes your correction is correct. I guess we have a typo in the wiki. Can you please send me the wiki link so that we correct it?
• build.sh should bring all necessary controller handlers under the same folder. Therefore after running the build handler your dir should be as below
  .
  ├── build.sh
  ├── change_persistence.py
  ├── change_stats_period.py
  ├── clean.sh
  ├── flowmods_configure.py
  ├── get_controller.sh
  ├── get_flows.py
  ├── get_hosts.py
  ├── get_links.py
  ├── get_switches.py
  ├── init.py
  ├── start_droptestDS.sh
  ├── start_droptestRPC.sh
  ├── start_no_dlux.sh
  ├── start.sh
  ├── status.sh
  ├── stop.sh
  └── xml_utils.py

[lucar91]

• sure there is no problem, this is the link, https://github.com/intracom-telecom-sdn/nstat/wiki/Controller-stability-test-with-active-MT-Cbench-switches
  but the problem is also in the other links of the "stress test" section of the same page.

• Sorry maybe I misplaced the question. When starting build.sh it creates all the dir files as you posted, but the problem is that build.sh only starts if I put sudo before "git clone ..." in the build.sh file. Without the permission sudo, does not start, so do not download all the contents of the dir... Maybe now I wait a bit 'and see if I give a mistake ..

[konstantinos-papadopoulos]

-It should not behave like this. If you run this within a container, the path is /opt/nstat/emulators/... or /opt/nstat/controllers/.. for which we have prepared /opt with proper rights so that sudo is not needed.
What I mean is to

• ssh to your container where controller/emulator runs
• run the build.sh scripts (i.e /opt/nstat/controllers) it should clone without any issues.

-If your controller/emulator node is your localhost then we have most probably to open a bug since in the build.sh script we do not check for directory rights.
If you plan
Did you progress?

[lucar91]

I tried to access the content odl_boron_pb via ssh, the command ./build.sh not start and after a few seconds this error appears:

fatal: unable to access 'https://github.com/intracom-telecom-sdn/nstat-sdn-controllers.git/': Failed to connect to 172.28.40.9 port 3128: Connection timed out
[build.sh] Cloning nstat-sdn-controllers failed. Exiting 1 ...

  If I start with sudo ./build.sh privileges, it starts ...

The error could be due to the command "sudo usermod -a -G docker $ (whoami)"?
maybe when I started it something did not work and now I do not have permissions on docker?

[lucar91]

I modified the build.sh file in

/opt/nstat/controllers/odl_boron_pb
and in
/opt/nstat/emulators/mt_cbench/

adding sudo before git clone ..., and the test start.

A curiosity, how much RAM does the test need? Started the basic test and my 4GB of ram were immediately filled ...
However, thank you for the help. I think that given the low amount of ram and the problem of compatibility with other controllers (I do not have much time to work) i will only do some tests with ODL. Thank you very much and I hope I did not waste time unnecessarily.

[konstantinos-papadopoulos]

Hi Luca,
the sample test needs minimum 8Gb of RAM to run and get a feeling of how the test runs so that to get a few sample results and the final html report.
As I said in my previous comment sudo is not needed as long as you have set proper rights to the /opt folder.
Other controllers are not supported at the moment.
Feel free to ask whatever regarding nstat
Thanks for trying using it.