delay on fixture

[lornat75]

iCubSim takes some time to stabilize framerate: first execution of camera test fails because of this. Maybe we should add the possibilty to add a delay after the fixture:

<environment>--robotname icubSim</environment>
<fixture param="--fixture icubsim-fixture.xml"> yarpmanager </fixture>
<--! add delay about here -->
<!-- Camera -->
<test type="dll" param="--from camera_right.ini"> CameraTest </test>

[apaikan]

@lornat75 and @randaz81

• Probably the delay mechanism should be implemented in the fixture manager (yarpmanager application xml file). right?
• A RPC control command like the one we have for the robotInterface can be an alternative option?

Which option you prefer?

1) having state port?

    <module>
        <name>iCub_SIM</name>
        <node>localhost</node>
        <ensure command="state" reply="ready">/icubSim/rpc</ensure>
    </module>    

2) having time delay?

    <module>
        <name>iCub_SIM</name>
        <node>localhost</node>
        <wait>5.0</wait>
    </module>    

3) having both mechanisms?

[apaikan]

I implemented a simple waiting mechanism (in yarpmanager) to "ensure" that enough time is given to a module to get ready after launching it (please update to robotology/yarp@becdbbe).

Using the following example, the fixture manager waits for 10s after launching the iCub_SIM and before running the test cases.

    <module>
        <name>iCub_SIM</name>
        <parameters></parameters>
        <node>localhost</node>
        <ensure>
            <wait>10</wait>
        </ensure>
    </module>    

Notice: The camera tests still fail (some times) on my machine even waiting for 30s before launching the test cases!

@traversaro How is the behavior in Gazebo?

[lornat75]

👍

We should check why it takes so much to get to the correct framerate. Have you tried just to run the simulator alone and connect yarp ping --rate to one of the cameras?

[apaikan]

I have tested the icubSim cameras rate using our port frequency test and yarpview. On my machine, both the cameras have 45 hrz in average.

<?xml version="1.0" encoding="UTF-8"?>
<suit name="Basic Tests Suite">
    <description>Testing robot's basic features</description>
    <environment>--robotname icubSim</environment>
    <fixture param="--fixture icubsim-fixture.xml"> yarpmanager </fixture>

    <!-- Port frequency (camera) -->
    <test type="dll" param="--from cam_stream.ini"> PortsFrequency </test>

    <!-- Camera -->
    <test type="dll" param="--from camera_right.ini"> CameraTest </test>
    <test type="dll" param="--from camera_left.ini"> CameraTest </test> 
</suit>

Staring test runner.
Test suit Basic Tests Suite started...
[INFO]  (Basic Tests Suite) reports: yarpmanager is setuping the fixture...
Test case PortsFrequency started...
||| did not find cam_stream.ini
||| did not find cam_stream.ini
[INFO]  (Camera Frequency) reports: 
[INFO]  (Camera Frequency) reports: Checking port /icubSim/cam/left ...
[INFO]  (Camera Frequency) reports: Time delay between sender/receiver is 0.0079 s. (min: 0.0246, max: 04)
[INFO]  (Camera Frequency) reports: Sender frequency 45 hrz. (min: 29, max: 60)
[INFO]  (Camera Frequency) reports: Receiver frequency 45 hrz. (min: 32, max: 64)
[FAIL]  (Camera Frequency) checking (diff < ports[i].tolerance): Receiver frequency is outside the desired range [25 .. 35]
[INFO]  (Camera Frequency) reports: 
[INFO]  (Camera Frequency) reports: Checking port /icubSim/cam/right ...
[INFO]  (Camera Frequency) reports: Time delay between sender/receiver is 0.0081 s. (min: 0.0354, max: 04)
[INFO]  (Camera Frequency) reports: Sender frequency 44 hrz. (min: 19, max: 60)
[INFO]  (Camera Frequency) reports: Receiver frequency 45 hrz. (min: 25, max: 62)
[FAIL]  (Camera Frequency) checking (diff < ports[i].tolerance): Receiver frequency is outside the desired range [25 .. 35]
Test case Camera Frequency failed!
Test case CameraTest started...
||| did not find camera_right.ini
||| did not find camera_right.ini
[INFO]  (Test Right Camera) reports: Listening to camera for 5 seconds
[INFO]  (Test Right Camera) reports: connecting from /CameraTest/image:i to /icubSim/cam/right
[INFO]  (Test Right Camera) reports: Reading images...
[INFO]  (Test Right Camera) reports: Received 223 frames, expecting 300
[FAIL]  (Test Right Camera) checking (abs(frames-expectedFrames)<tolerance): checking number of received frames
Test case Test Right Camera failed!
Test case CameraTest started...
||| did not find camera_left.ini
||| did not find camera_left.ini
[INFO]  (Test Left Camera) reports: Listening to camera for 5 seconds
[INFO]  (Test Left Camera) reports: connecting from /CameraTest/image:i to /icubSim/cam/left
[INFO]  (Test Left Camera) reports: Reading images...
[INFO]  (Test Left Camera) reports: Received 224 frames, expecting 300
[FAIL]  (Test Left Camera) checking (abs(frames-expectedFrames)<tolerance): checking number of received frames
Test case Test Left Camera failed!
[INFO]  (Basic Tests Suite) reports: yarpmanager is tearing down the fixture...
Test suit Basic Tests Suite failed!
Ending test runner.

---------- results -----------
Total number of test suites  : 1
Number of passed test suites : 0
Number of failed test suites : 1
Total number of test cases   : 3
Number of passed test cases  : 0
Number of failed test cases  : 3

[traversaro]

In theory this kind of wait should not be necessary on Gazebo, however I obtained strange results when running the Gazebo tests on Travis (see robotology/gazebo-yarp-plugins#194 ). I could add a waiting time to see if the problem persist.