Add support to check tests on Travis CI

[efernandez]

First temptative for running tests on Travis CI for ROS Control.

I'll ping someone for review once it's working.

[efernandez]

A working configuration is now available, but it doesn't work well because the log is huge: https://s3.amazonaws.com/archive.travis-ci.org/jobs/124081962/log.txt

We need to silence most of the commands, so they only report errors.

@bmagyar I'd like to have another pair of eyes looking into this before I update the previous issues and PRs. From the log everything looks fine, but here in github I still see 'some checks haven't completed yet'.

BTW I don't think this is blocking for ROS Kinetic, although it's a generally important thing to have. I also don't know if Travis already support an Ubuntu distribution supporting ROS Kinetic, like Wily or Xenial. Do you know how to check that (other than just trying)?

I've just updated the PR to use --no-status with catkin, which hopefully it's enough to reduce the output on the logs. I still have to look with apt-get install -qq still spam the logs, and how to silence wstool. If you know how, please comment on this PR.

[bmagyar]

It is true that this is not blocking the Kinetic releases, I'm running the prerelease tests for 16.06 anyways which runs tests too. Let's put this one on hold and concentrate on the releases for now.

[efernandez]

Now it's actually worked! :)

It's saying failure because of the jerk limits test in diff_drive_controller, which I know it's still failing sometimes, and I should disable for now :P

Now I'm going to start working... do you mind raising this to the others on the previous issues or PRs?

[bmagyar]

How's this one? I've been getting fails all the time (even local pre-releases) for the jerk test of the diff_drive_controller. It seemed that using 0.1 as the jerk tolerance would make it work. How about that @efernandez ?

[efernandez]

@bmagyar Yeah, let's allow for a little more jerk error for now.

[efernandez]

All tests pass! 😃

@bmagyar Do you want to be on the email notification list?
@adolfo-rt I think your PAL Robotics email is no longer valid, right? Which one should I use instead? (you guys can reach me on hangout if you don't want to have your email typed in plain text here)

Any other one I should add to the mailing list?

[bmagyar]

Yeah please add me to the list.

@ros-controls/ros_control , now is the time to subscribe for ros-control-related spam from Travis!

[bmagyar]

Just checked this one too and it seems the Travis build is against Indigo.

[efernandez]

@bmagyar it should be jade, isn't it?

Then I'll try with kinetic.

[efernandez]

@bmagyar OK, I see it now. I think the problem is here:

https://raw.githubusercontent.com/ros-controls/ros_control/jade-devel/ros_control.rosinstall

which still points to indigo-devel

Same issue with kinetic.

[efernandez]

We should re-run the Travis job once ros-controls/ros_control#239 gets merged.

@bmagyar Could you have a look at it and merge it once its own Travis job succeeds?

[bmagyar]

I pulled triggers both on ros-controls/ros_control#239 and ros-controls/ros_control#240

[efernandez]

Thanks!

I've just triggered the job again in Travis. Let's see if the raw log now shows jade-devel instead of indigo-devel (cross-fingers)

[efernandez]

These guys are failing:

Exception caught during install: Error processing 'control_msgs' : [control_msgs] Checkout of https://github.com/ros-controls/control_msgs.git version jade-devel into /home/travis/ros/ws_ros_controllers/src/control_msgs failed.
Error processing 'control_toolbox' : [control_toolbox] Checkout of https://github.com/ros-controls/control_toolbox.git version jade-devel into /home/travis/ros/ws_ros_controllers/src/control_toolbox failed.
Error processing 'realtime_tools' : [realtime_tools] Checkout of https://github.com/ros-controls/realtime_tools.git version jade-devel into /home/travis/ros/ws_ros_controllers/src/realtime_tools failed.

ERROR in config: Error processing 'control_msgs' : [control_msgs] Checkout of https://github.com/ros-controls/control_msgs.git version jade-devel into /home/travis/ros/ws_ros_controllers/src/control_msgs failed.
Error processing 'control_toolbox' : [control_toolbox] Checkout of https://github.com/ros-controls/control_toolbox.git version jade-devel into /home/travis/ros/ws_ros_controllers/src/control_toolbox failed.
Error processing 'realtime_tools' : [realtime_tools] Checkout of https://github.com/ros-controls/realtime_tools.git version jade-devel into /home/travis/ros/ws_ros_controllers/src/realtime_tools failed.

Do we have a jade-devel for them? It surprise me that we don't.

[efernandez]

I've just checked a couple of them, and indeed we don't have jade-devel for control_msgs and realtime_tools, and probably the others. Is it fine to just create a jade-devel for them out of indigo-devel? @bmagyar

If not, the rosinstall for jade-devel should mix jade- and indigo-devel branches! :(

[bmagyar]

Oh, shoot! For a long time Indigo was tracking Jade in ros_control, but eventually Jade got ahead. I'm fine with mixing them, if you can make a comment in the rosinstall on those saying that there's no jade-devel that'd be great. I think it's better to have it that way than spawning branches that are no different from the former one, also we are not likely to get new code in for Jade nor Indigo.

[efernandez]

It seems that I have to run it myself though, to have all green ;)

[efernandez]

Tests will run again as I've just updated the PR with @adolfo-rt and @bmagyar 's emails.

[efernandez]

This is what failed:

[diff_drive_controller.rosunit-diff_drive_wheel_separation_param_test/testTurn][FAILURE]

/home/travis/ros/ws_ros_controllers/src/ros_controllers/diff_drive_controller/test/diff_drive_test.cpp:116

The difference between fabs(yaw_new - yaw_old) and 3.14159265358979323846 is 0.24504422698002415, which exceeds ORIENTATION_TOLERANCE, where

fabs(yaw_new - yaw_old) evaluates to 2.896548426609769,

3.14159265358979323846 evaluates to 3.1415926535897931, and

ORIENTATION_TOLERANCE evaluates to 0.029999999999999999.

Running the test again to see if it's an intermittent or a consistent failure.

[efernandez]

It's intermittent. I'd say it's another test that depends on timing issues on the testing machine. We could increase the tolerance, but since it's the first time I've seen it failing, I'd simply ignore it for now.

@bmagyar If you have the same opinion, please hit the green button.

Then I'll sort of cherry pick this PR into kinetic, and the other repos of both jade and kinetic.

[bmagyar]

Besides, a 14 degree error in orientation shouldn't be tolerated ;)

Merging...