Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fisheye model in ros2 #547

Closed
wants to merge 10 commits into from
Closed

fisheye model in ros2 #547

wants to merge 10 commits into from

Conversation

soeroesg
Copy link
Contributor

implemented the fisheye mono and fisheye stereo calibration based on the code in the melodic branch.
(Please test it because I have no ROS2 to test)

@soeroesg
Copy link
Contributor Author

soeroesg commented Jul 6, 2020

@SteveMacenski can you check why all checks have failed? there is actually no problem inside the logs.
I do not have ROS2 so if you can, please test my implementation there.

@SteveMacenski
Copy link
Member

@soeroesg ignore the ros2.build.com ones, only look at the Circle CI builds. the build farm is failing because of some cv_bridge issues that are killing camera calibration for us.

Run that test locally and see if it passes

@JWhitleyWork
Copy link
Collaborator

@soeroesg In the camera_calibration folder, add a file called pytest.ini and put the following text in it:

junit_family=xunit1

This should avoid the error that was generated. Don't worry about the build farm failures at this time, just pay attention to the CircleCI test.

Also, you can test the build in docker. Here are some basic instructions:

  1. Install Docker. See https://docs.docker.com/engine/install/ubuntu/ and also run the Post-Installation Steps for Linux and restart.
  2. Create a ROS2 workspace (mkdir -p ~/ros2_ws/src) and clone this branch into the src folder.
  3. cd into your workspace.
  4. docker run -it -v $(pwd):/root/workspace ros:foxy-ros-base
  5. cd /root/workspace
  6. apt-get update && rosdep update
  7. rosdep install --from-paths src --ignore-src -y
  8. colcon build
  9. colcon test
  10. colcon test-result --verbose

@SteveMacenski
Copy link
Member

I think the remaining issues are from the opencv issue. Though Circle should pass even if the PR builders fail in that case. @soeroesg have you pulled in ros2 master to get the https://github.com/ros-perception/image_pipeline/blob/ros2/tools/ros2_dependencies.repos file so we're building from source?

@soeroesg
Copy link
Contributor Author

hm, I only see that the test of camera_calibration fails, but where can I see the actual error message?

@SteveMacenski
Copy link
Member

SteveMacenski commented Jul 15, 2020

https://github.com/ros-perception/image_pipeline/blob/ros2/.circleci/config.yml#L30 it should be printing test-results but we don't store artifacts. Do you not see this failure locally? Have you pulled in ros2 head? We added the source builds of vision_opencv that should let Circle pass from the PR builder issues recently.

@soeroesg
Copy link
Contributor Author

I pulled in the upstream/ros2 but I don't have access to Docker at the moment so i cannot test it locally... but I will find a way

@JWhitleyWork
Copy link
Collaborator

I pulled in the upstream/ros2 but I don't have access to Docker at the moment so i cannot test it locally... but I will find a way

Try rebasing again now that #564 is in. It likely won't fix the issue but it's worth a try.

https://github.com/ros-perception/image_pipeline/blob/ros2/.circleci/config.yml#L30 it should be printing test-results but we don't store artifacts. Do you not see this failure locally? Have you pulled in ros2 head? We added the source builds of vision_opencv that should let Circle pass from the PR builder issues recently.

The problem is not that we don't store artifacts, it's that colcon test (the line before the one you mentioned) fails with a non-0 exit code which ends the test before colcon test-result can be run because CircleCI runs the commands with #!/bin/bash -eo pipefail. I'll see if I can get around it later.

I'll test this change locally and see if I get the same results.

@soeroesg
Copy link
Contributor Author

I pulled in the latest upstream and tried the docker container. Colcon test gives a strange error (ModuleNotFoundError) which I don't know how to handle, so I I attach here the full console output, I hope it helps.
(there are some compiler warnings too but they are not related)

root@31e372d636e3:~/workspace# rosdep install --from-paths src --ignore-src -y
#All required rosdeps installed successfully
root@31e372d636e3:~/workspace# colcon build
Starting >>> cv_bridge
Starting >>> image_transport
Starting >>> image_geometry
Starting >>> camera_calibration_parsers
Finished <<< camera_calibration_parsers [0.20s]                                                                                         
Starting >>> camera_info_manager
Finished <<< cv_bridge [0.24s]
Finished <<< image_geometry [0.23s]
Starting >>> camera_calibration
Starting >>> opencv_tests
Starting >>> vision_opencv
Finished <<< image_transport [0.27s]
Starting >>> image_proc
Starting >>> depth_image_proc
Starting >>> image_rotate
Starting >>> image_view
Finished <<< camera_info_manager [0.49s]                                                                                                                              
Starting >>> image_publisher
Starting >>> image_common
Finished <<< vision_opencv [0.49s]
Finished <<< image_rotate [0.56s]                                                                                                                      
Finished <<< image_common [0.15s]
Finished <<< depth_image_proc [0.58s]
Finished <<< camera_calibration [0.71s]                                                                                                                                
Finished <<< opencv_tests [0.71s]
Finished <<< image_publisher [1.47s]                                                                                              
--- stderr: image_proc                                                                          
CMake Deprecation Warning at /opt/ros/foxy/share/ament_cmake_export_interfaces/cmake/ament_export_interfaces.cmake:37 (message):
  ament_export_interfaces() is deprecated, use ament_export_targets() instead
Call Stack (most recent call first):
  CMakeLists.txt:206 (ament_export_interfaces)


In file included from /root/workspace/src/image_pipeline/image_proc/include/image_proc/crop_decimate.hpp:37,
                 from /root/workspace/src/image_pipeline/image_proc/src/crop_decimate.cpp:33:
/opt/ros/foxy/include/image_transport/image_transport.h:41:89: note: #pragma message: Warning: This header is deprecated. Use 'image_transport.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'image_transport.hpp' instead")
      |                                                                                         ^
---
Finished <<< image_proc [4.38s]
Starting >>> stereo_image_proc
Starting >>> image_pipeline
Finished <<< image_pipeline [0.13s]                                                                                             
--- stderr: stereo_image_proc                                                                         
CMake Deprecation Warning at /opt/ros/foxy/share/ament_cmake_target_dependencies/cmake/ament_target_dependencies.cmake:92 (message):
  Package image_proc is exporting the variable image_proc_INTERFACES which is
  deprecated, it should export

              image_proc_TARGETS instead
Call Stack (most recent call first):
  CMakeLists.txt:39 (ament_target_dependencies)


---
Finished <<< stereo_image_proc [2.26s]
--- stderr: image_view                                  
CMake Deprecation Warning at /opt/ros/foxy/share/ament_cmake_export_interfaces/cmake/ament_export_interfaces.cmake:37 (message):
  ament_export_interfaces() is deprecated, use ament_export_targets() instead
Call Stack (most recent call first):
  CMakeLists.txt:80 (ament_export_interfaces)


In file included from /root/workspace/src/image_pipeline/image_view/include/image_view/video_recorder_node.hpp:19,
                 from /root/workspace/src/image_pipeline/image_view/src/video_recorder_node.cpp:15:
/opt/ros/foxy/include/image_transport/image_transport.h:41:89: note: #pragma message: Warning: This header is deprecated. Use 'image_transport.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'image_transport.hpp' instead")
      |                                                                                         ^
In file included from /root/workspace/src/image_pipeline/image_view/include/image_view/image_saver_node.hpp:55,
                 from /root/workspace/src/image_pipeline/image_view/src/image_saver_node.cpp:49:
/opt/ros/foxy/include/image_transport/image_transport.h:41:89: note: #pragma message: Warning: This header is deprecated. Use 'image_transport.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'image_transport.hpp' instead")
      |                                                                                         ^
In file included from /root/workspace/src/image_pipeline/image_view/src/video_recorder_node.cpp:21:
/opt/ros/foxy/include/camera_calibration_parsers/parse.h:41:79: note: #pragma message: Warning: This header is deprecated. Use 'parse.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'parse.hpp' instead")
      |                                                                               ^
In file included from /root/workspace/src/image_pipeline/image_view/include/image_view/stereo_view_node.hpp:55,
                 from /root/workspace/src/image_pipeline/image_view/src/stereo_view_node.cpp:49:
/opt/ros/foxy/include/image_transport/subscriber_filter.h:41:91: note: #pragma message: Warning: This header is deprecated. Use 'subscriber_filter.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'subscriber_filter.hpp' instead")
      |                                                                                           ^
/root/workspace/src/image_pipeline/image_view/src/video_recorder_node.cpp: In member function ‘void image_view::VideoRecorderNode::callback(const ConstSharedPtr&)’:
/root/workspace/src/image_pipeline/image_view/src/video_recorder_node.cpp:137:23: warning: catching polymorphic type ‘class cv_bridge::Exception’ by value [-Wcatch-value=]
  137 |   } catch (cv_bridge::Exception) {
      |                       ^~~~~~~~~
In file included from /root/workspace/src/image_pipeline/image_view/src/image_saver_node.cpp:55:
/opt/ros/foxy/include/camera_calibration_parsers/parse.h:41:79: note: #pragma message: Warning: This header is deprecated. Use 'parse.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'parse.hpp' instead")
      |                                                                               ^
/root/workspace/src/image_pipeline/image_view/src/image_saver_node.cpp: In member function ‘bool image_view::ImageSaverNode::saveImage(const ConstSharedPtr&, std::string&)’:
/root/workspace/src/image_pipeline/image_view/src/image_saver_node.cpp:112:23: warning: catching polymorphic type ‘class cv_bridge::Exception’ by value [-Wcatch-value=]
  112 |   } catch (cv_bridge::Exception) {
      |                       ^~~~~~~~~
In file included from /root/workspace/src/image_pipeline/image_view/include/image_view/image_view_node.hpp:21,
                 from /root/workspace/src/image_pipeline/image_view/src/image_view_node.cpp:49:
/opt/ros/foxy/include/image_transport/image_transport.h:41:89: note: #pragma message: Warning: This header is deprecated. Use 'image_transport.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'image_transport.hpp' instead")
      |                                                                                         ^
In file included from /root/workspace/src/image_pipeline/image_view/src/extract_images_node.cpp:53:
/opt/ros/foxy/include/image_transport/image_transport.h:41:89: note: #pragma message: Warning: This header is deprecated. Use 'image_transport.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'image_transport.hpp' instead")
      |                                                                                         ^
/root/workspace/src/image_pipeline/image_view/src/extract_images_node.cpp: In member function ‘void image_view::ExtractImagesNode::image_cb(const ConstSharedPtr&)’:
/root/workspace/src/image_pipeline/image_view/src/extract_images_node.cpp:114:23: warning: catching polymorphic type ‘class cv_bridge::Exception’ by value [-Wcatch-value=]
  114 |   } catch (cv_bridge::Exception) {
      |                       ^~~~~~~~~
In file included from /root/workspace/src/image_pipeline/image_view/include/image_view/video_recorder_node.hpp:19,
                 from /root/workspace/src/image_pipeline/image_view/src/video_recorder.cpp:19:
/opt/ros/foxy/include/image_transport/image_transport.h:41:89: note: #pragma message: Warning: This header is deprecated. Use 'image_transport.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'image_transport.hpp' instead")
      |                                                                                         ^
In file included from /root/workspace/src/image_pipeline/image_view/include/image_view/extract_images_node.hpp:53,
                 from /root/workspace/src/image_pipeline/image_view/src/extract_images.cpp:49:
/opt/ros/foxy/include/image_transport/image_transport.h:41:89: note: #pragma message: Warning: This header is deprecated. Use 'image_transport.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'image_transport.hpp' instead")
      |                                                                                         ^
In file included from /root/workspace/src/image_pipeline/image_view/include/image_view/image_saver_node.hpp:55,
                 from /root/workspace/src/image_pipeline/image_view/src/image_saver.cpp:53:
/opt/ros/foxy/include/image_transport/image_transport.h:41:89: note: #pragma message: Warning: This header is deprecated. Use 'image_transport.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'image_transport.hpp' instead")
      |                                                                                         ^
In file included from /root/workspace/src/image_pipeline/image_view/include/image_view/stereo_view_node.hpp:55,
                 from /root/workspace/src/image_pipeline/image_view/src/stereo_view.cpp:53:
/opt/ros/foxy/include/image_transport/subscriber_filter.h:41:91: note: #pragma message: Warning: This header is deprecated. Use 'subscriber_filter.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'subscriber_filter.hpp' instead")
      |                                                                                           ^
In file included from /root/workspace/src/image_pipeline/image_view/include/image_view/image_view_node.hpp:21,
                 from /root/workspace/src/image_pipeline/image_view/src/image_view.cpp:53:
/opt/ros/foxy/include/image_transport/image_transport.h:41:89: note: #pragma message: Warning: This header is deprecated. Use 'image_transport.hpp' instead
   41 | #pragma message ("Warning: This header is deprecated. Use 'image_transport.hpp' instead")
      |                                                                                         ^
---
Finished <<< image_view [14.8s]

Summary: 16 packages finished [15.3s]
  3 packages had stderr output: image_proc image_view stereo_image_proc
root@31e372d636e3:~/workspace# colcon test
Starting >>> cv_bridge
Starting >>> image_transport
Starting >>> image_geometry
Starting >>> camera_calibration_parsers
Finished <<< image_geometry [0.16s]                                                                                                       
Finished <<< image_transport [0.54s]                                                                              
Finished <<< camera_calibration_parsers [1.21s]                                          
Starting >>> camera_info_manager
Finished <<< camera_info_manager [0.98s]                                          
Starting >>> image_common
Finished <<< image_common [1.49s]                                          
Finished <<< cv_bridge [12.7s]             
Starting >>> image_proc
Starting >>> camera_calibration
Starting >>> depth_image_proc
Starting >>> image_rotate
Starting >>> image_publisher
Starting >>> image_view
Starting >>> opencv_tests
Starting >>> vision_opencv
Finished <<< vision_opencv [0.38s]                                                                                                                              
--- stderr: opencv_tests                                                                                                                                        
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3/dist-packages/pytest.py", line 102, in <module>
    raise SystemExit(pytest.main())
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 63, in main
    config = _prepareconfig(args, plugins)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 206, in _prepareconfig
    return pluginmanager.hook.pytest_cmdline_parse(
  File "/usr/lib/python3/dist-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 92, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 83, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3/dist-packages/_pytest/helpconfig.py", line 94, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 739, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 947, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 893, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 297, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/usr/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/ros/foxy/lib/python3.8/site-packages/launch_testing_ros/__init__.py", line 19, in <module>
    from .test_runner import LaunchTestRunner
  File "/usr/lib/python3/dist-packages/_pytest/assertion/rewrite.py", line 304, in load_module
    exec(co, mod.__dict__)
  File "/opt/ros/foxy/lib/python3.8/site-packages/launch_testing_ros/test_runner.py", line 17, in <module>
    import launch_testing.test_runner
  File "/opt/ros/foxy/lib/python3.8/site-packages/launch_testing/__init__.py", line 15, in <module>
    from . import tools
  File "/opt/ros/foxy/lib/python3.8/site-packages/launch_testing/tools/__init__.py", line 18, in <module>
    from .process import launch_process
  File "/opt/ros/foxy/lib/python3.8/site-packages/launch_testing/tools/process.py", line 18, in <module>
    import launch.actions
ModuleNotFoundError: No module named 'launch.actions'
---
Finished <<< opencv_tests [0.57s]	[ with test failures ]
Failed   <<< camera_calibration [1.29s, exited with code 2]                                                                                                     
Finished <<< image_publisher [2.53s]                                                                                                                           
Finished <<< image_rotate [2.56s]
Finished <<< depth_image_proc [3.15s]                                                                            
Finished <<< image_view [3.28s]                                                        
Finished <<< image_proc [3.53s]                        
Starting >>> stereo_image_proc
Starting >>> image_pipeline
Finished <<< image_pipeline [0.03s]
Finished <<< stereo_image_proc [5.13s]                        

Summary: 15 packages finished [21.7s]
  1 package failed: camera_calibration
  1 package had stderr output: opencv_tests
  1 package had test failures: opencv_tests
root@31e372d636e3:~/workspace# 

@JWhitleyWork
Copy link
Collaborator

The error you're seeing is because opencv_tests missed a <test_depend> in the package.xml file. Should be solvable by installing ros-foxy-launch.

@JWhitleyWork
Copy link
Collaborator

@soeroesg We have switched this repository to use Github Actions instead of CircleCI. Would you mind rebasing your branch on the ros2 branch? Github Actions should run then.

@DavidTorresOcana
Copy link
Contributor

DavidTorresOcana commented Aug 2, 2020

Please beware of #580 and #592

@soeroesg
Copy link
Contributor Author

I rebased again on the latest ros2 and I still get an error, but a new one (see below, Enum seems to be missing). I don't know why it worked before then... The line from enum import Enum is just above the erroneous line.
@JWhitleyWork any idea?

/usr/local/lib/python3.8/dist-packages/_pytest/assertion/rewrite.py:170: in exec_module
exec(co, module.dict)
test/test_directed.py:45: in
from camera_calibration.calibrator import MonoCalibrator, StereoCalibrator,
src/camera_calibration/calibrator.py:49: in
class CAMERA_MODEL(Enum):
E NameError: name 'Enum' is not defined

  • generated xml file: /__w/image_pipeline/image_pipeline/ros_ws/build/camera_calibration/pytest.xml -

----------- coverage: platform linux, python 3.8.2-final-0 -----------
Coverage XML written to file coverage.xml

=========================== short test summary info ============================
ERROR - NameError: name 'Enum' is not defined
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.90s ===============================

@soeroesg
Copy link
Contributor Author

I am just blind and I was inspecting the melodic branch :S
So I added now the two missing import lines and github actions run through successfully.
Can I ignore CircleCI from now on?

@JWhitleyWork
Copy link
Collaborator

Can I ignore CircleCI from now on?

Yes. I wish I could disable the check but someone with higher access than me has to disable it on the repo.

@JWhitleyWork
Copy link
Collaborator

@soeroesg This is still awaiting testing, correct?

@JWhitleyWork JWhitleyWork added the needs testing Currently awaiting testing on physical hardware label Aug 19, 2020
@soeroesg
Copy link
Contributor Author

yes, this still needs to be tested by a volunteer, I do not have ROS2 on my computer at the moment

@pmusau17
Copy link
Contributor

pmusau17 commented Jun 22, 2021

Tested this on ros2 foxy, it passes all tests. Issued one warning.

Warning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
-- Docs: https://docs.pytest.org/en/latest/warnings.html

@JWhitleyWork
Copy link
Collaborator

@pmusau17 Thanks for testing this. Unfortunately, this PR is very old and needs a manual rebase before it can be merged. @soeroesg Are you still around to be able to do this?

@soeroesg
Copy link
Contributor Author

soeroesg commented Jun 30, 2021

@JWhitleyWork @pmusau17 I merged the upstream changes

@clalancette clalancette changed the base branch from ros2 to rolling July 15, 2022 14:52
@JWhitleyWork
Copy link
Collaborator

@soeroesg This needs a serious rebase but could probably still be merged (sorry for the huge delay). Please let me know if you are still able to work on it.

@mikeferguson
Copy link
Member

This was actually rebased and merged in #677

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs testing Currently awaiting testing on physical hardware ros2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants