Added support for Eigen cross-compilation #13337

sergiud · 2018-11-30T08:54:34Z

This PR fixes two (closely related) issues:

Eigen is initially completely disabled during cross-compilation. There is no reason for this.
During cross-compilation, the CMake option WITH_EIGEN can still be enabled. In this case, however, CMake will use the Eigen version from the host system (typically from /usr or /usr/local) even if the toolchain explicitly disables this behavior (and rightly so). The libraries in the CMAKE_FIND_ROOT_PATH set by toolchains must be located with higher priority.

This PR uses the Eigen package config if it's available and falls back to previous find_path behavior if the package config is not available. The latter behavior also respects the search mode for packages requested by toolchains now.

buildworker:Custom=linux-1,linux-2,linux-4
docker_image:Custom=powerpc64le

alalek · 2018-11-30T09:37:03Z

CMakeLists.txt

 OCV_OPTION(WITH_CUBLAS         "Include NVidia Cuda Basic Linear Algebra Subprograms (BLAS) library support" ON IF (NOT IOS AND NOT WINRT) )
 OCV_OPTION(WITH_NVCUVID        "Include NVidia Video Decoding library support"                               ON IF (NOT IOS AND NOT APPLE) )
-OCV_OPTION(WITH_EIGEN          "Include Eigen2/Eigen3 support"               (NOT CV_DISABLE_OPTIMIZATION)       IF (NOT WINRT AND NOT CMAKE_CROSSCOMPILING) )
+OCV_OPTION(WITH_EIGEN          "Include Eigen2/Eigen3 support"               (NOT CV_DISABLE_OPTIMIZATION)       IF (NOT WINRT) )


It is fine to allow Eigen for cross-compilation.
But lets continue enabling it explicitly via -DWITH_EIGEN=ON.

(NOT CV_DISABLE_OPTIMIZATION AND NOT CMAKE_CROSSCOMPILING)

So why does WITH_EIGEN defaults to OFF when cross-compiling?

Because of include directory.

What is going with cross-compilation if you add /usr/include (where Eigen is located on host system) into the list of include directories?

This is the point of this PR. The host version of Eigen will not be used during cross-compilation anymore.

I mean that result of compilation is unpredictable if you add host /usr/include (or /usr/local/include) into compiler list of include directories.

This dangerous behavior should be disabled by default.

Yes, that's why this PR removed unconditional /usr/include and /usr/local/include Eigen search prefixes, which are set by CMake anyway. But not during cross-compilation.

The question is now why do you only exlude Eigen (and presumably few other libraries) during cross-compliation, but not other? This is inconsistent.

alalek · 2018-11-30T09:40:07Z

cmake/OpenCVFindLibsPerf.cmake

-            PATH_SUFFIXES include/eigen3 include/eigen2 Eigen/include/eigen3 Eigen/include/eigen2
-            DOC "The path to Eigen3/Eigen2 headers"
-            CMAKE_FIND_ROOT_PATH_BOTH)
+  find_package (Eigen3)


There is no such package in CMake 2.8.12 (Ubuntu 14.04) at least.
Need to guard this statement by check with proper CMake version and "OR DEFINED Eigen3_DIR".

Use "QUIET" option to avoid unnecessary messages.

The package is not part of CMake but installed by Eigen itself.

alalek · 2018-11-30T09:46:48Z

cmake/OpenCVFindLibsPerf.cmake

+
+  if (TARGET Eigen3::Eigen)
+    # Use Eigen3 improted target if possible
+    ocv_include_directories ($<TARGET_PROPERTY:Eigen3::Eigen,INTERFACE_INCLUDE_DIRECTORIES>)


Code style: please avoid "space" before bracket "("

seiko2plus · 2018-11-30T22:49:47Z

This patch causes massive shadow warnings on newest versions of GCC,

/usr/include/eigen3/Eigen/src/SVD/JacobiSVD.h:613:111: warning: declaration of 'rows' shadows a member of 'Eigen::JacobiSVD<MatrixType, QRPreconditioner>' [-Wshadow]
/usr/include/eigen3/Eigen/src/SVD/BDCSVD.h:119:5: warning: declaration of 'cols' shadows a member of 'Eigen::BDCSVD<MatrixType>' [-Wshadow]
/usr/include/eigen3/Eigen/src/SVD/BDCSVD.h:119:5: warning: declaration of 'rows' shadows a member of 'Eigen::BDCSVD<MatrixType>' [-Wshadow]
/usr/include/eigen3/Eigen/src/SVD/BDCSVD.h:215:90: warning: declaration of 'cols' shadows a member of 'Eigen::BDCSVD<MatrixType>' [-Wshadow]
/usr/include/eigen3/Eigen/src/SVD/BDCSVD.h:215:90: warning: declaration of 'rows' shadows a member of 'Eigen::BDCSVD<MatrixType>' [-Wshadow]
/usr/include/eigen3/Eigen/src/Core/DenseBase.h:253:5: warning: declaration of 'cols' shadows a member of 'Eigen::DenseBase<Derived>' [-Wshadow]
/usr/include/eigen3/Eigen/src/Core/DenseBase.h:253:5: warning: declaration of 'rows' shadows a member of 'Eigen::DenseBase<Derived>' [-Wshadow]
/usr/include/eigen3/Eigen/src/Core/CwiseNullaryOp.h:790:103: warning: declaration of 'cols' shadows a member of 'Eigen::MatrixBase<Derived>' [-Wshadow]
/usr/include/eigen3/Eigen/src/Core/CwiseNullaryOp.h:790:103: warning: declaration of 'rows' shadows a member of 'Eigen::MatrixBase<Derived>' [-Wshadow]

GCC 6.5: https://ocv-power.imavr.com/#/builders/1/builds/182
GCC 5.5: https://ocv-power.imavr.com/#/builders/9/builds/343

alalek · 2018-11-30T23:27:42Z

We prefer to follow this policy about CMake options for 3rdparty dependencies:

WITH_* enables auto-detection code from CMake scripts.
HAVE_* enables feature. It is assumed that corresponding variables (libraries, include dirs) are valid.

You can bypass (or disable) auto-detection code and setup required variables manually (including HAVE_*=ON variable).

alalek · 2018-11-30T23:30:04Z

Warnings from external headers are usually handled by using of system (-isystem) include directories (common "/usr/include" is already system, but not for cross-compilation mode)

sergiud · 2018-12-01T09:39:39Z

@seiko2plus

The PR does not do anything differently besides changing the WITH_EIGEN default value. Hence, the -Wshadow warnings were always there. The CI just did not build OpenCV with Eigen.

sergiud · 2018-12-01T10:02:47Z

We prefer to follow this policy about CMake options for 3rdparty dependencies:

WITH_* enables auto-detection code from CMake scripts.

HAVE_* enables feature. It is assumed that corresponding variables (libraries, include dirs) are valid.

You can bypass (or disable) auto-detection code and setup required variables manually (including HAVE_*=ON variable).

The PR changes only the WITH_EIGEN default during cross-compilation, so I'm not sure how is this relevant.

sergiud · 2018-12-03T07:43:10Z

Warnings from external headers are usually handled by using of system (-isystem) include directories (common "/usr/include" is already system, but not for cross-compilation mode)

Looking closely at the CI logs of @seiko2plus, he does not seem to cross-compile. The host system is just a ppc64.

sergiud · 2019-01-09T14:05:16Z

@alalek PTAL

seiko2plus · 2019-01-10T12:17:04Z

@sergiud,

The PR does not do anything differently besides changing the WITH_EIGEN default value. Hence, the -Wshadow warnings were always there. The CI just did not build OpenCV with Eigen.

The PR does change search directory option for Eigen path from "-isystem" to "-I" and that disabled shadow suppressing in

opencv/modules/core/include/opencv2/core/private.hpp

Line 58 in 581d0ef

# pragma GCC diagnostic ignored "-Wshadow"

The reason of changing gcc option is coming from passing generator expression to ocv_include_directories() and that made ocv_is_opencv_directory() fail to determine if the path part of opencv or not, since cmake expression is evaluated during build.

opencv/cmake/OpenCVFindLibsPerf.cmake

Line 48 in 3af178e

    
           ocv_include_directories($<TARGET_PROPERTY:Eigen3::Eigen,INTERFACE_INCLUDE_DIRECTORIES>)

I suggest you to use EIGEN3_INCLUDE_DIR with ocv_include_directories instead of generator expression since even if you going to use include_directories(AFTER SYSTEM ) directly, the problem will still remain during generate pch.

sergiud · 2019-01-10T12:43:39Z

@seiko2plus That does not make any sense. The Eigen path was never part of OpenCV, hence the problem must be in ocv_include_directories. How is using EIGEN3_INCLUDE_DIR supposed to fix the problem? Please explain.

sergiud · 2019-01-10T12:46:12Z

The PR does change search directory option for Eigen path from "-isystem" to "-I" and that disabled shadow suppressing in

This statement is obviously incorrect. The PR does not (actively) do such thing.

seiko2plus · 2019-01-10T13:29:34Z

@sergiud,

That does not make any sense. The Eigen path was never part of OpenCV, hence the problem must be in ocv_include_directories. How is using EIGEN3_INCLUDE_DIR supposed to fix the problem? Please explain.

The problem isn't in ocv_include_directories but in using generator expressions with it $<TARGET_PROPERTY:Eigen3::Eigen,INTERFACE_INCLUDE_DIRECTORIES>.

ocv_include_directories will not be able to extract the path from generator expression to be tested by ocv_is_opencv_directory in configuration stage and vice versa with EIGEN3_INCLUDE_DIR.

This statement is obviously incorrect. The PR does not (actively) do such thing.

Indirect way it does

sergiud · 2019-01-10T14:13:46Z

I understand this. However, this does not change the fact that the issue is caused by ocv_include_directories. More precisely, ocv_is_subdir does not check whether the provided path actually exists:

opencv/cmake/OpenCVUtils.cmake

Lines 227 to 230 in 7e2ebec

    
           function(ocv_is_subdir res dir sub ) 
        
             get_filename_component(dir "${dir}" ABSOLUTE) 
        
             get_filename_component(sub "${sub}" ABSOLUTE) 
        
             file(TO_CMAKE_PATH "${dir}" dir)

Instead, the generator expression is converted to an absolute path by prepending the directory OpenCV resides in. Therefore the subdirectory check succeeds and the SYSTEM modifier is not added by ocv_include_directories.

This is obviously a separate issue. Using EIGEN3_INCLUDE_DIR works only if Eigen was actually installed. It does not work for local Eigen builds.

orivej · 2019-07-26T21:54:26Z

This breaks the build of OpenCV 3.4.7 with Eigen and ENABLE_PRECOMPILED_HEADERS in Nixpkgs:

[  4%] Building CXX object modules/core/CMakeFiles/opencv_core_pch_dephelp.dir/opencv_core_pch_dephelp.cxx.o
In file included from /tmp/nds-build-opencv3/source/modules/core/src/precomp.hpp:55:0,
                 from /tmp/nds-build-opencv3/source/build/modules/core/opencv_core_pch_dephelp.cxx:1:
/tmp/nds-build-opencv3/source/modules/core/include/opencv2/core/private.hpp:66:12: fatal error: Eigen/Core: No such file or directory
 #  include <Eigen/Core>
            ^~~~~~~~~~~~

OPENCV_LINKER_LIBS are set correctly (OPENCV_LINKER_LIBS=dl;m;pthread;rt;Eigen3::Eigen) but they are not passed to the compilation of opencv_core_pch_dephelp.cxx.

This issue was reported in #14868. The author has closed it because they found a workaround. It has not been fixed in OpenCV.

The workaround is to configure cmake with -DEIGEN_INCLUDE_PATH=path/to/include/eigen3.

orivej · 2019-07-26T22:34:03Z

The build gets a bit further if I add target_link_libraries(${_targetName}_pch_dephelp LINK_PRIVATE ${OPENCV_LINKER_LIBS}) here, then it fails at:

[6/1046] cd /tmp/nds-build-opencv3/source/build/modules/fuzzy && /nix/store/9ifxn9p05l98aq7ia04b0ndh8ffb8im4-cmake-3.14.5/bin/cmake -E make_directory /tmp/nds-build-opencv3/source/build/m
odules/fuzzy/precomp.hpp.gch && chmod +x /tmp/nds-build-opencv3/source/build/modules/fuzzy/precomp.hpp.command.sh && /tmp/nds-build-opencv3/source/build/modules/fuzzy/precomp.hpp.command.
sh
FAILED: modules/fuzzy/precomp.hpp.gch/opencv_fuzzy_Release.gch 
cd /tmp/nds-build-opencv3/source/build/modules/fuzzy && /nix/store/9ifxn9p05l98aq7ia04b0ndh8ffb8im4-cmake-3.14.5/bin/cmake -E make_directory /tmp/nds-build-opencv3/source/build/modules/fu
zzy/precomp.hpp.gch && chmod +x /tmp/nds-build-opencv3/source/build/modules/fuzzy/precomp.hpp.command.sh && /tmp/nds-build-opencv3/source/build/modules/fuzzy/precomp.hpp.command.sh
In file included from /tmp/nds-build-opencv3/source/build/modules/fuzzy/precomp.hpp:45:0:
/tmp/nds-build-opencv3/source/modules/core/include/opencv2/core/private.hpp:66:12: fatal error: Eigen/Core: No such file or directory
 #  include <Eigen/Core>
            ^~~~~~~~~~~~
compilation terminated.

sergiud changed the title ~~added support for Eigen cross-compilation~~ Added support for Eigen cross-compilation Nov 30, 2018

alalek reviewed Nov 30, 2018

View reviewed changes

sergiud force-pushed the eigen-cross-compile branch 2 times, most recently from 104c4b5 to d0b00ad Compare November 30, 2018 11:39

added support for Eigen cross-compilation

3af178e

sergiud force-pushed the eigen-cross-compile branch from d0b00ad to 3af178e Compare January 7, 2019 09:46

cmake: fix Eigen detection

f388c11

alalek approved these changes May 20, 2019

View reviewed changes

alalek merged commit bc4b2d5 into opencv:3.4 May 20, 2019

This was referenced May 21, 2019

sfm: fix build with eigen opencv/opencv_contrib#2119

Merged

Merge 3.4 #14615

Merged

alalek mentioned this pull request Jun 9, 2019

Build fails, incorrect include path in odometry.cpp opencv/opencv_contrib#2144

Closed

alalek mentioned this pull request Jun 22, 2019

fatal error: Eigen/Core: No such file or directory #14868

Closed

Uh oh!

Added support for Eigen cross-compilation #13337

Added support for Eigen cross-compilation #13337

Uh oh!

Conversation

sergiud commented Nov 30, 2018 • edited by alalek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sergiud Dec 1, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seiko2plus commented Nov 30, 2018

Uh oh!

alalek commented Nov 30, 2018

Uh oh!

alalek commented Nov 30, 2018

Uh oh!

sergiud commented Dec 1, 2018

Uh oh!

sergiud commented Dec 1, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sergiud commented Dec 3, 2018

Uh oh!

sergiud commented Jan 9, 2019

Uh oh!

seiko2plus commented Jan 10, 2019

Uh oh!

sergiud commented Jan 10, 2019

Uh oh!

sergiud commented Jan 10, 2019

Uh oh!

seiko2plus commented Jan 10, 2019

Uh oh!

sergiud commented Jan 10, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

orivej commented Jul 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

orivej commented Jul 26, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sergiud commented Nov 30, 2018 •

edited by alalek

Loading

sergiud Dec 1, 2018 •

edited

Loading

sergiud commented Dec 1, 2018 •

edited

Loading

sergiud commented Jan 10, 2019 •

edited

Loading

orivej commented Jul 26, 2019 •

edited

Loading