Fix objPoints order in GridBoard and CharucoBoard #3174

AleksandrPanov · 2022-02-11T00:37:26Z

This PR:

Fixed objPoints order in GridBoard and CharucoBoard.
In the _drawPlanarBoardImpl() function the y coordiantes of the charuco markers get horizontally mirrored by pf.y = (1.0f - pf.y / sizeY) * float(out.rows); Now pf.y = pf.y / sizeY * float(out.rows);

Drawing correctness was checked by example.cpp.txt

Before fix:

Arrays objPoints and ids for 3x4 aruco::GridBoard are in the wrong order
Arrays objPoints and ids for 3x4 aruco::CharucoBoard are in the wrong order. Also objPoints coordinates are incorrect.

After fix:

Everything is drawn correctly.

Fixed generation of rotated images (in ArUco tests). Now rotated images are generated using the "aircraft principal axes".

Also fixed objPoints[markerIndex] order (from CCW to CW) in GridBoard and CharucoBoard. Because tutorial and all ArUco algs use CW order.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

modules/aruco/include/opencv2/aruco.hpp

asmorkalov · 2022-02-14T06:19:57Z

modules/aruco/test/test_aruco_utils.hpp

+
+            // rotate camera in pitch axis
+            Mat rotPitch(3, 1, CV_64FC1);
+            rotPitch.ptr< double >(0)[0] = pitch;


rotPitch.at< double >(i, j) is better.

alalek · 2022-02-16T09:30:43Z

modules/aruco/include/opencv2/aruco.hpp


    /// array of object points of all the marker corners in the board
-    /// each marker include its 4 corners in CCW order. For M markers, the size is Mx4.
+    /// each marker include its 4 corners in CW order. For M markers, the size is Mx4.


It doesn't make sense to mention CW or CCW without specifying used coordinate system.

Please take a look on calib3d module documentation.

Added this definition:

/// each marker include its 4 corners in this order: ///- objPoints[i][0] - left-top point of i-th marker ///- objPoints[i][1] - right-top point of i-th marker ///- objPoints[i][2] - right-bottom point of i-th marker ///- objPoints[i][3] - left-bottom point of i-th marker

modules/aruco/test/test_aruco_utils.hpp

alalek · 2022-02-16T09:34:09Z

modules/aruco/src/aruco.cpp

-            corners[2] = corners[0] + Point3f(markerLength, -markerLength, 0);
-            corners[3] = corners[0] + Point3f(0, -markerLength, 0);
+            corners[2] = corners[0] + Point3f(markerLength, markerLength, 0);
+            corners[3] = corners[0] + Point3f(0, markerLength, 0);


You just changed the coordinate system by flipping Y.

This changes was added to make coordinate systems consistent.
Now Board::create() uses this coordinate system:

But Board::draw() uses another coordinate system:

Also, Board::create() sets corners for marker in objPoints with "CCW" order:

But detectMarkers() finds markers and reorders corners to "CW" order:

This makes coordinate systems not consistent and makes the code hard to read

alalek · 2022-02-16T09:36:06Z

modules/aruco/test/test_aruco_utils.hpp

+namespace opencv_test {
+    namespace {
+
+        static inline double deg2rad(double deg) { return deg * CV_PI / 180.; }


avoid indentation in namespaces

AleksandrPanov · 2022-02-18T08:11:32Z

@alalek thanks for the review)
do you have any more questions?

asmorkalov · 2022-03-03T05:22:35Z

modules/aruco/src/aruco.cpp

+    axis.push_back(Point3f(0.f, 0.f, 0.f));
+    axis.push_back(Point3f(length, 0.f, 0.f));
+    axis.push_back(Point3f(0.f, length, 0.f));
+    axis.push_back(Point3f(0.f, 0.f, -length));


why -length? Your function differs from standard drawFrameAxes with opposite direction of the third axis.

This was added to improve appearance. The drawAxis() function doesn't make sense, as it is just calls drawFrameAxes().

asmorkalov · 2022-03-03T06:27:56Z

modules/aruco/test/test_aruco_utils.hpp

+    axis.push_back(Point3f(offset, offset, 0.f));
+    axis.push_back(Point3f(length+offset, offset, 0.f));
+    axis.push_back(Point3f(offset, length+offset, 0.f));
+    axis.push_back(Point3f(offset, offset, -length));


The same, the result axis direction is opposite to what standard drawFrameAxes does.

AleksandrPanov · 2022-03-14T00:54:32Z

I am not fan of the drawCustomAxis() because it feels unnecessary for me. I would prefer some good API for handling SE3/SO3 transformations with some good tutorials. You can also shoot yourself in the foot if you use axisCoefficient=(-1,0,0) or any other non valid values.

Moreover, the extra functionality of drawCustomAxis() is not used in your PR (only for debugging purposes), no?

What about composeRT(), can it do the same job?

@catree, thx)
I removed drawCustomAxis() and added cv::drawFrameAxes()'s link to docs and replaced old images with new ones. The tutorial now shows the correct direction and position of the axes.

AleksandrPanov · 2022-03-14T11:03:41Z

@catree, I think that good API for handling SE3/SO3 transformations should add into another PR. This PR fixes only ArUco problems.

AleksandrPanov · 2022-03-16T13:25:57Z

@alalek could you recheck PR? I fixed catree's issue.

alalek

Thank you 👍

…oints Fix objPoints order in GridBoard and CharucoBoard * fix gridBoard * fix charucoBoard * add rightBottomBorder * add test_aruco_utils and code refactoring/fix tests * fix axes and add charuco dict * add axes test, remove drawAxis(), update tutorial

…oints Fix objPoints order in GridBoard and CharucoBoard * fix gridBoard * fix charucoBoard * add rightBottomBorder * add test_aruco_utils and code refactoring/fix tests * fix axes and add charuco dict * add axes test, remove drawAxis(), update tutorial (cherry picked from commit 56d492c)

Licoze · 2022-05-05T16:00:58Z

Hello, @AleksandrPanov
FYI:
Removing drawAxis breaks backward compatibility. (for example for some wrappers https://github.com/shimat/opencvsharp)
Wouldn't it be better to move it to the next major release?

mabl · 2022-06-23T06:38:02Z

It looks like this change makes interpolateCornersCharuco return a different number for board corners. See image here. Before this, this corner id mapped to the index of board->chessboardCorners.

Is this change intentional? If so, how can I recover the corresponding index for board->chessboardCorners from interpolateCornersCharuco markerIds, i.e. the list of identifiers for each marker in corners.

AleksandrPanov added 3 commits February 11, 2022 01:37

fix gridBoard

85e70f6

fix charucoBoard

c2a869d

add rightBottomBorder

e11a6df

AleksandrPanov self-assigned this Feb 11, 2022

AleksandrPanov force-pushed the fix_gridboard_objPoints branch 3 times, most recently from 3bd672c to f824b1b Compare February 14, 2022 00:57

asmorkalov reviewed Feb 14, 2022

View reviewed changes

AleksandrPanov force-pushed the fix_gridboard_objPoints branch 3 times, most recently from 9e0a992 to 03dd8c5 Compare February 15, 2022 19:33

AleksandrPanov marked this pull request as ready for review February 15, 2022 20:50

asmorkalov requested review from alalek and asmorkalov February 16, 2022 06:24

asmorkalov approved these changes Feb 16, 2022

View reviewed changes

asmorkalov mentioned this pull request Feb 16, 2022

Charuco seems inaccurate. #3175

Closed

AleksandrPanov removed their assignment Feb 16, 2022

alalek reviewed Feb 16, 2022

View reviewed changes

AleksandrPanov force-pushed the fix_gridboard_objPoints branch from 03dd8c5 to 6e81f89 Compare February 16, 2022 18:52

add test_aruco_utils and code refactoring/fix tests

4970a6d

AleksandrPanov force-pushed the fix_gridboard_objPoints branch from 6e81f89 to 4970a6d Compare February 16, 2022 19:17

AleksandrPanov marked this pull request as draft February 24, 2022 14:29

AleksandrPanov force-pushed the fix_gridboard_objPoints branch from 53e8e38 to f18dcbd Compare March 3, 2022 01:25

fix axes and add charuco dict

c8936ad

AleksandrPanov force-pushed the fix_gridboard_objPoints branch from f18dcbd to 63204ac Compare March 3, 2022 03:01

asmorkalov reviewed Mar 3, 2022

View reviewed changes

asmorkalov requested changes Mar 3, 2022

View reviewed changes

AleksandrPanov force-pushed the fix_gridboard_objPoints branch 2 times, most recently from 3b22360 to de40e05 Compare March 3, 2022 22:33

AleksandrPanov force-pushed the fix_gridboard_objPoints branch from 30824a4 to 0f44c79 Compare March 10, 2022 19:06

add axes test, remove drawAxis(), update tutorial

f8470c6

AleksandrPanov force-pushed the fix_gridboard_objPoints branch from 0f44c79 to f8470c6 Compare March 11, 2022 01:43

AleksandrPanov mentioned this pull request Mar 15, 2022

Ambiguity problem in the detection of the ArUco marker when it is parallel to the camera plane #3190

Open

AleksandrPanov requested a review from alalek March 15, 2022 11:27

AleksandrPanov mentioned this pull request Mar 16, 2022

aruco: documentation for Board class contradicts tutorial #3197

Closed

AleksandrPanov linked an issue Mar 16, 2022 that may be closed by this pull request

aruco: documentation for Board class contradicts tutorial #3197

Closed

alalek approved these changes Mar 16, 2022

View reviewed changes

alalek assigned asmorkalov Mar 16, 2022

alalek merged commit 56d492c into opencv:3.4 Mar 16, 2022

AleksandrPanov mentioned this pull request Mar 24, 2022

merge aruco changes from 3.4 to 4.x #3213

Closed

6 tasks

alalek mentioned this pull request Mar 26, 2022

(4.x) Merge 3.4 #3215

Merged

opencv-pushbot mentioned this pull request Apr 23, 2022

(5.x) Merge 4.x #3237

Merged

This was referenced Jul 6, 2022

Charuco Behavior in opencv-python-contrib opencv/opencv-python#686

Closed

CharucoBoard_create regression / incompatible patterns (only even row count) with 4.6.0 #3291

Closed

j3soon mentioned this pull request Dec 31, 2022

Update README & Fix CharucoBoard bug tony2guo/charuco-pose-estimation#2

Merged

AleksandrPanov mentioned this pull request Jul 4, 2023

Charuco board detection is not suitable for calibration? opencv/opencv#23873

Open

4 tasks

Fix objPoints order in GridBoard and CharucoBoard #3174

Fix objPoints order in GridBoard and CharucoBoard #3174

Uh oh!

Conversation

AleksandrPanov commented Feb 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AleksandrPanov Feb 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AleksandrPanov commented Feb 18, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AleksandrPanov commented Mar 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AleksandrPanov commented Mar 14, 2022

Uh oh!

AleksandrPanov commented Mar 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

Licoze commented May 5, 2022

Uh oh!

mabl commented Jun 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

AleksandrPanov commented Feb 11, 2022 •

edited

Loading

AleksandrPanov Feb 16, 2022 •

edited

Loading

AleksandrPanov commented Mar 14, 2022 •

edited

Loading

AleksandrPanov commented Mar 16, 2022 •

edited

Loading