Contrib XPhoto Inpaint: Segmentation fault using FSR algorithms (fast or best)

[danopdev]

System information (version)

• OpenCV => 4.5.5 (but found in previous versions too)
• Operating System / Platform => Linux 64 Bit
• Compiler => gcc 9.3.0

Detailed description

I used the sticker to create a panorama and I tried to fill the resulting black areas using the inpaint algoritms from contrib/xphoto.
The application crashed.

Steps to reproduce

This is the source code that does only the inpaint part.

   #include <opencv2/core.hpp>
   #include <opencv2/imgcodecs.hpp>
   #include <opencv2/highgui.hpp>
   #include <opencv2/photo.hpp>
   #include <opencv2/xphoto/inpainting.hpp>
   
   using namespace cv;
   using namespace std;
   
   int main() {
       Mat photo = imread("test.png");
       Mat mask = imread("mask.png", IMREAD_GRAYSCALE); 
       Mat output;
    
       xphoto::inpaint(photo, mask, output, cv::xphoto::INPAINT_FSR_FAST);
       imwrite("output.png", output);
   
       return 0;
   }

Issue submission checklist

• [ x ] I report the issue, it's not a question
• [ x ] I checked the problem with documentation, FAQ, open issues,
  forum.opencv.org, Stack Overflow, etc and have not found any solution
• [ x ] I updated to the latest OpenCV version and the issue is still there
• [ x ] There is reproducer code and related data files: videos, images, onnx, etc

[danopdev]

The issue is in the function icvGetTodoBlocks. Some indexes from set_todo are out or the range of sigma_n_array.

Since I don't know the algorithm details I can't explain why it happens, but checking the range seems to fix the problem.

Suggested patch:

@@ -318,7 +318,7 @@ icvGetTodoBlocks(Mat& sampled_img, Mat& sampling_mask, std::vector< std::tuple<
         Mat error_mask_2d = sampling_mask(Range(yblock_counter*block_size - top_border, std::min(img_height, (yblock_counter*block_size + block_size + bottom_border))), Range(xblock_counter*block_size - left_border, std::min(img_width, (xblock_counter*block_size + block_size + right_border))));
 
         // determine normalized and weighted standard deviation
-        if (block_size > block_size_min)
+        if (block_size > block_size_min && xblock_counter < sigma_n_array.cols && yblock_counter < sigma_n_array.rows)
         {
             double sigma_n = icvStandardDeviation(distorted_block_2d, error_mask_2d);
             sigma_n_array.at<double>( yblock_counter, xblock_counter) = sigma_n;

[alalek]

Feel free to prepare PR with your patch.

relates #2296

[danopdev]

PR done. Please let me know if I miss something.

[orbisvicis]

How did you find this? Not sure I have the same issue. I encounter a segfault during the destruction of Mat sigma_n_array upon leaving icvDetermineProcessingOrder... at least that's what gdb shows. Running 4.5.5; haven't tried your patch yet.

[danopdev]

I used the attached images and valgrind. It didn't crashed on small version of the images.
As you said maybe there are other issues too.

[orbisvicis]

I tried your patch and it works! I'm glad, and it's a relief since Valgrind doesn't run on Cygwin. My images are small, 100x100. Possibly the out-of-bounds write was modifying an important location for cv::Mat::deallocate? No matter. I don't think your patch breaks the algorithm - the results of inpainting are very promising, much better than nearest neighbor interpolation or INPAINT_TELEA/INPAINT_NS. Thanks!

[danopdev]

I found SHIFTMAP event better, but it's just a metter of taste.