Incorrect types 16S for ximgproc::DisparityFilter in documentation

[Rektino]

Describe the doc issue

OpenCV version : 4.5.4
OS : Ubuntu 22.04

I was following this example https://docs.opencv.org/4.x/d3/d14/tutorial_ximgproc_disparity_filtering.html
to implement a WLS filter on a disparity map.

The doxygen comments on the function virtual void cv::ximgproc::DisparityFilter::filter() mention that the left disparity and left view images should be of type CV_16S. It also mentions that right disparity map is optional. I ran into runtime assertion fail when trying that, which was :

[bash-5] terminate called after throwing an instance of 'cv::Exception'
[bash-5] what(): OpenCV(4.5.4) ./contrib/modules/ximgproc/src/disparity_filters.cpp:301: error: (-215:Assertion failed) !disparity_map_right.empty() && (disparity_map_right.depth() == CV_32F) && (disparity_map_right.channels() == 1) in function 'filter'_

Fix suggestion

No response

[Kumataro]

Hello, thank you for your report.

As first it is better to reference document about you want to use OpenCV version.
If you use 4.5.4, please could you reference to https://docs.opencv.org/4.5.4/db/d72/classcv_1_1ximgproc_1_1DisparityFilter.html .

If possible, could you share your test code to reproduce ?
Therefore, it makes the problem more clearly.

---

OpenCV Documents said.
https://docs.opencv.org/4.5.4/db/d72/classcv_1_1ximgproc_1_1DisparityFilter.html#a3dc752f1760b375f1ea12173c1f07a9e

disparity_map_left - disparity map of the left view, 1 channel, CV_16S type.
left_view - left view of the original stereo-pair to guide the filtering process, 8-bit single-channel or three-channel image.
right_view - optional argument, some implementations might also use the right view of the original stereo-pair.

I think.

• disparity_map_left must be CV_16SC1.
• left_view must be CV_8UC1 or CV_8UC3
• right_view can be omitted, but its type is not described.

And I read code. right_view is convert to CV_32F automatic without no scaling.

opencv_contrib/modules/ximgproc/src/disparity_filters.cpp

Lines 236 to 242 in 2bc3e1a

	if (!disparity_map_right.empty() && use_confidence)
	{
	if(disparity_map_right.depth() != CV_32F)
	disparity_map_right.getMat().convertTo(right, CV_32F);
	else
	right = disparity_map_right.getMat();
	}

I guess you set not single channel image(e.g. CV_8UC3) as right_view argument ?
If so, it will occurs error because (disparity_map_right.channels() == 1) is failed.

[Rektino]

Hi Kumataro,

Thanks for the tips. The code snippet in question looks like this :
left_matcher_ptr_-> compute(rectified_left_gray, rectified_right_gray ,disparity_map_16S_left2right_); right_matcher_ptr_->compute(rectified_right_gray, rectified_left_gray, disparity_map_16S_right2left_); wls_filter_ptr_->filter(disparity_map_16S_left2right_ , rectified_left_gray , filtered_disparity_map_) ;

where the variable naming show the type : I use grayscale rectified left and right image, and use the 16-bit disparity map computed by stereo->compute().

I notice in the source code of disparity_filters.cpp, that the function that is called performs the right image checks only because this condition if(!use_confidence) is false. I saw it nowhere else in the code. Hope this helps!

[Rektino]

Also, adding one simple conversion to CV_32F for the left and right disparity maps, along with passing all the parameters, fixed the assertion failure. This confirms that the types expected are different than stated, at least when the WLS filter is created with the simple :
wls_filter_ptr_ = cv::ximgproc::createDisparityWLSFilter(left_matcher_ptr_) ;

Also, the function you shared is not the one getting called ! The one getting called is the "filter_" with an underscore.

[Kumataro]

Also, the function you shared is not the one getting called ! The one getting called is the "filter_" with an underscore.

My recognize is your application call filter() which is published by OpenCV, and filter() calls filter_(). If it is incorrect, I'm sorry.

Your snipplet is here.

wls_filter_ptr_->filter(disparity_map_16S_left2right_ , rectified_left_gray , filtered_disparity_map_) ;

opencv_contrib/modules/ximgproc/src/disparity_filters.cpp

Lines 220 to 244 in 2bc3e1a

	void DisparityWLSFilterImpl::filter(InputArray disparity_map_left, InputArray left_view, OutputArray filtered_disparity_map, InputArray disparity_map_right, Rect ROI, InputArray)
	{
	CV_Assert(!disparity_map_left.empty() && (disparity_map_left.channels() == 1));
	CV_Assert(!left_view.empty() && (left_view.depth() == CV_8U) && (left_view.channels() == 3 || left_view.channels() == 1));
	Mat left, right, filt_disp;
	if (disparity_map_left.depth() != CV_32F)
	{
	disparity_map_left.getMat().convertTo(left, CV_32F);
	}
	else
	{
	left = disparity_map_left.getMat();
	filt_disp = filtered_disparity_map.getMat();
	}
	if (!disparity_map_right.empty() && use_confidence)
	{
	if(disparity_map_right.depth() != CV_32F)
	disparity_map_right.getMat().convertTo(right, CV_32F);
	else
	right = disparity_map_right.getMat();
	}
	filter_(left, left_view, filt_disp, right, ROI);

void DisparityWLSFilterImpl::filter(InputArray disparity_map_left, InputArray left_view, OutputArray filtered_disparity_map, InputArray disparity_map_right, Rect ROI, InputArray)
{
    CV_Assert(!disparity_map_left.empty() && (disparity_map_left.channels() == 1));
    CV_Assert(!left_view.empty() && (left_view.depth() == CV_8U) && (left_view.channels() == 3 || left_view.channels() == 1));
    Mat left, right, filt_disp;


    if (disparity_map_left.depth() != CV_32F)
    {
        disparity_map_left.getMat().convertTo(left, CV_32F);
    }
    else
    {
        left = disparity_map_left.getMat();
        filt_disp = filtered_disparity_map.getMat();
    }


    if (!disparity_map_right.empty() && use_confidence)
    {
        if(disparity_map_right.depth() != CV_32F)
            disparity_map_right.getMat().convertTo(right, CV_32F);
        else
            right = disparity_map_right.getMat();
    }


    filter_(left, left_view, filt_disp, right, ROI);

[Kumataro]

And I think the OpenCV maintainers will move this issue to the opencv_contrib repository.

I would appreciate it if you could create a PR to fix the documentation appropriately.

[Rektino]

I 've never done a PR before, but there's always a first time. I ll look into how it's done, thanks.