run simpleMontage.py get error ITK ERROR: Object factory failed to instantiate class itk::RealToHalfHermitianForwardFFTImageFilter on windows 10

[zhusihan-python]

hi, i build the lastest verion itk 5.3.0 and itk montage 0.8.1 on windows 10 python 3.10.10
run examples/simpleMontage.py get error like below

Warning: Unknown parameter 'itk::Matrix<float,4,4>' in template 'itk::MetaDataObject'
Computing tile registration transforms
WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC270C0): Trying to delete object with non-zero reference count. 
                                                                                       
WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC289C0): Trying to delete object with non-zero reference count. 
                                                                                       
WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC24820): Trying to delete object with non-zero reference count. 
                                                                                       
WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC2AC20): Trying to delete object with non-zero reference count. 

WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC29320): Trying to delete object with non-zero reference count.

WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC25180): Trying to delete object with non-zero reference count.

WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC257C0): Trying to delete object with non-zero reference count.

WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC27D40): Trying to delete object with non-zero reference count.

WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC286A0): Trying to delete object with non-zero reference count.

WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC26A80): Trying to delete object with non-zero reference count.

WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC2B260): Trying to delete object with non-zero reference count.

WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182

Traceback (most recent call last):
  File "E:\projects\ITKMontage-master\examples\SimpleMontage.py", line 66, in <module>
WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC2BBC0): Trying to delete object with non-zero reference count.

WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC2B580): Trying to delete object with non-zero reference count.

    montage.Update()
WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC26DA0): Trying to delete object with non-zero reference count.

RuntimeError: E:\projects\ITK-master\Modules\Filtering\FFT\include\itkRealToHalfHermitianForwardFFTImageFilter.h:85:
ITK ERROR: Object factory failed to instantiate class itk::RealToHalfHermitianForwardFFTImageFilter<class itk::Image<float,2>,class itk::Image<class std::complex<float>,2> >
WARNING: In E:\projects\ITK-master\Modules\Core\Common\src\itkLightObject.cxx, line 182
LightObject (00000214ECC29000): Trying to delete object with non-zero reference count.

itk cmake configure

itk montage configure

expectation: itk 5.3.0 and itk montage 0.8.1 run simpleMontage successfully, get a stitched image

[dzenanz]

@tbirdso Could this be related to FFT refactoring?

[tbirdso]

@dzenanz Sounds possible. I will work to reproduce on my end.

[tbirdso]

Hi @zhusihan-python , I am having difficulty in reproducing the error that you describe. The examples/simpleMontage.py example runs to completion on one of the ITKMontage sample datasets with versions itk==5.3.0, itk-montage==0.8.1:

> python SimpleMontage.py D:\repos\ITKMontage-bld\ExternalData\test\Input\MNML_5_500x_101 D:\repos\ITKMontage-bld\Testing\Temporary\py output-montage.mhd
Computing tile registration transforms

Iteration 1  Writing tile transforms
Producing the mosaic
Resampling complete

A couple of questions to help us investigate:

1. Could you please provide more information on the dataset you are using to run the example? Are you using one of the samples provided by ITKMontage?
2. When you built ITK, did you build the default set of modules?
3. Are you able to reproduce the error if you install and use the ITKMontage Python release from PyPI rather than your local build? pip install itk itk-montage

I have identified a small error later in simpleMontage.py for which I will submit a patch, but I do not expect that it will impact this issue.

[zhusihan-python]

@tbirdso hi sir
my dataset is group of jpg files, you can download it here if need

i build itk by myself, and i set ITK_BUILD_DEFAULT_MODULES ON, you can see that on the itk configure picture i upload

i can't reproduce the error use the ITKMontage Python release from PyPI, i try to reinstall itk itk-montage from pypi, then tried the MNML_5_500x_101 files, it get another error like this:

(py310) E:\projects\ITKMontage-master\examples>python SimpleMontage.py E:\projects\ITKMontage-master\test\Input\MNML_5_500x_101 E:\data\testing output-montage.mhd
Computing tile registration transforms

Iteration 1  Writing tile transforms
Producing the mosaic
Traceback (most recent call last):
  File "C:\Users\allen\.conda\envs\py310\lib\site-packages\itk\support\template_class.py", line 525, in __getitem__
    this_item = self.__template__[key]
KeyError: (<class 'itk.itkImagePython.itkImageRGBUC2'>,)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\projects\ITKMontage-master\examples\SimpleMontage.py", line 85, in <module>
    resampleF = itk.TileMergeImageFilter[type(color_images[0])].New()
  File "C:\Users\allen\.conda\envs\py310\lib\site-packages\itk\support\template_class.py", line 529, in __getitem__
    raise itk.TemplateTypeError(self, key)
itk.support.extras.TemplateTypeError: itk.TileMergeImageFilter is not wrapped for input type `itk.Image[itk.RGBPixel[itk.UC],2]`.

To limit the size of the package, only a limited number of
types are available in ITK Python. To print the supported
types, run the following command in your python environment:

    itk.TileMergeImageFilter.GetTypes()

Possible solutions:
* If you are an application user:
** Convert your input image into a supported format (see below).
** Contact developer to report the issue.
* If you are an application developer, force input images to be
loaded in a supported pixel type.

    e.g.: instance = itk.TileMergeImageFilter[itk.Image[itk.SS,2], itk.D].New(my_input)

* (Advanced) If you are an application developer, build ITK Python yourself and
turned to `ON` the corresponding CMake option to wrap the pixel type or image
dimension you need. When configuring ITK with CMake, you can set
`ITK_WRAP_${type}` (replace ${type} with appropriate pixel type such as
`double`). If you need to support images with 4 or 5 dimensions, you can add
these dimensions to the list of dimensions in the CMake variable
`ITK_WRAP_IMAGE_DIMS`.

Supported input types:

itk.Image[itk.SS,2]
itk.Image[itk.UC,2]
itk.Image[itk.US,2]
itk.Image[itk.F,2]
itk.Image[itk.D,2]
itk.Image[itk.RGBPixel[itk.UC],2]
itk.Image[itk.RGBAPixel[itk.UC],2]
itk.Image[itk.SS,3]
itk.Image[itk.UC,3]
itk.Image[itk.US,3]
itk.Image[itk.F,3]
itk.Image[itk.D,3]
itk.Image[itk.RGBPixel[itk.UC],3]
itk.Image[itk.RGBAPixel[itk.UC],3]
itk.Image[itk.SS,4]
itk.Image[itk.UC,4]
itk.Image[itk.US,4]
itk.Image[itk.F,4]
itk.Image[itk.D,4]
itk.Image[itk.RGBPixel[itk.UC],4]
itk.Image[itk.RGBAPixel[itk.UC],4]

itk.TileMergeImageFilter is not wrapped for input type itk.Image[itk.RGBPixel[itk.UC],2], maybe some data type is missing on windows?

one of my environment is windows 10, python 3.10.10, i remove itk i build and reinstall itk itk-montage from pypi,
another environment is conda virtual env python 3.10.9 which i want to make sure it's not related to itk uninstalled incorrectly, both environments get the same error using the official build itk

SimpleMontage.py is the same as in this repo

import sys
import os
import itk
from pathlib import Path

if len(sys.argv) < 4:
  print(f"Usage: {sys.argv[0]} <directoryWithInputTiles> <outputDirectory> <outputFilename>")
  sys.exit(1)

input_path = Path(sys.argv[1])
output_path = Path(sys.argv[2])
out_file = Path(sys.argv[3])
if not out_file.is_absolute():
  out_file = (output_path / out_file).resolve()


dimension = 2

stage_tiles = itk.TileConfiguration[dimension]()
stage_tiles.Parse(str(input_path / "TileConfiguration.txt"))

color_images = [] # for mosaic creation
grayscale_images = [] # for registration
for t in range(stage_tiles.LinearSize()):
  origin = stage_tiles.GetTile(t).GetPosition()
  filename = str(input_path / stage_tiles.GetTile(t).GetFileName())
  image = itk.imread(filename)
  spacing = image.GetSpacing()

  # tile configurations are in pixel (index) coordinates
  # so we convert them into physical ones
  for d in range(dimension):
    origin[d] *= spacing[d]

  image.SetOrigin(origin)
  color_images.append(image)

  image = itk.imread(filename, itk.F) # read as grayscale
  image.SetOrigin(origin)
  grayscale_images.append(image)

# only float is wrapped as coordinate representation type in TileMontage
montage = itk.TileMontage[type(grayscale_images[0]), itk.F].New()
montage.SetMontageSize(stage_tiles.GetAxisSizes())
for t in range(stage_tiles.LinearSize()):
  montage.SetInputTile(t, grayscale_images[t])

print("Computing tile registration transforms")
montage.Update()

print("Writing tile transforms")
actual_tiles = stage_tiles # we will update it later
for t in range(stage_tiles.LinearSize()):
  index = stage_tiles.LinearIndexToNDIndex(t)
  regTr = montage.GetOutputTransform(index)
  tile = stage_tiles.GetTile(t)
  itk.transformwrite([regTr], str(output_path / (tile.GetFileName() + ".tfm")))

  # calculate updated positions - transform physical into index shift
  pos = tile.GetPosition()
  for d in range(dimension):
    pos[d] -= regTr.GetOffset()[d] / spacing[d]
  tile.SetPosition(pos)
  actual_tiles.SetTile(t, tile)
actual_tiles.Write(str(output_path / "TileConfiguration.registered.txt"))

print("Producing the mosaic")
resampleF = itk.TileMergeImageFilter[type(color_images[0])].New()
resampleF.SetMontageSize(stage_tiles.GetAxisSizes())
for t in range(stage_tiles.LinearSize()):
  resampleF.SetInputTile(t, color_images[t])
  index = stage_tiles.LinearIndexToNDIndex(t)
  resampleF.SetTileTransform(index, montage.GetOutputTransform(index))
resampleF.Update()
itk.imwrite(resampleF.GetOutput(), str(out_file))
print("Resampling complete")

[zhusihan-python]

it works fine on linux

get expected output files

my linux version

i think it didn't work on windows and WSL by my test

[tbirdso]

Hi @zhusihan-python , thank you for the detailed description. The Windows error you described in SimpleMontage.py was fixed yesterday in #215, your issue should be resolved by either downloading the most recent version of SimpleMontage.py or running git pull in your ITKMontage local repository.

[zhusihan-python]

Hi @zhusihan-python , thank you for the detailed description. The Windows error you described in SimpleMontage.py was fixed yesterday in #215, your issue should be resolved by either downloading the most recent version of SimpleMontage.py or running git pull in your ITKMontage local repository.

thank you for the help, it works perfect on windows now

in montage 0.7.3 the rgb image support should be like this
image = itk.imread(filename, itk.RGBPixel[itk.US])
0.8.1 it should be like this, read directly
image = itk.imread(filename)

[tbirdso]

thank you for the help, it works perfect on windows now

Great, glad to hear it. I'll leave this issue open for now in case someone has bandwidth to investigate the C++ object factory instantiation failure. It sounds like using the Python distribution is a sufficient workaround for now.

[EricAtORS]

I'm posting here for posterity's sake. There were not a lot hits with the exact issue. There was also a forum post on the official ITK forum, but I've decided to post here to make it easier to reference across the github ecosystem.

I had the exact same issue using RTK with this:

ITK ERROR: Object factory failed to instantiate class itk::RealToHalfHermitianForwardFFTImageFilter<class itk::Image<float,2>,class itk::Image<class std::complex<float>,2> >

and I also found that the python code did work, though migrating to python was not a viable option for us.

I tracked down the issue to our conan build of ITK.
We were using a variant of the one on conancenter to enable RTK, but the issue with conan is it essentially deletes all the supporting cmake files in the final build.

Once I got that, I found that we were never registering the factories. I suspected issues with the static variables and this discussion confirmed my suspicions InsightSoftwareConsortium/ITK#2864 and explains why it works in python but not in our C++ code.

Finally, the fix, there are 2 possible fixes:

1. In your cmake configuration, add this to your librariry/exectuable that depend on ITK:

# assuming that you are not using conan.
include("${PACKAGE_DIR}/lib/cmake/ITK-5.3/ITKConfig.cmake")
include("${PACKAGE_DIR}/lib/cmake/ITK-5.3/UseITK.cmake")

where PACKAGE_DIR is where you installed ITK.

2. I didn't feel like updating our conan recipe, so the quick and dirty solution is to include this once somewhere in your code:
   I stuck it in a header and included it once in one of our cpp files.
   This code is generated by the function itk_generate_factory_registration() of the UseITK.cmake file.

/*=========================================================================
 *
 *  Copyright NumFOCUS
 *
 *  Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 *  You may obtain a copy of the License at
 *
 *         https://www.apache.org/licenses/LICENSE-2.0.txt
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 *
 *=========================================================================*/

#ifndef itkFFTImageFilterInitFactoryRegisterManager_h
#define itkFFTImageFilterInitFactoryRegisterManager_h

namespace itk {

//
//  The following code is intended to be expanded at the end of the
//  itk<config>FFTImageFilter.h files
//
void  VnlFFTImageFilterInitFactoryRegister__Private();

class FFTImageFilterInitFactoryRegisterManager
{
  public:
  explicit FFTImageFilterInitFactoryRegisterManager(void (* const list[])(void))
  {
    for(;*list != nullptr; ++list)
      {
      (*list)();
      }
    }
};

//
// The code below registers available FFT helpers using static initialization in
// application translation units. Note that this code will be expanded in the
// ITK-based applications and not in ITK itself.
//
void (* const FFTImageFilterInitFactoryRegisterList[])(void) = {
  VnlFFTImageFilterInitFactoryRegister__Private,
  nullptr};
const FFTImageFilterInitFactoryRegisterManager FFTImageFilterInitFactoryRegisterManagerInstance(FFTImageFilterInitFactoryRegisterList);

}

#endif