Seems like the imageio uses more memory than needed

[hinerm]

Needed over 3 gigs of memory to convert one plane of a 1.3 GB ome.tiff dataset.

[mabruce]

Seconded. Writing three 150 MB files somehow needs 4-5 GB.

[phcerdan]

Seconded. An error sample going out of memory using SCIFIO-ITK bridge.

terminate called after throwing an instance of 'itk::ExceptionObject' what(): /home/phc/Software/ITK/build-devtoolset/Modules/Remote/SCIFIO/src/itkSCIFIOImageIO.cxx:906: itk::ERROR: SCIFIOImageIO(0xc4b1c0): SCIFIOImageIO: 'SCIFIOITKBridge read' exited abnormally. Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at loci.common.DataTools.allocate(DataTools.java:829) at loci.formats.ChannelFiller.openBytes(ChannelFiller.java:183) at io.scif.itk.SCIFIOITKBridge.read(SCIFIOITKBridge.java:429) at io.scif.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:153) at io.scif.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:109) at io.scif.itk.SCIFIOITKBridge.waitForInput(SCIFIOITKBridge.java:84) at io.scif.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:164) at io.scif.itk.SCIFIOITKBridge.main(SCIFIOITKBridge.java:696)

[ctrueden]

I pushed a small performance improvement for reading. But it won't affect the writing performance. There is surely more going on here than just lack of buffer reuse. Best would be to profile the Java-side code. @hinerm has mad skills there—but finding the time is another issue...

[emmenlau]

I guess this is still relevant? Did somebody follow this up further?

In a similar benchmark with Java 1.6.0 and BioFormats 4.x I have found that the JVM can accommodate with significantly less memory if low limits are imposed already during startup. It seems that the JVM can make excessive use of memory when no tight limits are set, or at least it seems its not calling the garbage collector frequently enough.

Of course this policy of low initial limits has the flaw that if the limit is too low, the execution will eventually fail. The best policy I could come up with was to test with continuously growing memory limits until the image writing would work.