Bioformats: Fastest way to get RGB of an extra large image

[kazak]

Hello,

I'm using Bioformats and ImageJ to process extra large images (~75000 x 75000 px) such as .vsi extension. I currently use imp = BF.openImagePlus(options)[0]; with crop options. Then I use the method convertToRGB of the ImageConverter class to convert the ImagePlus and getRGB method of the ColorProcessor class to retrieve the RGB values.

My problem is that the conversion process seems very time consuming, whereas I don't really need this ImagePlus but only the RGB arrays.

I wondered what would be the fastest (minimal) way to scan any type of image format, tile by tile, and for each tile retrieving only the RGB arrays?

Best regards

Gilles

[bramalingam]

Hi,

The Bio-Formats macro extensions for ImageJ, are limited in their coverage.
So to get any sort of performance improvements, you should consider using the Bio-Formats API (in Jython, or etc).

If you prefer to work within the Macro world, you can try the following suggestion,

The thing that is affecting your performance is this,
imp = BF.openImagePlus(options)[0];

and you can access the backend code for the openImagePlus method here,
https://github.com/openmicroscopy/biofo ... ns/BF.java

Calling the setId method iteratively does not help your cause!

setId - "setId initializes the file set, reads all of the metadata, and sets the reader up for reading planes. for large file sets, it can be slow (several seconds to minutes)"

So using larger tile sizes to crop your images (which would mean, lesser number of setId calls) might slightly improve your performance.

Hope that helps,
Best,
Balaji

[kazak]

Hi Balaji,

First thank you for your answer. I did not noticed the difference between the API and the macro extension...

I finally followed this example: https://www.openmicroscopy.org/site/sup ... xport.html with success. The reader.openBytes() does exactly what I was looking for.

It takes now about 3 seconds to read tiles of 5000 x 5000 px, and I have images in the range of 75000 x 75000 px to process. Thus, total reading time is still larger than 10 minutes. I thought about parallelization but I will have to limit the number of threads in order to avoid out of memory.

Do you have any tips to improve the reading performances?

Best regards

Gilles

[bramalingam]

Hi,

Try the following methods to optimise your tile width and tile height parameters,

reader.getOptimalTileWidth()
reader.getOptimalTileHeight()

Incorporating the values obtained from these methods might improve your performance slightly.
Hope that helps.

Best,
Balaji

[kazak]

Hello Balaji and everyone,

I tried the reader.getOptimalTileWidth() to improve the speed of reading, without success. It seems that large tiles are faster, but there is of course the limit of the heap space memory.

Does anyone know if it is possible to map the file into memory for faster access? I see: "DEBUG loci.common.NIOByteBufferProvider - Using mapped byte buffer? false" when I run the reader and I wonder what this is?

Can someone tell me if it is feasable and where I can start?

I found also: http://downloads.openmicroscopy.org/bio ... vider.html

Actually I already tested with a ssd, but the performances are quite similar to my standard hard drive.

Gilles

[mtbc]

I believe that you can set the use of mapped byte buffers with System.setProperty("mappedBuffers", "true") or by using -DmappedBuffers=true on the Java command line. However, I believe that users have seen bugs with this mode so, if what I am saying works at all, definitely use it with caution; I'm not sure it's even supported, let alone recommended. Somebody who knows more than I should be along in due course, with better help!

Cheers,
Mark