Skip to content

Generating multiple readers with Bio-Formats can be very slow #865

@petebankhead

Description

@petebankhead

Bug report

Describe the bug
BioFormatsImageServer lazily creates a new reader for each thread as required. This is ok if the readers are fast to initialize, but can be a major bottleneck if they are not.

This was noticed when working with large CZI images (>30 GB). Initializing a reader took ~3-5s. However, because the method was synchronized and 32 threads were wanting to get pixels for the viewer, many other threads were blocked. This meant that even viewing the image properly was not possible for well over a minute.

Once the readers were created, performance was fine.

To Reproduce
Unfortunately, I'm not aware of any public images images that can be used to test this. It might be evident with any large Axioscan images (I'm not entirely sure).

Once the image is open, zoom in and wait for tiles to appear. If experiencing the problem, this will take an unreasonable amount of time. VisualVM indicates that the bottleneck is initializing readers.

Expected behavior
No major delay in requesting tiles once the image has been opened.

Desktop (please complete the following information):

  • OS: macOS (but likely to be all)
  • QuPath Version 0.3.0

Screenshots
Compare the 'total time' (>100s) with the actual time spent using the CPU (3.6s) for a thread requesting image tiles.

Screenshot 2021-12-15 at 12 49 12

Additional context
A few things could help:

  • Reduce the maximum number of tile request threads
  • Reduce synchronization when creating readers
  • Limit the number of readers Bio-Formats can create, independently of the number of threads making tile requests
  • Reduce the calls to isThisType() when creating a reader (when the class of the reader can be known)

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions