SVS: Incorrect pyramid levels with 6.8.0

[dgault]

Issue was raised on the forum thread https://forum.image.sc/t/problem-about-opening-some-svs-slides-in-qupath-v0-3-1-bio-formats-6-8-0/61404/15

A sample file was provided on the thread which reproduces the problem. The problem seems to be as a result of changes to the handling of macro and label images in #3733

Using the provided sample file with 6.8.0 you get:

Series count = 1
Series #0 -- :
	Resolutions = 6
		sizeX[0] = 62094
		sizeX[1] = 15523
		sizeX[2] = 3880
		sizeX[3] = 1024
		sizeX[4] = 1549
		sizeX[5] = 685
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 62094
	Height = 36729
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 256 x 256
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = false
	-----
	Plane #0 <=> Z 0, C 0, T 0

With 6.8.0 and flattened resolutions you get:

Series #0 -- Series 1:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 62094
	Height = 36729
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 256 x 256
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = false
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #1 -- Series 2:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 15523
	Height = 9182
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 256 x 256
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #2 -- Series 3:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 3880
	Height = 2295
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 256 x 256
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #3 -- Series 4:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 1024
	Height = 605
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 1024 x 605
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #4 -- Series 5:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 1549
	Height = 580
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 1549 x 580
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #5 -- Series 6:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 685
	Height = 530
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 685 x 530
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

With 6.7.0 you get:

Series count = 3
Series #0 -- :
	Resolutions = 4
		sizeX[0] = 62094
		sizeX[1] = 15523
		sizeX[2] = 3880
		sizeX[3] = 1024
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 62094
	Height = 36729
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 256 x 256
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = false
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #1 -- label image:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 685
	Height = 530
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 685 x 530
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

Series #2 -- macro image:
	Image count = 1
	RGB = true (3) 
	Interleaved = false
	Indexed = false (true color)
	Width = 1549
	Height = 580
	SizeZ = 1
	SizeT = 1
	SizeC = 3 (effectively 1)
	Tile size = 1549 x 580
	Thumbnail size = 685 x 530
	Endianness = intel (little)
	Dimension order = XYCZT (uncertain)
	Pixel type = uint8
	Valid bits per pixel = 8
	Metadata complete = true
	Thumbnail series = true
	-----
	Plane #0 <=> Z 0, C 0, T 0

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/problem-about-opening-some-svs-slides-in-qupath-v0-3-1-bio-formats-6-8-0/61404/17

[sbesson]

Thanks to @petebankhead for the investigation on the source of this issue. https://forum.image.sc/t/problem-about-opening-some-svs-slides-in-qupath-v0-3-1-bio-formats-6-8-0/61404/15 seems to indicate this impacts a subset of SVS files.

As the testing of #3733 suggests, we only have representative SVS sample files in the curated OME QA repository where label and macro are stored in the metadata and allow to identify these ancillary images. Collecting representative samples for the new PFF variant(s?) is definitely a good first step.

An issue here is that reverting #3733 would solve this bug but cause a regression in other use cases.

Solving this issue will require to improve the label/macro detection logic (https://github.com/ome/bioformats/pull/3733/files#diff-314c2ae6ccef06aa694ffa85cf5843ba9a42c62e312a561914894648d61ff09aR305) to handle the different scenarios. For complete reference, the Aperio specification document section related to label and macro images:

Optionally at the end of an SVS file there may be a slide label image, which is a low resolution picture taken of the slide’s label, and/or a macro camera image, which is a low resolution picture taken of the entire slide. The label and macro images are always stripped. If present the label image is compressed with LZW compression, and the macro image with JPEG compression. By storing the label and macro images in the same file as the high resolution slide data there is no chance of an accidental mix-up between the slide contents and the slide label information.

A few initial thoughts:

• check the ImageDescription to distinguish between different implementations - feels brittle
• check the ordering of the resolution dimensions - in the example above, it is clear that 1549 is greater than 1024 and does not correspond to the next resolution level
• check other metadata in agreement with the spec above (compression, strips/tiles)

Additionally, the label/macro detection logic could be controlled via a reader-specific option allowing the user to turn it on/off.

[petebankhead]

Thanks @dgault and @sbesson
Using tiffinfo to look at the example posted on image.sc I get

TIFF Directory at offset 0x114d39fe (290273790)
  Subfile Type: (0 = 0x0)
  Image Width: 62094 Image Length: 36729 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: Aperio Leica Biosystems GT450 v1.0.0 
62094x36729 [0,0,62094x36729] (256x256) JPEG/YCC Q=91|AppMag = 40|Date = 03/17/2021|Exposure Scale = 0.000001|Exposure Time = 8|Filtered = 3|Focus Offset = -0.500000|Gamma = 2.2|Left = 32.792793273926|MPP = 0.263852|Rack = 2|ScanScope ID = SS12042|Slide = 12|StripeWidth = 4096|Time = 13:27:46|Time Zone = GMT+0800|Top = 14.066833496094
  ICC Profile: <present>, 13113264 bytes
TIFF Directory at offset 0x121f3558 (304035160)
  Subfile Type: (0 = 0x0)
  Image Width: 1024 Image Length: 605
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  Samples/Pixel: 3
  Rows/Strip: 605
  Planar Configuration: single image plane
  ImageDescription: Aperio Leica Biosystems GT450 v1.0.0 
1024x605 [0,0,1024x605] (1024x605) JPEG/YCC Q=100|AppMag = 40|Date = 03/17/2021|Exposure Scale = 0.000001|Exposure Time = 8|Focus Offset = 0.000000|Left = 4.3758044242859|MPP = 21.345880|Rack = 2|ScanScope ID = SS12042|Slide = 12|StripeWidth = 4096|Time = 13:27:46|Time Zone = GMT+0800|Top = 32.792793273926
  ICC Profile: <present>, 13113264 bytes
TIFF Directory at offset 0x14082a62 (336079458)
  Subfile Type: (0 = 0x0)
  Image Width: 15523 Image Length: 9182
  Tile Width: 256 Tile Length: 256
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: Aperio Leica Biosystems GT450 v1.0.0 
15523x9182 [0,0,15523x9182] (256x256) JPEG/YCC Q=91|AppMag = 40|Date = 03/17/2021|Exposure Scale = 0.000001|Exposure Time = 8|Filtered = 3|Focus Offset = -0.500000|Gamma = 2.2|Left = 32.792793273926|MPP = 1.055409|Rack = 2|ScanScope ID = SS12042|Slide = 12|StripeWidth = 4096|Time = 13:27:46|Time Zone = GMT+0800|Top = 14.066569328308
  ICC Profile: <present>, 13113264 bytes
TIFF Directory at offset 0x14e58466 (350585958)
  Subfile Type: (0 = 0x0)
  Image Width: 3880 Image Length: 2295
  Tile Width: 256 Tile Length: 256
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: Aperio Leica Biosystems GT450 v1.0.0 
3880x2295 [0,0,3880x2295] (256x256) JPEG/YCC Q=91|AppMag = 40|Date = 03/17/2021|Exposure Scale = 0.000001|Exposure Time = 8|Filtered = 3|Focus Offset = -0.500000|Gamma = 2.2|Left = 32.792793273926|MPP = 4.221636|Rack = 2|ScanScope ID = SS12042|Slide = 12|StripeWidth = 4096|Time = 13:27:46|Time Zone = GMT+0800|Top = 14.064458847046
  ICC Profile: <present>, 13113264 bytes
TIFF Directory at offset 0x15b6e470 (364307568)
  Subfile Type: reduced-resolution image (1 = 0x1)
  Image Width: 685 Image Length: 530
  Bits/Sample: 8
  Compression Scheme: LZW
  Photometric Interpretation: RGB color
  Samples/Pixel: 3
  Rows/Strip: 530
  Planar Configuration: single image plane
  Predictor: horizontal differencing 2 (0x2)
TIFF Directory at offset 0x15b899a0 (364419488)
  Subfile Type: reduced-resolution image (9 = 0x9)
  Image Width: 1549 Image Length: 580
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: RGB color
  Samples/Pixel: 3
  Rows/Strip: 580
  Planar Configuration: single image plane

The subfile type looks potentially useful, as does the (lack of) ImageDescription. Although I am faintly aware that some other software claims to write svs (e.g. 3DHistech's Slide Converter) and I don't know how closely it follows the Aperio specification document.

In case QuPath could be useful for testing, I've just merged a PR that makes it easier to switch between Bio-Formats versions. The application can be launched from the source directory with

./gradlew run -Pbioformats-version=6.8.0

or

./gradlew run -Pbioformats-version=6.7.0

The latest code uses the trick of eliminating the 'impossible' resolutions, i.e. where the downsample value decreases at the next level. However this was still not enough to get things to work in 6.8.0: it seems that the levels that were requested were incorrect.

In the meantime, I plan to make a QuPath release that reverts to 6.7.0 soon.

[dgault]

Reopening this as there is still an ongoing issue in the thread which will require some further investigation: https://forum.image.sc/t/problem-about-opening-some-svs-slides-in-qupath-v0-3-1-bio-formats-6-8-0/61404/26

The file in question is located at https://dbarchive.biosciencedbc.jp/data/open-tggates-pathological-images/LATEST/images/2-nitrofluorene/Kidney/65141.svs

[sbesson]

Cross-linking to Pete's answer on the thread, the way OpenSlide makes the assignment between WSI and label/macro for the sample file mentioned in #3757 (comment) is based on the usage of tiles vs strips. This is in agreement with the relevant line of the specification - see #3757 (comment) and might be the next thing to try out in the reader

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/fiji-wont-recognize-svs-files/63528/2