QuPath with CUDA doesn’t release GPU memory after StarDist segmentation Usage & Issues qupath

[MarkZaidi]

QuPath is not closing the loaded StarDist model when running on a GPU (QuPath built with CUDA). Each time a StarDist script is run, the model will be repeatedly loaded but never closed, leading to an out-of-memory exception when requesting memory greater than the available memory remaining in the VRAM.

Steps to reproduce outlined in: https://forum.image.sc/t/qupath-with-cuda-doesnt-release-gpu-memory-after-stardist-segmentation/57462

[petebankhead]

Thanks @MarkZaidi my guess is that DnnModel will need to be updated to implement AutoCloseable in QuPath v0.4.0, so that there is a standardised way to support cleanup.

In the meantime I'll try to investigate if there is a workaround.

[MarkZaidi]

In the 0.3.1 change log (https://github.com/qupath/qupath/blob/main/CHANGELOG.md#version-031) it seems to state that this has been fixed (at least from my understanding), however I'm able to reproduce the same out-of-memory error as in https://forum.image.sc/t/qupath-with-cuda-doesnt-release-gpu-memory-after-stardist-segmentation/57462 by running a StarDist script on an image over and over until no VRAM is available.

Previously, the fix was to add dnn.getPredictionFunction().net.close() after the segmentation has been performed, which still resolves the issue in 0.3.1 just as it had in previous versions with no issues whatsoever.

So my question is, what's changed in 0.3.1 in relation to the following change log entry?

DnnModel implements AutoCloseable, so that calling DnnModel.close() can resolve

    GPU memory not freed when using OpenCV DNN (https://github.com/qupath/qupath/issues/841)
    QuPath with CUDA doesn’t release GPU memory after StarDist segmentation (https://github.com/qupath/qupath-extension-stardist/issues/11)

[petebankhead]

v0.3.1 just adds the core change so that this issue can be resolved, but I still need to find time to update the extension to use this (which is why this issue remains open).

From the perspective of the workaround, you should now be able to call dnn.close() - which is a public method - rather than needing to access private fields and methods to close the model. Or alternatively use the DnnModel within a try-with-resources block.