Skip to content

FileResourceManagers inadvertently leave directories open #594

@petebankhead

Description

@petebankhead

Bug report

Describe the bug

The following code in FileResourceManager leaves an open connection to the directory:

return Files.list(path)
	.filter(p -> Files.isRegularFile(p) && p.toString().endsWith(ext))
	.collect(Collectors.toMap(p -> nameWithoutExtension(p, ext), p -> p));

It was expected that the collect terminal operation would close the connection... it it seems it does not.

This causes real-world problems when running scripts for many images, e.g. see https://forum.image.sc/t/error-too-many-open-files/41628

To Reproduce
Steps to reproduce the behavior:

  1. Create a pixel or object classifier
  2. Run a script using loadPixelClassifier('Anything')
  3. Check for open files... on macOS this can be through Activity monitor or lsof

Each time the script is called, the directory containing the pixel classifiers is left open.

Expected behavior
Directories aren't left open unnecessarily.

Desktop (please complete the following information):

  • QuPath v0.2 -- most problematic on Linux, where the number of open files is limited.

Additional context
The error can be easily fixed by using the following code instead:

try (var stream = Files.list(path)) {
	return stream.filter(p -> Files.isRegularFile(p) && p.toString().endsWith(ext))
		.collect(Collectors.toMap(p -> nameWithoutExtension(p, ext), p -> p));
}

However, the code should be checked for other instances of this pattern. Also, DefaultProject should perhaps cache resource managers rather than creating them anew on each request.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions