Skip to content

sqlite fixtures unlink the file without closing the connection, flaky on Windows #11

Description

@simontaurus

Symptom

test_local_db_periodic_cleanup errors at teardown on Windows runners:

ERROR at teardown of test_local_db_periodic_cleanup
E   PermissionError: [WinError 32] The process cannot access the file
    because it is being used by another process:
    'C:\Users\RUNNER~1\AppData\Local\Temp\tmpXXXXXXXX.sqlite'
tests\test_controller.py:87: PermissionError

The tests themselves pass: 141 passed, 11 skipped, 1 error. Only the
teardown fails, so the job goes red on an otherwise green run.

Seen on the v0.42.8.post1000002004006 tag build, on test (3.11, windows-latest), after the same commit had passed on the PR and on
main. Re-running the job made it pass, which is what makes it a nuisance
rather than a blocker.

Cause

Both sqlite fixtures unlink the temp file without closing the controller
first:

    db = LocalTimeSeriesDatabaseController(..., db_path=path)
    yield db
    os.unlink(path)          # tests/test_controller.py:87

and the v1 variant at tests/test_controller.py:608. Windows refuses to
unlink a file with an open handle, so whether teardown succeeds depends
on whether the connection, or the periodic cleanup task the test starts,
has been finalised yet. On POSIX the unlink succeeds regardless, which is
why it only shows up on Windows and only sometimes.

Suggested fix

Close the connection in the fixture before unlinking, and tolerate a
failed unlink:

    yield db
    await db.close()          # or the sync equivalent
    with contextlib.suppress(PermissionError, FileNotFoundError):
        os.unlink(path)

The periodic_cleanup task started by the test should also be cancelled
and awaited before teardown, otherwise it can re-open the database after
the close.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions