Enable WAL mode - #4644
Open
uclaros wants to merge 6 commits into
Open
Conversation
QGIS will now enable accessing geopackages in WAL mode
…esent. Copy uses sqlite backup to make sure that WAL contents are also copied along. Rename tries to also rename -wal and -shm files
Coverage Report for CI Build 31395445511Coverage increased (+0.08%) to 59.179%Details
Uncovered ChangesNo uncovered changes found. Coverage Regressions49 previously-covered lines in 4 files lost coverage.
Coverage Stats
💛 - Coveralls |
📦 Build Artifacts Ready
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
QGIS uses WAL mode for accessing local geopackages.
By removing the
OGR_SQLITE_JOURNAl=DELETEwe let QGIS use that default.Geopackages are opened in WAL mode so now readers do not block writers, fixing the issue that geodiff would find the database locked and create conflicted copies.
One thing to be aware of is that now commits may live in the
.gpkg-walfile until a sqlite checkpoint operation takes place writing the.gpkg-walcontents into the.gpkgfile, or the provider has closed the file. We must therefore treat the three files (.gpkg,.gpkg-wal,.gpkg-shm) as a group (hello shapefile!) when copying/renaming at the os level.Geodiff has a
GEODIFF_makeCopySqlite()function for copying gpkgs, which opens the source and creates a backup on the destination, so practically implicitly checkpointing along the way. This has been wrapped inGeodiffUtils::copyWalAware()and we need to be using this whenever we need to copy a geopackage that was opened and written to by qgis, like project layer geopackage, however we don't need to do it for copying the base file from.merginfolder or the diff geopackages.When renaming such gpkgs we can either do
GeodiffUtils::copyWalAware()and then delete the source (adding the overhead of a full copy) or useGeodiffUtils::renameWalAware()which bluntly tries to also rename-waland-shmfiles (this was chosen for this proposed implementation to avoid the copy overhead, however the first solution is probably the safest choice).Fixes #4577
Fixes #4583
Hopefully fixes #3866 (not tested)