Skip to content

Add client-side database sync infrastructure#418

Merged
arhowe00 merged 1 commit into
mainfrom
cloud
Feb 27, 2026
Merged

Add client-side database sync infrastructure#418
arhowe00 merged 1 commit into
mainfrom
cloud

Conversation

@posnova
Copy link
Copy Markdown

@posnova posnova commented Feb 13, 2026 •
edited by arhowe00
Loading

Noclose: there was no associated issue with this pull request.

Adds infrastructure for synchronizing openlifu database objects (protocols, subjects, sessions, volumes, transducers, solutions, runs, photoscans) with a cloud backend. Includes a rest client and bidirectional syncing logic

@posnova posnova requested a review from arhowe00 February 13, 2026 20:02
arhowe00
arhowe00 previously requested changes Feb 16, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@arhowe00 arhowe00 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Before merging, please ensure:

  1. CI passes (at minimum, linting must pass)
  2. Commits should follow the guidelines in CONTRIBUTING.md, specifically referencing issue numbers in commit titles or bodies. Alternatively, let us know if you're okay with me squash merging it

As a minor note: some brief documentation on the cloud sync model would help future maintainers. Many of the classes are repetitive and self-documenting and don't need comments. But some of the database DTO members like owner_uid, connected_system_id, and *_sync_date timestamps aren't immediately understood from variable names in the class definitions

Comment thread src/openlifu/cloud/api/databases_api.py
response = self._request.put("/databases/claim", dto)
return from_json(DatabaseDto, response)

def release_database(self, database_id: int):
Copy link
Copy Markdown
Contributor

@arhowe00 arhowe00 Feb 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't called anywhere - is it server-side timeout?

Comment thread src/openlifu/cloud/api/dto.py


@dataclass
class ClaimDbDto:
Copy link
Copy Markdown
Contributor

@arhowe00 arhowe00 Feb 16, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ownership model here isn't clear (without seeing server-side code), and documentation or at least a schematic to reference would help, e.g. brief comments explaining the intended semantics (especially around owner_uid and concurrent clients)

Comment thread src/openlifu/cloud/components/solutions.py
if not remote_id:
config = json.loads(data.decode())
protocol_local_id = config["protocol_id"]
transducer_local_id = config["transducer_id"]
Copy link
Copy Markdown
Contributor

@arhowe00 arhowe00 Feb 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the abstract_component.py reads the JSON from disk, that json file would have the entire transducer as the value and not just the id

@arhowe00 arhowe00 changed the title Cloud Add cloud database synchronization infrastructure Feb 16, 2026
@arhowe00 arhowe00 changed the title Add cloud database synchronization infrastructure Add client-side database sync infrastructure Feb 16, 2026
@arhowe00
Copy link
Copy Markdown
Contributor

arhowe00 commented Feb 19, 2026

Looks like linting passes. @dev-et , let me know when you're ready for things to merge and if you're okay with a squash merge. @peterhollender feel free to chime in here if you've gotten information on what is preferred and whether it's ready to be merged.

@peterhollender
Copy link
Copy Markdown
Contributor

peterhollender commented Feb 21, 2026

Some of the backend syncing is still under development, so we'll hold off on merging it until we've got the kinks ironed out. Either I or @posnova will ping you when it's ready!

@arhowe00 arhowe00 enabled auto-merge (squash) February 27, 2026 18:42
@arhowe00 arhowe00 disabled auto-merge February 27, 2026 18:42
@arhowe00 arhowe00 dismissed their stale review February 27, 2026 18:43

Review is dismissed to prioritize merging

@arhowe00 arhowe00 self-requested a review February 27, 2026 18:43
@arhowe00 arhowe00 enabled auto-merge (squash) February 27, 2026 18:45
@arhowe00
Add client-side database sync infrastructure (#418)
d68db32 
Add openlifu.cloud module for bidirectional sync between local OpenLIFU
databases and the cloud API (api.openwater.health). Introduces a REST
API client layer with typed DTOs for all entity types (subjects,
sessions, protocols, systems, transducers, volumes, runs, solutions,
users, photoscans). Sync engine uses a component tree
(AbstractComponent) with per-entity-type subclasses, a debounced worker
thread (SyncThread), filesystem watching via watchdog, and
WebSocket-based push notifications for real-time cloud-to-local
updates. Volumes and photoscans are download-only (pulled from cloud
then deleted server-side); all other entities sync bidirectionally.
Adds watchdog and python-socketio[client] dependencies.

Squashed commit of the following:

commit c11402d
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Mon Feb 23 10:43:55 2026 +0200

    fix: endless sync loop

commit 3894d67
Author: dev-et <dev-et>
Date:   Fri Feb 20 18:15:26 2026 -0800

    Set daemon to false

commit 565ef1c
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Fri Feb 20 18:12:35 2026 +0200

    fix: don't delete photoscans on cloud

commit 871188e
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Fri Feb 20 13:10:32 2026 +0200

    fix: download only finished photoscans

commit 7f490c2
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Fri Feb 20 12:51:23 2026 +0200

    fix: don't sync photocollections and don't upload volumes to cloud

commit 54cb278
Author: dev-et <dev-et>
Date:   Thu Feb 19 21:59:55 2026 -0800

    Disable daemon for sync thread

commit 6e91aa4
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Thu Feb 19 15:36:55 2026 +0200

    fix: getting MAC address on Windows

commit f3c104e
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Thu Feb 19 10:27:20 2026 +0200

    fix: disable photocollections sync

commit 63c2900
Author: dev-et <dev-et>
Date:   Wed Feb 18 10:08:06 2026 -0800

    Add temporary hardcoded mac address

commit 2b07ec9
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Wed Feb 18 13:57:25 2026 +0200

    fix: resolve lint errors

commit 9d41092
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Wed Feb 18 13:17:29 2026 +0200

    fix: resolve lint errors

commit 1edb8a5
Author: dev-et <dev-et>
Date:   Mon Feb 16 21:03:25 2026 -0800

    Add exception handling for errorous files

commit 7d4e84b
Author: dev-et <dev-et>
Date:   Mon Feb 16 01:37:06 2026 -0800

    Bug fixes, add additional logging, fix deadlock in sync_thread

commit d8f2894
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Fri Feb 13 15:19:33 2026 +0200

    One way sync for volumes, photocollections and photoscans and delete them on cloud after sync.

commit 174ba08
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Fri Feb 13 12:07:48 2026 +0200

    Fixes photocollection sync when photo added locally

commit 366ef47
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Fri Feb 13 11:50:10 2026 +0200

    Don't block the main thread when stopping the cloud

commit a28c622
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Thu Feb 12 13:34:54 2026 +0200

    Fixes sync error

commit 20da259
Author: dev-et <dev-et>
Date:   Mon Feb 9 13:09:39 2026 -0800

    Fix NameError: name 'AbstractComponent' is not defined exception

commit 08731b8
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Fri Feb 6 16:51:15 2026 +0200

    Don't delete database owner automatically when sync library stops

commit 20d7a48
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Fri Feb 6 15:10:06 2026 +0200

    Photoscans sync

commit 253dc32
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Fri Feb 6 12:47:46 2026 +0200

    Synchronization fixes

commit d84c2d0
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Thu Feb 5 18:25:17 2026 +0200

    Synchronization of sessions, volumes, photocollections, runs and solutions

commit 75103b3
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Mon Feb 2 11:17:32 2026 +0200

    update API URL

commit 8c01179
Author: Pavel Osnova <pvosnova@gmail.com>
Date:   Fri Jan 30 12:21:45 2026 +0200

    initial cloud library
@arhowe00 arhowe00 disabled auto-merge February 27, 2026 19:01
arhowe00
arhowe00 approved these changes Feb 27, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@arhowe00 arhowe00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution @dev-et ! Since linting and CI pass and this is a feature needed to be released soon, I'm going to go ahead and merge this. I've left my review comments as "unresolved" in case we would like to return to this in the future. I've squashed all your commits and added a summary at the top.

@arhowe00 arhowe00 merged commit abc30a4 into main Feb 27, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@arhowe00 arhowe00 arhowe00 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@posnova @arhowe00 @peterhollender