Read commit timestamps in UTC via linq2db and remove NormalizeTimestamp by myieye · Pull Request #2391 · sillsdev/languageforge-lexbox

myieye · 2026-06-30T08:30:46Z

Makes every DateTimeOffset we persist round-trip to the same UTC instant through both EF Core and linq2db, and removes the hand-rolled HistoryService.NormalizeTimestamp workaround for #2092.

Root cause: lexbox's second UseLinqToDB created a fresh MappingSchema that shadowed Harmony's, dropping Harmony's Commit.HybridDateTime.DateTime UTC converter — so linq2db read commit timestamps in local time and HistoryService reinterpreted the ticks by hand. The fix is to extend Harmony's existing schema (its own get-or-create pattern) instead of replacing it; the converter then applies, the redundant Commit column re-declaration is deleted, and NormalizeTimestamp is gone.

DateTimeOffsetOrmParityTests locks this in: the commit timestamp (EF + both HistoryService linq2db paths) and the plain columns (comment CreatedAt/UpdatedAt, UnreadComment.MarkedUnreadAt) must all report the same UTC instant at offset zero.
These bugs shift the instant by exactly the local UTC offset, so they're invisible on UTC (why Linq2Db uses wrong timezone when reading commit timestamps #2092 reached production). The FwLite test job now runs under Eastern (non-UTC, DST, negative offset) via a Set-TimeZone step; the test fails loudly rather than passing vacuously if ever run on UTC.
Applying Harmony's converter also changed how linq2db renders commit-timestamp parameters, exposing that SnapshotAtCommitService.DeleteCommitsAfter compared timestamps through linq2db — which is millisecond-grained (linq2db wraps SQLite timestamp comparisons in strftime('...%f')) and could delete the target commit itself (the PreserveAllFieldWorksCommits CI failure, ~25% of runs, timezone-independent). WhereAfter now runs through EF like Harmony's own CrdtRepository; linq2db keeps only the Json.Value author filter. The rule is documented in backend/FwLite/AGENTS.md. Bind SQLite timestamp parameters as the exact stored text harmony#78 hardens the parameter rendering upstream — this PR does not depend on it.

🤖 Generated with Claude Code

coderabbitai · 2026-06-30T08:30:55Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 8b9225be-7171-4e23-9f3d-4abbe5673f6d

📥 Commits

Reviewing files that changed from the base of the PR and between 915ca19 and d82cc33.

📒 Files selected for processing (6)

.github/workflows/fw-lite.yaml
backend/FwLite/AGENTS.md
backend/FwLite/LcmCrdt.Tests/DateTimeOffsetOrmParityTests.cs
backend/FwLite/LcmCrdt/HistoryService.cs
backend/FwLite/LcmCrdt/LcmCrdtKernel.cs
backend/FwLite/LcmCrdt/SnapshotAtCommitService.cs

📝 Walkthrough

Walkthrough

This PR fixes DateTimeOffset/timezone parity issues between EF Core and linq2db. It corrects linq2db mapping schema registration, removes a manual timestamp normalization workaround in HistoryService, refactors commit deletion to an ID-based approach, adds parity tests, forces a non-UTC CI timezone, and documents linq2db timestamp comparison caveats.

Changes

Timestamp/timezone parity fix

Layer / File(s)	Summary
LinqToDB mapping schema sharing fix `backend/FwLite/LcmCrdt/LcmCrdtKernel.cs`	Reuses the MappingSchema registered by `UseLinqToDbCrdt` instead of building a new one, throwing if missing, and removes the redundant `AddMappingSchema` call.
Remove NormalizeTimestamp workaround `backend/FwLite/LcmCrdt/HistoryService.cs`	`ProjectActivity` and `GetHistory` use raw `commit.HybridDateTime.DateTime` directly; the `NormalizeTimestamp` helper is deleted.
Refactor DeleteCommitsAfter to ID-based deletion `backend/FwLite/LcmCrdt/SnapshotAtCommitService.cs`	Materializes commit IDs via EF, optionally excludes FieldWorks-authored IDs via linq2db, deletes by ID via `RemoveRange`, and returns the ID count.
Parity tests and CI/docs support `backend/FwLite/LcmCrdt.Tests/DateTimeOffsetOrmParityTests.cs`, `.github/workflows/fw-lite.yaml`, `backend/FwLite/AGENTS.md`	Adds tests validating UTC instant parity for commits, comments, and deletions; forces a non-UTC CI timezone; documents linq2db timestamp precision caveats.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

sillsdev/languageforge-lexbox#2218: Both PRs modify SnapshotAtCommitService.DeleteCommitsAfter's logic for computing and returning deleted commit counts.

Suggested reviewers: rmunn, hahn-kev

Poem

A rabbit hopped through timezones tricky,
Found timestamps drifting, offsets sticky.
Now EF and linq2db agree as one,
UTC instants, no more undone.
🕐🐇 Hop, test, and ship — the fix is done!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title matches the main change: UTC commit timestamp handling and removal of NormalizeTimestamp.
Description check	✅ Passed	The description accurately summarizes the timestamp fixes, tests, timezone change, and deletion logic update.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch claude/date-parity-comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

hahn-kev

nice catch. I have some suggestions.

Locks in that every persisted DateTimeOffset round-trips to the same UTC instant through both EF Core and linq2db (#2092). linq2db skips the converter for the Harmony commit timestamp so HistoryService normalizes it by hand; comment dates use the global EF converter, which linq2db also applies. These errors equal the local UTC offset, so they vanish on UTC — hence the FwLite test job now runs under Eastern time (non-UTC, DST, negative offset). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

lexbox's second UseLinqToDB created a fresh MappingSchema that shadowed Harmony's, dropping Harmony's Commit.HybridDateTime.DateTime UTC converter. linq2db then read commit timestamps in local time, so HistoryService normalized them by hand (#2092). Extend Harmony's existing schema instead of replacing it: the converter now applies, the redundant Commit column re-declaration is gone, and HistoryService.NormalizeTimestamp is deleted. The parity test now also covers UnreadComment.MarkedUnreadAt and fails loudly instead of a silent no-op when run on UTC. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Collapse the repetitive per-column tuples into a generic AssertColumnsAreUtc<T> helper, and add a DeletedAt test that reads it from the deleted snapshot (EntityIsDeleted column) since it is never non-null in a projected column. Tightened comments. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

All plain DateTimeOffset columns share one global converter, so testing many proves nothing extra. Drop the generic helper and MarkedUnreadAt; keep one comment timestamp, the commit timestamp, and DeletedAt — the three distinct mechanisms. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Explain the conditional AddMappingSchema (UseLinqToDbCrdt already registered the schema); drop the duplicated DeletedAt note from the test header and state it once at the call site, including why that test isn't gated on a non-UTC zone. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

- Move RequireNonUtc() into InitializeAsync so it guards every test in the class (per hahn-kev); DeletedAt is now gated too, fine since it still round-trips correctly under a non-UTC zone. - Add an explicit linq2db read of the commit timestamp so the linq2db path stays covered even if HistoryService ever moves to EF. - Set the comment thread CreatedAt to a known non-UTC-offset instant and assert it exactly, testing that the write is recorded (not just the round-tripped return). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Falling back to a fresh MappingSchema silently dropped Harmony's Commit UTC conversion, reintroducing local-time timestamps (#2092). Fail loudly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

myieye · 2026-07-06T10:03:58Z

Rebased and resolved merge conflicts

argos-ci · 2026-07-06T10:06:51Z

The latest updates on your projects. Learn more about Argos notifications ↗︎

Build	Status	Details	Updated (UTC)
default (Inspect)	✅ No changes detected	-	Jul 7, 2026, 8:11 AM

linq2db renders timestamp parameters truncated to milliseconds while SQLite's strftime (which linq2db wraps around every timestamp comparison) rounds to the nearest millisecond, so WhereAfter via ToLinqToDB() classified targetCommit as after itself and deleted it — the PreserveAllFieldWorksCommits CI failure (~25% of runs, timezone-independent; reproducible on this branch, not on develop, because reusing Harmony's mapping schema changed how the parameter renders). sillsdev/harmony#78 fixes the parameter rendering, but linq2db comparisons stay millisecond-grained, so exact commit ordering belongs in EF regardless. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

CreateCommentThread requires a known user identity since Comments (#2382). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

github-actions Bot added the 💻 FW Lite issues related to the fw lite application, not miniLcm or crdt related label Jun 30, 2026

myieye mentioned this pull request Jun 30, 2026

Comments #2382

Merged

myieye changed the title ~~Add date-parity test and run FwLite tests under a non-UTC timezone~~ Read commit timestamps in UTC via linq2db and remove NormalizeTimestamp Jun 30, 2026

hahn-kev reviewed Jul 1, 2026

View reviewed changes

Comment thread backend/FwLite/LcmCrdt.Tests/DateTimeOffsetOrmParityTests.cs Outdated

hahn-kev requested changes Jul 1, 2026

View reviewed changes

Comment thread backend/FwLite/LcmCrdt.Tests/DateTimeOffsetOrmParityTests.cs

Comment thread backend/FwLite/LcmCrdt.Tests/DateTimeOffsetOrmParityTests.cs Outdated

myieye requested a review from hahn-kev July 1, 2026 12:59

hahn-kev reviewed Jul 2, 2026

View reviewed changes

Comment thread backend/FwLite/LcmCrdt/LcmCrdtKernel.cs Outdated

hahn-kev approved these changes Jul 2, 2026

View reviewed changes

Base automatically changed from comment-model to develop July 6, 2026 09:11

myieye and others added 7 commits July 6, 2026 12:00

Throw when Harmony's linq2db mapping schema is missing

1d8ffe9

Falling back to a fresh MappingSchema silently dropped Harmony's Commit UTC conversion, reintroducing local-time timestamps (#2092). Fail loudly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

myieye force-pushed the claude/date-parity-comments branch from 2cb4964 to 1d8ffe9 Compare July 6, 2026 10:03

myieye mentioned this pull request Jul 7, 2026

Bind SQLite timestamp parameters as the exact stored text sillsdev/harmony#78

Closed

myieye and others added 2 commits July 7, 2026 10:08

Set a last user in the comment timestamp parity test

d82cc33

CreateCommentThread requires a known user identity since Comments (#2382). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

myieye marked this pull request as ready for review July 7, 2026 08:09

myieye merged commit 431434b into develop Jul 7, 2026
21 checks passed

myieye deleted the claude/date-parity-comments branch July 7, 2026 13:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Read commit timestamps in UTC via linq2db and remove NormalizeTimestamp#2391

Read commit timestamps in UTC via linq2db and remove NormalizeTimestamp#2391
myieye merged 9 commits into
developfrom
claude/date-parity-comments

myieye commented Jun 30, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jun 30, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

Uh oh!

hahn-kev left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

myieye commented Jul 6, 2026

Uh oh!

argos-ci Bot commented Jul 6, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

myieye commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

Uh oh!

hahn-kev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

myieye commented Jul 6, 2026

Uh oh!

argos-ci Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

myieye commented Jun 30, 2026 •

edited

Loading

coderabbitai Bot commented Jun 30, 2026 •

edited

Loading

argos-ci Bot commented Jul 6, 2026 •

edited

Loading