Skip to content

[2/n] be more rigorous about when events are recorded in-memory vs durably#461

Open
sunshowers wants to merge 1 commit into
sunshowers/spr/main.2n-be-more-rigorous-about-when-events-are-recorded-in-memory-vs-durablyfrom
sunshowers/spr/2n-be-more-rigorous-about-when-events-are-recorded-in-memory-vs-durably
Open

[2/n] be more rigorous about when events are recorded in-memory vs durably#461
sunshowers wants to merge 1 commit into
sunshowers/spr/main.2n-be-more-rigorous-about-when-events-are-recorded-in-memory-vs-durablyfrom
sunshowers/spr/2n-be-more-rigorous-about-when-events-are-recorded-in-memory-vs-durably

Conversation

@sunshowers

Copy link
Copy Markdown

This fixes at least three different races, all related to #17.

Previously, the main executor loop and the worker task both updated parts of in-memory state separately. During this brief window of divergence, it was possible to observe torn state in a few different ways. See the included tests for some reproductions.

With this change, we make it so that:

  • in-memory state is always changed by the executor loop, atomically (while the lock is held)
  • durable state is always changed by the worker

For start events, in-memory state is updated before durable state. For completion events, durable state is updated before in-memory state. This seems like the most reasonable way to do things.

I had to introduce some additional injection infrastructure so tests can reproduce the races deterministically.

See the comments on #17 for detailed design discussion.

Created using spr 1.3.6-beta.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant