Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
623f158
try to correct resume messages with missing checkpoint
nicktrn Oct 17, 2024
11066b4
prevent creating checkpoints for outdated task waits
nicktrn Oct 17, 2024
f2b5c2a
prevent creating checkpoints for outdated batch waits
nicktrn Oct 17, 2024
d756a16
use heartbeats to check for and clean up any leftover containers
nicktrn Oct 17, 2024
5364558
Merge remote-tracking branch 'origin/main' into fix/invalid-resume-me…
nicktrn Oct 17, 2024
df15d6a
lint
nicktrn Oct 17, 2024
e003d25
improve exec logging
nicktrn Oct 17, 2024
9af6018
improve resume attempt logs
nicktrn Oct 17, 2024
4c8618d
fix for resuming parents of canceled child runs
nicktrn Oct 17, 2024
12ad920
separate SIGTERM from maybe OOM errors
nicktrn Oct 17, 2024
13faa69
Merge remote-tracking branch 'origin/main' into fix/resume-restore-bugs
nicktrn Oct 17, 2024
a9928be
pretty errors can have magic dashboard links
nicktrn Oct 17, 2024
2d84b7c
prevent uncancellable checkpoints
nicktrn Oct 17, 2024
34d9759
simplify task run error code enum export
nicktrn Oct 18, 2024
89ec5c8
grab the last, not the first child run
nicktrn Oct 18, 2024
5c262fd
Revert "prevent creating checkpoints for outdated batch waits"
nicktrn Oct 18, 2024
e6afbb4
Revert "grab the last, not the first child run"
nicktrn Oct 18, 2024
40d80f9
Revert "prevent creating checkpoints for outdated task waits"
nicktrn Oct 18, 2024
59d375b
more logs for resume message handling
nicktrn Oct 18, 2024
3604d83
add magic error link comment
nicktrn Oct 18, 2024
cdbf5c6
add changeset
nicktrn Oct 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
prevent uncancellable checkpoints
  • Loading branch information
nicktrn committed Oct 17, 2024
commit 2d84b7cdc41ba9c9e10fbccbddb00219b6486aa0
5 changes: 4 additions & 1 deletion apps/coordinator/src/checkpointer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -436,7 +436,10 @@ export class Checkpointer {
this.#logger.error("Error during cleanup", { ...metadata, error });
}

this.#abortControllers.delete(runId);
// Ensure only the current controller is removed
if (this.#abortControllers.get(runId) === controller) {
this.#abortControllers.delete(runId);
}
controller.signal.removeEventListener("abort", onAbort);
};

Expand Down