perf: cross-layer performance pass (dashboard, Python, Rust)#213
Merged
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds a persistent ChangesDependency tracking and storage optimization
Python middleware caching and serialization optimization
Dashboard React Query and component performance
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsStopped waiting for pipeline failures after 30000ms. One of your pipelines takes longer than our 30000ms fetch window to run, so review may not consider pipeline-failure results for inline comments if any failures occurred after the fetch window. Increase the timeout if you want to wait longer or run a Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/taskito-core/src/storage/postgres/mod.rs`:
- Around line 127-129: The loop calling common_migrations::backfill_statements()
currently calls migration_alter(&mut conn, sql) which swallows errors; change
this so backfill failures are propagated instead of only logged: update
migration_alter to return a Result (or use an existing fallible variant), and in
the loop check its Result and return Err (or propagate with ?) including the SQL
and underlying error if any; ensure the enclosing function's signature returns
Result so failures abort startup/migration rather than being ignored.
In `@crates/taskito-core/src/storage/sqlite/mod.rs`:
- Around line 118-120: The backfill loop currently calls migration_alter(&mut
conn, sql) and swallows failures; change this to fail fast by checking
migration_alter's result and returning or propagating an error (or panicking)
when it fails so startup doesn't continue with bad data. Specifically, in the
loop over common_migrations::backfill_statements() in mod.rs, replace the blind
call to migration_alter with error handling that inspects its return (or update
migration_alter to return a Result if it doesn't) and propagate a failure (e.g.,
return Err(...) from the surrounding init function or call panic!) when
migration_alter indicates an error so has_deps/backfill never silently fails.
In `@dashboard/src/components/layout/last-refreshed.tsx`:
- Around line 28-32: The scheduled timeout in schedule() currently uses coarse
fixed bucket durations and can let labels stay stale; replace the fixed delay
calculation with a delay that aligns to the next label boundary based on age: if
age < 60_000 use 1000 - (age % 1000), else if age < 3_600_000 use 60_000 - (age
% 60_000), otherwise use 3_600_000 - (age % 3_600_000). Update the id =
setTimeout(...) call in the same schedule() block so setTick and the recursive
schedule() run when the next label change actually occurs.
In `@py_src/taskito/app.py`:
- Around line 695-708: The batch-builder currently builds per-job option arrays
from mixed scalar or list inputs (delay_list, metadata_list, notes_list,
expires_list, result_ttl_list, unique_keys, idempotency_keys) without verifying
that any provided lists match the number of jobs (count or len(args_list)),
which allows mismatched lengths to produce confusing downstream errors (e.g., in
_inner.enqueue_batch). Before constructing notes_encoded and the other _src
lists, validate each provided per-job list (those ending with _list or the
unique_keys/idempotency_keys lists) has length == count; if any mismatch raise a
clear ValueError indicating which parameter has the wrong length and what the
expected length (count) is so callers get immediate, actionable feedback.
In `@py_src/taskito/mixins/decorators.py`:
- Around line 156-158: The middleware filtering in _get_middleware_chain
currently only checks mw.name, so dashboard disables that use class-path keys
don't match unnamed middleware; convert the incoming disabled iterable to a set
of strings and when filtering the chain check both getattr(mw, "name", "") and
the middleware class path (mw.__class__.__module__ + "." +
mw.__class__.__name__) against that set, removing any mw that matches either key
so unnamed middleware can be disabled by class path as admin discovery does.
In `@py_src/taskito/proxies/reconstruct.py`:
- Around line 30-35: The current shared ThreadPoolExecutor (_RECONSTRUCT_POOL)
cannot stop already-running reconstruction work because
Future.result(timeout=...) only stops waiting and future.cancel() won't kill
started threads; change the reconstruction to run in an isolated process that
can be terminated on timeout: replace the ThreadPoolExecutor-based
submit+future.result(timeout=...) usage with a per-task multiprocessing.Process
(or spawn via multiprocessing.get_context("spawn")), start the process for the
reconstruction function, join with the desired timeout, and if the join timed
out call process.terminate() and process.join() to free the worker; ensure the
reconstruction caller checks for a terminated process and raises/returns a
timeout error. Reference symbols: _RECONSTRUCT_POOL, ThreadPoolExecutor,
Future.result, future.cancel.
In `@tests/core/test_middleware_chain_cache.py`:
- Around line 16-33: The test intermittently fails because the middleware-chain
cache TTL (_MW_CHAIN_TTL) can expire during the loop; to fix, in
test_middleware_chain_caches_disable_reads set or monkeypatch the TTL or the
module's monotonic time so the TTL cannot elapse during the 50 iterations (e.g.,
assign a large value to Queue._MW_CHAIN_TTL or the module-level _MW_CHAIN_TTL,
or patch time.monotonic to a fixed value) before creating the Queue and running
the loop, then restore the original value if needed; target symbols:
test_middleware_chain_caches_disable_reads, Queue, _get_middleware_chain, and
_MW_CHAIN_TTL.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b8211a09-c927-42aa-a142-e459ca9d43bb
📒 Files selected for processing (37)
crates/taskito-core/src/job.rscrates/taskito-core/src/scheduler/poller.rscrates/taskito-core/src/storage/diesel_common/archival.rscrates/taskito-core/src/storage/diesel_common/dead_letter.rscrates/taskito-core/src/storage/diesel_common/jobs.rscrates/taskito-core/src/storage/diesel_common/migrations.rscrates/taskito-core/src/storage/models.rscrates/taskito-core/src/storage/postgres/mod.rscrates/taskito-core/src/storage/redis_backend/jobs/dequeue.rscrates/taskito-core/src/storage/schema.rscrates/taskito-core/src/storage/sqlite/mod.rscrates/taskito-core/src/storage/sqlite/tests.rscrates/taskito-python/src/async_worker.rscrates/taskito-python/src/py_worker.rsdashboard/src/components/layout/last-refreshed.tsxdashboard/src/components/ui/data-table.tsxdashboard/src/features/circuit-breakers/hooks.tsdashboard/src/features/jobs/components/job-overview-tab.tsxdashboard/src/features/jobs/components/job-table.tsxdashboard/src/features/overview/components/throughput-sparkline.tsxdashboard/src/features/resources/hooks.tsdashboard/src/features/tasks/hooks.tsdashboard/src/features/workers/hooks.tsdashboard/src/lib/query-client.tsdashboard/src/providers/theme-provider.tsxdashboard/src/routes/jobs/$id.tsxpy_src/taskito/app.pypy_src/taskito/async_support/executor.pypy_src/taskito/interception/registry.pypy_src/taskito/mixins/decorators.pypy_src/taskito/mixins/middleware_admin.pypy_src/taskito/notes.pypy_src/taskito/proxies/reconstruct.pypy_src/taskito/serializers.pypy_src/taskito/webhooks.pytests/core/test_middleware_chain_cache.pytests/worker/test_native_async.py
Collaborator
Author
|
Addressed all 7 review findings:
Verified: |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
py_src/taskito/dashboard/handlers/middleware.py (1)
50-53:⚠️ Potential issue | 🟠 Major | ⚡ Quick winUse
middleware_key()for PUT registration checksLine 51 validates against raw
mw.name, which can diverge from the keys returned by GET/list endpoints (now usingmiddleware_key). This can incorrectly 404 valid middleware toggles.Suggested fix
- names = {getattr(mw, "name", "") for mw in base_chain} + names = {middleware_key(mw) for mw in base_chain}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@py_src/taskito/dashboard/handlers/middleware.py` around lines 50 - 53, The PUT registration check currently builds names from mw.name which may differ from the canonical keys; update the check to use middleware_key(mw) instead: when constructing names from base_chain (from queue._global_middleware and queue._task_middleware.get(task_name, [])) call middleware_key on each middleware and compare mw_name against that set, leaving the _NotFound raising logic (and variables mw_name, task_name, base_chain) unchanged so valid middleware keys are not incorrectly 404ed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@py_src/taskito/dashboard/handlers/middleware.py`:
- Around line 50-53: The PUT registration check currently builds names from
mw.name which may differ from the canonical keys; update the check to use
middleware_key(mw) instead: when constructing names from base_chain (from
queue._global_middleware and queue._task_middleware.get(task_name, [])) call
middleware_key on each middleware and compare mw_name against that set, leaving
the _NotFound raising logic (and variables mw_name, task_name, base_chain)
unchanged so valid middleware keys are not incorrectly 404ed.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 7d5f4b07-9795-4e16-8d51-89de2a2f4265
📒 Files selected for processing (10)
crates/taskito-core/src/storage/postgres/mod.rscrates/taskito-core/src/storage/sqlite/mod.rsdashboard/src/components/layout/last-refreshed.tsxpy_src/taskito/app.pypy_src/taskito/dashboard/handlers/middleware.pypy_src/taskito/middleware.pypy_src/taskito/mixins/decorators.pypy_src/taskito/mixins/middleware_admin.pypy_src/taskito/proxies/reconstruct.pytests/core/test_middleware_chain_cache.py
🚧 Files skipped from review as they are similar to previous changes (5)
- tests/core/test_middleware_chain_cache.py
- dashboard/src/components/layout/last-refreshed.tsx
- py_src/taskito/app.py
- py_src/taskito/mixins/decorators.py
- py_src/taskito/proxies/reconstruct.py
A shared ThreadPoolExecutor force-joins its workers at interpreter exit (CPython issue 36780), so a hung reconstruction would block shutdown, and a bounded pool can be exhausted by hung tasks. A process pool can't return the non-picklable live objects reconstruction produces. Run each timed reconstruction on a daemon thread joined with the timeout instead: an overrun is abandoned without blocking the worker or shutdown. Adds timeout tests.
This was referenced May 29, 2026
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A low-risk-plus-moderate performance pass across all three layers, targeting per-job / per-enqueue / per-render hot paths. 17 focused commits, no public API or contract changes.
Dashboard — render & refetch overhead
refetchOnWindowFocusand add per-querystaleTimefor slow-moving data (no thundering-herd refetch on tab focus)JobTableerror-column scan, the throughput sparkline geometry (paths no longer recompute on hover; uniqueuseIdgradient), and theJobIntegrationslinks arrayReact.memo(DataTable)+ stable row callbacks; adaptiveLastRefreshedinterval (no more forced 1s header re-render); lazy-enable job-detail tab queries;ThemeProviderreadslocalStorageoncePython — enqueue & dispatch hot paths
cloudpickle.loads, ignoring@task(serializer=...); it now goes through_deserialize_payloadlike the sync pathenqueue/enqueue_manywhen no middleware is registeredThreadPoolExecutorper proxy arg); webhooknotifyno-subscriber fast path;cloudpicklebound once; interceptionTypeRegistryper-type resolution cacheRust core — storage, scheduler, worker
jobs.has_depscolumn (centralized migration + idempotent backfill) so dequeue skips the dependency lookup entirely for the common no-dependency job (SQLite/Postgres + Redis)reap_stale_jobsandpurge_completed_with_ttlpush the deadline / TTL predicate into SQL — they no longer load every candidate row's payload/result blobs just to filter in Rustenqueue_batchuses a single chunked multi-row INSERT instead of N single-row inserts (chunked to stay under SQLite's bound-parameter limit)MGETand reuses the connection for dependency checks; single GIL acquisition on the worker failure path;active_queuesborrows the queue list when nothing is pausedVerification
cargo check(default / postgres / redis / native-async),clippy --all-targets, 62 core tests (incl. 4 new regression tests for has_deps, reap, purge, batch chunking)Notes / scope
stats_by_queuewas intentionally not narrowed to active statuses — it also feeds the dashboard's completed/failed/dead counts, so narrowing would be a correctness regression.queuefield toJobResult::Failure(13 construction sites across 3 crates), PyO3 module-object caching, and Redis per-queue/per-task counter re-modeling.cargo fmthook normalized pre-existing signature-wrapping drift injobs.rs.Summary by CodeRabbit
Performance Improvements
Database & Migrations
Middleware & SDK
Dashboard Improvements
Tests