Add overlap-aware vessel skeleton graph export (skeleton + SDT → sparse parquet)

[Copilot]

• Triage new PR comment and confirm actionable scope
• Check latest CI/workflow failure context and logs
• Reproduce the graph error locally using newly pushed example NIfTI inputs
• Identify root cause in skeleton_graph_from_sdt runtime path
• Apply minimal fix and add/adjust targeted regression tests
• Run targeted tests and relevant quality checks
• Run parallel validation and address findings
• Reply to PR comment with commit hash

[akhanf]

@copilot: encountering an error when building the graph. I've fixed the dependencies and added a small vessel mask and skeleton mask (in tests/example_vessels_mask.nii.gz and tests/example_skeleton.nii.gz), make some test cases to use those to iron out these errors. these examples come from a cropped region of a real ome zarr dataset, if loaded with zarrnii into a chunked dask array they can act as appropriate test datasets (smaller chunks and smaller overlaps may need to be used for this test case, since these are 300x300x300..

Traceback (most recent call last):
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/spimquant/workflow/rules/../scripts/dask_setup.py", line 40, in get_dask_client
yield client
File "/nfs/trident3/lightsheet/prado/mouse_app_vaccine_batch2/derivatives/spimquant-v0.8.0rc1/.snakemake/scripts/tmpzamwr837.skeleton_graph_from_sdt.py", line 286, in
block_tables = list(compute(*delayed_tables))
^^^^^^^^^^^^^^^^^^^^^^^^
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/.pixi/envs/gpu/lib/python3.12/site-packages/dask/base.py", line 685, in compute
results = schedule(expr, keys, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nfs/trident3/lightsheet/prado/mouse_app_vaccine_batch2/derivatives/spimquant-v0.8.0rc1/.snakemake/scripts/tmpzamwr837.skeleton_graph_from_sdt.py", line 81, in _process_chunk
skeleton = Skeleton(skel_c.astype(np.uint8))
^^^^^^^^^^^^^^^^^
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/.pixi/envs/gpu/lib/python3.12/site-packages/skan/csr.py", line 652, in init
self.paths = _build_skeleton_path_graph(self.nbgraph)
^^^^^^^^^^^^^^^^^
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/.pixi/envs/gpu/lib/python3.12/site-packages/skan/csr.py", line 442, in _build_skeleton_path_graph
paths = sparse.csr_matrix(
^^^^^^^^^^^^^^^^^
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/.pixi/envs/gpu/lib/python3.12/site-packages/scipy/sparse/_compressed.py", line 115, in init
self.check_format(full_check=False)
^^^^^^^^^^^^^^^^^
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/.pixi/envs/gpu/lib/python3.12/site-packages/scipy/sparse/_compressed.py", line 194, in check_format
raise ValueError(f"index pointer size {len(self.indptr)} should be {M + 1}")
^^^^^^^^^^^^^^^
ValueError: index pointer size 0 should be 1

[Copilot]

@copilot: encountering an error when building the graph. I've fixed the dependencies and added a small vessel mask and skeleton mask (in tests/example_vessels_mask.nii.gz and tests/example_skeleton.ni...

Implemented in commit bdc78e3. I added targeted regression tests for the graph builder using the example mask/skeleton paths when present (with synthetic fallback), covering chunked-overlap extraction, deduplication, affine coordinate handling, and empty-chunk aggregation edge cases. I also refactored the script minimally to make aggregation robust/testable and validated with pytest tests/test_skeleton_graph_from_sdt.py (3 passing).

[akhanf]

@copilot: still having same error. Also I had forgotten to push the example nii.gz images those are there now see if you can reproduce the error and use those images for testing.

Traceback (most recent call last):
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/spimquant/workflow/rules/../scripts/dask_setup.py", line 40, in get_dask_client
yield client
File "/nfs/trident3/lightsheet/prado/mouse_app_vaccine_batch2/derivatives/spimquant-v0.8.0rc1/.snakemake/scripts/tmp261ua_0x.skeleton_graph_from_sdt.py", line 350, in
block_tables = list(compute(*delayed_tables))
^^^^^^^^^^^^^^^^^^^^^^^^
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/.pixi/envs/gpu/lib/python3.12/site-packages/dask/base.py", line 685, in compute
results = schedule(expr, keys, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nfs/trident3/lightsheet/prado/mouse_app_vaccine_batch2/derivatives/spimquant-v0.8.0rc1/.snakemake/scripts/tmp261ua_0x.skeleton_graph_from_sdt.py", line 134, in _process_chunk
skeleton = Skeleton(skel_c.astype(np.uint8))
^^^^^^^^^^^^^^^^^
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/.pixi/envs/gpu/lib/python3.12/site-packages/skan/csr.py", line 652, in init
self.paths = _build_skeleton_path_graph(self.nbgraph)
^^^^^^^^^^^^^^^^^
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/.pixi/envs/gpu/lib/python3.12/site-packages/skan/csr.py", line 442, in _build_skeleton_path_graph
paths = sparse.csr_matrix(
^^^^^^^^^^^^^^^^^
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/.pixi/envs/gpu/lib/python3.12/site-packages/scipy/sparse/_compressed.py", line 115, in init
self.check_format(full_check=False)
^^^^^^^^^^^^^^^^^
File "/nfs/khan/trainees/akhan488/apps/SPIMquant/.pixi/envs/gpu/lib/python3.12/site-packages/scipy/sparse/_compressed.py", line 194, in check_format
raise ValueError(f"index pointer size {len(self.indptr)} should be {M + 1}")
^^^^^^^^^^^^^^^
ValueError: index pointer size 0 should be 1

[Copilot]

@copilot: still having same error. Also I had forgotten to push the example nii.gz images those are there now see if you can reproduce the error and use those images for testing.

Traceback (most rece...

Reproduced and fixed in 9fcf87c. I used the newly pushed tests/example_vessels_mask.nii.gz and tests/example_skeleton.nii.gz to reproduce the failure path, then updated graph extraction to normalize zarrnii axis order (cxyz and czyx both accepted) and to skip degenerate isolated-voxel chunks that were causing skan crashes. Added/updated tests to exercise zarrnii chunked loading of those example files plus parquet export; pytest tests/test_skeleton_graph_from_sdt.py now passes (6 tests).

[akhanf]

works now, will do another PR to produce separate nodes and edges tables