Skip to content

STAR-compat batch (--alignEndsType, --outSAMorder, chimeric/transcriptome) + PGO CI fix - #145

Merged
Psy-Fer merged 4 commits into
mainfrom
feat/star-compat-batch
Jul 28, 2026
Merged

STAR-compat batch (--alignEndsType, --outSAMorder, chimeric/transcriptome) + PGO CI fix#145
Psy-Fer merged 4 commits into
mainfrom
feat/star-compat-batch

Conversation

@Psy-Fer

@Psy-Fer Psy-Fer commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

STAR-compat batch (--alignEndsType, --outSAMorder, chimeric/transcriptome) + PGO CI fix

Works through five STAR-faithfulness items surfaced in an older deep-dive review, plus fixes the flaky linux-x86_64-v4 release build. Two logically independent changesets on one branch — split into two commits if you'd rather land the CI fix on its own.

Gate (whole branch): 560 lib + 26 integration = 586 tests pass · clippy --all-targets 0 warnings · fmt clean. Yeast SE Local 8788/8926 and PE 8390 both-mapped / 0 half-mapped — identical to baseline.


Part 1 — STAR-compatibility batch

1. --alignEndsType (read-end extension mode)

Implements Local (default), EndToEnd, Extend5pOfRead1, Extend5pOfReads12, Extend3pOfRead1 — STAR's alignEndsType.ext[iMate][iEnd] boolean matrix (Parameters.cpp, extendAlign.cpp, stitchWindowAligns.cpp).

  • extend_alignment gains an extend_to_end branch that forces extension over the full remaining read (no max-score tracking, no mismatch-limit break; chromosome-boundary hit kills the transcript), matching STAR's extendToEnd path.
  • finalize_transcript derives per-end flags from ext[imate][iEnd]. Because it runs per-mate-slice, original_is_reverse equals STAR's (Str != iMate), so the mapping reduces to ext[imate][is_reverse] (start) / ext[imate][!is_reverse] (end). Internal stitch extensions stay Local; forcing is applied only at the read-end extension (additive scoring makes deferring to finalize equivalent to STAR's in-stitch forcing).

Validation: Local default byte-identical to baseline (SE 8788/8926, PE 8390/0). --alignEndsType EndToEnd vs STAR --alignEndsType EndToEnd on 10k yeast: 98.4% position agreement (7924/8054), rustar 8821 vs STAR 8817 mapped — at parity with Local's tie character. +2 unit tests (test_extend_to_end_*).

2. --outSAMorder

--outSAMorder (Paired | PairedKeepInputOrder). rustar's align pipeline already emits records in exact input (FASTQ) order regardless of thread count (verified empirically on a 4-thread run — identical ordering). The flag is now accepted (both values, effectively a no-op) for STAR/pipeline compatibility; unknown values are rejected. +3 param tests.

3. PE chimeric diffMates gap relaxation

The specific "gap check wrongly filters inter-mate chimeras" claim doesn't hold under rustar's architecture: detect_chimeric_old runs strictly per-mate, and inter-mate is handled by detect_inter_mate_chimeric, which imposes no inter-mate read-gap. Still made the function STAR-faithful: detect_chimeric_old_impl gains an optional mate_boundary and waives the read-gap across it (STAR ReadAlign_chimericDetectionOld.cpp:67-71). The public detect_chimeric_old and all per-mate callers pass Nonezero behavior change. +1 unit test asserting the gap is enforced without a boundary and waived with one.

4. Chimeric score-drop read_len (verify)

Verified self-consistent, no fix needed: the score-drop gate uses the length of the read passed in, so it scales correctly for both modes — per-mate length for rustar's PE pools (an intra-mate chimera spans one mate) and combined length for a combined read (matching STAR's readLength[0]+readLength[1]). Added a clarifying comment.

5. Multi-exon minus-strand transcriptome projection (test gap)

Added the missing regression test for a 2-exon reverse-strand transcript projection (align_to_one_transcript tr_strand==2 branch): exon-order reversal, t-space/read-coord inversion relative to transcript/read length, and N-op stripping. Passes first try — the projection code was already correct.


Part 2 — CI fix: flaky linux-x86_64-v4 (and preventive neoverse-v1)

Symptom: the linux-x86_64-v4 release job intermittently died with Illegal instruction (core dumped) / exit 132 in the PGO training run; re-running sometimes passed.

Cause: the PGO instrumented binary was built at the shipped ISA (-Ctarget-cpu=x86-64-v4, AVX-512) and then executed to collect the profile. GitHub's ubuntu-latest runners don't reliably have AVX-512, so on a non-AVX-512 runner the training run hit an AVX-512 instruction and SIGILL'd — flaky by which runner the job landed on.

Fix: decouple the training ISA from the shipped ISA. scripts/pgo-build.sh gains PGO_TRAIN_EXTRA_RUSTFLAGS (defaults to PGO_EXTRA_RUSTFLAGS), applied to the instrumented build only. The release matrix trains at a runner-safe ISA and ships at the optimized one:

target train ISA ship ISA
linux-x86_64-v4 x86-64-v3 (AVX2) x86-64-v4 (AVX-512)
linux-aarch64-neoverse-v1 generic (armv8-a) neoverse-v1 (SVE)

PGO profiles are behavioral (block/branch frequencies) and transfer across -Ctarget-cpu levels, so the shipped binaries remain fully optimized. The neoverse-v1 entry gets the same split preventively — identical failure mode on any non-SVE aarch64 runner.

Validation: ran a full local train=v3 / ship=v4 PGO cycle — completes, ships a working v4-optimized binary (the -pgo-warn-missing-function notices are the expected benign cross-ISA function mismatches). rustc confirmed to accept -Ctarget-cpu=generic for aarch64; workflow re-validated as valid YAML.


Files

  • src/params/mod.rsAlignEndsType type + --alignEndsType/--outSAMorder params, validation, tests
  • src/align/score.rsalign_ends_type on AlignmentScorer
  • src/align/stitch.rsextend_to_end branch, finalize_transcript ext-flag wiring, tests
  • src/align/read_align.rs — per-mate imate args to finalize_transcript
  • src/chimeric/detect.rsdetect_chimeric_old_impl + diffMates, score-drop comment, test
  • src/quant/transcriptome.rs — multi-exon minus-strand projection test
  • scripts/pgo-build.sh, .github/workflows/release.yml — PGO train/ship ISA split
  • README.md--alignEndsType and input-order/--outSAMorder feature bullets

@Psy-Fer
Psy-Fer enabled auto-merge (squash) July 28, 2026 13:04
@Psy-Fer
Psy-Fer merged commit 176908e into main Jul 28, 2026
10 checks passed
@Psy-Fer
Psy-Fer deleted the feat/star-compat-batch branch July 28, 2026 13:09
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 28, 2026
…ead-length mismatch cap

Adds the flags this theme needs that scverse#145 did not bring:
`alignEndsProtrude`, `alignSoftClipAtReferenceEnds`, `alignInsertionFlush`,
`alignTranscriptsPerReadNmax`, `outFilterMismatchNoverReadLmax`,
`seedNoneLociPerWindow`, `seedSplitMin`. Off-menu values are rejected loudly
rather than silently ignored.

`--alignEndsType` and its `ext[mate][end]` matrix come from scverse#145; this branch
builds on that rather than duplicating it. The scorer gains only the three
fields scverse#145 does not have: `flush_right`, `soft_clip_at_reference_ends` and
`p_mm_max_read`.

Two behaviours land here:

- `--alignInsertionFlush Right` now works. The insertion-placement scan accepts
  ties as well as strict improvements, then walks the insertion further right
  for as long as the read keeps matching, rejecting when it runs out of read on
  the B side (STAR -1000009). `None`, the default, is bit-identical to before.
- `--outFilterMismatchNoverReadLmax` is enforced. STAR's
  `outFilterMismatchNmaxTotal` is the tightest of three caps: absolute, a
  fraction of the mapped length, and a fraction of the read length. Only the
  first two were applied, so the flag had nothing to bind on. Now in
  `AlignmentScorer::mismatch_nmax_total`.

Note for a follow-up: the pre-existing `--clipAdapterType` validation sits
inside the `solo_enabled()` branch of `try_parse_from`, so it only fires for
STARsolo runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 28, 2026
An alignment may no longer be soft-clipped past the end of its chromosome when
`--alignSoftClipAtReferenceEnds No` is given. `Yes`, the default, leaves
behaviour unchanged.

Sits directly after the end-to-end boundary check from scverse#145: both are "this
extension may not run off the reference" rules and share the extension results.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 28, 2026
scverse#145 handles the same case with an explicit EXTEND_TO_END_KILL sentinel checked
before the score clamp, so there is no divergence left to record.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 28, 2026
…it now owns

scverse#145 landed `--outSAMorder` and `--alignEndsType` while this branch was open.
The duplicate `--outSAMorder` declaration and its validation are removed, along
with the test that covered them, and `alignEndsType` moves out of
`NOT_YET_ACCEPTED` since it is now implemented upstream.

The parameter-surface test needed no other change, which is the point of it:
it reports the new number by itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 28, 2026
`extend_alignment` gained an `extend_to_end` argument when --alignEndsType
landed. The two long-read chaining call sites pass `false`: they score a seed's
own extension potential during chaining, and --alignEndsType applies at the read
ends, not there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 29, 2026
…ead-length mismatch cap

Adds the flags this theme needs that scverse#145 did not bring:
`alignEndsProtrude`, `alignSoftClipAtReferenceEnds`, `alignInsertionFlush`,
`alignTranscriptsPerReadNmax`, `outFilterMismatchNoverReadLmax`,
`seedNoneLociPerWindow`, `seedSplitMin`. Off-menu values are rejected loudly
rather than silently ignored.

`--alignEndsType` and its `ext[mate][end]` matrix come from scverse#145; this branch
builds on that rather than duplicating it. The scorer gains only the three
fields scverse#145 does not have: `flush_right`, `soft_clip_at_reference_ends` and
`p_mm_max_read`.

Two behaviours land here:

- `--alignInsertionFlush Right` now works. The insertion-placement scan accepts
  ties as well as strict improvements, then walks the insertion further right
  for as long as the read keeps matching, rejecting when it runs out of read on
  the B side (STAR -1000009). `None`, the default, is bit-identical to before.
- `--outFilterMismatchNoverReadLmax` is enforced. STAR's
  `outFilterMismatchNmaxTotal` is the tightest of three caps: absolute, a
  fraction of the mapped length, and a fraction of the read length. Only the
  first two were applied, so the flag had nothing to bind on. Now in
  `AlignmentScorer::mismatch_nmax_total`.

Note for a follow-up: the pre-existing `--clipAdapterType` validation sits
inside the `solo_enabled()` branch of `try_parse_from`, so it only fires for
STARsolo runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 29, 2026
An alignment may no longer be soft-clipped past the end of its chromosome when
`--alignSoftClipAtReferenceEnds No` is given. `Yes`, the default, leaves
behaviour unchanged.

Sits directly after the end-to-end boundary check from scverse#145: both are "this
extension may not run off the reference" rules and share the extension results.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 29, 2026
scverse#145 handles the same case with an explicit EXTEND_TO_END_KILL sentinel checked
before the score clamp, so there is no divergence left to record.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 29, 2026
…it now owns

scverse#145 landed `--outSAMorder` and `--alignEndsType` while this branch was open.
The duplicate `--outSAMorder` declaration and its validation are removed, along
with the test that covered them, and `alignEndsType` moves out of
`NOT_YET_ACCEPTED` since it is now implemented upstream.

The parameter-surface test needed no other change, which is the point of it:
it reports the new number by itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 29, 2026
`extend_alignment` gained an `extend_to_end` argument when --alignEndsType
landed. The two long-read chaining call sites pass `false`: they score a seed's
own extension potential during chaining, and --alignEndsType applies at the read
ends, not there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant