Fix guanlab_dengkw_pm - #32
Merged
Merged
Conversation
* Only apply the mod2 SVD re-mapping when that SVD was fitted * Exit 99 for modality pairs the method has no hyperparameters for * Add src/utils/exit_codes.py embedder_mod2 was only defined inside the if, but used unconditionally, so a target with fewer features than n_mod2 hit a NameError.
…-branch # Conflicts: # CHANGELOG.md
The port fitted one model per left-out batch (or a single fixed two-way split for ADT), rather than the original's five reshuffles into halves. Results depended on batch ordering and no seed was set. * Average ten models over five reshuffled two-way splits, as in submission_170636 * Expose `--n_repeats` and `--seed`; drop the unused `--distance_method`/`--n_pcs`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes
Three problems in
guanlab_dengkw_pm, checked against the original submission.1. The consensus scheme had been replaced. The original averages ten kernel ridge models -- five reshuffles of the batches into two halves, seeded with 1000:
The port did something else entirely: one fixed two-way split when either modality is ADT, leave-one-batch-out otherwise, with no reshuffling and no seed. So the result depended on the order the batches happened to arrive in, and for ADT pairs it averaged two models where the original averaged ten. Restored the original, with
--n_repeatsand--seedas arguments.Effect on the test resources -- 23 of 32 metric/dataset combinations improve:
bmmc_multiome/normalis the one that gets slightly worse, which makes sense -- that is the pair where the old code's leave-one-batch-out trained each model on more batches. But the point is fidelity to the published method rather than picking whichever variant scores best, and the differences here are muted anyway: these fixtures have few batches, while the real datasets have nine or more, where "half the batches" and "all but one batch" differ a lot more.2.
embedder_mod2is defined conditionally but used unconditionally.Present in the original too. The
elsefires when the target has<= n_mod2features (50 or 70), which no dataset we currently ship does -- ADT is 134, GEX ~14k, ATAC 10k -- so it has never been hit. And the re-mapping is wrong there anyway: in that branch the KRR already predicts in the target space. Now tracked withembedder_mod2 = Noneand the re-mapping skipped:3.
n_comp_dictcovers only the four NeurIPS pairs. Anything else is a bareKeyError, which reads as "the method is broken" rather than "the method doesn't apply here". Switched toexit_non_applicable()(exit 99), which also means addingsrc/utils/exit_codes.py, matchingtask_batch_integration.Also dropped
--distance_methodand--n_pcs, which were declared but never read.One thing deliberately not ported
The original clips ATAC predictions to
[0, 1]:That assumed the starter kit's binarised ATAC. Here the ATAC target is
log_cp10k, reaching 8.8 with 2.9% of values above 1, so the clip would cap most of the signal. Left out.All four pairs run green:
Checklist before requesting a review
I have performed a self-review of my code
Check the correct box. Does this PR contain:
Proposed changes are described in the CHANGELOG.md
CI Tests succeed and look good!