Skip to content

test(suite): exhaustive test suite expansion (~559 tests)#85

Merged
johnmarktaylor91 merged 1 commit intomainfrom
test/exhaustive-test-suite-expansion
Mar 2, 2026
Merged

test(suite): exhaustive test suite expansion (~559 tests)#85
johnmarktaylor91 merged 1 commit intomainfrom
test/exhaustive-test-suite-expansion

Conversation

@johnmarktaylor91
Copy link
Copy Markdown
Owner

Summary

  • Add 64 new toy model classes and tests covering attention/transformers, containers (ModuleList/Dict), conditional/dynamic ops, normalization variants (LayerNorm/GroupNorm/InstanceNorm), Conv1d/Conv3d, residual blocks, parameter sharing, in-place chains, type casting, scalar/broadcasting, packed sequences, custom autograd, exemption registry stress tests, architecture patterns (VAE, GAN, UNet, TCN, PixelShuffle, PointNet, ActorCritic, TwoTower, DepthEstimator), and 16 adversarial edge-case models targeting specific torchlens code paths
  • Add 12 new real-world model tests: DistilBERT, ELECTRA, MobileViT, MoE (built-in), T5, BART, RoBERTa, BLIP, Whisper, ViT-MAE, Conformer, SentenceTransformer
  • Merge test_real_world_models_slow.py into test_real_world_models.py organized by architecture/modality, using @pytest.mark.slow for heavy tests
  • Register slow pytest marker in pyproject.toml
  • Add 4 new fixtures (seq_input, token_input, input_3d, input_1d_seq) and 5 vis output subdirs

Test plan

  • pytest tests/test_toy_models.py -v --timeout=120 — verify new toy model tests
  • pytest tests/test_real_world_models.py -m "not slow" -v --timeout=300 — verify fast real-world tests
  • pytest tests/test_real_world_models.py -m "slow" -v --timeout=600 — verify slow real-world tests
  • pytest tests/ -v --timeout=600 — full regression
  • Review adversarial edge-case failures and decide which need exemptions vs xfail vs fixes

Add ~76 new tests covering major gaps in architecture and edge-case coverage:
- 64 new toy model tests (attention, transformers, containers, conditionals,
  normalization variants, residual/param sharing, in-place/type ops, scalar/
  broadcasting, exemption stress tests, architecture patterns, adversarial
  edge cases)
- 12 new real-world model tests (DistilBERT, ELECTRA, MobileViT, MoE, T5,
  BART, RoBERTa, BLIP, Whisper, ViT-MAE, Conformer, SentenceTransformer)

Merge test_real_world_models_slow.py into test_real_world_models.py organized
by architecture/modality, using @pytest.mark.slow for heavy tests. Register
the slow marker in pyproject.toml.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@johnmarktaylor91 johnmarktaylor91 merged commit 90023fa into main Mar 2, 2026
1 check passed
@johnmarktaylor91 johnmarktaylor91 deleted the test/exhaustive-test-suite-expansion branch March 2, 2026 20:26
johnmarktaylor91 added a commit that referenced this pull request Mar 4, 2026
- #95: fix mixed LayerLog/LayerPassLog format in vis module containment check
- #83: LayerLog.parent_layer_arg_locs returns strings (not sets) for consistency
- #99: warn on tensor shape mismatch in fast-path source tensor logging
- #23: add case-insensitive exact match and substring lookup for layers/modules
- #85: confirmed not-a-bug (any special arg correctly explains output invariance)
- #39, #41, #42: confirmed already fixed or correct as-is
- Add 6 regression tests for the above fixes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant