Add exact-vs-estimated arity certificates and provenance (CAP0-04, SLM-80)#330
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughAdds CAP0-04 arity certificate models, provenance and digest handling, deterministic Markdown/CSV rendering, optional CLI emission using verified CAP0-03 witnesses, public exports, and regression tests. ChangesCAP0-04 arity certificate flow
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CLI
participant ArityAnalysis
participant CodeVerification
participant CertificateBuilder
participant Renderers
CLI->>ArityAnalysis: Generate canonical arity report
CLI->>CodeVerification: Verify candidate coding construction
CodeVerification-->>CLI: Verification witness
CLI->>CertificateBuilder: Build certificate bundle
CertificateBuilder-->>CLI: Bundle with provenance and digests
CLI->>Renderers: Render requested outputs
Renderers-->>CLI: Markdown, CSV, or one-line output
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Tyler-R-Kendrick
force-pushed
the
slm-79-cap0-03-coding
branch
from
July 18, 2026 08:39
c17d9e2 to
71e9175
Compare
…M-80) Redesigned onto main's canonical arity API + the CAP0-03 coding API (the source branch coupled to a retired arity stub that #354 replaced). - certificate.py: the pure certificate/evidence/provenance dataclasses (ArityCertificate/ArityResult/ExactEvidence/EstimatedEvidence/...) plus a ReportView bridge over ExactArityReport (frame_id/total_states/ minimized_states/digest) — no edit to canonical report.py. exact_certificate_from_report now takes an explicit CAP0-03 CodeVerification witness (from verify_code over a named construction) instead of a report-attached CodingMetadata, emitting EXACT_LOCAL evidence. - render.py: markdown/csv/one-line certificate renderers. - analyze_grammar_arity.py: optional, dormant CAP0-04 flags (canonical CLI behavior unchanged by default). - 12 tests (dataclass guards, report-backed exact-local, digest stability, renderers) rebuilt on main's analyze() + verify_code; canonical + CAP0-03 suites unchanged (37 passed). The retired stub (analyzer/explorer/types, ArityReport, CodingMetadata) is not reintroduced. Mechanism increment, no ship claim. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017NWyjZwGMUPHXZhtqQVzUJ
Tyler-R-Kendrick
force-pushed
the
slm-80-cap0-04-certificates
branch
from
July 18, 2026 09:17
9197057 to
fef03b1
Compare
Tyler-R-Kendrick
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/slm_training/dsl/analysis/arity/certificate.py`:
- Around line 36-42: The certificate schema must represent incomplete arity
analysis instead of silently omitting requested artifacts. In
src/slm_training/dsl/analysis/arity/certificate.py:36-42, add a supported
representation for EvidenceKind.INCOMPLETE; in
src/slm_training/dsl/analysis/arity/certificate.py:111-118, allow it in
ArityResult with status unknown. In scripts/analyze_grammar_arity.py:211-213 and
:247-259, propagate explicit certificate-emission failures and either emit an
honest incomplete result or return a nonzero status when requested outputs
cannot be produced.
- Around line 276-283: Validate in the bundle class’s __post_init__ that the
bundled certificate’s report_digest and frame_id match the corresponding values
in self.report before computing bundle_digest. Reject mismatches at construction
time, then preserve the existing canonical serialization and digest calculation
for valid pairs.
- Around line 345-383: Validate the coding witness before constructing
ExactEvidence: require verification.required_size to equal view.minimized_states
and, when report.capacity["d"] is present, require verification.n to equal its
integer value. Reject mismatches as an invalid or diagnostic result rather than
issuing a supported exact-local certificate, and keep the existing
constraints/evidence flow for matching witnesses.
- Around line 151-201: Update the nested _convert helper in Certificate.to_dict
to reject unsupported values instead of returning repr(value). Preserve
recursive handling for supported primitives, collections, enums, dataclasses,
ExactEvidence, EstimatedEvidence, and ArityResult, while raising the established
validation error for sets, custom objects, and other unrecognized values so
serialization remains deterministic.
In `@src/slm_training/dsl/analysis/arity/render.py`:
- Around line 15-22: Update the dictionary branch of _fmt to iterate over
mapping entries in sorted key order before constructing the formatted string,
while preserving the existing separator and value formatting for other input
types.
In `@tests/test_dsl/test_arity_certificate.py`:
- Around line 311-337: Update
test_cli_emits_certificate_without_disturbing_report to first run the
bounded-expr CLI with the canonical arguments excluding certificate/render
flags, capture the resulting out and durable contents, then run the
certificate-enabled invocation and compare both outputs against that baseline.
Keep the existing certificate artifact and successful exit assertions while
ensuring the comparison detects identical changes to both files.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 7ea62b76-9bb4-498d-bcb8-f26352c34e2b
📒 Files selected for processing (5)
scripts/analyze_grammar_arity.pysrc/slm_training/dsl/analysis/arity/__init__.pysrc/slm_training/dsl/analysis/arity/certificate.pysrc/slm_training/dsl/analysis/arity/render.pytests/test_dsl/test_arity_certificate.py
Comment on lines
+36
to
+42
| class EvidenceKind(str, Enum): | ||
| """How an arity claim was established.""" | ||
|
|
||
| EXACT_LOCAL = "exact_local" | ||
| EXACT_EXTERNAL = "exact_external" | ||
| ESTIMATED = "estimated" | ||
| INCOMPLETE = "incomplete" |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Implement the declared unknown/incomplete certificate path.
The schema cannot represent EvidenceKind.INCOMPLETE, so the CLI silently omits requested artifacts when no verified construction exists.
src/slm_training/dsl/analysis/arity/certificate.py#L36-L42: add an evidence representation forINCOMPLETE, or remove the unsupported enum contract.src/slm_training/dsl/analysis/arity/certificate.py#L111-L118: allow that evidence inArityResultwith anunknownstatus.scripts/analyze_grammar_arity.py#L211-L213: propagate certificate-emission failure when explicit outputs cannot be produced.scripts/analyze_grammar_arity.py#L247-L259: emit an honest incomplete result instead of silently skipping, or return a nonzero status.
📍 Affects 2 files
src/slm_training/dsl/analysis/arity/certificate.py#L36-L42(this comment)src/slm_training/dsl/analysis/arity/certificate.py#L111-L118scripts/analyze_grammar_arity.py#L211-L213scripts/analyze_grammar_arity.py#L247-L259
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/dsl/analysis/arity/certificate.py` around lines 36 - 42, The
certificate schema must represent incomplete arity analysis instead of silently
omitting requested artifacts. In
src/slm_training/dsl/analysis/arity/certificate.py:36-42, add a supported
representation for EvidenceKind.INCOMPLETE; in
src/slm_training/dsl/analysis/arity/certificate.py:111-118, allow it in
ArityResult with status unknown. In scripts/analyze_grammar_arity.py:211-213 and
:247-259, propagate explicit certificate-emission failures and either emit an
honest incomplete result or return a nonzero status when requested outputs
cannot be produced.
Comment on lines
+151
to
+201
| def to_dict(self) -> dict[str, Any]: | ||
| """Serialize to a JSON-safe dict.""" | ||
|
|
||
| def _convert(value: object) -> object: | ||
| if isinstance(value, (str, int, float, bool)) or value is None: | ||
| return value | ||
| if isinstance(value, (list, tuple)): | ||
| return [_convert(v) for v in value] | ||
| if isinstance(value, dict): | ||
| return {str(k): _convert(v) for k, v in value.items()} | ||
| if isinstance(value, Enum): | ||
| return value.value | ||
| if isinstance(value, (ConstraintFrame, ArityProvenance)): | ||
| return _convert(asdict(value)) | ||
| if isinstance(value, ExactEvidence): | ||
| return { | ||
| "evidence_kind": value.evidence_kind.value, | ||
| "theorem_or_algorithm": value.theorem_or_algorithm, | ||
| "constraints": _convert(value.constraints), | ||
| "complete": value.complete, | ||
| "witness_or_proof_hash": value.witness_or_proof_hash, | ||
| "work_counters": _convert(value.work_counters), | ||
| "source_uri": value.source_uri, | ||
| } | ||
| if isinstance(value, EstimatedEvidence): | ||
| return { | ||
| "evidence_kind": value.evidence_kind.value, | ||
| "constraints": _convert(value.constraints), | ||
| "dataset_ids": _convert(value.dataset_ids), | ||
| "trace_ids": _convert(value.trace_ids), | ||
| "checkpoint_ids": _convert(value.checkpoint_ids), | ||
| "sample_count": value.sample_count, | ||
| "sampling_design": value.sampling_design, | ||
| "coverage": _convert(value.coverage), | ||
| "estimator": value.estimator, | ||
| "confidence_interval": _convert(value.confidence_interval), | ||
| "tail_metric": _convert(value.tail_metric), | ||
| "seed_ids": _convert(value.seed_ids), | ||
| } | ||
| if isinstance(value, ArityResult): | ||
| return { | ||
| "metric_name": value.metric_name, | ||
| "value": _convert(value.value), | ||
| "units": value.units, | ||
| "evidence": _convert(value.evidence), | ||
| "status": value.status, | ||
| } | ||
| return repr(value) | ||
|
|
||
| base = asdict(self) | ||
| return {k: _convert(v) for k, v in base.items()} |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Reject unsupported values instead of hashing their repr.
ArityResult.value is Any, and Line 198 serializes unsupported objects using repr(). Sets, custom objects, and address-bearing representations make JSON and certificate digests non-deterministic.
Proposed fix
- return repr(value)
+ raise TypeError(
+ f"unsupported certificate value type: {type(value).__name__}"
+ )As per coding guidelines, “Always perform input validation at trust boundaries.”
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def to_dict(self) -> dict[str, Any]: | |
| """Serialize to a JSON-safe dict.""" | |
| def _convert(value: object) -> object: | |
| if isinstance(value, (str, int, float, bool)) or value is None: | |
| return value | |
| if isinstance(value, (list, tuple)): | |
| return [_convert(v) for v in value] | |
| if isinstance(value, dict): | |
| return {str(k): _convert(v) for k, v in value.items()} | |
| if isinstance(value, Enum): | |
| return value.value | |
| if isinstance(value, (ConstraintFrame, ArityProvenance)): | |
| return _convert(asdict(value)) | |
| if isinstance(value, ExactEvidence): | |
| return { | |
| "evidence_kind": value.evidence_kind.value, | |
| "theorem_or_algorithm": value.theorem_or_algorithm, | |
| "constraints": _convert(value.constraints), | |
| "complete": value.complete, | |
| "witness_or_proof_hash": value.witness_or_proof_hash, | |
| "work_counters": _convert(value.work_counters), | |
| "source_uri": value.source_uri, | |
| } | |
| if isinstance(value, EstimatedEvidence): | |
| return { | |
| "evidence_kind": value.evidence_kind.value, | |
| "constraints": _convert(value.constraints), | |
| "dataset_ids": _convert(value.dataset_ids), | |
| "trace_ids": _convert(value.trace_ids), | |
| "checkpoint_ids": _convert(value.checkpoint_ids), | |
| "sample_count": value.sample_count, | |
| "sampling_design": value.sampling_design, | |
| "coverage": _convert(value.coverage), | |
| "estimator": value.estimator, | |
| "confidence_interval": _convert(value.confidence_interval), | |
| "tail_metric": _convert(value.tail_metric), | |
| "seed_ids": _convert(value.seed_ids), | |
| } | |
| if isinstance(value, ArityResult): | |
| return { | |
| "metric_name": value.metric_name, | |
| "value": _convert(value.value), | |
| "units": value.units, | |
| "evidence": _convert(value.evidence), | |
| "status": value.status, | |
| } | |
| return repr(value) | |
| base = asdict(self) | |
| return {k: _convert(v) for k, v in base.items()} | |
| def to_dict(self) -> dict[str, Any]: | |
| """Serialize to a JSON-safe dict.""" | |
| def _convert(value: object) -> object: | |
| if isinstance(value, (str, int, float, bool)) or value is None: | |
| return value | |
| if isinstance(value, (list, tuple)): | |
| return [_convert(v) for v in value] | |
| if isinstance(value, dict): | |
| return {str(k): _convert(v) for k, v in value.items()} | |
| if isinstance(value, Enum): | |
| return value.value | |
| if isinstance(value, (ConstraintFrame, ArityProvenance)): | |
| return _convert(asdict(value)) | |
| if isinstance(value, ExactEvidence): | |
| return { | |
| "evidence_kind": value.evidence_kind.value, | |
| "theorem_or_algorithm": value.theorem_or_algorithm, | |
| "constraints": _convert(value.constraints), | |
| "complete": value.complete, | |
| "witness_or_proof_hash": value.witness_or_proof_hash, | |
| "work_counters": _convert(value.work_counters), | |
| "source_uri": value.source_uri, | |
| } | |
| if isinstance(value, EstimatedEvidence): | |
| return { | |
| "evidence_kind": value.evidence_kind.value, | |
| "constraints": _convert(value.constraints), | |
| "dataset_ids": _convert(value.dataset_ids), | |
| "trace_ids": _convert(value.trace_ids), | |
| "checkpoint_ids": _convert(value.checkpoint_ids), | |
| "sample_count": value.sample_count, | |
| "sampling_design": value.sampling_design, | |
| "coverage": _convert(value.coverage), | |
| "estimator": value.estimator, | |
| "confidence_interval": _convert(value.confidence_interval), | |
| "tail_metric": _convert(value.tail_metric), | |
| "seed_ids": _convert(value.seed_ids), | |
| } | |
| if isinstance(value, ArityResult): | |
| return { | |
| "metric_name": value.metric_name, | |
| "value": _convert(value.value), | |
| "units": value.units, | |
| "evidence": _convert(value.evidence), | |
| "status": value.status, | |
| } | |
| raise TypeError( | |
| f"unsupported certificate value type: {type(value).__name__}" | |
| ) | |
| base = asdict(self) | |
| return {k: _convert(v) for k, v in base.items()} |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/dsl/analysis/arity/certificate.py` around lines 151 - 201,
Update the nested _convert helper in Certificate.to_dict to reject unsupported
values instead of returning repr(value). Preserve recursive handling for
supported primitives, collections, enums, dataclasses, ExactEvidence,
EstimatedEvidence, and ArityResult, while raising the established validation
error for sets, custom objects, and other unrecognized values so serialization
remains deterministic.
Source: Coding guidelines
Comment on lines
+276
to
+283
| def __post_init__(self) -> None: | ||
| canonical = { | ||
| "report": self.report.to_dict(), | ||
| "certificate": self.certificate.to_dict(), | ||
| } | ||
| payload = json.dumps(canonical, sort_keys=True, separators=(",", ":")) | ||
| digest = hashlib.sha256(payload.encode("utf-8"), usedforsecurity=False).hexdigest()[:32] | ||
| object.__setattr__(self, "bundle_digest", digest) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Validate that the certificate belongs to the bundled report.
The public bundle constructor accepts mismatched report_digest or frame_id, then computes a valid-looking bundle digest over that inconsistent pair.
Proposed fix
def __post_init__(self) -> None:
+ if self.certificate.report_digest != self.report.digest:
+ raise ValueError("certificate report_digest does not match report")
+ if self.certificate.frame_id != self.report.frame_id:
+ raise ValueError("certificate frame_id does not match report")
canonical = {As per coding guidelines, “Always perform input validation at trust boundaries.”
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def __post_init__(self) -> None: | |
| canonical = { | |
| "report": self.report.to_dict(), | |
| "certificate": self.certificate.to_dict(), | |
| } | |
| payload = json.dumps(canonical, sort_keys=True, separators=(",", ":")) | |
| digest = hashlib.sha256(payload.encode("utf-8"), usedforsecurity=False).hexdigest()[:32] | |
| object.__setattr__(self, "bundle_digest", digest) | |
| def __post_init__(self) -> None: | |
| if self.certificate.report_digest != self.report.digest: | |
| raise ValueError("certificate report_digest does not match report") | |
| if self.certificate.frame_id != self.report.frame_id: | |
| raise ValueError("certificate frame_id does not match report") | |
| canonical = { | |
| "report": self.report.to_dict(), | |
| "certificate": self.certificate.to_dict(), | |
| } | |
| payload = json.dumps(canonical, sort_keys=True, separators=(",", ":")) | |
| digest = hashlib.sha256(payload.encode("utf-8"), usedforsecurity=False).hexdigest()[:32] | |
| object.__setattr__(self, "bundle_digest", digest) |
🧰 Tools
🪛 ast-grep (0.44.1)
[info] 280-280: use jsonify instead of json.dumps for JSON output
Context: json.dumps(canonical, sort_keys=True, separators=(",", ":"))
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/dsl/analysis/arity/certificate.py` around lines 276 - 283,
Validate in the bundle class’s __post_init__ that the bundled certificate’s
report_digest and frame_id match the corresponding values in self.report before
computing bundle_digest. Reject mismatches at construction time, then preserve
the existing canonical serialization and digest calculation for valid pairs.
Source: Coding guidelines
Comment on lines
+345
to
+383
| view = report_view(report) | ||
| capacity_dim = report.capacity.get("d") | ||
| dimensions = None if capacity_dim is None else int(capacity_dim) | ||
|
|
||
| constraints = ConstraintFrame( | ||
| grammar_hash=report.grammar_hash, | ||
| parser_version=str(report.parser_version), | ||
| codec_version=str(report.codec_version), | ||
| state_signature_version=str(report.signature_version), | ||
| generation_order="preorder", | ||
| ast_bounds={ | ||
| "max_ast_nodes": report.bounds.max_ast_nodes, | ||
| "max_ast_depth": report.bounds.max_ast_depth, | ||
| }, | ||
| scope_bounds={"max_live_bindings": report.bounds.max_live_bindings}, | ||
| template_classes=report.bounds.template_classes, | ||
| latent_role=None, | ||
| dimensions=dimensions, | ||
| noise_model=None, | ||
| packing_assumption=None, | ||
| ) | ||
| evidence = ExactEvidence( | ||
| evidence_kind=EvidenceKind.EXACT_LOCAL, | ||
| theorem_or_algorithm=construction, | ||
| constraints=constraints, | ||
| complete=True, | ||
| witness_or_proof_hash=_witness_hash(construction, verification), | ||
| work_counters={ | ||
| "states": view.minimized_states, | ||
| "code_size": verification.size, | ||
| "minimum_distance": verification.minimum_distance, | ||
| "required_size": verification.required_size, | ||
| "required_distance": verification.required_distance, | ||
| }, | ||
| source_uri=None, | ||
| ) | ||
| status: Literal["supported", "infeasible", "unknown", "diagnostic"] = ( | ||
| "supported" if verification.ok else "infeasible" | ||
| ) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Bind the coding witness to the report’s certified frame.
The helper never checks that verification.required_size == minimized_state_count or that verification.n matches report.capacity["d"]. This is reachable from scripts/analyze_grammar_arity.py, whose fixed witnesses use lengths 4 and 7 regardless of --dimensions, allowing an exact-local certificate under the wrong frame.
Proposed validation
dimensions = None if capacity_dim is None else int(capacity_dim)
+ if verification.required_size != view.minimized_states:
+ raise ValueError("verification target does not match minimized state count")
+ if dimensions is not None and verification.n != dimensions:
+ raise ValueError("verification code length does not match report dimensions")As per coding guidelines, “Always perform input validation at trust boundaries.”
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| view = report_view(report) | |
| capacity_dim = report.capacity.get("d") | |
| dimensions = None if capacity_dim is None else int(capacity_dim) | |
| constraints = ConstraintFrame( | |
| grammar_hash=report.grammar_hash, | |
| parser_version=str(report.parser_version), | |
| codec_version=str(report.codec_version), | |
| state_signature_version=str(report.signature_version), | |
| generation_order="preorder", | |
| ast_bounds={ | |
| "max_ast_nodes": report.bounds.max_ast_nodes, | |
| "max_ast_depth": report.bounds.max_ast_depth, | |
| }, | |
| scope_bounds={"max_live_bindings": report.bounds.max_live_bindings}, | |
| template_classes=report.bounds.template_classes, | |
| latent_role=None, | |
| dimensions=dimensions, | |
| noise_model=None, | |
| packing_assumption=None, | |
| ) | |
| evidence = ExactEvidence( | |
| evidence_kind=EvidenceKind.EXACT_LOCAL, | |
| theorem_or_algorithm=construction, | |
| constraints=constraints, | |
| complete=True, | |
| witness_or_proof_hash=_witness_hash(construction, verification), | |
| work_counters={ | |
| "states": view.minimized_states, | |
| "code_size": verification.size, | |
| "minimum_distance": verification.minimum_distance, | |
| "required_size": verification.required_size, | |
| "required_distance": verification.required_distance, | |
| }, | |
| source_uri=None, | |
| ) | |
| status: Literal["supported", "infeasible", "unknown", "diagnostic"] = ( | |
| "supported" if verification.ok else "infeasible" | |
| ) | |
| view = report_view(report) | |
| capacity_dim = report.capacity.get("d") | |
| dimensions = None if capacity_dim is None else int(capacity_dim) | |
| if verification.required_size != view.minimized_states: | |
| raise ValueError("verification target does not match minimized state count") | |
| if dimensions is not None and verification.n != dimensions: | |
| raise ValueError("verification code length does not match report dimensions") | |
| constraints = ConstraintFrame( | |
| grammar_hash=report.grammar_hash, | |
| parser_version=str(report.parser_version), | |
| codec_version=str(report.codec_version), | |
| state_signature_version=str(report.signature_version), | |
| generation_order="preorder", | |
| ast_bounds={ | |
| "max_ast_nodes": report.bounds.max_ast_nodes, | |
| "max_ast_depth": report.bounds.max_ast_depth, | |
| }, | |
| scope_bounds={"max_live_bindings": report.bounds.max_live_bindings}, | |
| template_classes=report.bounds.template_classes, | |
| latent_role=None, | |
| dimensions=dimensions, | |
| noise_model=None, | |
| packing_assumption=None, | |
| ) | |
| evidence = ExactEvidence( | |
| evidence_kind=EvidenceKind.EXACT_LOCAL, | |
| theorem_or_algorithm=construction, | |
| constraints=constraints, | |
| complete=True, | |
| witness_or_proof_hash=_witness_hash(construction, verification), | |
| work_counters={ | |
| "states": view.minimized_states, | |
| "code_size": verification.size, | |
| "minimum_distance": verification.minimum_distance, | |
| "required_size": verification.required_size, | |
| "required_distance": verification.required_distance, | |
| }, | |
| source_uri=None, | |
| ) | |
| status: Literal["supported", "infeasible", "unknown", "diagnostic"] = ( | |
| "supported" if verification.ok else "infeasible" | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/dsl/analysis/arity/certificate.py` around lines 345 - 383,
Validate the coding witness before constructing ExactEvidence: require
verification.required_size to equal view.minimized_states and, when
report.capacity["d"] is present, require verification.n to equal its integer
value. Reject mismatches as an invalid or diagnostic result rather than issuing
a supported exact-local certificate, and keep the existing constraints/evidence
flow for matching witnesses.
Source: Coding guidelines
Comment on lines
+15
to
+22
| def _fmt(value: Any) -> str: | ||
| if value is None: | ||
| return "" | ||
| if isinstance(value, (list, tuple)): | ||
| return ", ".join(str(v) for v in value) | ||
| if isinstance(value, dict): | ||
| return "; ".join(f"{k}={v}" for k, v in value.items()) | ||
| return str(value) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Canonicalize mapping order in deterministic renderers.
Line 21 preserves dictionary insertion order, so equivalent certificate values can render differently. Sort keys before formatting.
Proposed fix
if isinstance(value, dict):
- return "; ".join(f"{k}={v}" for k, v in value.items())
+ return "; ".join(
+ f"{k}={v}" for k, v in sorted(value.items(), key=lambda item: str(item[0]))
+ )📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def _fmt(value: Any) -> str: | |
| if value is None: | |
| return "" | |
| if isinstance(value, (list, tuple)): | |
| return ", ".join(str(v) for v in value) | |
| if isinstance(value, dict): | |
| return "; ".join(f"{k}={v}" for k, v in value.items()) | |
| return str(value) | |
| def _fmt(value: Any) -> str: | |
| if value is None: | |
| return "" | |
| if isinstance(value, (list, tuple)): | |
| return ", ".join(str(v) for v in value) | |
| if isinstance(value, dict): | |
| return "; ".join( | |
| f"{k}={v}" for k, v in sorted(value.items(), key=lambda item: str(item[0])) | |
| ) | |
| return str(value) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/dsl/analysis/arity/render.py` around lines 15 - 22, Update
the dictionary branch of _fmt to iterate over mapping entries in sorted key
order before constructing the formatted string, while preserving the existing
separator and value formatting for other input types.
Comment on lines
+311
to
+337
| def test_cli_emits_certificate_without_disturbing_report(tmp_path): | ||
| # The optional CAP0-04 CLI flags emit a certificate bundle + renders while | ||
| # leaving the canonical CAP0-02 report outputs byte-identical (guards the | ||
| # #354/#327 report contract under the new flags). | ||
| from scripts.analyze_grammar_arity import main | ||
|
|
||
| out = tmp_path / "scratch.json" | ||
| durable = tmp_path / "durable.json" | ||
| cert = tmp_path / "cert.json" | ||
| md = tmp_path / "cert.md" | ||
| csv_path = tmp_path / "cert.csv" | ||
| code = main( | ||
| [ | ||
| "--fixture", "bounded-expr", | ||
| "--max-ast-nodes", "6", | ||
| "--max-live-bindings", "2", | ||
| "--dimensions", "4", | ||
| "--out", str(out), | ||
| "--durable-out", str(durable), | ||
| "--certificate-out", str(cert), | ||
| "--out-md", str(md), | ||
| "--out-csv", str(csv_path), | ||
| ] | ||
| ) | ||
| assert code == 0 | ||
| assert out.exists() and durable.exists() | ||
| assert out.read_text(encoding="utf-8") == durable.read_text(encoding="utf-8") |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Compare against a baseline invocation without certificate flags.
out == durable only compares two files from the same certificate-enabled run. A regression where the new flags alter both files identically would pass despite violating the stated canonical-output contract.
Proposed regression check
def test_cli_emits_certificate_without_disturbing_report(tmp_path):
from scripts.analyze_grammar_arity import main
+ base_args = [
+ "--fixture", "bounded-expr",
+ "--max-ast-nodes", "6",
+ "--max-live-bindings", "2",
+ "--dimensions", "4",
+ ]
+ baseline = tmp_path / "baseline.json"
out = tmp_path / "scratch.json"
durable = tmp_path / "durable.json"
cert = tmp_path / "cert.json"
md = tmp_path / "cert.md"
csv_path = tmp_path / "cert.csv"
+
+ assert main([*base_args, "--out", str(baseline)]) == 0
code = main(
[
- "--fixture", "bounded-expr",
- "--max-ast-nodes", "6",
- "--max-live-bindings", "2",
- "--dimensions", "4",
+ *base_args,
"--out", str(out),
"--durable-out", str(durable),
"--certificate-out", str(cert),
"--out-md", str(md),
"--out-csv", str(csv_path),
]
)
assert code == 0
assert out.exists() and durable.exists()
+ assert baseline.read_bytes() == out.read_bytes()
assert out.read_text(encoding="utf-8") == durable.read_text(encoding="utf-8")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def test_cli_emits_certificate_without_disturbing_report(tmp_path): | |
| # The optional CAP0-04 CLI flags emit a certificate bundle + renders while | |
| # leaving the canonical CAP0-02 report outputs byte-identical (guards the | |
| # #354/#327 report contract under the new flags). | |
| from scripts.analyze_grammar_arity import main | |
| out = tmp_path / "scratch.json" | |
| durable = tmp_path / "durable.json" | |
| cert = tmp_path / "cert.json" | |
| md = tmp_path / "cert.md" | |
| csv_path = tmp_path / "cert.csv" | |
| code = main( | |
| [ | |
| "--fixture", "bounded-expr", | |
| "--max-ast-nodes", "6", | |
| "--max-live-bindings", "2", | |
| "--dimensions", "4", | |
| "--out", str(out), | |
| "--durable-out", str(durable), | |
| "--certificate-out", str(cert), | |
| "--out-md", str(md), | |
| "--out-csv", str(csv_path), | |
| ] | |
| ) | |
| assert code == 0 | |
| assert out.exists() and durable.exists() | |
| assert out.read_text(encoding="utf-8") == durable.read_text(encoding="utf-8") | |
| def test_cli_emits_certificate_without_disturbing_report(tmp_path): | |
| # The optional CAP0-04 CLI flags emit a certificate bundle + renders while | |
| # leaving the canonical CAP0-02 report outputs byte-identical (guards the | |
| # `#354/`#327 report contract under the new flags). | |
| from scripts.analyze_grammar_arity import main | |
| base_args = [ | |
| "--fixture", "bounded-expr", | |
| "--max-ast-nodes", "6", | |
| "--max-live-bindings", "2", | |
| "--dimensions", "4", | |
| ] | |
| baseline = tmp_path / "baseline.json" | |
| out = tmp_path / "scratch.json" | |
| durable = tmp_path / "durable.json" | |
| cert = tmp_path / "cert.json" | |
| md = tmp_path / "cert.md" | |
| csv_path = tmp_path / "cert.csv" | |
| assert main([*base_args, "--out", str(baseline)]) == 0 | |
| code = main( | |
| [ | |
| *base_args, | |
| "--out", str(out), | |
| "--durable-out", str(durable), | |
| "--certificate-out", str(cert), | |
| "--out-md", str(md), | |
| "--out-csv", str(csv_path), | |
| ] | |
| ) | |
| assert code == 0 | |
| assert out.exists() and durable.exists() | |
| assert baseline.read_bytes() == out.read_bytes() | |
| assert out.read_text(encoding="utf-8") == durable.read_text(encoding="utf-8") |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_dsl/test_arity_certificate.py` around lines 311 - 337, Update
test_cli_emits_certificate_without_disturbing_report to first run the
bounded-expr CLI with the canonical arguments excluding certificate/render
flags, capture the resulting out and durable contents, then run the
certificate-enabled invocation and compare both outputs against that baseline.
Keep the existing certificate artifact and successful exit assertions while
ensuring the comparison detects identical changes to both files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CAP0-04 (SLM-80): exact-vs-estimated arity certificates and provenance
Redesigned onto main's canonical arity API + the merged CAP0-03 coding API — the source branch coupled to a retired arity stub (
ArityAnalyzer/ArityReport/CodingMetadata) that #354 replaced, so a mechanical port was impossible.What's here (5 files, on current
main)arity/certificate.py— the pure certificate/evidence/provenance dataclasses (ArityCertificate/ArityResult/ExactEvidence/EstimatedEvidence/ConstraintFrame/ArityProvenance+certificate_digest, reused verbatim from Add exact-vs-estimated arity certificates and provenance (CAP0-04, SLM-80) #330) plus aReportViewbridge overExactArityReport(derivesframe_id/total_states/minimized_states/digest) — no edit to canonicalreport.py.exact_certificate_from_reportnow takes an explicit CAP0-03CodeVerificationwitness (fromverify_codeover a named construction, e.g.build_mds_7_4_2_3) instead of a report-attachedCodingMetadata, emittingEXACT_LOCALevidence.arity/render.py— markdown/csv/one-line certificate renderers (verbatim).arity/__init__.py— additive CAP0-04 exports (canonical + CAP0-03 exports retained).scripts/analyze_grammar_arity.py— optional dormant CAP0-04 flags; default canonical CLI behavior unchanged.tests/test_dsl/test_arity_certificate.py— 12 tests (dataclass guards, report-backed exact-local, digest stability, renderers), rebuilt on main'sanalyze()+verify_codewithout weakening assertions.Honesty
The retired stub (
analyzer/explorer/types,ArityReport,CodingMetadata) is not reintroduced. #330'sINCOMPLETE-evidence branch was dead (it would raise against its ownEstimatedEvidenceguard) — not resurrected or faked; the builder yieldsEXACT_LOCALfrom a verified witness (its live behavior). Mechanism increment, no ship claim.Verification (torch-free)
test_arity_analysis/test_arity_coding/test_arity_suggest37 passed unchanged. Newtest_arity_certificate12 passed (49 total).ruff/repo_policy/diff-checkclean.🤖 Generated with Claude Code
Summary by CodeRabbit