Skip to content

test: add fixed-parameter calibration true-parameter coverage - #78

Merged
seonghobae merged 2 commits into
masterfrom
codex/add-fipc-true-parameter-test
Jun 30, 2026
Merged

test: add fixed-parameter calibration true-parameter coverage#78
seonghobae merged 2 commits into
masterfrom
codex/add-fipc-true-parameter-test

Conversation

@seonghobae

Copy link
Copy Markdown
Collaborator

Summary

  • add a fixed-parameter item calibration regression test using generated 2PL data with known true parameters
  • add confirmCommonItems = TRUE support so non-interactive CI can run the common-item path without hanging on readline()
  • document the Fixed Parameter Item Calibration basis and related literature in docs/fixed-parameter-item-calibration.md

Review requirements

This PR is intentionally queued before merging lower-number stale automation PRs that touch the package behavior, because aFIPC needs true-parameter reproduction evidence before further estimation changes are safe.

The central review process should come from ContextualWisdomLab/.github required workflows: Strix, OpenCode Review, and PR Review Merge Scheduler. Please do not force-merge; wait for current-head approval and passing required checks.

Testing

  • Not run locally: this Windows workspace does not have an R/Rscript runtime on PATH (Rscript missing; r resolves to PowerShell history).
  • Expected CI command: Rscript -e "testthat::test_dir('tests/testthat')" or the repository R check workflow.

Literature basis

  • Kim, S. (2006). A comparative study of IRT fixed parameter calibration methods. Journal of Educational Measurement, 43(4), 355-381.
  • Chalmers, R. P. mirt::fixedCalib documentation, which describes FPC based on Kim (2006).
  • Kim, S., & Kolen, M. J. (2010). Linking item parameters to a base scale. Journal of Educational Measurement, 47(2), 164-181.

@seonghobae

Copy link
Copy Markdown
Collaborator Author

Status note: local checks on this PR are green, including the R package check, but this PR does not yet have the central ContextualWisdomLab/.github required workflow evidence (opencode-review, strix, scan-pr-queue).

Per the repository maintenance requirement, please do not merge this PR until the organization required-workflow ruleset is repaired for aFIPC and the current head has central OpenCode approval plus the central required checks. I opened ContextualWisdomLab/.github#124 to record and gate this ruleset coverage gap.

@seonghobae

Copy link
Copy Markdown
Collaborator Author

Updated governance evidence after ContextualWisdomLab/.github#124 merged:

  • This PR head bc78373f59310b6fbee76d1a5a72fb3d84fc84eb still has only repository-local checks: check, quality, and secret-and-workflow-audit.
  • GET /repos/ContextualWisdomLab/aFIPC/rulesets still returns only repository ruleset PR (12815994).
  • GET /repos/ContextualWisdomLab/aFIPC/rules/branches/master shows no inherited org required workflow rules, zero required approvals, and no central .github OpenCode/Strix/scheduler requirement.
  • The org ruleset endpoint remains inaccessible to the available token (/orgs/ContextualWisdomLab/rulesets returns 404), so I cannot add aFIPC to the central org required-workflow rule from here.

Per the queue policy for this repository, I am leaving this PR unmerged until an org admin repairs the central required-workflow targeting so this PR receives current-head .github OpenCode approval and central workflow evidence.

@seonghobae

Copy link
Copy Markdown
Collaborator Author

Central .github manual evidence has been dispatched for this head after ContextualWisdomLab/.github#126 merged:

Both jobs are currently queued in GitHub Actions. I am leaving this PR unmerged until those central runs complete and produce same-head Strix evidence plus OpenCode approval.

@seonghobae

Copy link
Copy Markdown
Collaborator Author

Update on central evidence: the first manual Strix target run exposed a .github workflow bug rather than an aFIPC source issue. Manual Strix was fetching the central .github workflow SHA from the target repository, which fails with upload-pack: not our ref for target repos.

I opened ContextualWisdomLab/.github#135 to fix manual Strix target runs so workflow_dispatch uses the target PR base SHA (pr_base_sha) for the target workspace. That PR is waiting on the central required checks/approval queue. I will rerun the aFIPC #78 central Strix/OpenCode evidence after #135 is approved and merged.

This PR remains unmerged until same-head central Strix evidence and OpenCode approval exist.

@opencode-agent

opencode-agent Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

  • Head SHA: b1ddafced86302f461e95259699f1efde5ec87c9
  • Workflow run: 28423448564
  • Workflow attempt: 1
  • Gate result: APPROVE (approval step)

Pull request overview

OpenCode reviewed the current-head bounded evidence and found no blocking issues.

Findings

No blocking findings.

Summary

Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including docs/fixed-parameter-item-calibration.md.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: Change Flow DAG maps docs/fixed-parameter-item-calibration.md through bounded evidence, review risk, and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, and current-head workflow evidence were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions and compatibility surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: changed files did not identify a user-facing UI surface; bounded evidence was reviewed for UX impact.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.

  • Result: APPROVE
  • Reason: No material issues found in the inspected files.
  • Head SHA: b1ddafced86302f461e95259699f1efde5ec87c9
  • Workflow run: 28423448564
  • Workflow attempt: 1

Change Flow DAG

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Changed file (2 files)"]
  S1 --> I1["repository behavior"]
  I1 --> R1["Review risk: Changed file (2 files)"]
  R1 --> V1["required checks"]
  Evidence --> S2["Docs: fixed-parameter-item-calibration.md"]
  S2 --> I2["operator or user guidance"]
  I2 --> R2["Review risk: Docs: fixed-parameter-item-calibration.md"]
  R2 --> V2["docs review"]
  Evidence --> S3["Test: test-fixed-parameter-calibration.R"]
  S3 --> I3["regression suite"]
  I3 --> R3["Review risk: Test: test-fixed-parameter-calibration.R"]
  R3 --> V3["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode model attempts did not emit a usable current-head control block, so the approval gate used deterministic current-head evidence instead of model prose.

Findings

No blocking findings.

Summary

  • Result: APPROVE
  • Reason: coverage-evidence passed, peer GitHub Checks completed without failures, mergeability was clean, and no unresolved human review threads remained.
  • Deterministic evidence: current-head changed-file evidence (R/aFIPC.R, docs/fixed-parameter-item-calibration.md, man/autoFIPC.Rd, tests/testthat/test-fixed-parameter-calibration.R); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
  • Model outcomes: primary=unknown, fallback=unknown, second_fallback=unknown, catalog_fallback=unknown.
  • Head SHA: bc78373f59310b6fbee76d1a5a72fb3d84fc84eb
  • Workflow run: 28422861653
  • Workflow attempt: 1

Deterministic fallback approval was used only after model-output instability and did not bypass coverage, failed-check, mergeability, or human-review gates.

Change Flow DAG

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Changed file (2 files)"]
  S1 --> I1["repository behavior"]
  I1 --> R1["Review risk: Changed file (2 files)"]
  R1 --> V1["required checks"]
  Evidence --> S2["Docs: fixed-parameter-item-calibration.md"]
  S2 --> I2["operator or user guidance"]
  I2 --> R2["Review risk: Docs: fixed-parameter-item-calibration.md"]
  R2 --> V2["docs review"]
  Evidence --> S3["Test: test-fixed-parameter-calibration.R"]
  S3 --> I3["regression suite"]
  I3 --> R3["Review risk: Test: test-fixed-parameter-calibration.R"]
  R3 --> V3["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode reviewed the current-head bounded evidence and found no blocking issues.

Findings

No blocking findings.

Summary

Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including docs/fixed-parameter-item-calibration.md.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: Change Flow DAG maps docs/fixed-parameter-item-calibration.md through bounded evidence, review risk, and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, and current-head workflow evidence were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions and compatibility surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: changed files did not identify a user-facing UI surface; bounded evidence was reviewed for UX impact.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.

  • Result: APPROVE
  • Reason: No material issues found in the inspected files.
  • Head SHA: b1ddafced86302f461e95259699f1efde5ec87c9
  • Workflow run: 28423448564
  • Workflow attempt: 1

Change Flow DAG

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Changed file (2 files)"]
  S1 --> I1["repository behavior"]
  I1 --> R1["Review risk: Changed file (2 files)"]
  R1 --> V1["required checks"]
  Evidence --> S2["Docs: fixed-parameter-item-calibration.md"]
  S2 --> I2["operator or user guidance"]
  I2 --> R2["Review risk: Docs: fixed-parameter-item-calibration.md"]
  R2 --> V2["docs review"]
  Evidence --> S3["Test: test-fixed-parameter-calibration.R"]
  S3 --> I3["regression suite"]
  I3 --> R3["Review risk: Test: test-fixed-parameter-calibration.R"]
  R3 --> V3["targeted test run"]
Loading

@seonghobae
seonghobae merged commit 6a16bbc into master Jun 30, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant