Skip to content

Add fixture-backed regression coverage for prior-update behavior and IPD anchor filtering#87

Closed
seonghobae with Copilot wants to merge 3 commits into
masterfrom
copilot/find-key-articles-for-software
Closed

Add fixture-backed regression coverage for prior-update behavior and IPD anchor filtering#87
seonghobae with Copilot wants to merge 3 commits into
masterfrom
copilot/find-key-articles-for-software

Conversation

Copilot AI commented Jul 1, 2026

Copy link
Copy Markdown

Current coverage did not explicitly lock down two high-risk behaviors in autoFIPC(): prior-distribution update semantics during linking and quantitative IPD/anchor-quality filtering. This PR adds deterministic regression fixtures and targeted assertions for both.

  • Prior-update regression fixture (PARSCALE-vs-BILOG behavior surface)

    • Added deterministic fixture config for shifted new-form ability distribution.
    • Added regression test that contrasts:
      • freeMEAN=TRUE, forceNormalZeroOne=FALSE (mean/cov can move), vs
      • forceNormalZeroOne=TRUE (group mean/cov fixed to 0/1).
    • Asserts expected GROUP parameter estimation/fixing behavior to prevent silent drift.
  • IPD/anchor-quality quantitative regression

    • Added deterministic fixture with one intentionally drifted common item.
    • Added regression test with checkIPD=TRUE asserting:
      • drifted anchor is removed from retained common-item set,
      • retained anchors remain numerically fixed to old-form a1/d values in linked calibration.
  • Fixture baseline and roadmap alignment

    • Introduced fixture file under tests/testthat/fixtures/ to centralize scenario parameters.
    • Updated architecture roadmap language to reflect that baseline prior-update/IPD-anchor fixture coverage now exists and can be extended.
linked_fixed <- aFIPC::autoFIPC(
  newformXData = new_model,
  oldformYData = old_model,
  newformCommonItemNames = new_common_items,
  oldformCommonItemNames = old_common_items,
  itemtype = "2PL",
  checkIPD = FALSE,
  tryEM = TRUE,
  freeMEAN = TRUE,
  forceNormalZeroOne = TRUE,
  confirmCommonItems = TRUE
)

@seonghobae seonghobae marked this pull request as ready for review July 1, 2026 08:40
@seonghobae seonghobae self-requested a review as a code owner July 1, 2026 08:40
Copilot AI review requested due to automatic review settings July 1, 2026 08:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds deterministic, fixture-backed regression coverage to lock down two high-risk autoFIPC() behaviors (prior-distribution update semantics during linking and quantitative IPD/anchor filtering), and updates project docs to reflect the new baseline.

Changes:

  • Adds two regression tests that (1) distinguish free-mean vs fixed-normal linking behavior and (2) verify drifted anchors are removed and retained anchors stay fixed on the old-form scale.
  • Introduces centralized fixture parameters under tests/testthat/fixtures/ to keep scenarios deterministic and auditable.
  • Updates ARCHITECTURE.md roadmap language and expands AGENTS.md with FIPC literature + algorithmic guardrails.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
tests/testthat/test-regression-fixtures.R Adds deterministic regression tests for prior-update semantics and IPD anchor filtering assertions.
tests/testthat/fixtures/fipc-regression-fixtures.R Defines reusable fixture parameter sets (seeds, sample sizes, drift index, etc.).
ARCHITECTURE.md Updates roadmap to reflect the new regression-fixture baseline.
AGENTS.md Adds FIPC literature references and development constraints tied to prior-update/IPD behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@seonghobae

Copy link
Copy Markdown
Collaborator

Superseded by #90.

Reason: the markdown failure in this PR is mechanical, but the IPD regression assertion is not safe to repair in place. Current CI showed the synthetic drifted anchor remains in IPDCommonItemList, so changing R/aFIPC.R to satisfy that exact assertion would be an algorithmic behavior change without maintainer-approved regression evidence. #90 keeps the regression lane additive and verifies prior-update behavior plus the IPD-filtered anchors that are actually fixed in the linked model.

@seonghobae seonghobae closed this Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants