Skip to content

feat(sim): placement-synced operator action events with causal ground truth - #99

Closed
Nitjsefnie wants to merge 5 commits into
bamdadd:mainfrom
Nitjsefnie-OSC:feat/34-placement-synced-actions
Closed

feat(sim): placement-synced operator action events with causal ground truth#99
Nitjsefnie wants to merge 5 commits into
bamdadd:mainfrom
Nitjsefnie-OSC:feat/34-placement-synced-actions

Conversation

@Nitjsefnie

@Nitjsefnie Nitjsefnie commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Placement-synced operator action events for the assembly-station scene: the operator's tracked hand dips at t_i − δ before each item placement, so a reach-and-return strict local minimum is recoverable straight off the manifest, and an interactions.json sidecar carries the true action↔change pairs as ground truth. Opt-in behind --placement-synced; the default output is byte-identical to today.

Related Issues and Pull Requests

Fixes #34

Changes

  • src/multicam_sim/actions.py (new) — CausalTiming (δ and the lag window as explicit typed parameters), DipSchedule (the dip profile and the validators that keep every minimum strict), ActionChange, ActionGroundTruth, build_action_ground_truth() and write_actions_json(). Frozen pydantic, validators in the motion.py idiom, pydantic and numpy only.
  • src/multicam_sim/__init__.py — exports.
  • examples/assembly_station.py — an opt-in --placement-synced preset. Several functions gained parameters and rewritten bodies; with the preset off the behaviour is unchanged, which is the claim I can actually stand behind.
  • tests/test_actions.py (new) and additions to tests/test_assembly_example.py.

DipSchedule.author() rewrites the tracked joint's whole height channel, so the authored dips are the only strict local minima in it — the recoverability property holds by construction rather than by luck.

On prior art, because the name is not new: order.ActionEvent already exists on main and its docstring already says "An operator action synced to an assembly change (causal-fusion GT)". What is new is the substance — upstream's wrist motion is a +y sinusoid with z constant, so the height channel is flat and there are no recoverable dips at all, and there is no δ lag, no negatives and no causal sidecar.

Testing

ruff check, ruff format --check, mypy src clean. pytest: 294 passed on upstream/main310 here. Preset-off outputs of all four examples are byte-identical to upstream/main, compared by sha256 and byte length.

The negatives are the point, and in the first version they did not work. An adversarial review implemented the associator a consumer would actually write — "pair each dip with the next placement inside the lag window" — and it scored precision 1.000, recall 1.000. The distractor dip simply went unpaired, which that rule already tolerates, and the distractor placement was never the next one for any dip. Both were invisible to the only sensible causal rule, so the data could confirm an associator but never falsify one.

Repositioned: the distractor dip at frame 10 places nothing, and part_d's uncaused move at 11 sits inside its lag window. The same naive associator now scores TP=3, FP=1, FN=0, precision 0.750, and a manifest-only test asserts (10, 11, part_d) as that specific false positive. part_d stays in the BOM so order verification still reports fulfilled, and the complementary-visibility story holds (worktop 13/13, overview 0/13).

Also from that review, and fixed here:

  • A validator-legal combination emitted unauthorable ground truth. CausalTiming(action_lag=5, lag_window=5) with a placement at frame 3 produced action_frame=-2 — a pair referencing a dip no DipSchedule can author and no manifest can contain, silently breaking the one-dip-per-pair contract. build_action_ground_truth now raises at construction when placed_at − δ < 1, naming the item and frames, with boundary tests at δ and δ+1. The shipped preset could never hit it; it is guarded anyway.
  • The bite evidence was weaker than it looked. Reverting production made every new test fail at import time, which proves they need the module, not that they assert its behaviour. With the module importable but authoring broken to a constant height channel, the manifest-recoverability test now fails on its own assertion — assert [] == [1, 4, 7, 10].

Independently verified in that review: the recovered minima are exactly the authored dips with boundaries clean, and 30 legal DipSchedule combinations recovered the authored dips every time.

Follow-ups / Known Limitations

  • The sidecar lists only true pairs. Distractors live in the manifest and deliberately not in interactions.json, so any association a consumer makes beyond that list is a false positive by construction — that is what makes the file usable as ground truth rather than as a hint.
  • Timing is in frames throughout, matching the repo's other sidecars.
  • δ, the lag window, the dip depth and half-width are a first pass chosen to keep the dips separable at this scene's frame count. Happy to take different numbers.

Nitjsefnie and others added 5 commits July 31, 2026 12:52
…add#34)

Add multicam_sim.actions: CausalTiming (delta + lag window as typed
parameters), DipSchedule (reach-and-return strict local minima authored
into a tracked joint's height channel), and ActionGroundTruth — the
(actor, item, action_frame, change_frame) pairs for an interactions.json
sidecar. Pure additive module; the byte-golden manifest is untouched.

Co-Authored-By: Kimi K3 <[email protected]>
…amdadd#34)

--placement-synced swaps the continuous sinusoidal wrist reach for
discrete hand dips at placed_at - delta per placed item, adds a
distractor dip that assembles nothing and a distractor item (part_d)
that moves outside the causal lag window, and writes interactions.json
with the true causal pairs. Off by default: without the flag the scene
and every emitted file are byte-identical to before.

Co-Authored-By: Kimi K3 <[email protected]>
…amdadd#34)

Library tests pin the dip profile with pasted literals and assert the
authored trajectory yields exactly the authored strict local minima.
Example tests play a manifest-only consumer: recover dips and item
changes off manifest.json, associate within the lag window, and match
interactions.json exactly — with the distractor dip and the late item
as negatives. Also pins that the default run emits no sidecar.

Co-Authored-By: Kimi K3 <[email protected]>
…ps (bamdadd#34)

Reposition the distractors so the naive causal-forward associator (pair
each dip with the next change inside the lag window) actually errs: the
distractor dip at frame 10 places nothing, but part_d's uncaused move at
frame 11 follows inside the window, so the rule pairs them and
interactions.json says otherwise (was: dip 11 / move 10, which no causal
rule mis-pairs). Also reject placements at frame <= action_lag in
build_action_ground_truth — their dip would land where no DipSchedule
can author one. Docstrings now credit order.ActionEvent with the
placement-synced action-event concept; what is new here is the delta-
lagged dips, the negatives, and the pair sidecar.

Co-Authored-By: Kimi K3 <[email protected]>
…ies (bamdadd#34)

Implement the naive causal-forward associator a consumer would write and
run it on the preset output read from disk: TP=3 FP=1 FN=0, precision
0.750, with the dip-10 -> part_d@11 pairing asserted as THE false
positive. Add boundary tests for the unauthorable-dip guard (placement
exactly at delta and just above it).

Co-Authored-By: Kimi K3 <[email protected]>
@bamdadd

bamdadd commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Merged as a1af401. Thank you @Nitjsefnie — the design instinct here is excellent. Making the authored dips the only strict local minima in the height channel so recoverability holds by construction, and then deliberately planting negatives (the distractor dip at 10 and part_d's uncaused move at 11) so a naive next-in-window associator is forced down to precision 0.750 rather than a meaningless 1.000 — that is exactly how you build ground truth that can falsify a consumer instead of flattering it. The frame<=delta guard and the by-name false-positive assertion (10, 11, part_d) are the right kind of rigor. Opt-in, byte-identical default, all deterministic. 357 passed locally, ruff/format/mypy clean.

@bamdadd

bamdadd commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Landed via local squash-merge (author preserved as @Nitjsefnie) in a1af401 on main; closing this fork PR.

@bamdadd bamdadd closed this Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Assembly-station: emit placement-synced operator action events (for causal action↔change fusion)

2 participants