feat(gt): per-entity activity-state ground-truth channel (opt-in sidecar) - #95
Merged
bamdadd merged 2 commits intoJul 31, 2026
Merged
Conversation
Add multicam_sim.activity: a typed ActivityState StrEnum (standing / crouching / reaching) labeling an entity over half-open frame intervals, an ActivityTimeline with per-entity per-frame state_at_frame queries, and write_activity_json — following the order.py / possession.py sidecar precedent. Attached to Scene via an optional additive field, so the byte-golden manifest is unchanged when the channel is absent. Co-Authored-By: Kimi K3 <[email protected]>
SceneBuilder.activity(entity_id, state, start, end) records an ActivitySegment into the scene's activity sidecar without touching motion or geometry. Tests cover enum string serialisation, segment/timeline validation, half-open per-frame queries, builder validation, sidecar round-trips, and byte-identical manifests with/without opt-in. Co-Authored-By: Kimi K3 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a per-entity, per-frame activity-state ground-truth channel (
standing/crouching/reaching) so an activity-recognition head has a target to score against. Opt-in and additive — the byte-golden manifests are unchanged.Related Issues and Pull Requests
Closes #65
Changes
src/multicam_sim/activity.py(new):ActivityState(StrEnum),ActivitySegment(half-open[start, end)),ActivityTimelinewithstate_at_frame, andwrite_activity_json.src/multicam_sim/scene.py: optional additiveScene.activityfield.src/multicam_sim/dsl/builder.py:SceneBuilder.activity(...)opt-in hook; touches neither motion nor the manifest.src/multicam_sim/__init__.py: exports.tests/test_activity.py: 10 tests.It deliberately mirrors
possession.py/order.pyone-to-one rather than introducing a second idiom — same validators, same overlap rejection, same(entity_id, start_frame)sort key, sameto_json/write_*_jsonshape, andstate_at_framemirrorsholder_at_frameincluding its early break. Per #29's note, this is only the labeled GT channel; it generates no motion and does not depend on #29.Testing
uv run ruff check .,uv run ruff format --check .,uv run mypy srcall clean; plainuv run pytest(not-q, whichaddoptswould turn into-qqand silence) → 294 passed, 4 skipped, against a 284 passed, 4 skipped baseline onmainunder the same plainuv sync. The +10 is exactly the new test module.Byte-golden verified by hashing rather than asserted: the smoke, mtmc and assembly manifests built at
mainand on this branch without opting in are byte-identical (same sha256 and same byte length on all three), and a test pins manifest equality with and without opt-in.build_manifestreads onlytopology/cameras/entities/occluders/fps/num_frames, so neitheractivitynorpossessionis on the manifest path at all.Boundary behaviour is tested at the exact half-open edges (frames 3/4, 6/7, 9/10 handovers), not at interval midpoints, since an off-by-one at an edge is the obvious failure mode here.
Follow-ups / Known Limitations
One precision point on "open to more without a schema fork", which is worth your call. Adding a state is genuinely additive producer-side — a new enum member serialises as a plain string, and existing files with known states keep deserialising fine. But a consumer using the shipped typed model does not accept a state it has never heard of:
ActivityTimeline.model_validate_json(...)with"state": "walking"raisesValidationError. So forward-compatibility holds for raw-JSON consumers, not for typed ones.I have matched the merged
OrderStatusprecedent exactly here, which has the identical property by construction, rather than making this one module lenient and creating a second idiom for the same concept. If you would rather the typed models accept unknown states, that is a repo-wide decision affectingOrderStatustoo and I am happy to do it as a separate PR — it did not feel like mine to make inside this one.Generated by Claude Opus 5 (brief, review), Kimi K3 (implementation)