Current detection-correctness metric (see docs/evaluation.md) is presence-based: per scripted scene, was the correct label detected at all (yes/no). A more rigorous version is possible since evaluation runs in Gazebo sim: project each object's known 3D pose/extents (from the world SDF) into the camera frame using the same intrinsics/TF the semantic detector already computes (inverse of the existing 2D→3D unprojection in semantic_detector.py), producing a synthetic ground-truth 2D bbox per visible object per frame. Compare against GroundingDINO's actual output frame-by-frame, threshold IoU at 0.5, compute per-class precision/recall (TP/FP/FN).
Open problem: naive projection will place a ground-truth bbox for an object even when something else occludes it in the actual view (e.g. a shelf blocking a pallet), which would incorrectly count as a false negative. Needs raycasting or some occlusion heuristic to be correct.
Not started — scoped as future work, not current priority.
Current detection-correctness metric (see
docs/evaluation.md) is presence-based: per scripted scene, was the correct label detected at all (yes/no). A more rigorous version is possible since evaluation runs in Gazebo sim: project each object's known 3D pose/extents (from the world SDF) into the camera frame using the same intrinsics/TF the semantic detector already computes (inverse of the existing 2D→3D unprojection insemantic_detector.py), producing a synthetic ground-truth 2D bbox per visible object per frame. Compare against GroundingDINO's actual output frame-by-frame, threshold IoU at 0.5, compute per-class precision/recall (TP/FP/FN).Open problem: naive projection will place a ground-truth bbox for an object even when something else occludes it in the actual view (e.g. a shelf blocking a pallet), which would incorrectly count as a false negative. Needs raycasting or some occlusion heuristic to be correct.
Not started — scoped as future work, not current priority.