Skip to content

fix(analyzer): retain fenced SKILL.md findings#276

Open
rodboev wants to merge 1 commit into
NVIDIA:mainfrom
rodboev:pr/skill-md-fenced-example-filtering
Open

fix(analyzer): retain fenced SKILL.md findings#276
rodboev wants to merge 1 commit into
NVIDIA:mainfrom
rodboev:pr/skill-md-fenced-example-filtering

Conversation

@rodboev

@rodboev rodboev commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Summary

The static runner now retains TM1 findings from fenced commands in canonical SKILL.md. It keeps existing code-example filtering for ordinary Markdown, including non-canonical *skill.md filenames, and leaves the unchanged analyzer-local fenced-example bypasses disclosed as residual scope.

Root cause

run_static_patterns() treated SKILL.md as non-executable Markdown, then dropped findings whose context contained triple backticks. The TM1 analyzer emitted the finding, but the runner removed it before reporting.

Changes

  • Kept the existing loose _is_skill_md(path) helper at the three legacy SKILL.md doc-filter exemption sites, and added _is_canonical_skill_md(path) for the new exact-basename runner exemption.
  • Skipped the generic code-example hard-drop and downweight branch only for files literally named SKILL.md.
  • Added runner coverage for fenced SKILL.md, fenced guide.md, non-canonical *skill.md paths that must stay filtered, and the loose-vs-canonical path-helper boundary.
  • Added a no-LLM CLI regression that writes the issue-shaped fixture to tmp_path / "SKILL.md" and asserts TM1 plus nonzero before/after filtering counts.

Scope

This implements only the fenced SKILL.md example-filtering slice of issue #268. It doesn't change meta_analyzer.py, scoring bands, report schema, analyzer registration, provider code, or other issue slices. The retained finding remains in the current LOW/SAFE aggregate band; this change surfaces and counts it without retuning scoring. Analyzer-local fenced-example bypasses still remain for TM4, EA2, E5, AR, and MP because those modules filter upstream of the runner.

Verification

  • pytest tests/nodes/analyzers/test_static_runner_filtering.py tests/unit/test_cli.py -v, 58 passed
  • pytest tests/unit/test_cli.py::test_cli_scan_no_llm_reports_fenced_skill_md_tool_misuse -q, 1 passed
  • pytest -m "not integration and not provider" tests/ -q, 1321 passed, 12 skipped, 34 deselected, 6 xfailed
  • uv run ruff check src/ tests/
  • uv run ruff format --check src/ tests/
  • git diff --check

Base reproduction: issues: [], findings_before_filtering: 0, and findings_after_filtering: 0.
Head reproduction: the CLI regression asserts TM1 in issues and both filtering counts greater than zero.

Upstream

Refs #268

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant