NIXL EP: comparison report

[podkidyshev]

Summary

• NIXL EP comparison report added

Test Plan

• Automated CI
• Manual runs

Additional Notes

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: e3c496b5-d031-45d4-88eb-b7670405f1db

📥 Commits

Reviewing files that changed from the base of the PR and between 6f678f7 and 7f975b0.

📒 Files selected for processing (7)

• src/cloudai/registration.py
• src/cloudai/workloads/nixl_ep/__init__.py
• src/cloudai/workloads/nixl_ep/log_parsing.py
• src/cloudai/workloads/nixl_ep/nixl_ep_comparison_report.py
• tests/test_init.py
• tests/workloads/nixl_ep/test_comparison_report.py
• tests/workloads/nixl_ep/test_log_parsing.py

---

📝 Walkthrough

Walkthrough

This PR introduces a new NixlEPComparisonReport class to generate comparison reports and visualizations for NIXL EP benchmarks. It updates log parsing to accept optional timing fields, implements data extraction and aggregation from per-node logs, creates Rich tables and Bokeh bar charts for metric comparisons, registers the report in the system, and validates the feature through comprehensive tests.

Changes

NIXL EP Comparison Report Feature

Layer / File(s)	Summary
Log parsing regex enhancement src/cloudai/workloads/nixl_ep/log_parsing.py	_KINETO_BW_RE now accepts optional avg_t=... us timing fields around dispatch and combine bandwidth values, preserving the rank and bandwidth GB/s capture groups.
NixlEPComparisonReport implementation src/cloudai/workloads/nixl_ep/nixl_ep_comparison_report.py	New comparison report class filters test runs to NixlEPTestDefinition, extracts per-node metrics from log files, computes aggregated means, dynamically detects available bandwidth/time columns, builds Rich comparison tables, and generates Bokeh categorical bar charts with tooltips and y-axis scaling based on observed maxima.
Module exports and registry wiring src/cloudai/workloads/nixl_ep/__init__.py, src/cloudai/registration.py	NixlEPComparisonReport is re-exported from the nixl_ep module and registered as a scenario report named nixl_ep_comparison with an enabled ComparisonReportConfig.
Test coverage and validation tests/workloads/nixl_ep/test_comparison_report.py, tests/workloads/nixl_ep/test_log_parsing.py, tests/test_init.py	New test validates comparison report HTML generation with synthetic test runs and node logs; log parsing test updated to reflect optional timing fields; registry structure tests extended to include the new scenario report key and config entry.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

• srivatsankrishnan
• jeffnvidia
• amaslenn

Poem

🐰 A new report hops into view,
Parsing kineto logs, both old and new,
Tables and charts in Bokeh's glow,
NIXL EP benchmarks steal the show! ✨

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding a comparison report feature for NIXL EP workload.
Description check	✅ Passed	The description is related to the changeset, accurately describing that a NIXL EP comparison report was added.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch ipod/nixlep-comparison

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}