Skip to content

add bench test#220

Merged
EmmasBox merged 10 commits into
Mainframe-Renewal-Project:devfrom
yinjie-z:feat/add-performance-test
Mar 15, 2026
Merged

add bench test#220
EmmasBox merged 10 commits into
Mainframe-Renewal-Project:devfrom
yinjie-z:feat/add-performance-test

Conversation

@yinjie-z

@yinjie-z yinjie-z commented Mar 15, 2026

Copy link
Copy Markdown
Collaborator

💡 Issue Reference

Issue: #217

💻 What does this address?

This PR introduces performance optimizations and formal benchmarks for the sear library. It specifically targets the modernization of IRRSMO00, IRRSEQ00, and IRRSDL00 request generation.

📟 Implementation Details

CMake Infrastructure

  • New Build Option: Introduced the SEAR_BUILD_BENCHMARKS toggle (Default: OFF) to prevent benchmark overhead in standard builds.
  • Target bench_runner: Created a dedicated benchmark executable.
  • Optimization: Forced -O3 optimization levels for the benchmark target to ensure timing reflects production-grade performance rather than unoptimized debug code.

CI/CD Integration

  • Automated Benchmarking: Integrated a GitHub Workflow that triggers on every Pull Request.
  • Insightful Reporting: Configured the workflow to parse benchmark results and automatically post a Performance Summary Table as a PR comment.
  • Regression Tracking: Set a threshold to alert developers if a code change causes a significant increase in execution time or memory growth.

📋 Is there a test case?

cmake --preset linux-bench -DBUILD_SHARED_LIBS=on
cmake --build --preset linux-bench --target sear
cmake --build --preset linux-bench --target bench_runner

@github-actions github-actions Bot added build Related to packaging, compiling, and GitHub actions tests This tag is for pull requests and issues relating to unit tests or functional tests labels Mar 15, 2026
@yinjie-z yinjie-z force-pushed the feat/add-performance-test branch from 06fe0f1 to 644cdec Compare March 15, 2026 12:27
@yinjie-z

Copy link
Copy Markdown
Collaborator Author

The Benchmark Test workflow currently encounters a 403 Forbidden error (Resource not accessible by integration) when attempting to post the results table.

This is due to GitHub's security model for on: pull_request triggers originating from forks, which restricts the GITHUB_TOKEN to read-only access.

I am updating the workflow trigger to on: pull_request_target. This will allow the runner to access the base repository's write permissions safely, enabling the automated benchmark summary to be posted directly to this PR.

@EmmasBox EmmasBox left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@EmmasBox EmmasBox merged commit bb5cf56 into Mainframe-Renewal-Project:dev Mar 15, 2026
6 checks passed
@yinjie-z yinjie-z deleted the feat/add-performance-test branch March 23, 2026 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Related to packaging, compiling, and GitHub actions tests This tag is for pull requests and issues relating to unit tests or functional tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants