Average multi microbenchmarks results

[VincentBu]

This PR aims at calculating average value of multiple microbenchmarks results. The work revolves around:

1. Reduce memory usage.
2. Change namespace of some classes and rename them for future work.

[Copilot]

Pull request overview

This PR updates the GC microbenchmark infrastructure to support aggregating (averaging) results across multiple microbenchmark runs/iterations, while also renaming/refactoring parts of the analysis/presentation pipeline and introducing an outlier-removal helper.

Changes:

• Add configurable microbenchmark iteration count (iterations) and wire it into suite creation and execution.
• Replace the previous single-result comparison flow with a new per-benchmark aggregation/comparison pipeline (MicrobenchmarkResultComparison, GCTraceMetrics, GCTraceMetricComparisonResult).
• Refactor output generation to primarily emit JSON (markdown generation currently disabled).

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 18 comments.

Show a summary per file

File	Description
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure/Commands/RunCommand/CreateSuiteCommand.cs	Reads configured iteration count and applies it to microbenchmark suite environment.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure/Commands/RunCommand/BaseSuite/MicrobenchmarksToRun.txt	Updates baseline suite benchmark list.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure/Commands/RunCommand/BaseSuite/Microbenchmarks.yaml	Renames environment iteration setting to iterations.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure/Commands/Microbenchmark/MicrobenchmarkCommand.cs	Runs microbenchmarks for iterations and switches to new aggregation/comparison logic before presenting results.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure/Commands/Microbenchmark/MicrobenchmarkAnalyzeCommand.cs	Updates analysis-only command to use the new aggregation/comparison logic.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Presentation/Microbenchmarks/Presentation.cs	Changes presentation API to accept precomputed grouped results; markdown output path currently disabled.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Presentation/Microbenchmarks/Markdown.cs	Markdown generation code is commented out.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Presentation/Microbenchmarks/Json/JsonOutput.cs	Removes unused placeholder type.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Presentation/Microbenchmarks/Json.cs	Moves JSON generator to Microbenchmarks presentation namespace and updates signature for grouped results.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Configurations/Microbenchmarks.Configuration.cs	Renames iteration to iterations in microbenchmark environment configuration.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Configurations/InputConfiguration.cs	Adds iterations map to input configuration.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Analysis/Microbenchmarks/MicrobenchmarkResultsAnalyzer.cs	Removes old analyzer/comparison pipeline.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Analysis/Microbenchmarks/MicrobenchmarkResultComparison.cs	Adds new JSON/trace mapping, per-benchmark analysis, and aggregation/grouping logic.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Analysis/Microbenchmarks/MicrobenchmarkResult.cs	Introduces new MicrobenchmarkResult model (namespace currently mismatched vs usage).
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Analysis/Microbenchmarks/MicrobenchmarkComparisonResult.cs	Updates comparison to support averaged values/outlier removal and new trace-metric comparisons.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Analysis/GCTraceMetrics.cs	Adds trace-derived metric extraction (includes reflection/stat bugs).
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Analysis/GCTraceMetricComparisonResult.cs	Adds averaged comparison for trace metrics (baseline vs comparand).
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Analysis/GCTraceMetricComparison.cs	Adds helper wrapper for metric comparison construction.
src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Analysis/BdnJsonResult.cs	Refactors BDN JSON model types; renames top-level to BdnJsonResult.
src/benchmarks/gc/GC.Infrastructure/GC.Analysis.API/Statistics.cs	Adds RemoveOutliers helper (IQR method).
src/benchmarks/gc/GC.Infrastructure/Configurations/Run.yaml	Adds iteration configuration block (currently mismatched with new iterations input model).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 13 comments.

[Copilot]

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 11 comments.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 8 comments.

Comments suppressed due to low confidence (1)

src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Presentation/Microbenchmarks/Json.cs:15

• Json.Generate doesn’t use the configuration parameter, and the using GC.Analysis.API; / using GC.Infrastructure.Core.Presentation.GCPerfSim; directives are unused. Consider removing the unused parameter/usings to avoid warnings and keep the API surface minimal.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (2)

src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Analysis/Microbenchmarks/MicrobenchmarkResultComparison.cs:155

• When trace collection is enabled, tracePath falls back to an empty string if the JSON path isn’t present in jsonToTraceMap, and that empty path is passed to AnalyzerManager.GetAnalyzer(...), which will fail with a less actionable error. Prefer validating the mapping result (e.g., TryGetValue) and throwing an exception that includes the missing JSON path / benchmark name, or ensuring MapJsonToTrace guarantees coverage for every jsonPath.

                if ((!excludeTraces) && configuration.TraceConfigurations.Type != "none")
                {
                    string outputPathForRun = Path.Combine(configuration.Output.Path, run.Name!);
                    string tracePath = jsonToTraceMap.GetValueOrDefault(jsonPath, "");

                    using (var analyzer = AnalyzerManager.GetAnalyzer(tracePath))
                    {

src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Presentation/Microbenchmarks/Markdown.cs:181

• GetValueOrDefault(column) on Dictionary<string,double> returns 0 when the metric is missing, so baselineValue.HasValue is always true and missing metrics render as 0 (and deltaPercent can divide by 0). This can silently produce incorrect tables when a column can’t be computed (e.g., GC-derived columns when traces are unavailable). Use TryGetValue and treat missing metrics as null/empty, and guard the percent calculation when the baseline value is 0.

                        foreach (var column in configuration.Output.Columns)
                        {
                            double? baselineValue = lr.AveragedBaselineOtherMetrics.GetValueOrDefault(column);
                            double? comparandValue = lr.AveragedComparandOtherMetrics.GetValueOrDefault(column);

                            string baselineResult = baselineValue.HasValue ? Math.Round(baselineValue.Value, 4).ToString() : string.Empty;
                            string comparandResult = comparandValue.HasValue ? Math.Round(comparandValue.Value, 4).ToString() : string.Empty;
                            double? delta = baselineValue.HasValue && comparandValue.HasValue ? comparandValue.Value - baselineValue.Value : null;
                            string deltaResult = delta.HasValue ? Math.Round(delta.Value, 4).ToString() : string.Empty;

                            double? deltaPercent = delta.HasValue ? (delta / baselineValue.Value) * 100 : null;
                            string deltaPercentResult = deltaPercent.HasValue ? Math.Round(deltaPercent.Value, 4).ToString() : string.Empty;

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 6 comments.

Comments suppressed due to low confidence (1)

src/benchmarks/gc/GC.Infrastructure/GC.Infrastructure.Core/Analysis/BdnJsonResult.cs:25

• The BDN JSON POCOs declare many non-nullable reference-type properties (e.g., HostEnvironmentInfo.BenchmarkDotNetCaption, Benchmark.Measurements, BdnJsonResult.Benchmarks) without initialization. With <Nullable>enable</Nullable>, this produces CS8618 warnings and also makes it easy for deserialization to yield nulls at runtime if fields are absent. Consider making these properties nullable where appropriate or initializing with = null!; / empty collections for required fields.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.