Skip to content

Print benchmark comment body to stdout on fork PRs#183

Merged
NullVoxPopuli merged 2 commits intomainfrom
copilot/fix-ci-print-stdout-comments
Mar 18, 2026
Merged

Print benchmark comment body to stdout on fork PRs#183
NullVoxPopuli merged 2 commits intomainfrom
copilot/fix-ci-print-stdout-comments

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 18, 2026

Fork PRs can't post PR comments due to permissions, but the CI also wasn't printing the comment content to the job log—so the only way to see results was navigating to the job summary.

  • Print the full formatted benchmark comment via core.info() when fork is detected, with delimiters for readability

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Fix CI to print expected comments for forks Print benchmark comment body to stdout on fork PRs Mar 18, 2026
Copilot AI requested a review from NullVoxPopuli March 18, 2026 03:35
@NullVoxPopuli NullVoxPopuli marked this pull request as ready for review March 18, 2026 03:54
@github-actions
Copy link
Copy Markdown
Contributor

🏎️ Benchmark Comparison

Benchmark Control (avg) Experiment (avg) Δ
🟡 gts small 2.32 ms 2.23 ms -4.3%
🟢 gts medium 7.83 ms 7.40 ms -5.5%
gts large 20.11 ms 20.26 ms +0.7%
🟡 gjs small 1.61 ms 1.63 ms +1.5%
🟡 gjs medium 6.28 ms 6.21 ms -1.1%
🔴 gjs large 14.13 ms 14.94 ms +5.7%
🟢 hbs small 188.62 µs 176.78 µs -6.3%
hbs medium 2.73 ms 2.72 ms -0.4%
hbs large 14.93 ms 14.81 ms -0.8%

🟢 faster · 🔴 slower · 🟡 within 5% · ⚪ within 1%

Full mitata output
clk: ~3.34 GHz
cpu: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
runtime: node 24.14.0 (x64-linux)

benchmark                   avg (min … max) p75 / p99    (min … top 1%)
------------------------------------------- -------------------------------
gts small (control)            2.32 ms/iter   2.78 ms ▇█                   
                       (1.15 ms … 12.49 ms)   5.80 ms ██▆▃▂▂               
                    ( 45.68 kb …   2.74 mb) 635.36 kb ██████▇▄▄▃▃▄▅▅▆▂▁▂▂▂▁

gts small (experiment)         2.23 ms/iter   2.60 ms ▅█▃                  
                       (1.14 ms … 11.96 ms)   6.05 ms ███▄▃                
                    ( 74.88 kb …   1.86 mb) 624.70 kb █████▅▅▄▃▃▃▃▆▃▄▂▂▂▁▁▂

                             ┌                                            ┐
                             ╷ ┌────────┬───┐                           ╷
         gts small (control) ├─┤        │   ├───────────────────────────┤
                             ╵ └────────┴───┘                           ╵
                             ╷ ┌───────┬──┐                               ╷
      gts small (experiment) ├─┤       │  ├───────────────────────────────┤
                             ╵ └───────┴──┘                               ╵
                             └                                            ┘
                             1.14 ms            3.59 ms             6.05 ms

summary
  gts small (experiment)
   1.04x faster than gts small (control)

------------------------------------------- -------------------------------
gts medium (control)           7.83 ms/iter   8.30 ms  █▇▃                 
                       (4.10 ms … 30.06 ms)  24.17 ms ▅███                 
                    (509.30 kb …  11.78 mb)   2.62 mb █████▆▄▄▂▅▄▂▂▂▁▁▁▁▁▁▂

gts medium (experiment)        7.40 ms/iter   8.78 ms  █                   
                       (4.04 ms … 24.94 ms)  24.31 ms  █                   
                    (527.63 kb …  11.80 mb)   2.62 mb ▅█▇▇▅▇▃▂▄▂▂▁▂▁▁▁▁▁▁▁▂

                             ┌                                            ┐
                             ╷  ┌────┬┐                                   ╷
        gts medium (control) ├──┤    │├───────────────────────────────────┤
                             ╵  └────┴┘                                   ╵
                             ╷ ┌────┬───┐                                 ╷
     gts medium (experiment) ├─┤    │   ├─────────────────────────────────┤
                             ╵ └────┴───┘                                 ╵
                             └                                            ┘
                             4.04 ms           14.18 ms            24.31 ms

summary
  gts medium (experiment)
   1.06x faster than gts medium (control)

------------------------------------------- -------------------------------
gts large (control)           20.11 ms/iter  22.83 ms    █                 
                      (10.49 ms … 42.44 ms)  42.13 ms ▃▆ █ ▆               
                    (  5.53 mb …   8.53 mb)   6.90 mb ████▁█▄▄█▄▁▁▁▄█▁▁▁▄▁▄

gts large (experiment)        20.26 ms/iter  23.13 ms ▂█ ██  ▂             
                       (9.90 ms … 44.10 ms)  41.08 ms ██ ██▅ █▅           ▅
                    (  3.90 mb …  19.74 mb)   7.33 mb ██▇███▁██▇▇▁▇▁▁▇▁▁▇▁█

                             ┌                                            ┐
                              ╷   ┌────────┬───┐                          ╷
         gts large (control)  ├───┤        │   ├──────────────────────────┤
                              ╵   └────────┴───┘                          ╵
                             ╷  ┌──────────┬───┐                         ╷
      gts large (experiment) ├──┤          │   ├─────────────────────────┤
                             ╵  └──────────┴───┘                         ╵
                             └                                            ┘
                             9.90 ms           26.02 ms            42.13 ms

summary
  gts large (control)
   1.01x faster than gts large (experiment)

------------------------------------------- -------------------------------
gjs small (control)            1.61 ms/iter   1.47 ms █                    
                     (898.03 µs … 22.07 ms)   6.31 ms █▆                   
                    ( 66.76 kb …   6.43 mb) 572.21 kb ██▅▂▃▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁

gjs small (experiment)         1.63 ms/iter   1.46 ms █                    
                     (933.99 µs … 19.04 ms)   7.15 ms █▂                   
                    (105.77 kb …   3.81 mb) 559.26 kb ██▅▂▂▂▂▁▁▁▂▁▁▁▁▁▁▁▁▁▁

                             ┌                                            ┐
                             ┌────┬                                 ╷
         gjs small (control) │    │─────────────────────────────────┤
                             └────┴                                 ╵
                             ╷┌───┬                                       ╷
      gjs small (experiment) ├┤   │───────────────────────────────────────┤
                             ╵└───┴                                       ╵
                             └                                            ┘
                             898.03 µs           4.03 ms            7.15 ms

summary
  gjs small (control)
   1.01x faster than gjs small (experiment)

------------------------------------------- -------------------------------
gjs medium (control)           6.28 ms/iter   6.73 ms █▃                   
                       (3.33 ms … 30.05 ms)  30.04 ms ██▂                  
                    (  1.10 mb …   8.26 mb)   2.38 mb ███▆▄▄▂▂▃▁▁▁▁▁▁▁▁▂▁▁▂

gjs medium (experiment)        6.21 ms/iter   6.95 ms █▅                   
                       (3.19 ms … 29.21 ms)  25.14 ms ██▄▂                 
                    (  1.03 mb …  11.02 mb)   2.35 mb █████▆▃▃▃▂▁▁▁▁▁▁▁▁▁▁▂

                             ┌                                            ┐
                             ╷┌───┬┐                                      ╷
        gjs medium (control) ├┤   │├──────────────────────────────────────┤
                             ╵└───┴┘                                      ╵
                             ╷┌───┬┐                              ╷
     gjs medium (experiment) ├┤   │├──────────────────────────────┤
                             ╵└───┴┘                              ╵
                             └                                            ┘
                             3.19 ms           16.62 ms            30.04 ms

summary
  gjs medium (experiment)
   1.01x faster than gjs medium (control)

------------------------------------------- -------------------------------
gjs large (control)           14.13 ms/iter  16.03 ms  █                   
                       (7.85 ms … 38.49 ms)  35.66 ms  █ ▃ ▃  ▃            
                    (  4.38 mb …   7.89 mb)   6.19 mb ████▅█▅▅█▁▁▁▁▁▁▁▁▃▁▁▃

gjs large (experiment)        14.94 ms/iter  17.30 ms ██   ▄               
                       (7.80 ms … 37.71 ms)  34.34 ms ██▅ ▅██  █           
                    (  4.62 mb …  16.54 mb)   6.45 mb ███▅██████▁▅▁▁▁▁▁▁▅▁▅

                             ┌                                            ┐
                             ╷  ┌──────┬──┐                               ╷
         gjs large (control) ├──┤      │  ├───────────────────────────────┤
                             ╵  └──────┴──┘                               ╵
                             ╷ ┌─────────┬──┐                           ╷
      gjs large (experiment) ├─┤         │  ├───────────────────────────┤
                             ╵ └─────────┴──┘                           ╵
                             └                                            ┘
                             7.80 ms           21.73 ms            35.66 ms

summary
  gjs large (control)
   1.06x faster than gjs large (experiment)

------------------------------------------- -------------------------------
hbs small (control)          188.62 µs/iter 110.57 µs █                    
                      (80.70 µs … 23.32 ms)   2.76 ms █                    
                    (  2.77 kb …   1.83 mb) 137.43 kb █▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁

hbs small (experiment)       176.78 µs/iter 104.62 µs █                    
                      (76.31 µs … 20.58 ms)   2.50 ms █                    
                    (  2.58 kb …   2.94 mb) 137.54 kb █▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁

                             ┌                                            ┐
                             ┌─┬                                          ╷
         hbs small (control) │ │──────────────────────────────────────────┤
                             └─┴                                          ╵
                             ┌─┬                                      ╷
      hbs small (experiment) │ │──────────────────────────────────────┤
                             └─┴                                      ╵
                             └                                            ┘
                             76.31 µs            1.42 ms            2.76 ms

summary
  hbs small (experiment)
   1.07x faster than hbs small (control)

------------------------------------------- -------------------------------
hbs medium (control)           2.73 ms/iter   2.43 ms █                    
                       (1.31 ms … 21.24 ms)  19.36 ms █                    
                    (276.47 kb …   3.85 mb)   1.97 mb █▅▃▃▂▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁

hbs medium (experiment)        2.72 ms/iter   2.49 ms █                    
                       (1.31 ms … 20.75 ms)  19.91 ms █                    
                    ( 29.02 kb …   4.07 mb)   1.99 mb █▅▃▄▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁

                             ┌                                            ┐
                             ┌──┬                                        ╷
        hbs medium (control) │  │────────────────────────────────────────┤
                             └──┴                                        ╵
                             ┌──┬                                         ╷
     hbs medium (experiment) │  │─────────────────────────────────────────┤
                             └──┴                                         ╵
                             └                                            ┘
                             1.31 ms           10.61 ms            19.91 ms

summary
  hbs medium (experiment)
   1x faster than hbs medium (control)

------------------------------------------- -------------------------------
hbs large (control)           14.93 ms/iter  19.20 ms █                    
                       (8.29 ms … 29.86 ms)  29.70 ms █▆                   
                    (  7.33 mb …  10.62 mb)   9.14 mb ██▃▇▁▃█▁▅▁▃▁▁▃▅▁▅▇▁▃▃

hbs large (experiment)        14.81 ms/iter  17.04 ms █                    
                       (8.20 ms … 37.82 ms)  31.75 ms █  ▅                 
                    (  7.49 mb …  11.13 mb)   9.08 mb ██▃█▅▅▅▃▇▁▁▁▃▁▃▇▃▁▃▁▃

                             ┌                                            ┐
                             ╷┌───────────┬───────┐                   ╷
         hbs large (control) ├┤           │       ├───────────────────┤
                             ╵└───────────┴───────┘                   ╵
                             ╷┌───────────┬───┐                           ╷
      hbs large (experiment) ├┤           │   ├───────────────────────────┤
                             ╵└───────────┴───┘                           ╵
                             └                                            ┘
                             8.20 ms           19.97 ms            31.75 ms

summary
  hbs large (experiment)
   1.01x faster than hbs large (control)

@NullVoxPopuli NullVoxPopuli merged commit 84fb9ee into main Mar 18, 2026
34 checks passed
@NullVoxPopuli NullVoxPopuli deleted the copilot/fix-ci-print-stdout-comments branch March 18, 2026 03:56
@github-actions github-actions Bot mentioned this pull request Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants