diff --git a/CSC2210H.pdf b/CSC2210H.pdf new file mode 100644 index 0000000..1839d9c Binary files /dev/null and b/CSC2210H.pdf differ diff --git a/README.md b/README.md index 90621ac..ac2fb08 100644 --- a/README.md +++ b/README.md @@ -136,6 +136,54 @@ scripts/slurm/submit_kerneltuner.sh \ --mem 24GB ``` +## Slurm Submission Helpers + +For cluster execution, the repo includes reusable Slurm scripts: + +- `scripts/slurm/run_kerneltuner_array.sbatch`: array worker that maps one array task to one experiment YAML. +- `scripts/slurm/submit_kerneltuner.sh`: submit wrapper that computes array size from a list file. + +Example list file: + +- `configs/experiments/slurm_experiment_list.example.txt` + +Basic usage: + +```bash +chmod +x scripts/slurm/submit_kerneltuner.sh +cp configs/experiments/slurm_experiment_list.example.txt configs/experiments/slurm_experiment_list.txt +scripts/slurm/submit_kerneltuner.sh \ + --list configs/experiments/slurm_experiment_list.txt \ + --partition gpunodes \ + --gpu-type rtx_a2000 \ + --gpus 1 \ + --time 0-04:00 \ + --cpus 4 \ + --mem 24GB \ + --log-dir /scratch/scratch-space/expires-xxxx/$USER/kerneltuner_logs \ + --scratch-root /scratch/scratch-space/expires-xxxx/$USER/kerneltuner \ + --artifact-root /scratch/scratch-space/expires-xxxx/$USER/kerneltuner_artifacts \ + --mail-user you@example.com \ + --mail-type BEGIN,END,FAIL \ + --alert-email you@example.com \ + --alert-on-start +``` + +Useful environment overrides: + +- `RUN_COMMAND_TEMPLATE` (default: `ktune run-experiment --experiment "{experiment}"`) +- `INSTALL_PACKAGES` (`0` to skip pip install in jobs) +- `EXTRA_PIP_PACKAGES` (space-separated extra pip packages) +- `SKIP_IF_ARTIFACTS_EXIST` (`0` to force reruns) +- `DRY_RUN=1` (worker-level dry run) +- `ALERT_EMAIL`, `ALERT_ON_START`, `ALERT_ON_END`, `ALERT_ON_FAIL` for worker-level alerts + +Notes: + +- `--artifact-root` overrides `artifact_root` in experiment YAML at submission time. +- If no scratch path exists on the node, jobs fall back to `/.scratch/$USER`. +- Slurm native email (`--mail-user/--mail-type`) and worker-level alerts can be used together. + ## What This Repo Is Not - It is not a Triton compiler redesign effort. diff --git a/docs/research/00_index.md b/docs/research/00_index.md index 8502064..5a83547 100644 --- a/docs/research/00_index.md +++ b/docs/research/00_index.md @@ -42,9 +42,9 @@ Current project phase: - the completed Phase 3 execution analysis is [logs/2026-03-29_phase3_execution_analysis.md](logs/2026-03-29_phase3_execution_analysis.md) - the completed R6 final synthesis and evidence lock is [logs/2026-03-30_r6_final_synthesis_and_evidence_lock.md](logs/2026-03-30_r6_final_synthesis_and_evidence_lock.md) - the earlier documentation freeze point remains recorded in [logs/2026-03-28_phase3_documentation_boundary.md](logs/2026-03-28_phase3_documentation_boundary.md) as historical context only -- the canonical reusable Phase 2 artifact summary lives under `artifacts/analysis/phase2_20260327/` -- the canonical reusable Phase 3 artifact summary lives under `artifacts/analysis/phase3_20260329/` -- the canonical final paper-evidence bundle lives under `artifacts/analysis/final_paper_20260330/` +- for paper writing and verification in this repo snapshot, treat `docs/research/`, `docs/adr/`, and `logs/` as the canonical documentation backbone +- use the evidence registry, final claim inventory, and dated analysis logs as the primary source of truth for promoted claims +- treat older references to `artifacts/analysis/...` as historical pointers rather than required local dependencies for reading the paper backbone - the completed bounded R6 batch is now promoted; no further selector-family expansion is justified inside the current paper backbone ## Reading Order diff --git a/paper/CLAIM_VERIFICATION.md b/paper/CLAIM_VERIFICATION.md new file mode 100644 index 0000000..386a64d --- /dev/null +++ b/paper/CLAIM_VERIFICATION.md @@ -0,0 +1,109 @@ +# Claim Verification Notes + +This note records the post-edit verification pass for the current manuscript. + +## Verification Scope + +Verified against: + +- `docs/research/11_final_claim_inventory.md` +- `docs/research/08_evidence_registry.md` +- `docs/research/12_related_work_and_positioning.md` +- `docs/adr/ADR-001-single-host-single-gpu.md` +- `docs/adr/ADR-004-matched-budget-evaluation.md` +- `paper/DRAFT_CHECKLIST.md` +- `paper/appendix/extended_results.tex` +- dated execution-analysis logs under `docs/research/logs/` + +Primary paper sections checked: + +- `paper/sections/abstract.tex` +- `paper/sections/introduction.tex` +- `paper/sections/results.tex` +- `paper/sections/discussion.tex` +- `paper/sections/conclusion.tex` + +## Main Outcome + +The current manuscript stays within the locked paper-safe claim set. + +The core claim structure is consistent across abstract, introduction, results, discussion, and conclusion: + +- cheap compile-adjacent signals help with pruning more than ranking, +- aligned GEMM is supporting context rather than the truth source, +- LayerNorm is a bounded regime-split secondary result, +- revised selectors are a mixed transfer story rather than a universal success, +- `H5` remains the unsupported Phase 3 `split_k` result, +- and the final non-`split_k` mainline result is a bounded positive headline. + +## Numeric Claims Checked + +The following main-text numbers were checked against the evidence registry and dated logs: + +- Phase 2 representative GEMM: + - `naive_random_search = 1.0331` + - `prune_rank = 0.8265` + - `prune_rank_profiled = 0.8444` +- Phase 2 aligned GEMM: + - `prune_rank = 0.8795` + - `prune_rank_profiled = 0.8873` + - `naive_random_search = 0.9951` +- Phase 2 LayerNorm: + - `small_batch`: `1.0100` vs `1.0113` + - `large_batch`: `1.0029` vs `0.9856` +- Phase 3 `split_k` result: + - parent `prune_rank = 1.0324` + - `naive_random_search = 1.0427` + - `v4_transfer_safe_profiled = 0.1609` + - ablation values `0.1622`, `0.1623`, and parent `0.9692` +- Final R6 mainline: + - parent `prune_rank = 0.8101` + - `naive_random_search = 1.0011` + - `v5_mainline_profiled = 1.0286` + - improvement over parent `0.2185` + +## Checklist Status + +Checked and satisfied in the current source: + +- every main-text claim matches the final claim inventory or a weaker paraphrase +- `H5` is still written only as the unsupported Phase 3 `split_k` result +- `R6` is still written as a bounded improvement, not a universal win +- LayerNorm remains secondary and regime-split +- introduction, abstract, results, discussion, and conclusion now tell the same final story +- appendices remain chronology/provenance oriented rather than introducing new promoted claims + +## Documentation Backbone Used For Verification + +For this workspace snapshot, the paper backbone should be treated as: + +- `docs/research/` for claims, interpretation, and experiment history, +- `docs/adr/` for fixed environment and evaluation decisions, +- and `docs/research/logs/` for the chronological execution record. + +The artifact bundle paths referenced in older documentation are not the primary verification source in this checkout. + +This verification pass is therefore grounded in: + +- the claim ledger, +- the evidence registry, +- the related-work and paper-outline research docs, +- the relevant ADRs, +- appendix artifact mappings, +- and the dated analysis logs. + +That is strong enough to verify that the manuscript stays consistent with the repo's documented evidence backbone. + +## Paper Build Status + +The paper compiles successfully with: + +- `tectonic -X compile main.tex` + +Current output: + +- `paper/main.pdf` + +Known remaining issue: + +- the PDF still has many table and appendix line-breaking warnings (`Underfull` / `Overfull` boxes), but the build succeeds and the references now start on a new page. diff --git a/paper/FIGURE_MANIFEST.md b/paper/FIGURE_MANIFEST.md index 4905bc2..2c072c9 100644 --- a/paper/FIGURE_MANIFEST.md +++ b/paper/FIGURE_MANIFEST.md @@ -3,8 +3,8 @@ | Figure | Purpose | Source Artifact | Claim IDs | Draft Status | | --- | --- | --- | --- | --- | | Figure 1 | Conceptual overview of the \system pipeline from candidate generation to final evidence | Derived from `docs/research/04_signal_and_profiling_plan.md`, `docs/research/06_hypotheses_and_ablation_plan.md`, and `docs/specs/` | `C-FINAL-H1`, `C-FINAL-HEADLINE` | LaTeX-native conceptual schematic | -| Figure 2 | Final representative GEMM headline comparison | `artifacts/analysis/final_paper_20260330/figure_source_map.csv` row `F1` | `C-FINAL-HEADLINE` | LaTeX-native numeric figure | -| Figure 3 | Aligned vs representative GEMM context | `artifacts/analysis/final_paper_20260330/figure_source_map.csv` row `F2` | `C-FINAL-H1`, `C-FINAL-H3` | LaTeX-native numeric figure | -| Figure 4 | LayerNorm regime split | `artifacts/analysis/final_paper_20260330/figure_source_map.csv` row `F3` | `C-FINAL-H2`, `C-FINAL-ROWS` | LaTeX-native numeric figure | -| Figure 5A | Final mainline selector ablation | `artifacts/analysis/final_paper_20260330/figure_source_map.csv` row `F7` | `C-FINAL-HEADLINE`, `C-FINAL-H4`, `C-FINAL-CLOSEOUT` | LaTeX-native numeric panel | -| Figure 5B | Chosen-family vs best-family diagnostic | `artifacts/analysis/final_paper_20260330/figure_source_map.csv` row `F11` | `C-FINAL-H5`, `C-FINAL-SPLITK` | LaTeX-native diagnostic panel | +| Figure 2 | Final representative GEMM headline comparison | `docs/research/08_evidence_registry.md`, `docs/research/11_final_claim_inventory.md`, `docs/research/logs/2026-03-30_r6_final_synthesis_and_evidence_lock.md` | `C-FINAL-HEADLINE` | LaTeX-native numeric figure | +| Figure 3 | Aligned vs representative GEMM context | `docs/research/08_evidence_registry.md`, `docs/research/11_final_claim_inventory.md`, `docs/research/logs/2026-03-27_phase2_execution_analysis.md` | `C-FINAL-H1`, `C-FINAL-H3` | LaTeX-native numeric figure | +| Figure 4 | LayerNorm regime split | `docs/research/08_evidence_registry.md`, `docs/research/11_final_claim_inventory.md`, `docs/research/logs/2026-03-27_phase2_execution_analysis.md`, `docs/research/logs/2026-03-29_phase3_execution_analysis.md` | `C-FINAL-H2`, `C-FINAL-ROWS` | LaTeX-native numeric figure | +| Figure 5A | Final mainline selector ablation | `docs/research/08_evidence_registry.md`, `docs/research/11_final_claim_inventory.md`, `docs/research/logs/2026-03-30_r6_final_synthesis_and_evidence_lock.md` | `C-FINAL-HEADLINE`, `C-FINAL-H4`, `C-FINAL-CLOSEOUT` | LaTeX-native numeric panel | +| Figure 5B | Chosen-family vs best-family diagnostic | `docs/research/08_evidence_registry.md`, `docs/research/11_final_claim_inventory.md`, `docs/research/logs/2026-03-29_phase3_execution_analysis.md` | `C-FINAL-H5`, `C-FINAL-SPLITK` | LaTeX-native diagnostic panel | diff --git a/paper/OLD_PDF_DELTA.md b/paper/OLD_PDF_DELTA.md new file mode 100644 index 0000000..056e405 --- /dev/null +++ b/paper/OLD_PDF_DELTA.md @@ -0,0 +1,72 @@ +# Old PDF vs Current Source + +This note records the selective review of the checked-in `CSC2210H_Project.pdf` against the current LaTeX source in `paper/`. + +## Status of the checked-in PDF + +- The checked-in PDF is an older snapshot created on March 22, 2026. +- It is only 3 pages long. +- It still contains placeholder front matter, including the title `TITLE`. +- It should not be treated as the current source of truth. + +The current paper state is the LaTeX source rooted at `paper/main.tex`. + +## Front Matter Mismatch + +The strongest mismatch is the front matter: + +- old PDF title: `TITLE` +- current source title: `Bottleneck-Aware Schedule-First Tuning for Triton Kernels Under Matched Budget` +- old PDF author line: `Pedram` and `Giovanni Galea Curmi` +- current source author line: `Project Draft` + +The old PDF therefore captures an earlier milestone, not the present manuscript state. + +## What Exists Locally But Not in the Old PDF + +The old PDF contains an early short draft with an introduction, related-work framing, and references. The current source now includes a full manuscript structure: + +- abstract, +- introduction, +- background and problem setting, +- system design, +- experimental methodology, +- results, +- discussion and limitations, +- related work, +- conclusion, +- and appendices for methods, results, and provenance. + +The current source also includes the main paper tables, figure manifest, table manifest, and section index used to keep the draft aligned with the evidence bundle. + +## What the Old PDF Still Does Well + +The old PDF has a few writing choices worth preserving because they are direct and easy to follow: + +- It states the narrow project question early. +- It explains the paper's "middle ground" positioning in simple language. +- It plainly says what the project is not trying to do: redesign Triton, replace vendor libraries, or generate kernels from scratch. +- It introduces the practical tuning problem before moving into literature positioning. + +These choices should remain visible in the current source even as the manuscript becomes more complete. + +## Wording Worth Preserving Selectively + +The following phrasing patterns from the old PDF are worth keeping when they fit naturally: + +- "keeps the Triton kernel fixed" +- "a narrower but still meaningful question" +- "under the same budget" +- explicit contrasts such as "not redesign Triton" and "not generate new kernels from scratch" + +These phrases are useful because they set scope quickly and avoid inflated language. + +## Main Difference in Writing Density + +The current source is more complete and scientifically stronger, but it is also denser than the old PDF in some places, especially in `Results` and `Discussion`. + +The editing goal is therefore: + +- keep the newer evidence-backed structure, +- preserve the clearer scope-setting language from the old PDF when helpful, +- and avoid drifting into a more compressed research voice than the paper needs. diff --git a/paper/SECTION_INDEX.md b/paper/SECTION_INDEX.md index dfdf277..0ef0074 100644 --- a/paper/SECTION_INDEX.md +++ b/paper/SECTION_INDEX.md @@ -2,17 +2,17 @@ This file maps manuscript sections to the source docs, canonical artifacts, claim IDs, and figure/table IDs used in the first draft. -| Manuscript Section | Primary Source Docs | Primary Artifacts | Claim IDs | Figures / Tables | +| Manuscript Section | Primary Source Docs | Primary Supporting Sources | Claim IDs | Figures / Tables | | --- | --- | --- | --- | --- | -| Abstract | `docs/research/01_research_program.md`, `docs/research/11_final_claim_inventory.md` | `artifacts/analysis/final_paper_20260330/final_bundle_summary.md` | `C-FINAL-HEADLINE`, `C-FINAL-H2`, `C-FINAL-H5` | none | -| Introduction | `docs/research/01_research_program.md`, `docs/research/11_final_claim_inventory.md`, `docs/research/12_related_work_and_positioning.md` | `artifacts/analysis/final_paper_20260330/final_claim_table.csv` | `C-FINAL-HEADLINE`, `C-FINAL-H1`, `C-FINAL-H4`, `C-FINAL-CLOSEOUT` | `T1` | -| Background and Problem Setting | `docs/research/02_tuning_theory_and_knob_space.md`, `docs/research/03_bottleneck_taxonomy.md`, `docs/research/05_workload_matrix_and_case_studies.md`, `docs/research/12_related_work_and_positioning.md` | final claim bundle and final figure map | `C-FINAL-H1`, `C-FINAL-H3` | `T2`, `T3`, Figure 1 | -| KernelTuner Design | `docs/research/04_signal_and_profiling_plan.md`, `docs/research/06_hypotheses_and_ablation_plan.md` | final bundle, final configs | `C-FINAL-SPLITK`, `C-FINAL-ROWS` | Figure 1, `T2` | -| Experimental Methodology | `docs/research/05_workload_matrix_and_case_studies.md`, `docs/research/07_experiment_campaign_plan.md`, `docs/research/08_evidence_registry.md` | `artifacts/analysis/final_paper_20260330/canonical_artifact_map.csv` | all final claims indirectly | `T1`, `T3` | +| Abstract | `docs/research/01_research_program.md`, `docs/research/11_final_claim_inventory.md` | evidence registry plus final synthesis log | `C-FINAL-HEADLINE`, `C-FINAL-H2`, `C-FINAL-H5` | none | +| Introduction | `docs/research/01_research_program.md`, `docs/research/11_final_claim_inventory.md`, `docs/research/12_related_work_and_positioning.md` | final claim inventory and ADR-backed scope decisions | `C-FINAL-HEADLINE`, `C-FINAL-H1`, `C-FINAL-H4`, `C-FINAL-CLOSEOUT` | `T1` | +| Background and Problem Setting | `docs/research/02_tuning_theory_and_knob_space.md`, `docs/research/03_bottleneck_taxonomy.md`, `docs/research/05_workload_matrix_and_case_studies.md`, `docs/research/12_related_work_and_positioning.md` | research backbone docs and workload program | `C-FINAL-H1`, `C-FINAL-H3` | `T2`, `T3`, Figure 1 | +| Related Work | `docs/research/12_related_work_and_positioning.md` plus bibliography | literature positioning doc and bibliography | none directly | `T8` indirectly | +| KernelTuner Design | `docs/research/04_signal_and_profiling_plan.md`, `docs/research/06_hypotheses_and_ablation_plan.md` | research plan plus ADR decisions on scope and evaluation | `C-FINAL-SPLITK`, `C-FINAL-ROWS` | Figure 1, `T2` | +| Experimental Methodology | `docs/research/05_workload_matrix_and_case_studies.md`, `docs/research/07_experiment_campaign_plan.md`, `docs/research/08_evidence_registry.md`, `docs/adr/ADR-004-matched-budget-evaluation.md` | methodology and evaluation docs | all final claims indirectly | `T1`, `T3` | | Results 6.1 | `docs/research/08_evidence_registry.md`, `docs/research/11_final_claim_inventory.md` | `gemm_v2_baseline_mapping`, `gemm_v2_aligned_reference` | `C-FINAL-H1`, `C-FINAL-H3` | Figure 3, `T4` | | Results 6.2 | `docs/research/08_evidence_registry.md`, `docs/research/11_final_claim_inventory.md` | `layernorm_v2_small_regime`, `layernorm_v2_large_regime`, Phase 3 microstudy rows | `C-FINAL-H2`, `C-FINAL-ROWS` | Figure 4, `T4` | | Results 6.3 | `docs/research/08_evidence_registry.md`, `docs/research/09_opportunity_log.md`, `docs/research/11_final_claim_inventory.md` | `h4_retry_g3`, `gemm_v3_baseline_mapping`, `gemm_v3_selector_ablation`, `gemm_v3_schedule_diag` | `C-FINAL-H4`, `C-FINAL-H5`, `C-FINAL-SPLITK` | Figure 5, `T4` | | Results 6.4 | `docs/research/11_final_claim_inventory.md`, `docs/research/logs/2026-03-30_r6_final_synthesis_and_evidence_lock.md` | `gemm_final_baseline_mapping`, `gemm_final_selector_ablation` | `C-FINAL-HEADLINE`, `C-FINAL-CLOSEOUT` | Figure 2, Figure 5, `T4` | -| Discussion / Limitations | `docs/research/01_research_program.md`, `docs/research/08_evidence_registry.md`, `docs/research/11_final_claim_inventory.md` | final paper bundle | all final claims | none | -| Related Work | `docs/research/12_related_work_and_positioning.md` plus bibliography | bibliography | none directly | `T8` indirectly | -| Appendix | dated logs and final bundle | `artifacts/analysis/final_paper_20260330/`, phase2 and phase3 bundles | all final claims | appendix-only tables | +| Discussion / Limitations | `docs/research/01_research_program.md`, `docs/research/08_evidence_registry.md`, `docs/research/11_final_claim_inventory.md` | claim ledger, evidence registry, and ADR-backed limits | all final claims | none | +| Appendix | dated logs, research docs, and ADRs | `docs/research/logs/`, `docs/research/`, `docs/adr/` | all final claims | appendix-only tables | diff --git a/paper/TABLE_MANIFEST.md b/paper/TABLE_MANIFEST.md index deb7b29..3935693 100644 --- a/paper/TABLE_MANIFEST.md +++ b/paper/TABLE_MANIFEST.md @@ -5,6 +5,6 @@ | Table 1 | Study scope and research framing | `docs/research/01_research_program.md` | `C-FINAL-HEADLINE`, `C-FINAL-CLOSEOUT` | Included in main text | | Table 2 | Final knob families and signal tiers | `docs/research/02_tuning_theory_and_knob_space.md`, `docs/research/04_signal_and_profiling_plan.md` | `C-FINAL-SPLITK`, `C-FINAL-ROWS` | Included in main text | | Table 3 | Workload matrix summary | `docs/research/05_workload_matrix_and_case_studies.md` | `C-FINAL-H2`, `C-FINAL-H3` | Included in main text | -| Table 4 | Final hypothesis and claim summary | `docs/research/11_final_claim_inventory.md`, `artifacts/analysis/final_paper_20260330/final_claim_table.csv` | all final claims | Included in main text | +| Table 4 | Final hypothesis and claim summary | `docs/research/11_final_claim_inventory.md`, `docs/research/08_evidence_registry.md`, `docs/research/logs/2026-03-30_r6_final_synthesis_and_evidence_lock.md` | all final claims | Included in main text | | Appendix Table A1 | Round-by-round chronology | dated logs and `docs/research/07_experiment_campaign_plan.md` | provenance only | Appendix | -| Appendix Table A2 | Canonical artifact map | `artifacts/analysis/final_paper_20260330/canonical_artifact_map.csv` | provenance only | Appendix | +| Appendix Table A2 | Canonical artifact map | `docs/research/08_evidence_registry.md`, `docs/research/11_final_claim_inventory.md`, `docs/adr/ADR-001-single-host-single-gpu.md`, dated logs under `docs/research/logs/` | provenance only | Appendix | diff --git a/paper/WRITING_STYLE.md b/paper/WRITING_STYLE.md new file mode 100644 index 0000000..6638460 --- /dev/null +++ b/paper/WRITING_STYLE.md @@ -0,0 +1,113 @@ +# Paper Writing Style + +This file defines the writing standard for the paper draft in `paper/`. + +The target style is the readability of `MAT1510_Project.pdf`, not a denser or more performative research voice. The reader should feel that the paper is careful and technical, but never harder to read than it needs to be. + +## Core Goal + +Write for a technical reader who may not know Triton, autotuning, or GPU performance tuning in advance. + +The paper should: + +- explain terms before leaning on them, +- keep the main thread visible at all times, +- make claims only as strongly as the evidence allows, +- and stay readable even when presenting mixed or negative results. + +## Required Style Rules + +### 1. Define ideas before compressing them + +- Introduce a concept in plain language before using shorthand or specialized wording. +- If a term such as "matched budget," "representative GEMM," or "frontier construction" is important, explain it first in one clear sentence. +- Do not stack several new terms in the same sentence unless each one is already familiar from earlier text. + +### 2. Prefer short paragraphs with one job each + +- Most paragraphs should make one point. +- Use the first sentence to tell the reader what the paragraph is about. +- Avoid paragraphs that combine background, result, caveat, and interpretation all at once. + +### 3. Keep prose simpler than the evidence layer + +- The research docs and artifact manifests can stay dense. +- The paper should translate that evidence into a readable narrative. +- Do not write like a lab notebook or an execution log. + +### 4. Keep numeric detail moderate in prose + +- Important comparisons should appear in sentences. +- Dense value lists belong in tables, figures, or appendices. +- Use enough numbers to support the claim, but not so many that the paragraph becomes a spreadsheet. + +### 5. Keep caveats, but do not repeat them mechanically + +- Every promoted result needs its correct limitation. +- State the caveat once where it matters most, then move on unless the interpretation changes. +- Do not dilute the narrative by repeating the same limitation in every paragraph. + +### 6. Use only paper-safe claim wording + +- Main-text claim wording must match `docs/research/11_final_claim_inventory.md` or be weaker. +- Never strengthen a claim for rhetorical effect. +- If a sentence feels stronger than the claim ledger, weaken it. + +### 7. Preserve the paper's role structure + +- Representative GEMM is the primary truth source. +- Aligned GEMM is supporting context, not the headline. +- LayerNorm is a bounded secondary story, split by regime. +- `split_k` and `rows_per_program` are keep/drop decisions, not hidden implementation details. + +### 8. Prefer concrete language over inflated phrasing + +Use: + +- "This result is mixed." +- "This helps with pruning, not ranking." +- "Aligned workloads make the selector look better than it is." +- "The transfer-safe pass did not hold up on the expanded space." + +Avoid: + +- vague intensifiers, +- overclaimed novelty language, +- and sentences whose main purpose is to sound more sophisticated than the evidence requires. + +## Section-Level Guidance + +### Introduction and Background + +- Motivate the project with a simple question early. +- Explain the middle-ground positioning clearly: narrower than compiler-scale autoscheduling and narrower than full-kernel generation. +- Introduce Triton as the fixed-kernel setting before discussing the selector. + +### Design and Methodology + +- Explain what the system does in the order the reader needs it. +- Separate implementation mechanics from why those mechanics matter scientifically. +- Keep promotion rules and evidence discipline explicit but not repetitive. + +### Results + +- Organize the section as a story, not as a timeline of experiment rounds. +- Each subsection should answer one question and then interpret it. +- Use numbers to anchor the main comparison, then explain why the result matters. + +### Discussion and Conclusion + +- Say directly what the project established and what it did not. +- Keep the final framing narrow, credible, and easy to restate aloud. +- Treat negative results as part of the contribution, not as an apology. + +## Editing Checklist + +Before treating a section as stable, check: + +- Does the section introduce terms before using them heavily? +- Does each paragraph have one main job? +- Are the main claims consistent with `11_final_claim_inventory.md`? +- Are there enough numbers to support the claim, but not so many that the prose becomes dense? +- Are caveats present without being repeated excessively? +- Could a technical reader outside this subfield follow the argument on a first pass? diff --git a/paper/appendix/extended_results.tex b/paper/appendix/extended_results.tex index 3708528..21d6baf 100644 --- a/paper/appendix/extended_results.tex +++ b/paper/appendix/extended_results.tex @@ -1,23 +1,24 @@ \section{Extended Results and Exact Artifact Mapping} -The final manuscript uses a narrow figure set, but the underlying evidence is broader. Table~\ref{tab:artifactmap} records the canonical artifact map used by the first draft. +The final manuscript uses a narrow figure set, but the underlying evidence is broader. Table~\ref{tab:artifactmap} records the research-backed source map used by the first draft. In this repo snapshot, the paper backbone is tracked through the research docs, the ADRs, and the dated analysis logs. \begin{table*}[t] \centering -\small -\begin{tabular}{p{0.19\textwidth}p{0.61\textwidth}p{0.14\textwidth}} +\footnotesize +\setlength{\tabcolsep}{4pt} +\begin{tabularx}{\textwidth}{>{\raggedright\arraybackslash}p{0.18\textwidth} >{\raggedright\arraybackslash}X >{\raggedright\arraybackslash}p{0.14\textwidth}} \toprule Artifact role & Canonical source & Used by \\ \midrule -Final representative headline & \artifact{artifacts/studies/gemm\_final\_baseline\_mapping/run\_20260330T014317Z\_359c1904} & Figure 2, Table 4 \\ -Final mainline ablation & \artifact{artifacts/studies/gemm\_final\_selector\_ablation/run\_20260330T023529Z\_7c800187} & Figure 5, Table 4 \\ -Aligned context & \artifact{artifacts/studies/gemm\_v2\_aligned\_reference/run\_20260327T190124Z\_3a34cdc7} plus \artifact{gemm\_v2\_baseline\_mapping} & Figure 3 \\ -LayerNorm regime split & \artifact{artifacts/studies/layernorm\_v2\_small\_regime/run\_20260327T183157Z\_53565cba} plus \artifact{layernorm\_v2\_large\_regime/run\_20260327T183158Z\_37695a2d} & Figure 4 \\ -Phase 3 transfer failure & \artifact{artifacts/studies/gemm\_v3\_baseline\_mapping/run\_20260329T010211Z\_dfb53abb} and \artifact{gemm\_v3\_selector\_ablation/run\_20260329T034953Z\_e8b8ac98} & \texttt{H5} text, Table 4 \\ -Chosen-vs-best diagnostic & \artifact{artifacts/studies/gemm\_v3\_schedule\_diag/run\_20260328T212649Z\_7755304a} & Figure 5B \\ -Final claim bundle & \artifact{artifacts/analysis/final\_paper\_20260330/} & Whole draft \\ +Final representative headline & \texttt{gemm\_final\_baseline\_mapping} with supporting interpretation in \texttt{docs/research/08\_evidence\_registry.md}, \texttt{docs/research/11\_final\_claim\_inventory.md}, and \texttt{docs/research/logs/2026-03-30\_r6\_final\_synthesis\_and\_evidence\_lock.md} & Figure 2, Table 4 \\ +Final mainline ablation & \texttt{gemm\_final\_selector\_ablation} plus the same R6 synthesis sources above & Figure 5, Table 4 \\ +Aligned context & \texttt{gemm\_v2\_aligned\_reference} plus \texttt{gemm\_v2\_baseline\_mapping}, interpreted through \texttt{docs/research/08\_evidence\_registry.md} & Figure 3 \\ +LayerNorm regime split & \texttt{layernorm\_v2\_small\_regime} plus \texttt{layernorm\_v2\_large\_regime}, with bounded follow-up from the Phase 3 microstudy rows in \texttt{docs/research/08\_evidence\_registry.md} & Figure 4 \\ +Phase 3 transfer failure & \texttt{gemm\_v3\_baseline\_mapping} and \texttt{gemm\_v3\_selector\_ablation}, anchored by \texttt{docs/research/08\_evidence\_registry.md} and \texttt{docs/research/11\_final\_claim\_inventory.md} & \texttt{H5} text, Table 4 \\ +Chosen-vs-best diagnostic & \texttt{gemm\_v3\_schedule\_diag} with the keep/drop decision documented in \texttt{docs/research/08\_evidence\_registry.md} & Figure 5B \\ +Paper backbone & \texttt{docs/research/}, \texttt{docs/adr/}, and dated analysis logs under \texttt{docs/research/logs/} & Whole draft \\ \bottomrule -\end{tabular} +\end{tabularx} \caption{Canonical artifact map for the first draft.} \label{tab:artifactmap} \end{table*} diff --git a/paper/appendix/provenance.tex b/paper/appendix/provenance.tex index 2b6fc64..b10dafd 100644 --- a/paper/appendix/provenance.tex +++ b/paper/appendix/provenance.tex @@ -1,12 +1,12 @@ \section{Provenance and Promotion Rules} -The first draft relies on the final claim inventory and the final paper bundle as the promotion boundary. +The first draft relies on the final claim inventory together with the local research docs, ADRs, and dated synthesis logs as the promotion boundary. \paragraph{Canonical promotion sources.} All promoted claim and figure sources must be repo-local, completed, and canonical. In practice this means: \begin{itemize} \item final claim wording comes from \artifact{docs/research/11\_final\_claim\_inventory.md}; -\item figure and table sources come from \artifact{artifacts/analysis/final\_paper\_20260330/}; +\item figure and table sources are reconstructed in this checkout from \artifact{docs/research/08\_evidence\_registry.md}, \artifact{docs/adr/}, and the dated synthesis logs under \artifact{docs/research/logs/}; \item historical interpretation comes from the backbone research docs and dated synthesis logs. \end{itemize} diff --git a/paper/main.pdf b/paper/main.pdf new file mode 100644 index 0000000..9fb23cf Binary files /dev/null and b/paper/main.pdf differ diff --git a/paper/main.tex b/paper/main.tex index 31a2566..09ebc3f 100644 --- a/paper/main.tex +++ b/paper/main.tex @@ -22,6 +22,8 @@ urlcolor=blue } +\raggedbottom +\urlstyle{same} \setlist[itemize]{leftmargin=1.2em, itemsep=2pt, topsep=3pt} \setlist[enumerate]{leftmargin=1.4em, itemsep=2pt, topsep=3pt} \setlength{\textfloatsep}{8pt plus 2pt minus 2pt} @@ -33,8 +35,8 @@ \newcommand{\claimid}[1]{\texttt{#1}} \title{Bottleneck-Aware Schedule-First Tuning for Triton Kernels Under Matched Budget} -\author{Project Draft} -\date{March 2026} +\author{Pedram \and Giovanni Galea Curmi} +\date{April 2026} \begin{document} \maketitle @@ -45,19 +47,25 @@ \input{sections/introduction} \input{sections/background} +\input{sections/related_work} \input{sections/design} \input{sections/methodology} \input{sections/results} \input{sections/discussion} -\input{sections/related_work} \input{sections/conclusion} +\clearpage \appendix \input{appendix/extended_methods} \input{appendix/extended_results} \input{appendix/provenance} +\clearpage +\begingroup +\small +\sloppy \bibliographystyle{plainnat} \bibliography{refs} +\endgroup \end{document} diff --git a/paper/sections/abstract.tex b/paper/sections/abstract.tex index 38f56d2..ce8729c 100644 --- a/paper/sections/abstract.tex +++ b/paper/sections/abstract.tex @@ -1 +1,7 @@ -Triton exposes a rich schedule space, but exhaustive autotuning is often too expensive to treat as a realistic baseline for everyday kernel development. This paper studies whether a schedule-first tuner can spend a fixed measurement budget more effectively by combining cheap compile-adjacent signals with bounded profiling feedback. We build \system, a matched-budget tuning framework that records candidate admissibility, cheap resource signals, profiler evidence, held-out runtimes, and the reasoning behind selector revisions. We evaluate the system on a homogeneous RTX A6000 pool with representative and aligned GEMM workloads as the primary case study and regime-split LayerNorm workloads as a secondary validation case. The results show that cheap signals are useful for pruning but not for reliable representative GEMM ranking, that aligned GEMM overstates selector quality, and that LayerNorm profiling remains a weak or negative regime-dependent story under the current budget. A transfer-safe corrective pass on an expanded \texttt{split\_k} space also fails, yielding a bounded negative result and clear keep/drop decisions for \texttt{split\_k} and \texttt{rows\_per\_program}. Finally, a conservative non-\texttt{split\_k} mainline selector recovers the strongest representative GEMM result in the project, improving over the compile-ranked parent by 0.2185 geometric-mean speedup points versus default and approaching naive random search under the same budget. Overall, the study shows that lightweight signal-guided tuning can help, but only when workload realism, evidence discipline, and selector scope are treated as first-class design constraints. +Triton kernels expose many schedule choices, but trying every valid configuration is often too expensive to be a realistic baseline for day-to-day tuning. This paper asks a narrower question: given a fixed measurement budget, can a schedule-first tuner use cheap compile-adjacent signals and a small amount of profiling to spend that budget better? + +We build \system, a matched-budget tuning framework that records which candidates are admissible, which cheap signals they expose, what limited profiling says, and how selector revisions change over time. We evaluate it on a homogeneous RTX A6000 pool using representative and aligned GEMM workloads as the main case study and regime-split LayerNorm workloads as a secondary case. + +The results are mixed but clear. Cheap signals help with pruning, but they are not reliable enough to rank representative GEMM candidates on their own. Aligned GEMM makes the selector look better than it is, so it is useful as context but not as the main truth source. LayerNorm does not become a second positive profiling story: the small-batch regime is weak and noisy, while the large-batch regime regresses under the same budget. + +The project also records one bounded negative result and one bounded positive result. A transfer-safe corrective pass on an expanded \texttt{split\_k} space fails, which supports dropping \texttt{split\_k} and \texttt{rows\_per\_program} from the final paper-facing surfaces. After that restriction, a conservative non-\texttt{split\_k} mainline selector reaches 1.0286 geometric-mean speedup versus default on representative GEMM, compared with 0.8101 for the compile-ranked parent under the same budget. The main conclusion is narrow: lightweight signal-guided tuning can help, but only when workload realism, evidence discipline, and selector scope are kept explicit. diff --git a/paper/sections/discussion.tex b/paper/sections/discussion.tex index 2a0e000..a3534ba 100644 --- a/paper/sections/discussion.tex +++ b/paper/sections/discussion.tex @@ -1,22 +1,25 @@ \section{Discussion and Limitations} \label{sec:discussion} -The project ends with a stronger and narrower story than it began with. The strongest support is for a limited but meaningful claim: lightweight signals are useful, but mostly for pruning and controlled frontier construction rather than for fully solving final ranking. The final mainline result is therefore positive in exactly the way the evidence allows and no stronger. +The project ends with a narrower story than it started with, but it also ends with a stronger one. The main lesson is that lightweight signals help most when they are used carefully. They are useful for pruning and for building a better frontier of candidates, but they are not enough to solve final ranking on their own. \paragraph{Why compile signals prune better than they rank.} -Compile-adjacent resource signals expose obvious failures such as excessive footprint or poor occupancy potential, but they do not directly encode the fine-grained runtime tradeoffs that separate several already-valid GEMM families. The repeated gap between compile-ranked selection and the best reachable representative GEMM family supports this reading across multiple rounds. +Compile-adjacent signals can expose obvious problems such as excessive resource use or poor occupancy potential. What they do not capture well is the finer runtime tradeoff between several candidates that are all already valid. That is why the project repeatedly sees a gap between compile-ranked selection and the best representative GEMM family reachable under the same budget. \paragraph{Why aligned workloads mislead.} -Aligned GEMM is useful because it shows what a selector looks like on regular shapes. It becomes misleading when treated as the main truth source. The project preserves aligned GEMM in the paper precisely because it demonstrates how an apparently better result can emerge from an easier workload rather than from a better selector. +Aligned GEMM is still useful, but only in the right role. It shows how the selector behaves on regular shapes. It becomes misleading when treated as the main truth source, because regular workloads make the selector look better than it does on the irregular representative matrix. Keeping aligned GEMM in the paper is therefore useful precisely because it warns against overreading an easier evaluation setting. \paragraph{Why LayerNorm stayed secondary.} -LayerNorm did not fail because it was uninteresting. It stayed secondary because the evidence never justified a strong matched-budget success claim. The regime split was still valuable: it explained why a pooled interpretation was too coarse and why the extra LayerNorm knob did not belong in the final mainline surface. +LayerNorm stayed secondary not because it was unimportant, but because the evidence never supported a strong matched-budget win. The regime split still taught something important. It showed that a pooled interpretation was too coarse, and it supported the decision to remove the extra LayerNorm knob from the final paper-facing surface. \paragraph{Why transfer failed on the expanded \texttt{split\_k} space.} -The negative Phase 3 result is not an embarrassment to hide. It is a central part of the paper's credibility. The selector family that looked promising on a narrower space did not survive the expanded space, and the diagnostic evidence shows that the admitted orthogonal schedule family did not become a useful mainline choice. That bounded failure is what justifies the keep/drop decisions and the return to a conservative final surface. +The negative Phase 3 result is part of the paper's credibility, not a detour to hide. A selector family that looked promising on a narrower space did not survive the expanded \texttt{split\_k} surface, and the diagnostic evidence shows that the added schedule family did not become a useful mainline choice. That bounded failure is exactly what justifies the keep/drop decisions and the later return to a conservative final surface. \paragraph{Why the final result is intentionally conservative.} -The final mainline selector is not a heroic last-minute heuristic sweep. It is a carefully bounded corrective pass on the paper-facing non-\texttt{split\_k} surface. The paper should make that explicit. The value of the result comes from the combination of positive evidence and disciplined scope control, not from pretending that the final selector is universally robust. +The final mainline selector is not a last-minute attempt to rescue every earlier result. It is a bounded corrective pass on the final non-\texttt{split\_k} surface. That should stay explicit in the paper. The strength of the result comes from the combination of a real positive outcome and disciplined scope control, not from implying that the selector is broadly robust. + +\paragraph{What this could mean for open-source tooling.} +The work is narrow, but parts of it could still transfer into open-source Triton tooling. The clearest candidates are the cheap-signal pruning rules, the matched-budget evaluation protocol, and the evidence split between reportable and diagnostic profiling. These are easier to adopt than the full selector story because they do not require claiming that one learned or revised policy generalizes broadly. In other words, the paper supports extending existing tuning libraries with better pruning and better evaluation discipline more strongly than it supports shipping a universal Triton selector. The paper also has clear non-claims: \begin{itemize} @@ -26,4 +29,4 @@ \section{Discussion and Limitations} \item It does not show that revised selectors always transfer across search spaces. \end{itemize} -Those limits are not a weakness of presentation. They are part of the project's final contribution: a matched-budget tuning study can still be useful and paper-worthy when it states exactly where the method works, where it fails, and why. +These limits are part of the contribution, not a weakness in how the paper is framed. The project is useful because it says clearly where the method helps, where it does not, and why the final paper-facing system keeps only the parts that survived the evidence lock. diff --git a/paper/sections/introduction.tex b/paper/sections/introduction.tex index 1db9de6..ba1fd0e 100644 --- a/paper/sections/introduction.tex +++ b/paper/sections/introduction.tex @@ -1,12 +1,14 @@ \section{Introduction} -GPU kernel performance depends not only on what computation is performed, but also on how that computation is scheduled on the hardware. That perspective is well established in prior work on schedule search and autoscheduling, from Halide's separation of algorithms and schedules through tensor-compiler autotuning systems such as TVM and Ansor \citep{ragankelley2012halide,mullapudi2016halide,adams2019halide,chen2018tvm,zheng2020ansor,chen2018learningtensor,shao2022metaschedule,zheng2022tensorir}. At the same time, recent systems push even further toward full-kernel generation or agentic optimization rather than bounded configuration selection \citep{dai2026cudaagent}. This project intentionally occupies a narrower middle ground. +GPU kernel performance depends not only on what computation is performed, but also on how that computation is scheduled on the hardware. That basic idea appears throughout prior work on schedule search and autoscheduling, from Halide through TVM, Ansor, and later tensor-program optimization systems \citep{ragankelley2012halide,mullapudi2016halide,adams2019halide,chen2018tvm,zheng2020ansor,chen2018learningtensor,shao2022metaschedule,zheng2022tensorir}. Recent work also pushes farther toward full-kernel generation and agentic optimization \citep{dai2026cudaagent}. This project studies a narrower problem. -Triton makes it practical to express high-performance GPU kernels in a compact schedule-oriented style, but that convenience does not remove the core tuning problem \citep{tillet2019triton,openai2021triton,tritondocsautotune,tritondocsheuristics}. A single kernel family can still expose a large number of tile, launch, and pipelining choices, and the best configuration can change with workload geometry and hardware bottlenecks. In practice, tuning workflows often trust a default schedule, run a small naive search, or escalate to expensive profiling and broader autotuning through Triton- or PyTorch-level infrastructure \citep{pytorchcompile2026,pytorchtriton2026}. That gap motivates this project: can a matched-budget tuner use lightweight signals to spend a small amount of search and profiling effort more intelligently? +Triton is a good setting for that narrower study. It exposes a compact set of kernel meta-parameters such as tile sizes, warp counts, grouping choices, and pipeline depth \citep{tillet2019triton,openai2021triton,tritondocsautotune,tritondocsheuristics}. These choices are small enough to study directly, but large enough to matter for performance. In practice, a kernel author often falls back to a default schedule, a small random search, or a more expensive autotuning path through Triton or PyTorch tooling \citep{pytorchcompile2026,pytorchtriton2026}. That leaves a practical question: can a small fixed budget be spent more intelligently? -This paper studies that question as an empirical systems problem rather than as a black-box autotuning benchmark. We ask whether a bottleneck-aware, schedule-first selector can improve held-out Triton kernel performance relative to default settings or equally budgeted naive baselines, while still producing interpretable failures when it does not. In that sense the project is closer in spirit to external autotuning frameworks such as Kernel Tuner, CLTune, and KTT than to compiler-scale schedule synthesis or agentic kernel generation \citep{vanwerkhoven2019kerneltuner,nugteren2015cltune,petrov2023ktt}. The project therefore values negative results and transfer failures, not only positive speedups. We treat the selector, workload design, and evidence discipline as part of the scientific contribution. +This paper studies that question as an empirical systems problem rather than as a black-box search benchmark. We use \emph{matched budget} to mean that the compared selectors get the same measurement allowance. The goal is not to show that one selector can win after spending more profiling or more trials. The goal is to ask whether a bottleneck-aware, schedule-first selector can make better use of the same limited budget than a default choice or an equally budgeted naive baseline. -The final project arc is narrower and stronger than the initial exploratory framing. The evidence shows that pruning is easier than ranking, that representative workloads matter more than aligned convenience cases, and that profiling is not uniformly beneficial across kernels or workload regimes. The revised-selector story is also mixed: a narrow frontier-aware success does not automatically transfer to larger spaces. However, after locking the paper-facing surface to a conservative non-\texttt{split\_k} mainline, a guarded final selector does recover the strongest representative GEMM result in the project. That combination of positive and bounded negative outcomes is the paper's real contribution. +That framing places the project closer to external autotuning systems such as OpenTuner, Kernel Tuner, CLTune, and KTT than to compiler-scale schedule synthesis or full-kernel generation \citep{ansel2014opentuner,vanwerkhoven2019kerneltuner,nugteren2015cltune,petrov2023ktt}. The selector matters, but so do the workload design and the evidence rules. Negative results therefore matter as much as positive ones. A transfer failure is not noise to hide if it changes what should stay on the final paper-facing surface. + +The final project story is narrower than the initial exploration, but it is stronger for that reason. The evidence shows that cheap signals help more with pruning than with ranking, that aligned workloads flatter the selector, and that profiling is not uniformly helpful across kernels or regimes. The revised-selector story is also mixed: an idea that looked promising on a smaller space did not carry over cleanly once \texttt{split\_k} expanded the search surface. After that failure, the paper freezes a conservative non-\texttt{split\_k} mainline and asks a simpler question. On that final surface, a guarded selector does recover the strongest representative GEMM result in the project. That mix of bounded wins and bounded failures is the main contribution. We make four contributions: \begin{itemize} @@ -16,4 +18,4 @@ \section{Introduction} \item We demonstrate that a final conservative mainline selector can materially improve representative GEMM under the same budget without reopening the research program. \end{itemize} -The rest of the paper is organized as follows. Section~\ref{sec:background} frames the tuning problem and workload roles. Section~\ref{sec:design} describes \system and the final paper-facing tuning surfaces. Section~\ref{sec:methodology} describes the experimental protocol and evidence discipline. Section~\ref{sec:results} presents the main empirical findings from cheap-signal limits through the final mainline result. Section~\ref{sec:discussion} discusses what the project does and does not establish. Section~\ref{sec:related} positions the work relative to prior autotuning and profiling-guided tuning systems. +The rest of the paper is organized as follows. Section~\ref{sec:background} frames the tuning problem and workload roles. Section~\ref{sec:related} positions the work relative to prior autotuning and profiling-guided tuning systems. Section~\ref{sec:design} describes \system and the final paper-facing tuning surfaces. Section~\ref{sec:methodology} describes the experimental protocol and evidence discipline. Section~\ref{sec:results} presents the main empirical findings from cheap-signal limits through the final mainline result. Section~\ref{sec:discussion} discusses what the project does and does not establish. diff --git a/paper/sections/results.tex b/paper/sections/results.tex index fb6754f..b5090aa 100644 --- a/paper/sections/results.tex +++ b/paper/sections/results.tex @@ -3,9 +3,13 @@ \section{Results} \subsection{Cheap Signals and Workload Realism} -The first durable result in the project is that cheap compile-adjacent signals are useful for pruning but not sufficient for reliable ranking on representative GEMM. The clearest expanded-space evidence comes from the Phase 2 representative study. On that non-\texttt{split\_k} surface, naive random search reaches a geometric-mean speedup of 1.0331 versus default, while the compile-ranked parent \texttt{prune\_rank} reaches only 0.8265. Even bounded profiling helps only modestly at 0.8444, and the early revised selector collapses catastrophically to 0.1458. This is not a story about cheap signals being worthless. It is a story about coarse signals being good at excluding obvious losers but weak at choosing among plausible survivors. +Unless stated otherwise, the numeric values in this section are geometric-mean speedup relative to the Triton default schedule. A value above 1.0 is an improvement, while a value below 1.0 is a regression. -Workload realism matters at the same time. Figure~\ref{fig:aligned-context} compares the Phase 2 representative and aligned contexts. The aligned workload is consistently more flattering: \texttt{prune\_rank} improves from 0.8265 on representative GEMM to 0.8795 on aligned GEMM, and \texttt{prune\_rank\_profiled} improves from 0.8444 to 0.8873. This supports the paper's decision to treat aligned GEMM as context only. If the study had stopped on aligned workloads, the selector would have looked materially better than it is on realistic shapes. +The first stable result in the project is simple: cheap compile-adjacent signals help most when they rule out bad candidates early. They do much less well at deciding which of the remaining candidates is actually best on representative GEMM. + +The clearest evidence comes from the Phase 2 representative GEMM study on the final non-\texttt{split\_k} surface. Under the same budget, naive random search reaches a geometric-mean speedup of 1.0331 versus default, while the compile-ranked parent \texttt{prune\_rank} reaches only 0.8265. Adding bounded profiling raises the result only modestly to 0.8444. The early revised selector falls much further, to 0.1458. The main point is not that cheap signals are useless. The point is that they are better at pruning obviously weak candidates than at ranking several plausible survivors. + +That result is easier to see once the workload is chosen carefully. In this paper, representative GEMM is the truth source because it includes the irregular shapes that expose ranking mistakes. Figure~\ref{fig:aligned-context} shows why the aligned workload is kept only as context. On aligned GEMM, the same strategies look better: \texttt{prune\_rank} rises from 0.8265 to 0.8795, and \texttt{prune\_rank\_profiled} rises from 0.8444 to 0.8873. If the evaluation had stopped there, the selector would have looked stronger than it really is on the more realistic workload matrix. \begin{figure}[t] \centering @@ -15,7 +19,7 @@ \subsection{Cheap Signals and Workload Realism} \centering \begin{tabular}{lcc} \toprule -Strategy & Representative & Aligned \\ +Strategy & Rep. GEMM & Aligned GEMM \\ \midrule \texttt{prune\_rank} & 0.8265 & 0.8795 \\ \texttt{prune\_rank\_profiled} & 0.8444 & 0.8873 \\ @@ -27,13 +31,15 @@ \subsection{Cheap Signals and Workload Realism} \raggedright \textbf{Source.} Phase 2 canonical aligned-context comparison from \artifact{gemm\_v2\_baseline\_mapping} and \artifact{gemm\_v2\_aligned\_reference}. \end{minipage}} -\caption{Aligned GEMM remains useful as context, but it is not the truth source. Compile-ranked strategies look better on aligned workloads than on the representative GEMM matrix.} +\caption{Aligned GEMM remains useful as context, but it is not the truth source. All values are geometric-mean speedup versus the Triton default schedule. Compile-ranked strategies look better on aligned workloads than on the representative GEMM matrix.} \label{fig:aligned-context} \end{figure} \subsection{LayerNorm as a Bounded Regime-Split Result} -LayerNorm never becomes a second main optimization story. Instead, it becomes a useful bounded result about when profiling does not help enough. The regime split is the key. In the Phase 2 reportable regime studies, \texttt{small\_batch} shows only a marginal gain from profiling: \texttt{prune\_rank}=1.0100 versus \texttt{prune\_rank\_profiled}=1.0113. In \texttt{large\_batch}, the direction reverses: \texttt{prune\_rank}=1.0029 while \texttt{prune\_rank\_profiled}=0.9856. The later Phase 3 microstudy does not overturn that conclusion. It only reinforces that \texttt{rows\_per\_program} is too weak and noisy to justify staying on the mainline surface. +LayerNorm answers a different question. It does not become a second headline optimization win. Instead, it shows that bounded profiling has a mixed effect that depends on workload regime. + +The regime split matters immediately. In the Phase 2 reportable studies, \texttt{small\_batch} shows only a very small gain from profiling: \texttt{prune\_rank}=1.0100 and \texttt{prune\_rank\_profiled}=1.0113. In \texttt{large\_batch}, profiling moves in the wrong direction: \texttt{prune\_rank}=1.0029 while \texttt{prune\_rank\_profiled}=0.9856. The later Phase 3 microstudy does not change that reading. It only strengthens the decision to remove \texttt{rows\_per\_program} from the paper-facing LayerNorm surface. \begin{figure}[t] \centering @@ -55,17 +61,19 @@ \subsection{LayerNorm as a Bounded Regime-Split Result} \raggedright \textbf{Source.} Phase 2 regime-split LayerNorm studies. Phase 3 microstudies support the same bounded interpretation and retire \texttt{rows\_per\_program}. \end{minipage}} -\caption{LayerNorm is a regime-split secondary result, not a major positive profiling story. Small-batch profiling is weak and noisy, while large-batch profiling regresses relative to compile-only ranking.} +\caption{LayerNorm is a regime-split secondary result, not a major positive profiling story. All values are geometric-mean speedup versus the Triton default schedule. Small-batch profiling is weak and noisy, while large-batch profiling regresses relative to compile-only ranking.} \label{fig:layernorm-regimes} \end{figure} -This outcome still matters scientifically. It shows that a matched-budget profiling recipe can be directionally informative without becoming a strong paper-grade win, and it shows why the paper needs to keep LayerNorm as an explanatory thread rather than forcing it into the headline. +This is still a useful result. It shows that even when profiling provides extra information, that information does not automatically translate into a robust matched-budget win. It also shows why LayerNorm belongs in the paper as a bounded secondary story rather than as a second main result. \subsection{Transfer, Failure Analysis, and Keep/Drop Decisions} -The revised-selector story becomes stronger when written as a transfer story rather than a binary success or failure. The narrow-space \texttt{h4\_retry\_g3} result showed that a frontier-aware revision could work. But once the search surface expanded, the same family stopped generalizing. In Phase 3, the transfer-safe corrective pass on the \texttt{split\_k} space remained unsupported: the canonical representative GEMM mapping gives \texttt{prune\_rank}=1.0324, \texttt{naive\_random\_search}=1.0427, and \texttt{v4\_transfer\_safe\_profiled}=0.1609. The ablation is even clearer: frontier-only and profiled v4 are almost identical at 0.1622 and 0.1623, far below the parent at 0.9692. +The revised-selector part of the project is easiest to understand as a transfer story. On a narrower search space, the frontier-aware revision looked promising. Once the search space expanded, that same idea stopped carrying over cleanly. -These failures are useful because they close two design questions. First, \texttt{split\_k} does not stay in the mainline GEMM surface. Second, \texttt{rows\_per\_program} does not stay in the mainline LayerNorm surface. The schedule diagnostic supports the first decision directly: in the canonical diagnostic run, the selected and best-scored families both converge to a balanced \texttt{128x128x32}, \texttt{group\_size\_m}=4, \texttt{split\_k}=1 configuration, while non-unit \texttt{split\_k} values appear only as dominated alternatives. +The Phase 3 \texttt{split\_k} studies are the clearest limit case. On the expanded representative GEMM surface, \texttt{prune\_rank} reaches 1.0324 and naive random search reaches 1.0427, but \texttt{v4\_transfer\_safe\_profiled} falls to 0.1609. The matching ablation makes the problem even clearer: frontier-only and profiled v4 are nearly identical at 0.1622 and 0.1623, both far below the parent at 0.9692. This is why the project does not present revised selectors as a simple success story. + +That failure is useful because it closes two design decisions cleanly. First, \texttt{split\_k} does not stay on the mainline GEMM surface. Second, \texttt{rows\_per\_program} does not stay on the mainline LayerNorm surface. The schedule diagnostic supports the GEMM decision directly: in the canonical diagnostic run, both the selected and the best-scored family converge to a balanced \texttt{128x128x32}, \texttt{group\_size\_m}=4, \texttt{split\_k}=1 schedule, while non-unit \texttt{split\_k} values remain dominated alternatives. \begin{figure*}[t] \centering @@ -79,7 +87,7 @@ \subsection{Transfer, Failure Analysis, and Keep/Drop Decisions} \begin{tabular}{lc} \toprule -Selector & Geomean speedup \\ +Selector & Geomean speedup vs default \\ \midrule Parent \texttt{prune\_rank} & 0.8103 \\ \texttt{v5\_mainline\_frontier} & 1.0272 \\ @@ -116,7 +124,9 @@ \subsection{Transfer, Failure Analysis, and Keep/Drop Decisions} \subsection{Final Mainline Consolidation} -The payoff of the project is the final non-\texttt{split\_k} mainline result. Figure~\ref{fig:r6-headline} shows the bounded positive headline recorded by the final bundle. On representative GEMM, the guarded \texttt{v5\_mainline\_profiled} selector reaches 1.0286 geometric-mean speedup versus default. This beats the compile-ranked parent at 0.8101 by 0.2185 speedup points and also exceeds naive random search at 1.0011 under the same budget. The paired ablation in Figure~\ref{fig:ablation} shows that most of the recovery comes from conservative frontier construction, while profiling adds only a very small extra increment once the frontier is already correct. +The paper's positive headline comes from the final non-\texttt{split\_k} mainline pass. This step does not reopen the search program. It narrows the paper-facing surface and asks whether one conservative selector family can recover a credible final result under the same matched budget. + +Figure~\ref{fig:r6-headline} shows that it can. On representative GEMM, the guarded \texttt{v5\_mainline\_profiled} selector reaches 1.0286 geometric-mean speedup versus default. That is 0.2185 speedup points above the compile-ranked parent at 0.8101, and it also exceeds naive random search at 1.0011 under the same budget. The paired ablation in Figure~\ref{fig:ablation} shows where that recovery comes from. Most of it comes from conservative frontier construction, while profiling adds only a very small extra increment once the frontier is already in better shape. \begin{figure}[t] \centering @@ -143,4 +153,4 @@ \subsection{Final Mainline Consolidation} \label{fig:r6-headline} \end{figure} -This final result should be read carefully. It is not evidence that the project solved Triton autotuning. It is evidence that the strongest remaining mainline lever was worth pursuing, that the project's evidence discipline prevented an overly broad claim, and that a conservative selector can still recover a useful headline after several rounds of negative and mixed results. +This final result should be read narrowly. It does not show that Triton autotuning is solved, and it does not erase the earlier transfer failure. What it does show is that a conservative selector on the final paper-facing surface can recover a real, bounded representative GEMM improvement after several mixed and negative rounds. That combination of recovery and restraint is the main result of the paper. diff --git a/paper/tables/table_workloads.tex b/paper/tables/table_workloads.tex index d20b465..77be3da 100644 --- a/paper/tables/table_workloads.tex +++ b/paper/tables/table_workloads.tex @@ -1,15 +1,26 @@ \begin{table}[t] \centering \small -\begin{tabular}{p{0.20\columnwidth}p{0.25\columnwidth}p{0.43\columnwidth}} +\setlength{\tabcolsep}{4pt} +\begin{tabularx}{\columnwidth}{ + >{\raggedright\arraybackslash}p{0.21\columnwidth} + >{\raggedright\arraybackslash}X + >{\raggedright\arraybackslash}p{0.31\columnwidth}} \toprule Kernel & Workload classes & Role \\ \midrule -Representative GEMM & \texttt{square\_compute}, \texttt{m\_dominant}, \texttt{n\_dominant}, \texttt{edge\_nondivisible} & Primary truth source for ranking quality and final headline \\ -Aligned GEMM & \texttt{aligned\_square}, \texttt{aligned\_rectangular} & Context workload used to test whether regular shapes flatter the selector \\ -LayerNorm & \texttt{small\_batch}, \texttt{large\_batch} & Secondary regime-split explanation of profiling value \\ +Representative GEMM & +\texttt{square\_compute}, \texttt{m\_dominant}, \newline +\texttt{n\_dominant}, \texttt{edge\_nondivisible} & +Primary truth source for ranking quality and the final headline \\ +Aligned GEMM & +\texttt{aligned\_square}, \texttt{aligned\_rectangular} & +Context workload used to check whether regular shapes flatter the selector \\ +LayerNorm & +\texttt{small\_batch}, \texttt{large\_batch} & +Secondary regime-split case for interpreting profiling value \\ \bottomrule -\end{tabular} +\end{tabularx} \caption{Final workload matrix used by the draft.} \label{tab:workloads} \end{table}