docs(blog): add Prepared Image Specification cold-start post - #5934
docs(blog): add Prepared Image Specification cold-start post#5934Spencer Libbing (stl327) wants to merge 5 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new AKS Engineering blog post explaining how Prepared Image Specification (PIS) can reduce node/workload cold-start latency, along with the required author metadata entry.
Changes:
- Added a new blog article detailing PIS benefits, boundaries, and benchmark results plus a “Try PIS” walkthrough.
- Added a new author profile entry for Spencer Libbing in the shared blog author registry.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md | New blog post content for the PIS cold-start/scale-out latency topic (includes preview notice, methodology, and how-to links). |
| website/blog/authors.yml | Adds the spencer-libbing author record used by the new post. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Suppressed comments (2)
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:4
- The front-matter
descriptionis shorter than the 150–160 character SEO target used for this blog. Expand it slightly so it meets the length requirement.
title: "Reducing AKS Cold Starts with Prepared Image Specification"
description: "Learn how Prepared Image Specification can reduce AKS scale-out latency by preparing container images and host customizations ahead of time."
date: 2026-08-27
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:24
- This post is missing a hero image immediately after
<!-- truncate -->, which is the consistent pattern in other recent posts (for example,website/blog/2026-06-29-llm-routing-on-aks/index.mdandwebsite/blog/2026-04-08-agent-skills-for-aks/index.md). Add./hero-image.png(or another suitable hero asset) and include meaningful alt text.
<!-- truncate -->
In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively.
> **Preview notice:** PIS is currently an AKS preview feature. Preview features are opt-in, provided as-is and as-available, and not intended for production use. Review the current prerequisites, regions, limitations, and support policy before testing it.
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:136
- The unquoted
<digest>is interpreted as input redirection by Bash, causing the sample to fail with a missing-file error instead of passing an image reference to Azure CLI. Quote the complete placeholder argument so readers can replace the digest without changing the command structure.
--container-images myacr.azurecr.io/model-server@sha256:<digest> \
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.
Suppressed comments (7)
Previously missed (3) — in code that hasn't changed since the last review.
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:2
- This title uses headline-style capitalization, but repository guidance requires sentence-style capitalization for titles and headings. Lowercase “cold starts” to match that requirement.
title: "Reducing AKS Cold Starts with Prepared Image Specification"
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:56
- This row combines model readiness and first-token latency but reports only the 14% p95 value. Line 95 states that these endpoints had different p95 reductions (14% and 13%), so the table currently makes the first-token result ambiguous.
| Serve a model on a T4 GPU node | Model healthy and first token returned | **3% at median; 14% at p95** |
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:24
- Spell out GPU on first use. The repository’s acronym guidance requires the expanded term before the abbreviation.
This issue also appears in the following locations of the same file:
- line 40
- line 67
- line 73
In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively.
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:40
- Spell out SAS on first use so the security guidance is clear and follows the repository’s acronym requirements.
| Protect private dependencies | Managed identity and data-plane role-based access control (RBAC) can retrieve private artifacts without embedding keys or SAS tokens in scripts. |
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:67
- Spell out CA on first use. As written, readers encounter the abbreviation before “CA certificates” later in the post.
Not every task should be baked. A small Linux CA-and-policy setup was **14% slower at the median** in this test, even though its tail favored PIS. For small tasks, normal scale-out variation and other provisioning stages can outweigh the preparation that PIS removes.
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:73
- Spell out SLO on first use to meet the repository’s acronym guidance.
PIS moves that repeated work out of the burst path. In the three-node Windows comparison, this reduced median workload-start latency by 74%. For services with a strict scale-out SLO, that changes the capacity conversation from "How many idle nodes do we need?" to "How quickly can prepared capacity serve traffic?"
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:136
- The unquoted
<digest>placeholder is parsed by Bash as input/output redirection, so copying this example causes a shell error instead of passing the image reference to Azure CLI. Quote the complete placeholder argument (or assign it to a quoted variable).
--container-images myacr.azurecr.io/model-server@sha256:<digest> \
|  | ||
|
|
||
| In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively. | ||
|
|
There was a problem hiding this comment.
Regarding 3%/14% model-ready improvement for P50/P95, could you explain how this A/B test is run? Does it cover the GPU node provisioning, bootstrapping, inferencing engine container image pulling, model download/loading?
From what I have observed, most customers reserve GPU nodes for inferencing and training workloads for two reasons: GPU capacity scarcity and cold start latency. In that case, GPU nodes scale out/in may not be the primary use case for those customers, while GPU node OS upgrade could be a challenge. Can we measure how PIS helps in this scenario?
There was a problem hiding this comment.
Allen Wen (@AllenWen-at-Azure) , didn't you receive so much customer feedback about GPU scale up lately? What you are saying above is a bit at odds with the recent feedback. Maybe I am confused?
There was a problem hiding this comment.
This query suggests that most of the top 20 AKS GPU clusters reserve GPU nodes by either turning off cluster autoscaling or setting the min and max node values to be the same.
So, I don't think GPU node cold start is on the critical path of customer AI workload deployment, although it is still important for AKS to keep improving it.
I think what we should focus on instead is how PIS can help reduce disruption when customers upgrade their existing GPU nodes.
Nikhil Kaul (nikhilkaul1234)
left a comment
Nikhil Kaul (nikhilkaul1234)
left a comment
There was a problem hiding this comment.
Really strong post. The structure works, the results are presented honestly, and the "How to read these results" section is especially helpful. I left three small comments from the perspective of someone new to PIS, mostly about making the benchmark framing easier to follow. Otherwise, this looks great.
|
|
||
|  | ||
|
|
||
| In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively. |
There was a problem hiding this comment.
Reading this as a newcomer, I reached the percentages before I understood what they were being compared against or how to interpret median versus p90/p95. Could we add a short framing sentence here? That would also make the tail-latency point later in the post easier to appreciate on the first read.
There was a problem hiding this comment.
I had the same issue when I was trying to look at the numbers in the slide, but this paragraph made it easy for me to understand it.
|
|
||
| A PIS is an AKS-managed, versioned Azure resource that describes container images and Bash or PowerShell customizations to include in a prepared AKS node image. AKS builds on a supported AKS node image; this is not a bring-your-own-image workflow. | ||
|
|
||
| The feature creates a practical trade: pay a reusable preparation cost when a PIS-backed pool is created or updated, then avoid repeating the same downloads and initialization during later scale-outs. |
There was a problem hiding this comment.
"Pay a reusable preparation cost" initially sounded like a financial cost to me, especially because the post later discusses bill reduction. Could we clarify here that this means preparation time and operational effort, if that's the intent?
| - **Large, stable preparation paid off.** Image layers, a portable runtime, and a large dependency bundle were useful candidates because every standard node otherwise repeated the work. | ||
| - **GPU gains were real but smaller at the median.** Preloading the model and server image helped, but GPU VM creation, node registration, accelerator readiness, scheduling, and model initialization still dominated much of the end-to-end path. | ||
|
|
||
| Not every task should be baked. A small Linux CA-and-policy setup was **14% slower at the median** in this test, even though its tail favored PIS. For small tasks, normal scale-out variation and other provisioning stages can outweigh the preparation that PIS removes. |
There was a problem hiding this comment.
I appreciated seeing the case where PIS was slower - it makes the results feel balanced. Since it's the only result described in prose rather than the table, would it be worth adding it as a row so readers don't miss it?
|
|
||
|  | ||
|
|
||
| In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively. |
There was a problem hiding this comment.
I had the same issue when I was trying to look at the numbers in the slide, but this paragraph made it easy for me to understand it.
|
|
||
|  | ||
|
|
||
| In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively. |
There was a problem hiding this comment.
This test result is confusing. Why do we see 92% improvement in image pull in Linux, but the time to first token only improves by 3%? It isn't impressive enough to convince people to use the feature.
Also, 3% is so low that someone may question whether it is within margin of error. How many runs did we have here?
|  | ||
|
|
||
| In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively. | ||
|
|
There was a problem hiding this comment.
Allen Wen (@AllenWen-at-Azure) , didn't you receive so much customer feedback about GPU scale up lately? What you are saying above is a bit at odds with the recent feedback. Maybe I am confused?
| | --- | --- | | ||
| | Respond to demand sooner | New nodes can start with large image layers and stable host dependencies already present. | | ||
| | Reduce warm-capacity pressure | A faster, more predictable scale-out path may let a team hold fewer idle nodes solely as a cold-start buffer. | | ||
| | Make bootstrap repeatable | Versioned preparation replaces repeated best-effort downloads and installation on every new node. | |
There was a problem hiding this comment.
Isn't bootstrap repeatable already today? I think PIS eliminates repeated bootstrapping and provides the exact same image eliminating possible hiccups and variances.
Also the following two, improve rollout control and protect private dependencies, aren't the primary goals of customers. I wondered if those should be taken off this table.
| | Large image pulls during a three-node burst | Slowest new-node pull completed | **92% at Linux median; 98% at Windows median** | | ||
| | Install a pinned portable runtime | Runtime verified and workload started | **10% at Linux median; 34% at Windows median** | | ||
| | Prepare a large dependency bundle | Dependencies verified and workload started | **20% at Linux median; 5% at Windows median** | | ||
| | Serve a model on a T4 GPU node | Model healthy and first token returned | **3% at median; 14% at p95** | |
There was a problem hiding this comment.
You would expect this last test to be kind of in line with 10 to 74%.. But 3% is oddly low. Why is it so different?
| - **Windows gained the most.** Large Windows image and startup costs created more repeated work for PIS to remove, especially during multi-node bursts. | ||
| - **Tail latency often improved more than the median.** PIS made the slowest scale-out events less severe even when the typical Linux event changed only slightly. | ||
| - **Large, stable preparation paid off.** Image layers, a portable runtime, and a large dependency bundle were useful candidates because every standard node otherwise repeated the work. | ||
| - **GPU gains were real but smaller at the median.** Preloading the model and server image helped, but GPU VM creation, node registration, accelerator readiness, scheduling, and model initialization still dominated much of the end-to-end path. |
There was a problem hiding this comment.
Given this is a blog post, should we talk about how more work is being done to cut these layers and customers should stay tuned?
|
|
||
| Many node pools need more than a container image. They may require a portable runtime, policy files, CA certificates, model files, security tools, or other host-level dependencies. | ||
|
|
||
| Running that setup on every new node adds external dependencies to the critical path. A download endpoint can throttle, a package can change, or an install can fail halfway through. PIS lets teams prepare immutable inputs ahead of time and bind a pool to a version that has already been built. |
There was a problem hiding this comment.
I think these are a bit repeat... It feels very AI generated when we rediscuss the value prop and just add a few more sentences to it. Is there something else we can put in this section? For instance: "When should I use this feature?"
|
|
||
| The customer metric should be time to serve traffic, not merely time until a node reports Ready. Include accelerator availability, model health, and the first successful inference in the readiness objective. Larger models, slower artifact delivery, or different GPU SKUs may shift how much of that path PIS can remove. | ||
|
|
||
| ## Adopt PIS with a controlled rollout |
There was a problem hiding this comment.
I like this section. How do I adopt it?
|
|
||
| For Windows, use PowerShell and set `scriptType` to `PowerShell`. Keep secrets out of scripts and use managed identity with data-plane RBAC for private registries and artifacts. | ||
|
|
||
| ## How to read these results |
There was a problem hiding this comment.
This should have been right at the bottom of the table where results were discussed.
Summary
Publishes an AKS Engineering Blog article about Prepared Image Specification (PIS), an AKS preview feature that moves stable, repeated node preparation into an AKS-managed, versioned node image.
This change:
Context
A newly allocated AKS node must join the cluster, prepare host dependencies, pull container images, and start its workload before it can serve traffic. PIS prepares pinned images and stable Bash or PowerShell customizations ahead of recurring scale-outs. The article also states the boundary clearly: PIS does not remove VM allocation, node registration, networking, scheduling, driver initialization, or application startup.
The customer-visible findings highlighted in the post include:
The comparisons held VM size, AKS node image, networking, pinned artifacts, and workload constant between standard and PIS-backed pools. Image-caching tests used 100 rounds per arm; customization and GPU scenarios used 10 same-round pairs per reported cell. Tests used AKS 1.35.7 in East US 2 and West US 3. The article excludes PIS build and initial pool-creation time from recurring scale-out results and presents smaller-sample tail data as directional rather than as a production SLO or billing claim.
Changes
Validation
npm run buildnpm run typecheck