You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: hub/apps/windows-app-sdk/experimental-channel.md
+244-7Lines changed: 244 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,13 +19,250 @@ The experimental channel includes releases of the Windows App SDK with [experime
19
19
- If you'd like to upgrade an existing app from an older version of the Windows App SDK to a newer version, see [Update existing projects to the latest release of the Windows App SDK](update-existing-projects-to-the-latest-release.md).
20
20
- For documentation on experimental releases, see [Install tools for preview and experimental channels of the Windows App SDK](preview-experimental-install.md).
21
21
22
+
## Version 2.0 Experimental (2.0.0-Experimental3)
23
+
24
+
### Use on-device AI with Windows AI APIs
25
+
26
+
> [!IMPORTANT]
27
+
> The underlying ML models required for these APIs currently require your device to be running the latest Windows 11 Insider Preview Build on the Dev Channel. Additionally, these APIs require your device to be a Copilot+ PC. See [Copilot+ PCs Developer Guide](/windows/ai/npu-devices) to learn more about these devices. APIs will throw an exception when called on devices lacking the necessary support.
28
+
29
+
The Windows App SDK incorporates advanced Windows AI capabilities, enabling developers to seamlessly integrate intelligent features into their applications. These enhancements include local AI functionalities such as responding to incoming prompts, recognizing text within images, describing image contents, extracting objects from pictures, and more.
30
+
31
+
32
+
For information on responsible development practices utilized during the creation of the Windows AI APIs, which can also be applied when creating AI-assisted features, consult the [Developing Responsible Generative AI Applications and Features on Windows](/windows/ai/rai) guidance.
33
+
34
+
This release includes the following new and modified experimental APIs:
35
+
36
+
### Bug Fixes:
37
+
38
+
* Fixed bounding box calculation when text is rotated. In some circumstances, the OCR text matching within images reported inaccurate or empty region bounds when the text was rotated.
39
+
40
+
41
+
### Windows ML Updates
42
+
43
+
#### Renamed Types
44
+
45
+
* Renamed `WinMLCatalogModel` to `ModelCatalog`
46
+
* Renamed `CatalogModelSource` to `ModelCatalogSource`
47
+
48
+
#### Method Updates
49
+
50
+
*`CatalogModelInfo`: Renamed `GetInstance` to `GetInstanceAsync`
51
+
*`ModelCatalogSource`: Renamed `CreateFromUri` to `CreateFromUriAsync`
52
+
*`ModelCatalog`: Renamed `FindModel` to `FindModelAsync`
53
+
*`ModelCatalog`: Renamed `FindAllModels` to `FindAllModelsAsync`
54
+
55
+
56
+
#### Property Changes
57
+
58
+
* Updated `CatalogModelInfo.Size` to `CatalogModelInfo.ModelSizeInBytes`
59
+
60
+
#### Behavior Updates
61
+
62
+
* Retrieve instance from `CatalogModelInstanceResult` using .GetInstance()
63
+
*`CatalogModelStatus` now returns Ready or NotReady based on local availability
64
+
* Added `CatalogModelInstanceStatus` to separate instance status from model status
65
+
66
+
#### CatalogModelInfo Enhancements
67
+
68
+
* Renamed `Alias` to `Name`
69
+
* Renamed `Revision` to `Version`
70
+
* Added `Publisher`
71
+
* Removed `DisplayName`
72
+
73
+
#### JSON Changes
74
+
75
+
* Renamed `alias` to `id`.
76
+
* Removed `modelType` and `description`.
77
+
* Renamed `executionProvider` to `executionProviders`.
78
+
* Updated `executionProviders` to be an array of JSON objects instead of a comma-separated list.
79
+
80
+
#### Additional Changes
81
+
82
+
83
+
*`ModelCatalog` now returns a list of Execution Providers (EPs) when an instance is created.
84
+
* Added support for Windows 10 (1809) and above.
85
+
* Added support for local files, including both regular files and MSIX packages.
86
+
* Fixed crashes caused by invalid catalog JSON.
87
+
88
+
### AppContentIndexer Updates
89
+
90
+
* The previous `AppIndexQuery` type, which included `GetNextTextMatches` and `GetNextImageMatches` methods, has been split into two distinct types: `AppIndexTextQuery` and `AppIndexImageQuery`. The `AppContentIndexer.CreateQuery` method has been replaced with: `CreateTextQuery` and `CreateImageQuery`.
91
+
92
+
* These methods now return `AppIndexTextQuery` and `AppIndexImageQuery` respectively. To simplify usage, the options types have also been updated:
93
+
94
+
* Removed: `AppIndexQueryOptions`, `TextMatchOptions` and `ImageMatchOptions`
95
+
* Added: `TextQueryOptions` and `ImageQueryOptions`
96
+
97
+
* The APIs in the `AppContentIndex` namespace that previously returned arrays now return `IVectorView` for improved consistency and performance.
98
+
99
+
100
+
* The `AppContentIndexer.WaitForIndexingIdleAsync` method has been updated to accept a `TimeSpan` parameter instead of an integer, providing clearer and more flexible timeout handling.
101
+
102
+
### Video Super Resolution AI API
103
+
104
+
The `VideoScaler` API delivers real-time video enhancement through advanced AI upscaling, optimized for streams featuring people in conversation. It enables developers to provide sharper, clearer visuals across conferencing, streaming, and editing platforms, even under poor network conditions. The API supports customization of output resolution, frame rate, and regions of interest, with compatibility for multiple video formats including BGR, RGB, and NV12.
105
+
106
+
### Windows AI Text Rewriter Tone
107
+
108
+
The new RewriteCustomAsync API lets you provide an input string that guides Phi Silica in rewriting selected text. You can experiment with new creative styles like "Goofy" or "Pirate" to instantly transform your content.
109
+
110
+
111
+
### New split menu item for condensed functionality
112
+
113
+
The new experimental SplitMenuFlyoutItem control is designed to provide a split button experience within a menu flyout. This control will enable developers to expose a default primary action while also offering additional options through a submenu, ideal for condensing complex functionality into a smaller footprint and saving overall menu length.
114
+
115
+
Along with the capabilities of MenuFlyoutItem and MenuFlyoutSubItem, the control comes with two other properties : `SubMenuPresenterStyle` and `SubMenuItemStyle`, which allows the customization of the submenu, like using GridView for the submenu presenter.
116
+
117
+
```
118
+
<Button Text="Open File Button">
119
+
<Button.Flyout>
120
+
<MenuFlyout>
121
+
<SplitMenuFlyoutItem Text="Open with Notepad">
122
+
<MenuFlyoutItem Text="Visual Studio" />
123
+
<MenuFlyoutItem Text="VS Code" />
124
+
<MenuFlyoutItem Text="Wordpad" />
125
+
</SplitMenuFlyoutItem>
126
+
</MenuFlyout>
127
+
</Button.Flyout>
128
+
</Button>
129
+
```
130
+
131
+
### AI image generation
132
+
133
+
The `ImageGenerator` class leverages Stable Diffusion models to provide powerful image generation capabilities. It supports multiple generation scenarios:
134
+
135
+
-**Text-to-Image:** Generate images from descriptive text prompts.
136
+
-**Image-to-Image:** Transform existing images based on text descriptions.
137
+
-**Magic Fill:** Fill masked regions of images with AI-generated content.
138
+
-**Coloring Book Style:** Generate coloring-book-style images.
139
+
140
+
-**Restyle:** Change the artistic style of existing images while preserving structure.
141
+
142
+
All generated images are returned in **RGB8** format through [ImageBuffer](/windows/windows-app-sdk/api/winrt/microsoft.graphics.imaging.imagebuffer) objects. The API includes built-in [content safety filters](/azure/ai-services/content-safety/) and supports customizable generation parameters.
> If you previously installed Windows App SDK 2.0 Experimental 1, follow the [NuGet Uninstall](/nuget/consume-packages/install-use-packages-visual-studio#uninstall-a-package) guide to remove the `Microsoft.WindowsAppSDK` NuGet Metapackage with version `2.0.250930001-experimental1` from your project and the associated WinAppSDK component packages from that release before trying this new version, since the previously released Experimental package has a higher version number than the current one.
26
263
27
264
28
-
####App Content Search
265
+
### App Content Search
29
266
30
267
The AppContentIndexer APIs empower developers to efficiently index app content, including text and images for rapid and relevant retrieval. Supporting both lexical (keyword-based) and semantic (meaning-based) searches, these APIs allow apps to deliver fast, relevant results based on user intent and context rather than just exact keywords.
31
268
@@ -37,26 +274,26 @@ This capability unlocks the following advanced scenarios:
37
274
-**Retrieval-Augmented Generation (RAG)**
38
275
Indexed content can serve as a knowledge base for generative AI models. When a user asks a question, the app retrieves the most relevant documents or snippets from its index and feeds them into the model, enabling accurate, context-aware responses grounded in real data.
39
276
40
-
####Windows ML Model Catalog
277
+
### Windows ML Model Catalog
41
278
42
279
The Windows ML Model Catalog APIs enable your app or library to dynamically discover and download large AI model files from your own online model catalogs, eliminating the need to package these large files directly with your app or library. The model catalog helps ensure device compatibility by filtering models and downloading only those applicable for the specific Windows device in use.
43
280
44
-
####Persistent File and Folder Locations
281
+
### Persistent File and Folder Locations
45
282
The latest `Microsoft.Windows.Storage.Pickers` update streamlines file and folder selection by letting developers set initial and persistent folder locations, and by grouping filetype filters with clear labels for easier navigation.
46
283
47
-
####Relative Popup Positioning
284
+
### Relative Popup Positioning
48
285
49
286
The `PopupAnchor` API now allows `DesktopPopupSiteBridge` to support relative positioning by anchoring to its owning window or island, addressing the limitation where popups could only be positioned absolutely using screen coordinates.
50
287
51
-
####Input Routing for SystemVisual ContentIslands
288
+
### Input Routing for SystemVisual ContentIslands
52
289
53
290
The `InputUnderlyingWindowController` API enables developers to designate the target HWND for receiving input messages that were originally sent to a ContentIsland created from a SystemVisual (see [ContentIsland.CreateForSystemVisual](/windows/windows-app-sdk/api/winrt/microsoft.ui.content.contentisland.createforsystemvisual)).
54
291
55
-
####Flexible System Backdrop Placement
292
+
### Flexible System Backdrop Placement
56
293
57
294
`SystemBackdropHost` enables placing a system backdrop (acrylic/mica) anywhere within an application's visual tree.
58
295
59
-
####XAML Layout Sequential Positioning
296
+
### XAML Layout Sequential Positioning
60
297
61
298
The `WrapPanel` is a WinUI XAML layout panel that arranges child elements in a sequential position from left to right, items overflowing the line will break to the next line automatically at the end of the containing panel. It is useful for responsive layouts.
0 commit comments