deploy: aa18941

Author: alyssarosenzweig

2021/03/progress-report-january-february-2021/index.html

@@ -74,7 +74,7 @@ <h1 class=entry-title>Paving the Road to Vulkan on Asahi Linux</h1><ul class=blo
 <li>Create a framebuffer (possibly shareable), but don&rsquo;t share it yet.</li><li>Render stuff into the buffer.</li><li>Share it.</li></ol><p>When we submit the rendering command, it doesn&rsquo;t look like it&rsquo;s shared yet, so the driver doesn&rsquo;t do the implicit sync dance&mldr; and then when the app shares it, it&rsquo;s too late, and it doesn&rsquo;t have the right fence attached to it. Whoever is on the other side will try to use the buffer, and won&rsquo;t wait until the render is complete. Whoops!</p><p>I had to add a mechanism that keeps track of sync object IDs for all submitted but not complete batches, and attaches them to all buffers that are written. Then if those buffers are shared before we know those batches are complete, we can retroactively attach the fences.</p><p>Interestingly, when I brought this up with the Intel folks working on the Xe merge request&mldr; they hadn&rsquo;t heard of this before! It looks like their driver might have the same bug&mldr; I guess they might want to start testing with Sway ^^;;</p><p>Are we done yet? Mostly, though there are still bugs to squash&mldr; and we haven&rsquo;t even talked about the kernel yet!</p><h2 id=explicit-sync-meets-rust>Explicit Sync Meets Rust</h2><p>The previous version of the Asahi DRM kernel driver was pretty bare-bones in how it interacted with the rest of the kernel, since it had a very simple UAPI. I only had to add Rust abstractions for these DRM APIs:</p><ul>
 <li><code>drv</code> and <code>device</code>, the core of DRM drivers and handling devices.</li><li><code>file</code>, which is how DRM drivers interact with userspace.</li><li><code>gem</code>, which manages memory for GPUs with unified memory.</li><li><code>mm</code>, a generic memory range allocator which my driver uses for several things.</li><li><code>ioctl</code>, just some wrappers to calculate DRM ioctl numbers for the UAPI.</li></ul><p>To add proper explicit sync support, I had to add a bunch of new abstractions!</p><ul>
 <li><code>dma_fence</code>, the core Linux DMA fence mechanism.</li><li><code>syncobj</code>, DRM&rsquo;s sync object API.</li><li><code>sched</code>, which is the DRM component in charge of actually queuing GPU work and scheduling it.</li><li><code>xarray</code>, a generic kernel data structure that is basically an <code>int</code> → <code>void *</code> mapping, which I use to keep track of userspace UAPI objects like VMs and queues by their unique ID.</li></ul><p>I&rsquo;ve now <a href=https://lore.kernel.org/asahi/[email protected]/T/#t>sent out</a> all the DRM abstractions for initial review, so we can get them upstream as soon as possible and, after that, upstream the driver itself!</p><p>As part of this work, I even found two memory safety bugs in the DRM scheduler component that were causing kernel oopses for Alyssa and other developers, so the Rust driver work also benefits other kernel drivers that use this shared code! Meanwhile, I still haven&rsquo;t gotten any reports of kernel oopses due to bugs in the Rust code at all~ ✨</p><h2 id=even-more-stuff>Even more stuff!</h2><p>Explicit sync is the biggest change for this release, but there&rsquo;s even more! Since we want to get the UAPI as close as possible to the final version, I&rsquo;ve been working on adding lots more stuff:</p><ul>
-<li>Multiple GPU VMs (virtual memory address spaces) and GEM object binding based on the Xe UAPI model, to support future Vulkan requirements.</li><li>A result buffer, so the kernel driver can send GPU job execution results back to Mesa. This includes things like statistics and timings, but also whether the command succeeded and detailed fault information, so you can get verbose fault decoding right in Mesa!</li><li>Compute job support, to run compute shaders. We&rsquo;re still working on the Mesa side of this, but it should be enough to pass most tests and eventually add OpenCL support with <a href=https://www.phoronix.com/news/Rusticl-OpenCL-3.0-Conformance>Rusticl</a>!</li><li>The ability to submit multiple GPU jobs at once, and specify their dependencies directly, without using sync objects. This allows the GPU firmware to autonomously execute everything, which is a lot more efficient than going through the DRM scheduler every time. The Gallium driver doesn&rsquo;t use this yet, but it probably will in the future, and our upcoming Vulkan driver definitely will! There are <a href=https://github.com/AsahiLinux/docs/wiki/SW:AGX-driver-notes#queues>a lot of subtleties</a> around how all the queuing stuff works&mldr;</li><li>Stub support for blit commands. We don&rsquo;t know how these work yet, but at least we have some skeleton support in the UAPI.</li></ul><p>To make all this work on the driver side, I ended up refactoring the <a href=https://github.com/AsahiLinux/linux/blob/gpu/rust-wip/drivers/gpu/drm/asahi/workqueue.rs>workqueue</a> code and adding a whole new <a href=https://github.com/AsahiLinux/linux/tree/gpu/rust-wip/drivers/gpu/drm/asahi/queue>queue</a> module which adds all the infrastructure to use sync objects to track command dependencies and completions and manage work via the DRM scheduler. Phew!</p><h2 id=conclusions>Conclusions</h2><p>So what does this all mean for users of the Asahi Linux reference distro today? It means&mldr; things are way faster!</p><p>Since the Mesa driver no longer serializes GPU and CPU work, performance has improved a ton. Now we can run Xonotic at over 800 FPS, which is faster than macOS on the same hardware (M2 MacBook Air) at around 600*! This proves that open source reverse engineered GPU drivers really have the power to beat Apple&rsquo;s drivers in real-world scenarios!</p><p>Not only that, our driver passes 100% of the dEQP-GLES2 and dEQP-EGL conformance tests, which is better OpenGL conformance than macOS for that version. But we&rsquo;re not stopping there of course, with full GLES 3.0 and 3.1 support well underway thanks to Alyssa&rsquo;s tireless efforts! You can follow the driver&rsquo;s feature support progress over at the <a href=https://mesamatrix.net/>Mesa Matrix</a>. There have been many, many other improvements over the past few months, and we hope you find things working better and more smoothly across the board!</p><p>Of course, there are lots of new corner cases we can hit now that we have support for implicit sync with an explicit sync driver. We already know of at least one minor regression (brief magenta squares for a couple of frames when KDE starts up), and there&rsquo;s probably more, so please report any issues on <a href=https://github.com/AsahiLinux/linux/issues/72>the GitHub tracker bug</a>! The more issue reports we get, especially if they come with easy ways to reproduce the problem, the easier it is for us to debug these problems and fix them ^^.</p><p>* <em>Please don&rsquo;t take the exact number <strong>too</strong> seriously, as there are other differences too (Xonotic runs under Rosetta on macOS, but it was also rendering at a lower resolution there due to being a non-Retina app). The point is that the results are in the same league, and we will only keep improving our driver going forward!</em></p><h2 id=get-it>Get it!</h2><p>If you&rsquo;re already using the GPU drivers, just update your system and reboot to get the new version! Keep in mind that since the UAPI changed (a lot), apps will probably stop launching or will launch with software rendering until you reboot.</p><p>If you still haven&rsquo;t tried the new drivers, just install the packages:</p><pre tabindex=0><code>$ sudo pacman -Syu
+<li>Multiple GPU VMs (virtual memory address spaces) and GEM object binding based on the Xe UAPI model, to support future Vulkan requirements.</li><li>A result buffer, so the kernel driver can send GPU job execution results back to Mesa. This includes things like statistics and timings, but also whether the command succeeded and detailed fault information, so you can get verbose fault decoding right in Mesa!</li><li>Compute job support, to run compute shaders. We&rsquo;re still working on the Mesa side of this, but it should be enough to pass most tests and eventually add OpenCL support with <a href=https://www.phoronix.com/news/Rusticl-OpenCL-3.0-Conformance>Rusticl</a>!</li><li>The ability to submit multiple GPU jobs at once, and specify their dependencies directly, without using sync objects. This allows the GPU firmware to autonomously execute everything, which is a lot more efficient than going through the DRM scheduler every time. The Gallium driver doesn&rsquo;t use this yet, but it probably will in the future, and our upcoming Vulkan driver definitely will! There are <a href=/docs/SW-AGX-driver-notes#queues>a lot of subtleties</a> around how all the queuing stuff works&mldr;</li><li>Stub support for blit commands. We don&rsquo;t know how these work yet, but at least we have some skeleton support in the UAPI.</li></ul><p>To make all this work on the driver side, I ended up refactoring the <a href=https://github.com/AsahiLinux/linux/blob/gpu/rust-wip/drivers/gpu/drm/asahi/workqueue.rs>workqueue</a> code and adding a whole new <a href=https://github.com/AsahiLinux/linux/tree/gpu/rust-wip/drivers/gpu/drm/asahi/queue>queue</a> module which adds all the infrastructure to use sync objects to track command dependencies and completions and manage work via the DRM scheduler. Phew!</p><h2 id=conclusions>Conclusions</h2><p>So what does this all mean for users of the Asahi Linux reference distro today? It means&mldr; things are way faster!</p><p>Since the Mesa driver no longer serializes GPU and CPU work, performance has improved a ton. Now we can run Xonotic at over 800 FPS, which is faster than macOS on the same hardware (M2 MacBook Air) at around 600*! This proves that open source reverse engineered GPU drivers really have the power to beat Apple&rsquo;s drivers in real-world scenarios!</p><p>Not only that, our driver passes 100% of the dEQP-GLES2 and dEQP-EGL conformance tests, which is better OpenGL conformance than macOS for that version. But we&rsquo;re not stopping there of course, with full GLES 3.0 and 3.1 support well underway thanks to Alyssa&rsquo;s tireless efforts! You can follow the driver&rsquo;s feature support progress over at the <a href=https://mesamatrix.net/>Mesa Matrix</a>. There have been many, many other improvements over the past few months, and we hope you find things working better and more smoothly across the board!</p><p>Of course, there are lots of new corner cases we can hit now that we have support for implicit sync with an explicit sync driver. We already know of at least one minor regression (brief magenta squares for a couple of frames when KDE starts up), and there&rsquo;s probably more, so please report any issues on <a href=https://github.com/AsahiLinux/linux/issues/72>the GitHub tracker bug</a>! The more issue reports we get, especially if they come with easy ways to reproduce the problem, the easier it is for us to debug these problems and fix them ^^.</p><p>* <em>Please don&rsquo;t take the exact number <strong>too</strong> seriously, as there are other differences too (Xonotic runs under Rosetta on macOS, but it was also rendering at a lower resolution there due to being a non-Retina app). The point is that the results are in the same league, and we will only keep improving our driver going forward!</em></p><h2 id=get-it>Get it!</h2><p>If you&rsquo;re already using the GPU drivers, just update your system and reboot to get the new version! Keep in mind that since the UAPI changed (a lot), apps will probably stop launching or will launch with software rendering until you reboot.</p><p>If you still haven&rsquo;t tried the new drivers, just install the packages:</p><pre tabindex=0><code>$ sudo pacman -Syu
 $ sudo pacman -S linux-asahi-edge mesa-asahi-edge
 $ sudo update-grub
 </code></pre><p>Then if you&rsquo;re using KDE, make sure you have the Wayland session installed too:</p><pre tabindex=0><code>$ sudo pacman -S plasma-wayland-session