Disk Performance

[d-shehu]

Neat project, glad to see Apple is providing greater support for container dev.

One of the major hurdles with running containers (Docker, podman) or VMs on MacOS is relatively slow disk I/O, primarily with random read/write.

How's disk performance and if it's still slow is there a plan for robust passthrough of NVME disks especially over Thunderbolt? Thanks.

[jglogan]

Can you be more specific? Are you referring to disk I/O to the root disk or named volumes (virtioblk mounts)? Or to "bind mounts" (virtiofs mounts)?

What sort of "random read/write" workload are you referring to, and what are you comparing the container tools' performance to?

Knowing that will help us understand where the gaps might lie.

See also this issue recently filed on containerization that shows a more specific comparison of I/O performance (appears to be all for virtioblk devices and not for virtiofs). apple/containerization#729

[d-shehu]

It started with running my Docker stack with bind mounts 5-6 ago on my M1 laptop and continues to this day even with VirtioFS on my M3 Pro. I've tested running the VM on root disk or NVME over Thunderbolt 4. So I gave up on Docker or containers on MacOS. This summarizes things nicely.

https://www.paolomainardi.com/posts/docker-performance-macos-2025/

For the last 3-4 years, I've always run UTM VMs on my Mac so I can leverage my toolchain and keep things consistent with my Linux boxes. Performance was better as was compatibility. And I always used Apple's virtualization since QEMU virtualization with UTM was noticeably slower.

It's been tolerable for my little apps but recently I wanted to do more with (1) nested virtualization for a few special cases. And I've been testing apps that use (2) elastic search, (3) running K3s and minio nodes and other I/O bound use cases.

I especially noticed the sluggish behavior with container overlays in the nested VM scenario when pulling images for podman. It took upwards of 5+ minutes vs 20-30 secs on bare metal system with comparable SSD.

Basic tests revealed random read and writes were ~7-14x slower in the VM on M3 Pro Macbook again with virtualization and Arm Linux. Even with just 1 VM running a minimized headless Linux server!

I saw comparable slowdown with Intel Macbook 2019. So it doesn't seem specific to M series.

utmapp/UTM#7703

With AI bubble, I wanted to leverage spare Macs as Linux server. But aside from doing something crazy like trying to run Proxmox and then MacOS in a VM, I'm hitting a wall. Hope that helps clarify things.

TLDR
What is the best way to run with Apple virtualization? Perhaps I'm doing something wrong when configuring my VM?

What is the expected performance for random reads/writes in a VM vs MacOS host?

Thanks.

[jglogan]

TLDR
What is the best way to run with Apple virtualization? Perhaps I'm doing something wrong when configuring my VM?

"bind mounting" directories from the host uses virtiofs which is pretty slow for heavy I/O workloads like builds where writes are relatively small and there's a fair number of metadata writes.

Using a named volume will be significantly faster - it's using a virtioblk mount of an ext4 disk image file that resides on your host.

It's been a while since I've benchmarked but for builds, I'd typically see native clean build times be about 2-3x faster than virtioblk, which is in turn about 3x faster than virtiofs.

Random vs sequential isn't so much of a factor with SSDs. Very small writes are bad for performance and SSD wear but generally the OS buffer cache should be avoiding that.

[d-shehu]

Thanks, I've been using ext4 disk image as my roaming personal volume to share among machines (Linux VM on Mac/Win vs Linux). But based on limited benchmarking it's actually slower. Since I use it to quickly rsync large amounts of data between machines it's fine for that use case.

I'd hoped that Apple containers would be implemented more like native Linux containers with compatibility for OCI image or the ability to reuse Dockerfiles to build the "same" image on a Mac. Alternatively if we had options with Apple virtualization and could passthrough disks/NICs to the underlying VM that would be great since we could stick with native Linux tooling.

I hope this gets more publicity with 1.0 and we can re-evaluate performance.

[jglogan]

I'd hoped that Apple containers would be implemented more like native Linux containers with compatibility for OCI image or the ability to reuse Dockerfiles to build the "same" image on a Mac.

Apple containers are Linux containers that use a 1:1 container:VM model, as opposed to n:1.

See: https://github.com/apple/container/blob/main/docs/technical-overview.md#how-does-container-run-my-container

You use container build to build an OCI image, and container run to run an OCI image.

It's not clear what this has a to do with the I/O performance. Whether you run a traditional Linux VM using Parallels, UTM, or some other tool, or you run a Linux container in a Linux VM like container, if you're using Virtualization.framework for the VM, the performance is determined by the maximum IOPS and throughput from your workload, through the Linux storage stack and virtioblk driver, and down through the macOS kernel to the storage device on the host.