Flux sometimes fails spuriously before `encodeText` and then kills my test iPad entirely until restart.

[Bersaelor]

Model name

flux

Command run

This is a follow up of this thread I have now managed to complete many generations of images. Here's how I got there:

uv run coreai.diffusion.export flux2-klein-4b --platform iOS --overwrite

then compiled

{
    for m in VAEEncoder_half VAEDecoder_half TextEncoder Transformer_512; do
      xcrun coreai-build compile "$SRC/${m}.aimodel" \
        --platform iOS --architecture h16g --preferred-compute gpu \
        --output "$DST/${m}.aimodelc"
    done
}

imported into Xcode target and called via the documentation.

macOS / iOS target

macos 27 beta 2, ipadOS 27 beta 2

Xcode version

xcode 27 beta 2

Python / uv version

uv 0.11.25

Full error output

When prompting I sometimes get spurious:

Task 1123: signal SIGABRT

in this line of code of the coreai-models framework.

here's a full stack trace screenshot:

Anything else?

It would be really nice if the framework coreai-models could give more info about what exactly went wrong, or help the host app recovering instead of crashing.

[Bersaelor]

When this happens, my iPad is basically in a lost state.
Any further attempt to start the app via Xcode leads to just:

terminating due to uncaught exception of type std::bad_alloc: std::bad_alloc
Debug session ended with code 9: killed

I can also not open the app again from the homescreen.

Only thing that helps is restarting the iPad.

[Bersaelor]

Sometimes it does work though and I think the results are promising, so it'd be great tool for apps to have this working in a stable way:

[Bersaelor]

The native Image Playground works great too, but with that one the user has to help and click the sheets so I can't just create a set of images programmatically in iOS27.

[stikves]

Thank you for your report.

Did you have a chance to run the source models directly, triggering on device JIT, instead of pre-compilation?

Asking to help isolate the root cause.

[Bersaelor]

Thank you for your report.

Did you have a chance to run the source models directly, triggering on device JIT, instead of pre-compilation?

Asking to help isolate the root cause.

I tried that at first on my M4 iPad ( MVV83NF/A, 256GB, 11" ) I only have 8GB of RAM, so flux2-klein-4b --platform iOS always failed due to memory pressure.
I had to AIT compile it before I could get it to work.

[Bersaelor]

I think, it is easier to replicate this for me, since I always create a deck of cards.
While it has happened that the ipad froze on the first image generation, the chance seems rather low, maybe 1 in 20 to 50 generations.
But since I always generate at least 10 cards for a deck, the chance one deck-generation encounters this increases and then I have to restart my iPad before it works normally again.

[stikves]

I see

The iPads I used to test this model were all 16GB versions (M1 -> M5). It might be just at the memory boundary, and some recent change might have pushed it beyond 8GB limit.

I'll try to reproduce this on a 8GB device to test this hypothesis.

[Bersaelor]

Mhmm, i see. For what it's worth today I did 20 more Flux generations without any crashes, so they above estimate of 1 in 20 to 1 in 50 generations still seems a good range, if maybe a little low.

[stikves]

(Until I can acquire a test iPad...)

To clarify: does your app keep the whole pipeline in memory (text encoder, transformer, decoder)?
Or does it selectively unload and reload them on demand?

[Bersaelor]

To clarify: does your app keep the whole pipeline in memory (text encoder, transformer, decoder)?

Let me check this carefully, as I really just made minor tweaks to the sample code provided in the readme here.

My FluxEngine sits in a separate target, so I can weakly bind it (so my app can still run in the simulator allowing me to use the previews). In the FluxEngine I have warmUp(), generate() and unload() methods.
in warmUp() I run

pipeline = try await Flux2Pipeline(from: url, mode: .half)

and in unload() I run

        await pipeline?.unloadResources()
        pipeline = nil

but the unload() is only called AFTER all the cards are generated, not in between each generation. Memory stayed reasonable too, and didn't go up with increasing amounts of cards generated. The bug above happened even for very early cards as well as for later cards, it seemed to be extremely random.

That said, I can't currently test the app, as of Xcode beta 3, every time I try to run Flux generation I get:

generate: start 4 steps, seed 481532553, prompt='a playful cartoon illustration of a single Boulder, light blue background, no scenery, bold clean outlines, flat shading' (footprint 327 MB)
loc(fused<{call_stack = ["Embedding$1", "Qwen3Model$1", "Qwen3ForCausalLM$1", "Flux2TextEncoderWrapper$1"], identifiers = ["embedding"]}>["/Users/konradfeiler/Desktop/coreai-models/.venv/lib/python3.11/site-packages/torch/nn/modules/sparse.py":192:0, "/Users/konradfeiler/Desktop/coreai-models/.venv/lib/python3.11/site-packages/transformers/models/qwen3/modeling_qwen3.py":371:0, "/Users/konradfeiler/Desktop/coreai-models/.venv/lib/python3.11/site-packages/transformers/models/qwen3/modeling_qwen3.py":480:0, "/Users/konradfeiler/Desktop/coreai-models/python/src/coreai_models/diffusion/flux2.py":167:0, "coreai"("op_id":2718:0), "torch"("op_id":402:0), "coreai"("op_id":2718:0), "coreai"("op_id":2718:0)]): error: expected AICode versioned location, got: loc(fused<{call_stack = ["Embedding$1", "Qwen3Model$1", "Qwen3ForCausalLM$1", "Flux2TextEncoderWrapper$1"], identifiers = ["embedding"]}>["/Users/konradfeiler/Desktop/coreai-models/.venv/lib/python3.11/site-packages/torch/nn/modules/sparse.py":192:0, "/Users/konradfeiler/Desktop/coreai-models/.venv/lib/python3.11/site-packages/transformers/models/qwen3/modeling_qwen3.py":371:0, "/Users/konradfeiler/Desktop/coreai-models/.venv/lib/python3.11/site-packages/transformers/models/qwen3/modeling_qwen3.py":480:0, "/Users/konradfeiler/Desktop/coreai-models/python/src/coreai_models/diffusion/flux2.py":167:0, "coreai"("op_id":2718:0), "torch"("op_id":402:0), "coreai"("op_id":2718:0), "coreai"("op_id":2718:0)])
loc(fused<{call_stack = ["Embedding$1", "Qwen3Model$1", "Qwen3ForCausalLM$1", "Flux2TextEncoderWrapper$1"], identifiers = ["embedding"]}>["/Users/konradfeiler/Desktop/coreai-models/.venv/lib/python3.11/site-packages/torch/nn/modules/sparse.py":192:0, "/Users/konradfeiler/Desktop/coreai-models/.venv/lib/python3.11/site-packages/transformers/models/qwen3/modeling_qwen3.py":371:0, "/Users/konradfeiler/Desktop/coreai-models/.venv/lib/python3.11/site-packages/transformers/models/qwen3/modeling_qwen3.py":480:0, "/Users/konradfeiler/Desktop/coreai-models/python/src/coreai_models/diffusion/flux2.py":167:0, "coreai"("op_id":2718:0), "torch"("op_id":402:0), "coreai"("op_id":2718:0), "coreai"("op_id":2718:0)]): error: Failed to convert to versioned IR
LLVM ERROR: cannot unwrap empty `odiec_module_t`

I thought, huh, it's probably because I compiled the models to *.aimodelc using the Beta 2 metal toolchain, so I should run the metal toolchain for beta 3 again...
When I ran xcrun coreai-build I got the exact same error as on device, which got me to the idea to upgrade coreai-models as I was still on 1.7.2 to 1.74.

After upgrading and running uv run coreai.diffusion.export flux2-klein-4b --platform iOS again, I tried with the uncompiled *.aimodel files again, but still the same error on the 8GB iPad Pro:

guard let fn = try loadedModel.loadFunction(named: "main") else {

generate: start 4 steps, seed 2249184362, prompt='a playful cartoon illustration of a single Tractor, green background, no scenery, bold clean outlines, flat shading' (footprint 312 MB)
/Library/Caches/com.apple.xbs/68948C54-2B3B-4272-942A-8BD92FF3552E/TemporaryDirectory.54sxx3/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Runtimes/MPSRuntime/MemrefBufferizationRuntime.mm, line 202: error 'createMemrefHeap: newHeapWithDescriptor returned nil (size=3847225344, manualPlacement=1)'
/Library/Caches/com.apple.xbs/68948C54-2B3B-4272-942A-8BD92FF3552E/TemporaryDirectory.54sxx3/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Runtimes/MPSRuntime/MemrefBufferizationRuntime.mm:202: failed assertion `createMemrefHeap: newHeapWithDescriptor returned nil (size=3847225344, manualPlacement=1)'
/Library/Caches/com.apple.xbs/68948C54-2B3B-4272-942A-8BD92FF3552E/TemporaryDirectory.54sxx3/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Runtimes/MPSRuntime/MemrefBufferizationRuntime.mm:202: failed assertion `createMemrefHeap: newHeapWithDescriptor returned nil (size=3847225344, manualPlacement=1)'

so after compiling h16g versions of the 4 *.aimodelc again, putting them into the app, I get generations again. Will run some more long-term tests to calculate how often the crash happens statistically on the latest version.
@stikves I can however already answer your question, the unload() function is only called when the full deck of cards is generated.

Subjectively though, I feel like the previous Flux generations adhered better to the prompts, these ones are pretty wonky:
Prompt:

a playful cartoon illustration of a single Ant, green background, no scenery, bold clean outlines, flat shading'

[Bersaelor]

Also, I noticed despite xcode still being connected with the debugger, after the first generation, I don't see any logs anymore. App till runs, I can see health using the debug tab of xcode, and can confirm the memory usage does not increase. But no more logs. Does generating models stop logs from working?