Audit Open Issues - August 2026

[CarloLucibello]

Following a full pass over the backlog (133 issues reviewed, 92 closed as fixed / stale / out-of-scope / duplicate), 42 issues remain open. This is a classification of all of them, so we have a shared picture of what is left and where the leverage is.

Two axes:

• Impact — 🔴 high (silent wrong answers, or blocks real work), 🟡 medium (crashes with a workaround, perf, commonly-requested features), 🟢 low (cosmetic, meta, nice-to-have).
• Effort — 🟩 easy (localised, or the fix is already written in the thread), 🟨 moderate (needs design or touches NNlib), 🟥 hard (deep, or needs hardware/upstream work), ⛔ blocked upstream.

---

The headline: BatchNorm is 6 of the 42

Issue	Impact	Effort	Note
#2179 wrong gradient on GPU in testmode	🔴	🟨	Silent. Gradient is identically 0.0 on GPU where CPU gives ~1.0. Candidate NNlib fix from @paulnovo has been sitting in-thread since 2024.
#1992 variance → NaN on single-sample batches	🔴	🟩	Silent → NaN. The Bessel-correction guard is written out in the OP.
#2649 wrong output on AMDGPU	🔴	🟥	Silent. Needs AMD hardware to bisect.
#2122 ForwardDiff + destructure disagrees with Zygote	🔴	🟥	Silent. Running-stat mutation under a dual-number pass.
#2154 not twice-differentiable on GPU	🟡	🟥	Errors rather than lies, at least.
#2616 cuDNN NOT_SUPPORTED on large 2D inputs	🟡	🟨	Verified not memory exhaustion — fails on a 32 GB card with a 100 MiB input, so it is the 2D→4D reshape path. Fix is a size check plus fallback to the generic kernel.

Three of these are wrong answers with no error raised, which is the worst failure mode we ship. If we do one focused piece of work out of this audit, it should be this layer — and the two highest-impact entries both already have a proposed fix.

Correctness — other

Issue	Impact	Effort	Note
#2571 infinite recursion on f32 of a fieldless layer	🟡	🟩	f32, f16 and cpu all StackOverflow; fmap(identity, ...) is fine, so it is the adapt_structure method generated by @layer. Contributor has been waiting since January.
#2407 zero-sized arrays into Dense	🟡	🟨
#2648 zero-sized array after indexing, in a gradient	🟡	🟨	Reproduces on CPU, so not GPU-specific — plausibly the same root cause as #2407. Worth fixing the pair together.
#2431 SamePad() not adjoint for even-sized filters	🟡	🟨	Conv/ConvTranspose do not round padding the same way. Convention change, needs a deprecation story.
#2321 show miscounts shared parameters	🟢	🟨	Reports phantom "non-trainable" arrays. Cosmetic but actively misleading.
#1703 OneHotArray scalar indexing in a gradient context	🟡	🟨
#2061 scalar indexing feeding a view of a CuArray to LSTMCell	🟡	⛔	JuliaGPU/CUDA.jl#1542.
#2591 ROCm-aware MPI needs explicit AMDGPU.synchronize()	🟡	🟨	Wrong averaged gradients without it — silent, but AMD+MPI only.
#2658 broken AMDGPU support	🟡	🟩	Upstream blocker LuxDL/Lux.jl#1676 closed 2026-03-18. May already be fixed — needs a buildkite re-run before anything else.

Enzyme

Issue	Impact	Effort	Note
#2607 fails on a simple MLP on GPU	🟡	⛔	EnzymeAD/Enzyme.jl#2837.
#2448 fails with MultiHeadAttention on CUDA	🟡	⛔	Same family.
#2570 Vararg warning in tests	🟢	⛔
#2444 test Enzyme gradients for loss functions	🟢	🟩	Pure test-writing, no design. Good first contribution.
#2445 test Enzyme GPU support	🟢	🟩	Same.

The three real bugs are all waiting on Enzyme.jl. The two test tasks are unblocked and easy.

Performance / allocations

Issue	Impact	Effort	Note
#2508 depthwise conv allocates ~750× more than the old layer	🔴	🟥	22 555 vs 30 allocations. Directly penalises MobileNet-class architectures. Lives in the NNlib grouped-conv path.
#2523 CUDA memory usage grows over time	🟡	🟥	Largely lazy-GC semantics rather than a leak, but it recurs on Discourse often enough to deserve either a fix or a documented answer.
#1957 type-stable rrule for applychain(::Vector)	🟢	🟥

Features

Issue	Impact	Effort	Note
#2153 class weights in losses	🟡	🟩	Common ask; the OP sketches the implementation. Mostly an API-consistency decision across the loss suite.
#2454 opt-in error (not warning) on wrong input precision	🟡	🟩	A Preference flag, mirroring CUDA.allowscalar. Would save a lot of debugging.
#2336 padding masks in MultiHeadAttention	🟡	🟨	Only causal (seq, seq) masks work; (seq, batch) padding masks do not. Real gap for transformer users.
#1917 Conv with circular / reflect padding	🟡	🟨	NNlib has the padding functions already.
#1493 losses + OneHotArray	🟡	🟨
#395 max/mean unpooling	🟡	🟨	Labelled good first issue. NNlib-side work.
#2514 RNN feature requests (meta)	🟡	🟥	Remaining: cuDNN kernels, bidirectional. Keep as tracker.
#1335 per-layer RNG interface	🟢	🟨
#2013 missing functional equivalents for some layers	🟢	🟩	Labelled good first issue.
#2543 show fieldnames in containers	🟢	🟩	RNN(cell = RNNCell(4 => 6, tanh)). Small, self-contained.
#1999 drop type params of Chain children in compact show	🟢	🟨	Would make stacktraces far more readable.
#451 N-dimensional conv, N>3	🟢	🟥	NNlib + cuDNN.
#1312 batch renormalization	🟢	🟨
#91 ConvRNN / ConvLSTM cells	🟢	🟥	Better suited to RecurrentLayers.jl.

Docs & infrastructure

Issue	Impact	Effort	Note
#2410 PyTorch → Flux migration notes	🟡	🟩	Substantial material already written in the thread (init defaults, gain conventions). Needs someone to turn it into a docs page.
#1431 PyTorch feature parity (meta, 94 comments)	🟡	—	Keep as a tracker; do not try to close as a unit.
#1812 docs: when GPU weights can safely be saved/loaded	🟢	🟩	One-line wording fix in docs/src/guide/saving.md.
#2237 separate nightly CI from the posted badge	🟢	🟩

---

Suggested order of attack

Now, high value and cheap:

1. Batchnorm's variance becomes NaN when minibatch contains just one sample #1992 — fix is in the OP.
2. broken AMDGPU support #2658 — just re-run AMDGPU CI; may already be resolved.
3. infinite recursion when applying f32 on empty layer #2571 — a contributor is waiting, and the recursion is localised to @layer.
4. wrong gradient for BatchNorm on gpu in testmode #2179 — a candidate fix has been sitting unused for two years, and this is a silent wrong gradient.

Then, the two clusters worth a dedicated pass:

• BatchNorm (Batchnorm's variance becomes NaN when minibatch contains just one sample #1992, ForwardDiff + destructure is different from Zygote, on a model with BatchNorm #2122, BatchNorm is not twice-differentiable #2154, wrong gradient for BatchNorm on gpu in testmode #2179, BatchNorm Memory Blowup on Very Large CUDA Arrays #2616, [AMDGPU] wrong output from BatchNorm #2649) — six issues, one layer, three silent.
• Zero-sized arrays (Zero-sized arrays cannot be applied to Dense layers. #2407, Zero-sized Array after indexing causes exception during gradient #2648) — likely one root cause.

Good first issues, unblocked today: #2444, #2445, #2543, #2013, #1812, #2237, #2454.

Do not spend time on (⛔): #2607, #2448, #2570, #2061 — all upstream.

Corrections and re-prioritisations welcome; the point of writing it down is so we can argue about it.