You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix: use saturating add/sub in i16 DCT column pass to prevent overflow
When overshoot deringing pushes inputs to ±158, the column-pass butterfly
accumulates values that can exceed i16::MAX (e.g., 8 × 5056 = 40,448).
Wrapping add caused catastrophic sign flips in decoded pixels.
All butterfly adds/subs in dodct() now use _mm256_adds_epi16 /
_mm256_subs_epi16 (saturating). The column-pass descale rounding constant
add is also saturating to prevent 32767 + 2 wrapping to -32767.
Row pass uses wrapping _mm256_slli_epi16 (values naturally in safe range).
Worst-case saturation clamps coefficient [0,1] from 8294 to 8191 (1.2%),
which is at most 1 quantization step on extreme deringing patterns.
Verified: all 6 synthetic overflow patterns pass (max_diff ≤ 2 vs i32 ref).
See mozilla/mozjpeg#453.
0 commit comments