rpng: stream deflate output via multi-IDAT instead of one big buffer

Replaces rpng_save_image_stream's full-frame encode_buf + monolithic
deflate with per-row incremental deflate feeding a 16 KiB chunk buffer
that emits multiple IDAT chunks as it fills. Pixel output is unchanged;
only the internal memory shape differs.

Peak transient memory at 4K BGR24 drops from ~50 MiB (encode_buf +
deflate_buf, both sized to the full filtered frame) to ~70 KiB
(per-row filter scratch + one 16 KiB chunk buffer). Memory now scales
with width only, not width*height. Cost is ~0.07% more output bytes
from IDAT chunk headers (12 bytes per 16 KiB chunk):

  size       old bytes  new bytes  delta
  320x240      19,733     19,745   +12  (+0.06%)
  1024x768   100,995    101,067   +72  (+0.07%)
  1920x1080  181,909    182,041  +132  (+0.07%)

A 1920x1080 screenshot now produces 12 IDAT chunks vs. 1 before.
pngcheck accepts both old and new output at identical compression
ratio (97.8% on 1920x1080 pseudorandom).

Wall-clock performance: no regression. Measured on Linux x86_64
with /tmp on tmpfs, 5 iterations per config, two trials each:

  size       pattern    old avg    new avg    delta
  1280x720   solid      112.1 ms   112.2 ms   +0.1%
  1280x720   gradient   116.2 ms   112.7 ms   -3.0%
  1280x720   photo      497.7 ms   497.2 ms   -0.1%
  1280x720   random     129.3 ms   127.8 ms   -1.1%
  1920x1080  solid      244.9 ms   238.3 ms   -2.7%
  1920x1080  gradient   249.5 ms   240.5 ms   -3.6%
  1920x1080  random     270.8 ms   269.5 ms   -0.5%

Deltas below ~3% are within the noise floor observed for the same
encoder trial-to-trial. The overall shape is that the new encoder
is equal to or marginally faster than the old one across every
measured case, plausibly from reduced allocator pressure (no
~25 MiB malloc+free per encode) and better cache locality. Results
on spinning disks where each IDAT flush becomes a real write may
differ; not measured.

Filter selection, per-row scratch layout, and prev_encoded carry-over
are unchanged -- same five filters, same lowest-SAD pick. Only the
downstream "what happens to the filtered row" differs.

Three trans_stream_zlib subtleties the implementation accounts for:

  1. The backend reports AGAIN (not BUFFER_FULL) when avail_out and
     avail_in both hit zero on the same call -- BUFFER_FULL requires
     avail_in != 0. We detect this via chunk_fill >= IDAT_CHUNK_SIZE
     after every successful row feed and flush proactively; otherwise
     the next row's trans() would find avail_out=0 with fresh input
     waiting and return Z_BUF_ERROR.
  2. During the final Z_FINISH drain, AGAIN means "more output
     pending, buffer full" (set_in(NULL,0) ensures input is never
     the gating factor). We flush and retry until NONE.
  3. flush_idat_chunk tolerates payload_len=0 so a Z_STREAM_END
     landing on an exact chunk boundary doesn't emit a spurious
     empty IDAT.

Ships with a round-trip regression test
(libretro-common/samples/formats/png/rpng_roundtrip_test.c) verifying
pixel-level round-trip across all three public encode entry points
(argb, bgr24 top-down, bgr24 bottom-up via the negative-pitch trick
used by task_screenshot's viewport fast path) and three size tiers:

  - Hand-picked (45):  4x4, 37x29, 320x240 x 5 patterns x 3 entry
                       points. Patterns exercise different filter
                       selection outcomes.
  - Small-width (48):  widths {1,2,3,8,31,32,33,257} x heights {1,3}
                       x 3 entry points, pseudorandom. Catches
                       off-by-one bugs at narrow-width and near-
                       alignment boundaries.
  - Large (12):        1024x1, 1x1024, 2048x1, 1x12000 x 3 entry
                       points, pseudorandom. 1x12000 crosses zlib's
                       default 32 KiB sliding window.

105 subtests total. Exit 0 on success.

Every encoded PNG also passes a structural check: pngcheck(1) if
installed, otherwise a built-in fallback that walks chunks, verifies
CRCs via encoding_crc32 (already linked), and enforces chunk ordering.
pngcheck is not a build or runtime dependency. When the built-in
fallback is active a startup self-test generates a valid PNG,
confirms the fallback accepts it, corrupts a byte, confirms the
fallback rejects it -- aborting if the validator itself is broken
rather than running 105 subtests under a broken validator.

CI integration
(.github/workflows/Linux-libretro-common-samples.yml):
adds rpng_roundtrip_test to the RUN_TARGETS allowlist and extends
the Makefile target-extraction regex from TARGET_TEST to
TARGET_TEST[0-9]* so that TARGET_TEST2 is picked up by auto-discovery.
Strict superset -- extracted target lists are unchanged for every
other Makefile in libretro-common/samples/.

Verified:
  - 105/105 pass with pngcheck validation active.
  - 105/105 pass with built-in fallback active (stock ubuntu-latest
    runner scenario -- no pngcheck, zlib1g-dev only).
  - ASan+UBSan clean in both configurations.
  - pngcheck OK on every output file.
  - File sizes within +0.07% of old encoder.
  - No wall-clock regression on measured configurations.

Author: LibretroAdmin

.github/workflows/Linux-libretro-common-samples.yml

@@ -59,6 +59,7 @@ jobs:
             rtga_test
             rbmp_test
             rpng_chunk_overflow_test
+            rpng_roundtrip_test
           )
 
           # Per-binary run command (overrides ./<binary> if present).
@@ -130,8 +131,9 @@ jobs:
             #   TARGET := foo
             #   TARGETS = a b c
             #   TARGET_TEST := foo_test     (second target in same Makefile)
+            #   TARGET_TEST2 := foo_test2   (third target, fourth, ...)
             mapfile -t targets < <(
-              grep -hE '^(TARGET|TARGETS|TARGET_TEST)[[:space:]]*[:?]?=' "$d/Makefile" \
+              grep -hE '^(TARGET|TARGETS|TARGET_TEST[0-9]*)[[:space:]]*[:?]?=' "$d/Makefile" \
               | sed -E 's/^[^=]*=[[:space:]]*//' \
               | tr -s ' \t' '\n' \
               | grep -v '^$' \

libretro-common/formats/png/rpng_encode.c

@@ -204,26 +204,56 @@ static unsigned filter_paeth(uint8_t *target,
    return count_sad(target, width);
 }
 
+/* Size of the per-chunk deflate output buffer.  A screenshot-sized
+ * encode will fill this many times over and produce multiple IDAT
+ * chunks; smaller than zlib's default window (32 KiB) to keep
+ * peak memory low while large enough to amortise chunk-header
+ * overhead (12 bytes per IDAT). */
+#define IDAT_CHUNK_SIZE 16384
+
+/* Emit one IDAT chunk.  `chunk_buf` is laid out as:
+ *     bytes [0..4): length field (filled in here, big-endian)
+ *     bytes [4..8): literal "IDAT"
+ *     bytes [8..8+payload_len): the deflate output produced by
+ *                               this chunk's worth of trans() calls
+ * Matches the layout png_write_idat_string expects. */
+static bool flush_idat_chunk(intfstream_t *intf_s,
+      uint8_t *chunk_buf, size_t payload_len)
+{
+   if (payload_len == 0)
+      return true; /* empty chunk -- nothing to emit, not an error */
+   dword_write_be(chunk_buf + 0, (uint32_t)payload_len);
+   memcpy(chunk_buf + 4, "IDAT", 4);
+   return png_write_idat_string(intf_s, chunk_buf, payload_len + 8);
+}
+
 bool rpng_save_image_stream(const uint8_t *data, intfstream_t* intf_s,
       unsigned width, unsigned height, signed pitch, unsigned bpp)
 {
    unsigned h;
    struct png_ihdr ihdr = {0};
    bool ret = true;
    const struct trans_stream_backend *stream_backend = NULL;
-   size_t encode_buf_size  = 0;
-   uint8_t *encode_buf     = NULL;
-   uint8_t *deflate_buf    = NULL;
-   uint8_t *rgba_line      = NULL;
-   uint8_t *up_filtered    = NULL;
-   uint8_t *sub_filtered   = NULL;
-   uint8_t *avg_filtered   = NULL;
-   uint8_t *paeth_filtered = NULL;
-   uint8_t *prev_encoded   = NULL;
-   uint8_t *encode_target  = NULL;
-   void *stream            = NULL;
-   uint32_t total_in       = 0;
-   uint32_t total_out      = 0;
+   uint8_t *rgba_line        = NULL;
+   uint8_t *up_filtered      = NULL;
+   uint8_t *sub_filtered     = NULL;
+   uint8_t *avg_filtered     = NULL;
+   uint8_t *paeth_filtered   = NULL;
+   uint8_t *prev_encoded     = NULL;
+   /* filter_line holds [filter_byte][filtered_row] and is what we
+    * feed into deflate one row at a time. */
+   uint8_t *filter_line      = NULL;
+   /* chunk_buf is the IDAT-chunk staging buffer:
+    *   [0..4):        length field (filled in at flush time)
+    *   [4..8):        "IDAT"
+    *   [8..8+IDAT_CHUNK_SIZE): deflate output */
+   uint8_t *chunk_buf        = NULL;
+   void *stream              = NULL;
+   size_t line_len           = (size_t)width * bpp;
+   /* How many bytes deflate has produced into the current chunk_buf
+    * since the last set_out.  Reset to 0 after every flush_idat_chunk. */
+   size_t chunk_fill         = 0;
+   enum trans_stream_error err = TRANS_STREAM_ERROR_NONE;
 
    if (!intf_s)
       GOTO_END_ERROR();
@@ -233,133 +263,170 @@ bool rpng_save_image_stream(const uint8_t *data, intfstream_t* intf_s,
    if (intfstream_write(intf_s, png_magic, sizeof(png_magic)) != sizeof(png_magic))
       GOTO_END_ERROR();
 
-   ihdr.width = width;
-   ihdr.height = height;
-   ihdr.depth = 8;
+   ihdr.width      = width;
+   ihdr.height     = height;
+   ihdr.depth      = 8;
    ihdr.color_type = bpp == sizeof(uint32_t) ? 6 : 2; /* RGBA or RGB */
    if (!png_write_ihdr_string(intf_s, &ihdr))
       GOTO_END_ERROR();
 
-   encode_buf_size = (width * bpp + 1) * height;
-   encode_buf      = (uint8_t*)malloc(encode_buf_size);
-   if (!encode_buf)
+   /* Per-row scratch.  ~width*bpp each -- trivial compared to the
+    * frame-sized encode_buf the old full-buffer path allocated. */
+   prev_encoded   = (uint8_t*)calloc(1, line_len);
+   rgba_line      = (uint8_t*)malloc(line_len);
+   up_filtered    = (uint8_t*)malloc(line_len);
+   sub_filtered   = (uint8_t*)malloc(line_len);
+   avg_filtered   = (uint8_t*)malloc(line_len);
+   paeth_filtered = (uint8_t*)malloc(line_len);
+   filter_line    = (uint8_t*)malloc(line_len + 1);
+   chunk_buf      = (uint8_t*)malloc(IDAT_CHUNK_SIZE + 8);
+   if (!prev_encoded || !rgba_line || !up_filtered || !sub_filtered
+         || !avg_filtered || !paeth_filtered || !filter_line
+         || !chunk_buf)
       GOTO_END_ERROR();
 
-   prev_encoded = (uint8_t*)calloc(1, width * bpp);
-   if (!prev_encoded)
+   stream = stream_backend->stream_new();
+   if (!stream)
       GOTO_END_ERROR();
 
-   rgba_line      = (uint8_t*)malloc(width * bpp);
-   up_filtered    = (uint8_t*)malloc(width * bpp);
-   sub_filtered   = (uint8_t*)malloc(width * bpp);
-   avg_filtered   = (uint8_t*)malloc(width * bpp);
-   paeth_filtered = (uint8_t*)malloc(width * bpp);
-   if (!rgba_line || !up_filtered || !sub_filtered || !avg_filtered || !paeth_filtered)
-      GOTO_END_ERROR();
+   /* Point deflate's output at our chunk staging area (after the
+    * 8-byte chunk header).  We re-point it every time we flush
+    * a chunk so the driver doesn't need to know the chunk layout. */
+   stream_backend->set_out(stream,
+         chunk_buf + 8, (uint32_t)IDAT_CHUNK_SIZE);
 
-   encode_target = encode_buf;
-   for (h = 0; h < height;
-         h++, encode_target += width * bpp, data += pitch)
+   for (h = 0; h < height; h++, data += pitch)
    {
+      uint32_t rd, wn;
+      uint8_t filter;
+      unsigned none_score, up_score, sub_score, avg_score, paeth_score;
+      unsigned min_sad;
+      const uint8_t *chosen_filtered;
+
       if (bpp == sizeof(uint32_t))
          copy_argb_line(rgba_line, (const uint32_t*)data, width);
       else
          copy_bgr24_line(rgba_line, data, width);
 
-      /* Try every filtering method, and choose the method
-       * which has most entries as zero.
+      /* Filter selection unchanged from the previous implementation:
+       * try every filter, pick the one with lowest sum-of-abs-deviation. */
+      none_score  = count_sad(rgba_line, line_len);
+      up_score    = filter_up   (up_filtered,    rgba_line, prev_encoded, width, bpp);
+      sub_score   = filter_sub  (sub_filtered,   rgba_line,               width, bpp);
+      avg_score   = filter_avg  (avg_filtered,   rgba_line, prev_encoded, width, bpp);
+      paeth_score = filter_paeth(paeth_filtered, rgba_line, prev_encoded, width, bpp);
+
+      filter          = 0;
+      min_sad         = none_score;
+      chosen_filtered = rgba_line;
+      if (sub_score < min_sad)   { filter = 1; chosen_filtered = sub_filtered;   min_sad = sub_score;   }
+      if (up_score < min_sad)    { filter = 2; chosen_filtered = up_filtered;    min_sad = up_score;    }
+      if (avg_score < min_sad)   { filter = 3; chosen_filtered = avg_filtered;   min_sad = avg_score;   }
+      if (paeth_score < min_sad) { filter = 4; chosen_filtered = paeth_filtered;                        }
+
+      filter_line[0] = filter;
+      memcpy(filter_line + 1, chosen_filtered, line_len);
+      memcpy(prev_encoded,    rgba_line,       line_len);
+
+      /* Feed this row into deflate. The loop handles the case where
+       * our chunk buffer fills mid-row (BUFFER_FULL): flush IDAT,
+       * point deflate at a fresh output buffer, and keep going.
        *
-       * This is probably not very optimal, but it's very
-       * simple to implement.
-       */
+       * When trans() returns success with err=AGAIN, zlib has
+       * consumed what we gave it but hasn't finalized (no Z_FINISH
+       * was requested) -- that's the normal "ok, send more data
+       * next time" signal.  We break out and feed the next row. */
+      stream_backend->set_in(stream, filter_line, (uint32_t)(line_len + 1));
+      for (;;)
       {
-         unsigned none_score  = count_sad(rgba_line, width * bpp);
-         unsigned up_score    = filter_up(up_filtered, rgba_line, prev_encoded, width, bpp);
-         unsigned sub_score   = filter_sub(sub_filtered, rgba_line, width, bpp);
-         unsigned avg_score   = filter_avg(avg_filtered, rgba_line, prev_encoded, width, bpp);
-         unsigned paeth_score = filter_paeth(paeth_filtered, rgba_line, prev_encoded, width, bpp);
-
-         uint8_t filter       = 0;
-         unsigned min_sad     = none_score;
-         const uint8_t *chosen_filtered = rgba_line;
-
-         if (sub_score < min_sad)
-         {
-            filter = 1;
-            chosen_filtered = sub_filtered;
-            min_sad = sub_score;
-         }
+         bool ok = stream_backend->trans(stream, false, &rd, &wn, &err);
+         chunk_fill += wn;
 
-         if (up_score < min_sad)
+         if (ok)
          {
-            filter = 2;
-            chosen_filtered = up_filtered;
-            min_sad = up_score;
+            /* All input consumed.  If the output buffer also happens
+             * to be exactly full (avail_in=0 AND avail_out=0 on the
+             * same call, which the trans API reports as success
+             * with AGAIN rather than BUFFER_FULL), flush proactively
+             * -- otherwise the next row's trans() would find
+             * avail_out=0 and error out. */
+            if (chunk_fill >= IDAT_CHUNK_SIZE)
+            {
+               if (!flush_idat_chunk(intf_s, chunk_buf, chunk_fill))
+                  GOTO_END_ERROR();
+               chunk_fill = 0;
+               stream_backend->set_out(stream,
+                     chunk_buf + 8, (uint32_t)IDAT_CHUNK_SIZE);
+            }
+            break;
          }
 
-         if (avg_score < min_sad)
-         {
-            filter = 3;
-            chosen_filtered = avg_filtered;
-            min_sad = avg_score;
-         }
-
-         if (paeth_score < min_sad)
-         {
-            filter = 4;
-            chosen_filtered = paeth_filtered;
-         }
+         if (err != TRANS_STREAM_ERROR_BUFFER_FULL)
+            GOTO_END_ERROR();
 
-         *encode_target++ = filter;
-         memcpy(encode_target, chosen_filtered, width * bpp);
-
-         memcpy(prev_encoded, rgba_line, width * bpp);
+         /* Output filled mid-row.  chunk_fill should equal
+          * IDAT_CHUNK_SIZE.  Flush and re-point. */
+         if (!flush_idat_chunk(intf_s, chunk_buf, chunk_fill))
+            GOTO_END_ERROR();
+         chunk_fill = 0;
+         stream_backend->set_out(stream,
+               chunk_buf + 8, (uint32_t)IDAT_CHUNK_SIZE);
       }
    }
 
-   /* Deflate output is typically smaller than input; zlib guarantees it
-    * will not exceed input + 0.1% + 12 bytes.  We use 1.01x + 256 for
-    * safety, which is substantially less wasteful than the previous 2x. */
+   /* All rows consumed.  Drain deflate with Z_FINISH, emitting IDATs
+    * on BUFFER_FULL, final partial on NONE (Z_STREAM_END). */
+   stream_backend->set_in(stream, NULL, 0);
+   for (;;)
    {
-      size_t deflate_bound = (size_t)(encode_buf_size * 1.01) + 256;
-      deflate_buf = (uint8_t*)malloc(deflate_bound + 8); /* +8 for IDAT header */
-      if (!deflate_buf)
-         GOTO_END_ERROR();
+      uint32_t rd = 0, wn = 0;
+      bool ok = stream_backend->trans(stream, true, &rd, &wn, &err);
+      chunk_fill += wn;
 
-      stream = stream_backend->stream_new();
-
-      if (!stream)
+      if (!ok)
+      {
+         /* BUFFER_FULL during flush-drain with avail_in=0 shouldn't
+          * strictly be reachable, but handle defensively. */
+         if (err != TRANS_STREAM_ERROR_BUFFER_FULL)
+            GOTO_END_ERROR();
+         if (!flush_idat_chunk(intf_s, chunk_buf, chunk_fill))
+            GOTO_END_ERROR();
+         chunk_fill = 0;
+         stream_backend->set_out(stream,
+               chunk_buf + 8, (uint32_t)IDAT_CHUNK_SIZE);
+         continue;
+      }
+      if (err == TRANS_STREAM_ERROR_AGAIN)
+      {
+         /* Z_OK during Z_FINISH with avail_in=0 means deflate has
+          * more output to emit but our buffer ran out of space.
+          * Flush the full chunk and give it more room. */
+         if (!flush_idat_chunk(intf_s, chunk_buf, chunk_fill))
+            GOTO_END_ERROR();
+         chunk_fill = 0;
+         stream_backend->set_out(stream,
+               chunk_buf + 8, (uint32_t)IDAT_CHUNK_SIZE);
+         continue;
+      }
+      /* err == NONE: Z_STREAM_END.  Flush whatever's in the buffer
+       * and we're done.  flush_idat_chunk tolerates chunk_fill==0. */
+      if (!flush_idat_chunk(intf_s, chunk_buf, chunk_fill))
          GOTO_END_ERROR();
-
-      stream_backend->set_in(
-            stream,
-            encode_buf,
-            (unsigned)encode_buf_size);
-      stream_backend->set_out(
-            stream,
-            deflate_buf + 8,
-            (unsigned)deflate_bound);
+      break;
    }
 
-   if (!stream_backend->trans(stream, true, &total_in, &total_out, NULL))
-      GOTO_END_ERROR();
-
-   memcpy(deflate_buf + 4, "IDAT", 4);
-   dword_write_be(deflate_buf + 0,        ((uint32_t)total_out));
-   if (!png_write_idat_string(intf_s, deflate_buf, ((size_t)total_out + 8)))
-      GOTO_END_ERROR();
-
    if (!png_write_iend_string(intf_s))
       GOTO_END_ERROR();
+
 end:
-   free(encode_buf);
-   free(deflate_buf);
    free(rgba_line);
    free(prev_encoded);
    free(up_filtered);
    free(sub_filtered);
    free(avg_filtered);
    free(paeth_filtered);
+   free(filter_line);
+   free(chunk_buf);
 
    if (stream_backend)
    {