libretro-common/formats/json/rjson: integer overflow hardening + test

Audit of rjson.c (1569 lines; the JSON parser + writer used by
configs, manifests, RetroAchievements, cloud sync, etc.).  Six
integer-overflow/underflow hardenings in the parser and writer,
plus a regression test.  None are remotely exploitable in the
typical usage pattern -- they require inputs or outputs
approaching or exceeding 2 GiB -- but each is a real correctness
bug that can bite on 32-bit targets or on pathological
locally-generated JSON.

rjson: cap _rjson_grow_string doubling
  The growing string buffer doubled its capacity unconditionally:

    size_t new_string_cap = json->string_cap * 2;

  Once string_cap exceeds SIZE_MAX / 2, the doubling wraps to 0 or
  a tiny value.  realloc() with that argument is implementation-
  defined: glibc returns NULL (benign, caller errors out); some
  other libcs return a valid 0-byte pointer.  The following
  pushchar then writes at string[0], which is past the newly
  shrunk allocation.  Heap overflow.

  Reachable on 32-bit with a ~2 GiB JSON string token (and
  correspondingly much harder on 64-bit).  Fix: cap
  new_string_cap at _RJSON_MAX_SIZE (256 MiB) and fail cleanly
  when the growth would exceed it.

rjson: clamp rjson_open_user io_block_size
  The public entry point accepted any int as the input block size
  and allocated sizeof(rjson_t) - sizeof(input_buf) + io_block_size.
  A small io_block_size (including 0 or negative) underallocated
  input_buf to fewer bytes than the fixed struct expected, and
  the first read in _rjson_io_input ran off the end.

  The two internal callers (rjson_open_stream, rjson_open_rfile)
  already bound io_block_size to [1024, 4096], so this isn't
  reachable via them.  It is reachable from any consumer that
  calls rjson_open_user directly with attacker-influenced sizing,
  so clamp it here for defense-in-depth: io_block_size floored at
  16, ceilinged at _RJSON_MAX_SIZE.

rjsonwriter: size_t arithmetic in _rjsonwriter_memory_io
  new_cap = writer->buf_num + (is_append ? len : 0) + 512;

  All operands are signed int.  For a writer generating >2 GiB of
  output (local database dumps, long JSON-encoded state blobs)
  the sum overflows INT_MAX (UB).  Post-overflow the comparison
  "new_cap > buf_cap" misbehaves, the realloc is skipped, and the
  subsequent memcpy in the same function writes past the existing
  allocation.

  Fix: redo the math in size_t, overflow-check against
  _RJSON_MAX_SIZE, set error_text and return 0 on failure.  The
  realloc call then receives a sane positive int.

rjsonwriter: size_t arithmetic in rjsonwriter_raw
  Same class of bug:

    if (writer->buf_num + len > writer->buf_cap)
        rjsonwriter_flush(writer);

  Signed-int sum overflows for large writes and the flush is
  skipped, letting the downstream memcpy blow past the buffer.
  Also: nothing guarded against a negative len.  A caller passing
  len < 0 pre-patch advanced buf_num by a negative amount via a
  later "buf_num += len" and the next write corrupted the ring.

  Fix: reject negative len at the top, redo the bounds comparison
  in size_t.

rjsonwriter: size_t arithmetic in rjsonwriter_rawf
  Same class.  newcap = buf_num + need + 1 could overflow for a
  single oversized formatted value.  Same fix pattern.

Regression test: libretro-common/samples/formats/json/rjson_test.c
  Six subtests, three of which are true discriminators:

    1. parser happy path (nested object + array)
    2. long-string parse (forces _rjson_grow_string growth past
       the inline buffer) -- smoke for patch #1
    3. malformed inputs: 12 variants (truncated objects, dangling
       surrogates, bad numbers, ...) must NOT crash.  Smoke for
       general robustness.
    4. writer happy path
    5. rjsonwriter_raw with len = -99 -- DIRECT DISCRIMINATOR for
       the negative-len fix.  Pre-patch under ASan this fires:

          AddressSanitizer: negative-size-param: (size=-99)
            in memcpy at rjson.c:1392 (rjsonwriter_raw)

       Post-patch the call is a no-op and the output is exactly
       the expected 6-byte "AAABBB".
    6. rjson_open_user with tiny io_block_size (-10..8) -- DIRECT
       DISCRIMINATOR for the floor fix.  Pre-patch
       io_block_size=0 allocated zero bytes for input_buf and the
       first read was OOB (UB at C level, crashes on some libcs).
       Post-patch all values floor to 16 and the open+read+free
       cycle is clean.

  Built and ran under -Wall -pedantic -std=gnu99:
    All 6 subtests pass on patched code.
    ASan -O0 (patched): clean, exit 0.
    ASan -O0 (unpatched, for the negative-len subtest):
      "negative-size-param: (size=-99)" in memcpy -> immediate abort.

CI: libretro-common samples workflow updated.  rjson_test added
to RUN_TARGETS.  Full local dry-run under the GHA shell contract:
    Built: 15  Ran: 15  Failed: 0

VC6 compatibility: rjson.c is compiled on all platforms.  The
fix uses only size_t / ssize_t / int and the existing stdint
shim; <limits.h> (C89) is now explicitly included for INT_MAX.
_RJSON_MAX_SIZE is a size_t-cast literal, valid in C89.

Author: LibretroAdmin

.github/workflows/Linux-libretro-common-samples.yml

@@ -55,6 +55,7 @@ jobs:
             vfs_read_overflow_test
             cdrom_cuesheet_overflow_test
             http_parse_test
+            rjson_test
           )
 
           # Per-binary run command (overrides ./<binary> if present).

libretro-common/formats/json/rjson.c

@@ -25,8 +25,19 @@
 #include <stdio.h>  /* snprintf, vsnprintf */
 #include <stdarg.h> /* va_list */
 #include <string.h> /* memcpy */
-#include <stdint.h> /* int64_t */
+#include <stdint.h> /* int64_t, SIZE_MAX */
 #include <stdlib.h> /* malloc, realloc, atof, atoi */
+#include <limits.h> /* INT_MAX */
+
+/* Ceiling for the parser's growing string buffer and the writer's
+ * in-memory buffer.  Reached in practice only by pathologically large
+ * (and almost certainly malformed) JSON inputs on a 32-bit system,
+ * but having an explicit cap avoids undefined behaviour in the
+ * signed-int arithmetic of the writer and the size_t doubling in the
+ * parser wrapping past SIZE_MAX / 2.  Callers with legitimate giant
+ * payloads should not be using these APIs in the first place --
+ * stream in chunks. */
+#define _RJSON_MAX_SIZE ((size_t)256 * 1024 * 1024)
 
 #include <formats/rjson.h>
 #include <compat/posix_string.h>
@@ -139,7 +150,23 @@ static bool _rjson_io_input(rjson_t *json)
 static bool _rjson_grow_string(rjson_t *json)
 {
    char *string;
-   size_t new_string_cap = json->string_cap * 2;
+   size_t new_string_cap;
+   /* Bound the doubling so string_cap * 2 can never wrap past
+    * SIZE_MAX.  Pre-patch a pathological input on 32-bit (string_cap
+    * nearing 2 GiB) doubled to 0 and realloc() returned either NULL
+    * (benign) or a 0-byte pointer (heap overflow on next pushchar).
+    * Post-patch we fail cleanly before the arithmetic wrap. */
+   if (json->string_cap > _RJSON_MAX_SIZE / 2)
+   {
+      if (json->string_cap >= _RJSON_MAX_SIZE)
+      {
+         _rjson_error(json, "string token too large");
+         return false;
+      }
+      new_string_cap = _RJSON_MAX_SIZE;
+   }
+   else
+      new_string_cap = json->string_cap * 2;
    if (json->string != json->inline_string)
       string             = (char*)realloc(json->string, new_string_cap);
    else if ((string      = (char*)malloc(new_string_cap)) != NULL)
@@ -915,7 +942,16 @@ void _rjson_setup(rjson_t *json, rjson_io_t io, void *user_data, int input_len)
 
 rjson_t *rjson_open_user(rjson_io_t io, void *user_data, int io_block_size)
 {
-   rjson_t* json = (rjson_t*)malloc(
+   rjson_t* json;
+   /* Clamp io_block_size against negative / tiny / oversized values.
+    * The two internal callers (rjson_open_stream / rjson_open_rfile)
+    * already bound this, but this function is public and can be
+    * reached directly. */
+   if (io_block_size < 16)
+      io_block_size = 16;
+   else if ((size_t)io_block_size > _RJSON_MAX_SIZE)
+      io_block_size = (int)_RJSON_MAX_SIZE;
+   json = (rjson_t*)malloc(
          sizeof(rjson_t) - sizeof(((rjson_t*)0)->input_buf) + io_block_size);
    if (json) _rjson_setup(json, io, user_data, io_block_size);
    return json;
@@ -1290,7 +1326,24 @@ static int _rjsonwriter_memory_io(const void* buf, int len, void *user)
 {
    rjsonwriter_t *writer = (rjsonwriter_t *)user;
    bool is_append        = (buf != writer->buf);
-   int new_cap           = writer->buf_num + (is_append ? len : 0) + 512;
+   size_t append_len     = (is_append ? (size_t)len : 0);
+   size_t target;
+   int    new_cap;
+   /* Detect the int-overflow pre-patch, where buf_num + len + 512
+    * wrapped past INT_MAX and new_cap went negative.  The subsequent
+    * "new_cap > buf_cap" comparison then misbehaved and memcpy wrote
+    * past the existing allocation on a post-overflow buffer that
+    * hadn't been grown.  Do the arithmetic in size_t and cap at
+    * _RJSON_MAX_SIZE (which also fits comfortably in int). */
+   if (len < 0 || (size_t)writer->buf_num > _RJSON_MAX_SIZE - append_len
+                                           - 512)
+   {
+      if (!writer->error_text)
+         writer->error_text = "output buffer too large";
+      return 0;
+   }
+   target = (size_t)writer->buf_num + append_len + 512;
+   new_cap = (int)target;
    if (!writer->final_flush && (is_append || new_cap > writer->buf_cap))
    {
       bool can_realloc   = (writer->buf != writer->inline_buf);
@@ -1376,7 +1429,13 @@ const char *rjsonwriter_get_error(rjsonwriter_t *writer)
 
 void rjsonwriter_raw(rjsonwriter_t *writer, const char *buf, int len)
 {
-   if (writer->buf_num + len > writer->buf_cap)
+   /* Guard against negative len and against buf_num+len signed
+    * overflow.  A caller that somehow passes a huge len would
+    * pre-patch compute a negative sum, skip the flush, then memcpy
+    * past the existing buffer below. */
+   if (len < 0)
+      return;
+   if ((size_t)writer->buf_num + (size_t)len > (size_t)writer->buf_cap)
       rjsonwriter_flush(writer);
    if (len == 1)
    {
@@ -1424,6 +1483,14 @@ void rjsonwriter_rawf(rjsonwriter_t *writer, const char *fmt, ...)
       return;
    }
    rjsonwriter_flush(writer);
+   /* Guard signed-int overflow on newcap: buf_num + need + 1 could
+    * wrap past INT_MAX for a single very large formatted value. */
+   if ((size_t)writer->buf_num + (size_t)need + 1 > _RJSON_MAX_SIZE)
+   {
+      if (!writer->error_text)
+         writer->error_text = "output buffer too large";
+      return;
+   }
    if (writer->buf_num + need >= writer->buf_cap)
    {
       int newcap   = writer->buf_num + need + 1;

libretro-common/samples/formats/json/Makefile

@@ -0,0 +1,40 @@
+TARGET := rjson_test
+
+LIBRETRO_COMM_DIR := ../../..
+
+SOURCES := \
+	rjson_test.c \
+	$(LIBRETRO_COMM_DIR)/formats/json/rjson.c \
+	$(LIBRETRO_COMM_DIR)/compat/fopen_utf8.c \
+	$(LIBRETRO_COMM_DIR)/compat/compat_strl.c \
+	$(LIBRETRO_COMM_DIR)/compat/compat_posix_string.c \
+	$(LIBRETRO_COMM_DIR)/encodings/encoding_utf.c \
+	$(LIBRETRO_COMM_DIR)/encodings/encoding_crc32.c \
+	$(LIBRETRO_COMM_DIR)/file/file_path.c \
+	$(LIBRETRO_COMM_DIR)/file/file_path_io.c \
+	$(LIBRETRO_COMM_DIR)/streams/file_stream.c \
+	$(LIBRETRO_COMM_DIR)/streams/interface_stream.c \
+	$(LIBRETRO_COMM_DIR)/streams/memory_stream.c \
+	$(LIBRETRO_COMM_DIR)/streams/trans_stream.c \
+	$(LIBRETRO_COMM_DIR)/streams/trans_stream_pipe.c \
+	$(LIBRETRO_COMM_DIR)/streams/trans_stream_zlib.c \
+	$(LIBRETRO_COMM_DIR)/time/rtime.c \
+	$(LIBRETRO_COMM_DIR)/vfs/vfs_implementation.c
+
+OBJS := $(SOURCES:.c=.o)
+
+CFLAGS  += -Wall -pedantic -std=gnu99 -g -I$(LIBRETRO_COMM_DIR)/include
+LDLIBS  += -lz
+
+all: $(TARGET)
+
+%.o: %.c
+	$(CC) -c -o $@ $< $(CFLAGS)
+
+$(TARGET): $(OBJS)
+	$(CC) -o $@ $^ $(LDFLAGS) $(LDLIBS)
+
+clean:
+	rm -f $(TARGET) $(OBJS)
+
+.PHONY: clean