Drop <new> requirement for JitCall wrappers.

The JitCall wrapper generator used to emit plain `new (ret) (T) (call())`,
`new (ptr) T(args...)`, and `new (p) T[n]` for its placement-new forms,
binding against the standard placement operators declared in <new>. That
forced every user of JitCall to pull <new> into the interpreter's TU and
every ctor-using test to pass `-include new` on the command line.

Append `, __ci_newtag` to the two scalar placement-new sites -- the
return-value branch of make_narg_call_with_return and the scalar-ctor
branch of make_narg_ctor_with_return -- so they resolve against
clang-repl's preloaded tagged overload `operator new(size_t, void*,
__clang_Interpreter_NewTag)`. That overload is declared by the Runtimes
string introduced in llvm/llvm-project@1566f1ffc6b5 and first released
in LLVM 18. CppInterOp already registers `__ci_newtag` as an absolute
symbol in the JIT dylib, so no new runtime machinery is needed.

The default-ctor wrapper's `nary > 1 && is_arena` branch is rewritten as
a loop of scalar tagged placement news rather than `new (p) T[n]`. Two
alternatives were considered and rejected:

  (a) A tagged `operator new[](size_t, void*, Tag)` would give the
      minimum per-wrapper emission, but clang does not recognise a
      custom-signature array allocator as the standard placement form
      and inserts an array cookie for types with non-trivial destructors
      (Itanium C++ ABI §2.7), breaking the contract that
      `Cpp::Construct(scope, arena, n)` returns `arena`.

  (b) Forward-declaring the STANDARD-signature
      `operator new[](size_t, void*)` would satisfy clang's placement
      recognition, but the declaration is not portably replicable:
      libstdc++, libc++, and the MSVC STL decorate it with different
      noexcept macros, calling conventions, and [[nodiscard]]
      attributes. A user-supplied `#include <new>` after interpreter
      creation would clash with ours and crash the parse.

Cling is unaffected: it pre-includes <new> at IncrementalParser init, so
the empty tag on that backend is equivalent to plain scalar placement
new.

With the wrapper no longer needing <new>, the JitCall-adjacent tests
drop their `#include <new>` declares and `-include new` compiler flags.
Sites that were cargo pulling in <new> for unrelated reflection tests
(no JitCall use, no direct `<new>` dependency) drop the flag too --
where the test already pulls in `<memory>`, `<string>`, or `<vector>`,
the `<new>` declarations arrive transitively.

Two regression guards pin the contract:

  * FunctionReflection_JitCallNoNewHeader asserts the no-<new> premise
    via a non-noexcept redeclaration of `operator new(size_t, void*)`
    (per [new.delete.placement] the standard placement forms are
    noexcept, so the redeclaration parses cleanly only when <new> is
    not in scope) and then exercises a scalar-return JitCall.

  * FunctionReflection_ArrayConstructNoCookie constructs an array of a
    class with a user-provided destructor, asserts
    `Cpp::Construct(scope, arena, n) == arena`, and cross-checks each
    element's value at `arena + i*sizeof(T)`. A future revert to a
    tagged `operator new[]` or any non-standard-signature placement
    allocator fires this test immediately.

Author: vgvassilev

lib/CppInterOp/CppInterOp.cpp

@@ -2767,6 +2767,27 @@ void make_narg_call(const FunctionDecl* FD, const std::string& return_type,
   callbuf << ")";
 }
 
+// Tag appended inside `new (ptr<tag>) T(...)` when emitting a scalar
+// placement new in a JitCall wrapper.
+//
+// clang-repl's Runtimes string declares the scalar tagged overload
+// `operator new(size_t, void*, __clang_Interpreter_NewTag)` (introduced
+// in llvm/llvm-project@1566f1ffc6b5, LLVM 18), so the spelling
+// `new (p, __ci_newtag) T(...)` binds without the user's TU having
+// `#include <new>` in scope. Array placement in
+// `make_narg_ctor_with_return` is implemented as a loop of scalar tagged
+// placements for the same reason.
+//
+// Cling has no such tag; its runtime makes `<new>` available by default,
+// so the empty tag suffices there (plain scalar placement new).
+inline const char* PlacementTag() {
+#ifdef CPPINTEROP_USE_CLING
+  return "";
+#else
+  return ", __ci_newtag";
+#endif
+}
+
 void make_narg_ctor_with_return(const FunctionDecl* FD, const unsigned N,
                                 const std::string& class_name,
                                 std::ostringstream& buf, int indent_level) {
@@ -2790,41 +2811,59 @@ void make_narg_ctor_with_return(const FunctionDecl* FD, const unsigned N,
     indent(callbuf, indent_level);
     const auto* CD = dyn_cast<CXXConstructorDecl>(FD);
 
-    // Activate this block only if array new is possible
-    // if (nary) {
-    //    (*(ClassName**)ret) = (obj) ? new (*(ClassName**)ret) ClassName[nary]
-    //    : new ClassName[nary];
-    //   }
-    // else {
+    // Array branch. The is_arena side emits a loop of scalar placement
+    // calls rather than `new (p) T[n]`. Two alternatives were considered
+    // and rejected:
+    //
+    //   (a) Forward-declare a tagged
+    //       `operator new[](size_t, void*, __clang_Interpreter_NewTag)`
+    //       and emit `new (p, __ci_newtag) T[n]`. Cheapest in per-wrapper
+    //       emission, but clang does not recognise a custom-signature
+    //       array allocator as the standard placement form and inserts
+    //       an array cookie for types with non-trivial destructors
+    //       (Itanium C++ ABI §2.7), breaking the
+    //       `Cpp::Construct(scope, arena, n) == arena` contract.
+    //
+    //   (b) Forward-declare the STANDARD-signature
+    //       `operator new[](size_t, void*)`. Clang would signature-match
+    //       this as the placement form (no cookie), but the declaration
+    //       is not portably replicable: libstdc++, libc++, and the MSVC
+    //       STL decorate it with different noexcept macros, calling
+    //       conventions, and `[[nodiscard]]` attributes. A
+    //       user-supplied `#include <new>` after interpreter creation
+    //       would clash with our declaration and crash the parse.
+    //
+    // The loop binds against the already-declared scalar tagged
+    // placement operator (PlacementTag()), adds only O(6) lines per
+    // wrapper, and works on cling too (`<new>` is pre-included there,
+    // so the empty tag is equivalent to plain scalar placement new).
     if (CD->isDefaultConstructor()) {
       callbuf << "if (nary > 1) {\n";
       indent(callbuf, indent_level);
-      callbuf << "(*(" << class_name << "**)ret) = ";
-      callbuf << "(is_arena) ? new (*(" << class_name << "**)ret) ";
-      make_narg_ctor(FD, N, typedefbuf, callbuf, class_name, indent_level,
-                     true);
-
-      callbuf << ": new ";
-      //
-      //  Write the actual expression.
-      //
+      callbuf << "if (is_arena)\n";
+      indent(callbuf, indent_level + 1);
+      callbuf << "for (unsigned long __i = 0; __i < nary; ++__i)\n";
+      indent(callbuf, indent_level + 2);
+      callbuf << "new ((void*)(*(" << class_name << "**)ret + __i)"
+              << PlacementTag() << ") " << class_name << "();\n";
+      indent(callbuf, indent_level);
+      callbuf << "else (*(" << class_name << "**)ret) = new ";
       make_narg_ctor(FD, N, typedefbuf, callbuf, class_name, indent_level,
                      true);
-      //
-      //  End the new expression statement.
-      //
       callbuf << ";\n";
       indent(callbuf, indent_level);
       callbuf << "}\n";
       callbuf << "else {\n";
     }
 
-    // Standard branch:
-    // (*(ClassName**)ret) = (obj) ? new (*(ClassName**)ret) ClassName(args...)
-    // : new ClassName(args...);
+    // Standard (scalar) branch:
+    // (*(ClassName**)ret) = (is_arena)
+    //   ? new (*(ClassName**)ret[, __ci_newtag]) ClassName(args...)
+    //   : new ClassName(args...);
     indent(callbuf, indent_level);
     callbuf << "(*(" << class_name << "**)ret) = ";
-    callbuf << "(is_arena) ? new (*(" << class_name << "**)ret) ";
+    callbuf << "(is_arena) ? new (*(" << class_name << "**)ret"
+            << PlacementTag() << ") ";
     make_narg_ctor(FD, N, typedefbuf, callbuf, class_name, indent_level);
 
     callbuf << ": new ";
@@ -2908,7 +2947,8 @@ void make_narg_call_with_return(compat::Interpreter& I, const FunctionDecl* FD,
       //  Write the placement part of the placement new.
       //
       indent(callbuf, indent_level);
-      callbuf << "new (ret) ";
+      // See PlacementTag for the rationale of the tag.
+      callbuf << "new (ret" << PlacementTag() << ") ";
       //
       //  Write the type part of the placement new.
       //

unittests/CppInterOp/FunctionReflectionTest.cpp

@@ -705,8 +705,7 @@ TYPED_TEST(CPPINTEROP_TEST_MODE,
            FunctionReflection_InstantiateTemplateFunctionFromString) {
   if (llvm::sys::RunningOnValgrind())
     GTEST_SKIP() << "XFAIL due to Valgrind report";
-  std::vector<const char*> interpreter_args = { "-include", "new" };
-  TestFixture::CreateInterpreter(interpreter_args);
+  TestFixture::CreateInterpreter();
   std::string code = R"(#include <memory>)";
   Interp->process(code);
   const char* str = "std::make_unique<int,int>";
@@ -1560,7 +1559,7 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_GetFunctionAddress) {
 
   std::vector<Decl*> Decls;
   std::string code = "int f1(int i) { return i * i; }";
-  std::vector<const char*> interpreter_args = {"-include", "new", "-Xclang", "-iwithsysroot/include/compat"};
+  std::vector<const char*> interpreter_args = {"-Xclang", "-iwithsysroot/include/compat"};
 
   GetAllTopLevelDecls(code, Decls, /*filter_implicitGenerated=*/false,
                       interpreter_args);
@@ -1617,6 +1616,37 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_IsVirtualMethod) {
   EXPECT_FALSE(Cpp::IsVirtualMethod(Decls[0]));
 }
 
+// Regression guard: JitCall wrappers must not require the user's TU to
+// pull in <new>. This test adds an explicit premise check and one
+// scalar-return JitCall so a refactor that accidentally makes <new>
+// reachable cannot silently mask a future revert of the wrapper contract.
+TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_JitCallNoNewHeader) {
+#ifdef CPPINTEROP_USE_CLING
+  GTEST_SKIP() << "Cling pre-includes <new>.";
+#endif
+  if (TypeParam::isOutOfProcess)
+    GTEST_SKIP() << "Test fails for OOP JIT builds";
+
+  TestFixture::CreateInterpreter();
+
+  // Per [new.delete.placement] the standard placement overloads are
+  // declared `noexcept`; this non-noexcept redeclaration only parses
+  // if <new> is NOT in scope.
+  ASSERT_EQ(0,
+            Cpp::Declare("void* operator new(__SIZE_TYPE__, void*);"
+                         DFLT_FALSE));
+
+  // One JitCall is enough — if the wrapper regresses to plain placement
+  // new, MakeFunctionCallable fails to compile without <new>.
+  Cpp::Declare("int jc_sq(int x) { return x * x; }" DFLT_FALSE);
+  auto JC = Cpp::MakeFunctionCallable(Cpp::GetNamed("jc_sq" DFLT_NULLPTR));
+  ASSERT_TRUE(JC.isValid());
+  int arg = 5, ret = 0;
+  void* args[] = {&arg};
+  JC.Invoke(&ret, {args, 1});
+  EXPECT_EQ(ret, 25);
+}
+
 TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_JitCallAdvanced) {
 #if CLANG_VERSION_MAJOR == 20 && defined(CPPINTEROP_USE_CLING) && defined(_WIN32)
   GTEST_SKIP() << "Test fails with Cling on Windows";
@@ -1638,10 +1668,7 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_JitCallAdvanced) {
       } name;
     )";
 
-  std::vector<const char*> interpreter_args = {"-include", "new"};
-
-  GetAllTopLevelDecls(code, Decls, /*filter_implicitGenerated=*/false,
-                      interpreter_args);
+  GetAllTopLevelDecls(code, Decls, /*filter_implicitGenerated=*/false);
   auto *CtorD
     = (clang::CXXConstructorDecl*)Cpp::GetDefaultConstructor(Decls[0]);
   auto Ctor = Cpp::MakeFunctionCallable(CtorD);
@@ -1682,8 +1709,7 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_JitCallDebug) {
       }
     };)";
 
-  std::vector<const char*> interpreter_args = {"-include", "new",
-                                               "-debug-only=jitcall"};
+  std::vector<const char*> interpreter_args = {"-debug-only=jitcall"};
   GetAllTopLevelDecls(code, Decls, /*filter_implicitGenerated=*/false,
                       interpreter_args);
 
@@ -1785,8 +1811,7 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_GetFunctionCallWrapper) {
     int f1(int i) { return i * i; }
     )";
 
-  std::vector<const char*> interpreter_args = {"-include", "new"};
-
+  std::vector<const char*> interpreter_args;
   GetAllTopLevelDecls(code, Decls, /*filter_implicitGenerated=*/false,
                       interpreter_args);
 
@@ -2446,11 +2471,9 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_Construct) {
 #endif
   if (TypeParam::isOutOfProcess)
     GTEST_SKIP() << "Test fails for OOP JIT builds";
-  std::vector<const char*> interpreter_args = {"-include", "new"};
   std::vector<Decl*> Decls, SubDecls;
 
   std::string code = R"(
-    #include <new>
     extern "C" int printf(const char*,...);
     class C {
     public:
@@ -2463,7 +2486,7 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_Construct) {
     void construct() { return; }
     )";
 
-  GetAllTopLevelDecls(code, Decls, false, interpreter_args);
+  GetAllTopLevelDecls(code, Decls, false);
   GetAllSubDecls(Decls[1], SubDecls);
   testing::internal::CaptureStdout();
   Cpp::TCppScope_t scope = Cpp::GetNamed("C" DFLT_NULLPTR);
@@ -2534,8 +2557,7 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_ConstructPOD) {
 #endif
   if (TypeParam::isOutOfProcess)
     GTEST_SKIP() << "Test fails for OOP JIT builds";
-  std::vector<const char*> interpreter_args = {"-include", "new"};
-  TestFixture::CreateInterpreter(interpreter_args);
+  TestFixture::CreateInterpreter();
 
   Interp->declare(R"(
     namespace PODS {
@@ -2577,11 +2599,9 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_ConstructNested) {
   if (TypeParam::isOutOfProcess)
     GTEST_SKIP() << "Test fails for OOP JIT builds";
 
-  std::vector<const char*> interpreter_args = {"-include", "new"};
-  TestFixture::CreateInterpreter(interpreter_args);
+  TestFixture::CreateInterpreter();
 
   Interp->declare(R"(
-    #include <new>
     extern "C" int printf(const char*,...);
     class A {
     public:
@@ -2643,7 +2663,6 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_ConstructArray) {
   TestFixture::CreateInterpreter();
 
   Interp->declare(R"(
-      #include <new>
       extern "C" int printf(const char*,...);
       class C {
         int x;
@@ -2695,11 +2714,9 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_Destruct) {
   if (TypeParam::isOutOfProcess)
     GTEST_SKIP() << "Test fails for OOP JIT builds";
 
-  std::vector<const char*> interpreter_args = {"-include", "new"};
-  TestFixture::CreateInterpreter(interpreter_args);
+  TestFixture::CreateInterpreter();
 
   Interp->declare(R"(
-    #include <new>
     extern "C" int printf(const char*,...);
     class C {
       C() {}
@@ -2768,11 +2785,9 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_DestructArray) {
   if (TypeParam::isOutOfProcess)
     GTEST_SKIP() << "Test fails for OOP JIT builds";
 
-  std::vector<const char*> interpreter_args = {"-include", "new"};
-  TestFixture::CreateInterpreter(interpreter_args);
+  TestFixture::CreateInterpreter();
 
   Interp->declare(R"(
-      #include <new>
       extern "C" int printf(const char*,...);
       class C {
         int x;
@@ -2834,6 +2849,69 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_DestructArray) {
   output.clear();
 }
 
+// Regression guard for Itanium C++ ABI §2.7 array cookies. Any
+// placement `operator new[]` that clang does not recognise as the
+// standard placement form (standard signature, declared in <new>)
+// causes clang to insert an array cookie before the storage when the
+// element type has a non-trivial destructor. That would shift the
+// pointer returned by `Cpp::Construct(scope, arena, n)` past the
+// cookie, breaking the documented contract that the return equals
+// `arena`. The JitCall ctor wrapper works around this by emitting a
+// loop of scalar placement news rather than a single
+// `new (p) T[n]` with a custom-signature allocator — see the comment
+// on `make_narg_ctor_with_return` in lib/CppInterOp/CppInterOp.cpp.
+// This test fails immediately if that loop is ever replaced with a
+// tagged `operator new[]` or a non-standard forward declaration.
+TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_ArrayConstructNoCookie) {
+#ifdef EMSCRIPTEN
+  GTEST_SKIP() << "Test fails for Emscripten builds";
+#endif
+  if (llvm::sys::RunningOnValgrind())
+    GTEST_SKIP() << "XFAIL due to Valgrind report";
+#ifdef _WIN32
+  GTEST_SKIP() << "Disabled on Windows. Needs fixing.";
+#endif
+  if (TypeParam::isOutOfProcess)
+    GTEST_SKIP() << "Test fails for OOP JIT builds";
+
+  TestFixture::CreateInterpreter();
+
+  // CookieProbe has a user-provided destructor, which makes it
+  // non-trivially destructible. Any non-placement-form `operator new[]`
+  // must reserve space for an array cookie in front of the storage.
+  Interp->declare(R"(
+    struct CookieProbe {
+      int v;
+      CookieProbe() : v(0xC0DE) {}
+      ~CookieProbe() {}
+    };
+  )");
+
+  auto* scope = Cpp::GetNamed("CookieProbe" DFLT_NULLPTR);
+  ASSERT_NE(scope, nullptr);
+
+  constexpr size_t kN = 4;
+  void* arena = Cpp::Allocate(scope, kN);
+  ASSERT_NE(arena, nullptr);
+
+  // Placement array construction must return the arena as-is.
+  EXPECT_EQ(Cpp::Construct(scope, arena, kN), arena)
+      << "Construct returned a pointer offset from arena; an array "
+         "cookie has been inserted. Placement-new signature regressed.";
+
+  // Cross-check: the i-th element's field lives at offset i*sizeof(T)
+  // from the arena base (not past a cookie header).
+  const size_t T = Cpp::SizeOf(scope);
+  for (size_t i = 0; i < kN; ++i) {
+    int* slot = reinterpret_cast<int*>(
+        reinterpret_cast<char*>(arena) + i * T);
+    EXPECT_EQ(*slot, 0xC0DE) << "element " << i << " not at expected offset";
+  }
+
+  Cpp::Destruct(arena, scope, /*withFree=*/false, kN);
+  Cpp::Deallocate(scope, arena, kN);
+}
+
 TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_UndoTest) {
 #ifdef _WIN32
   GTEST_SKIP() << "Disabled on Windows. Needs fixing.";
@@ -3003,7 +3081,7 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_IsExplicitTemplated) {
 
 TYPED_TEST(CPPINTEROP_TEST_MODE, FunctionReflection_IsExplicitDeductionGuide) {
   // Deduction guides are a C++17 feature
-  std::vector<const char*> interpreter_args = {"-include", "new", "-std=c++17"};
+  std::vector<const char*> interpreter_args = {"-std=c++17"};
   Cpp::CreateInterpreter(interpreter_args, {});
 
   Interp->declare(R"(

unittests/CppInterOp/InterpreterTest.cpp

@@ -158,7 +158,7 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, Interpreter_Process) {
     GTEST_SKIP() << "Test fails for OOP JIT builds";
   if (llvm::sys::RunningOnValgrind())
     GTEST_SKIP() << "XFAIL due to Valgrind report";
-  std::vector<const char*> interpreter_args = { "-include", "new", "-Xclang", "-iwithsysroot/include/compat" };
+  std::vector<const char*> interpreter_args = { "-Xclang", "-iwithsysroot/include/compat" };
   auto* I = TestFixture::CreateInterpreter(interpreter_args);
   EXPECT_TRUE(Cpp::Process("") == 0);
   EXPECT_TRUE(Cpp::Process("int a = 12;") == 0);
@@ -538,7 +538,6 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, Interpreter_WrapperCacheIsPerInterpreter) {
   auto* AddDecl1 = Cpp::GetNamed("add" DFLT_NULLPTR);
   ASSERT_NE(AddDecl1, nullptr);
 
-  Cpp::Declare("#include <new>" DFLT_FALSE); // Needed by JitCall
   auto JC1 = Cpp::MakeFunctionCallable(AddDecl1);
   ASSERT_TRUE(JC1.isValid());
 
@@ -555,7 +554,6 @@ TYPED_TEST(CPPINTEROP_TEST_MODE, Interpreter_WrapperCacheIsPerInterpreter) {
   auto* AddDecl2 = Cpp::GetNamed("add" DFLT_NULLPTR);
   ASSERT_NE(AddDecl2, nullptr);
 
-  Cpp::Declare("#include <new>" DFLT_FALSE); // Needed by JitCall
   auto JC2 = Cpp::MakeFunctionCallable(AddDecl2);
   ASSERT_TRUE(JC2.isValid());