Skip to content

OTLP/GRPC support#1

Merged
lmangani merged 6 commits into
mainfrom
grpc
Jun 11, 2026
Merged

OTLP/GRPC support#1
lmangani merged 6 commits into
mainfrom
grpc

Conversation

@lmangani

@lmangani lmangani commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Implements OTLP/GRPC ingestion endpoint

Pros: full OTLP ingestion compatibility
Cons: long build times due to grpc libs

lmangani and others added 6 commits June 11, 2026 01:22
Implements the standard OpenTelemetry collector services (TraceService,
LogsService, MetricsService) in-process via gRPC:

- vendored opentelemetry-proto v1.3.2 definitions (proto/, Apache-2.0),
  compiled at build time with protoc + grpc_cpp_plugin
- requests convert through protobuf's canonical OTLP/JSON mapping and flow
  through the exact same native ingestion path as the HTTP API, so the two
  transports cannot diverge
- bearer-token auth and x-rawduck-{traces,logs,metrics}-table routing via
  gRPC metadata; OTLP partialSuccess with signal-specific rejected counts
- dependencies via vcpkg (grpc, protobuf; platform '!emscripten & !mingw');
  CMake auto-detects and degrades gracefully - builds without gRPC compile
  the same functions, which raise NotImplemented with guidance
- existing code paths untouched; wasm builds unaffected
- verified with grpcurl: export success, auth rejection, custom-table
  routing, flattened columns queryable; lifecycle sqllogictest gated behind
  RAWDUCK_GRPC_TESTS

Co-Authored-By: Claude Fable 5 <[email protected]>
The Catalog::GetEntry<T> template references T::Name (static constexpr
const char *), which our translation units materialized as strong
definitions colliding with DuckDB core's own under the Linux CI toolchain
(GCC 14, multiple definition of TableCatalogEntry::Name /
AggregateFunctionCatalogEntry::Name at libduckdb.so link time). Use the
non-template GetEntry(EntryLookupInfo) API + Cast<>() instead. Note:
EntryLookupInfo holds the entry name by reference, so it must outlive the
lookup.

Co-Authored-By: Claude Fable 5 <[email protected]>
The gRPC dependency pulls host-tool builds (protoc, grpc_cpp_plugin) under
the default host triplets, which build the entire grpc/protobuf/abseil
stack twice (debug + release). Overlay the default linux/osx triplets with
VCPKG_BUILD_TYPE=release so host builds are release-only, roughly halving
cold vcpkg time and cache size. Target triplets were already *-release;
subsequent runs restore from the binary cache either way.

Co-Authored-By: Claude Fable 5 <[email protected]>
gRPC/abseil require C++17 and CMake propagates that requirement through
the static extension library into DuckDB's own tools (plan_serializer),
while core stayed C++11. Mixed standards make 'static constexpr' members
duplicate definitions under GNU ld (BufferedFileWriter::DEFAULT_OPEN_FLAGS
et al). Setting CMAKE_CXX_STANDARD 17 in extension_config.cmake applies to
every target in the build (it is included in root scope before
add_subdirectory), making constexpr statics implicitly inline everywhere.
Verified: clean full local rebuild + 404 test assertions.

Co-Authored-By: Claude Fable 5 <[email protected]>
Newer DuckLake releases restrict ALTER type changes to their widening
promotion set (INTEGER -> BIGINT etc.); BIGINT -> DOUBLE is rejected with
'only widening type promotions are allowed', so asserting it made the test
depend on which ducklake version INSTALL resolves. Exercise INTEGER ->
BIGINT widening instead, which is supported across versions.

Co-Authored-By: Claude Fable 5 <[email protected]>
Mirrors the automatic wasm exclusion as an explicit user choice: the
OTLP/gRPC server is skipped at build time while OTLP/HTTP stays fully
functional, and raw_serve_grpc explains why it is unavailable.

Co-Authored-By: Claude Fable 5 <[email protected]>
@lmangani lmangani merged commit a6ea830 into main Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant