feat(logging): add telemetry registry and context examples

* feat(prometheus): expose scrape counters and collect duration

Add four built-in metrics to PrometheusHttpServerLogger so operators
can observe scrape health and collection latency:

- logit_prometheus_scrapes_total           (counter)
- logit_prometheus_scrape_errors_total     (counter)
- logit_prometheus_last_scrape_timestamp_ms (gauge)
- logit_prometheus_collect_duration_seconds (gauge)

Scrapes are counted in the HTTP handler before collect_payload().
Collection duration is measured inside collect_payload() using
steady_clock. Failed scrapes (exceptions from collect_payload())
increment scrape_errors_total.

All scrape metrics carry the logger="prometheus_http_server" label
for consistency with existing built-in families.

Not-tested: scrape_errors_total > 0 (requires an exception during
collect_payload, which is hard to trigger deterministically in the
current test setup).

Co-Authored-By: Claude Opus 4.7 <[email protected]>

* feat(logging): add telemetry registry and context examples

Add PrometheusRegistry for declarative custom application metrics and wire it into Prometheus examples and docs.

Add MDC/NDC context support based on the existing local LogContext design, including formatter tokens and coverage.

Fix OTLP graceful-shutdown worker wakeups and add zstd compression integration coverage.

* fix(context): make MDC and NDC opt-in

Add LOGIT_WITH_CONTEXT so diagnostic context support does not affect the default logging hot path.

Store context as an optional shared snapshot only when the thread context is non-empty, and keep ASan builds away from intentionally leaked thread-local storage.

* fix(prometheus): honor scrape metric label config

Apply PrometheusTextFormatConfig logger and instance label settings to PrometheusHttpServerLogger scrape diagnostics.

Add coverage for custom logger label names, instance labels, and disabled logger labels on scrape metrics.

* fix(prometheus): keep collecting healthy registry metrics

Continue collecting PrometheusRegistry entries after a value callback throws, append successfully collected samples, and then rethrow the first exception so Prometheus loggers still count the collection failure.

Update registry coverage and docs for the partial-success behavior.

---------

Co-authored-by: Claude Opus 4.7 <[email protected]>

Author: LimiNode

CMakeLists.txt

@@ -8,6 +8,7 @@ option(LOGIT_BENCH_WITH_SPDLOG "Enable spdlog comparison benchmarks" OFF)
 option(LOGIT_WITH_GZIP "Enable gzip via zlib" OFF)
 option(LOGIT_WITH_ZSTD "Enable zstd" OFF)
 option(LOGIT_WITH_FMT "Enable fmt support" OFF)
+option(LOGIT_WITH_CONTEXT "Enable MDC/NDC diagnostic context support" OFF)
 option(LOGIT_WITH_OTLP "Enable OTLP/HTTP log export via optional kurlyk dependency" OFF)
 option(LOGIT_WITH_PROMETHEUS "Enable Prometheus text payload support" OFF)
 option(LOGIT_WITH_PROMETHEUS_SERVER "Enable Prometheus HTTP server backend" OFF)
@@ -73,6 +74,10 @@ if(LOGIT_FORCE_ASYNC_OFF)
     target_compile_definitions(log-it-cpp INTERFACE LOGIT_DEFAULT_ASYNC_OFF=1)
 endif()
 
+if(LOGIT_WITH_CONTEXT)
+    target_compile_definitions(log-it-cpp INTERFACE LOGIT_WITH_CONTEXT=1)
+endif()
+
 if(LOGIT_WITH_SYSLOG AND (UNIX OR APPLE) AND NOT EMSCRIPTEN)
     target_compile_definitions(log-it-cpp INTERFACE LOGIT_HAS_SYSLOG=1)
 endif()
@@ -298,4 +303,4 @@ write_basic_package_version_file(
   install(FILES
       "${CMAKE_CURRENT_BINARY_DIR}/log-it-cpp.pc"
       DESTINATION share/pkgconfig
-  )
+  )

README.md

@@ -46,6 +46,13 @@ Internal headers under `logit/detail/` are private implementation details and sh
 
 See the macro examples below or browse the `examples/` folder for focused demonstrations, including queue tuning and crash handling.
 
+Recent focused examples include:
+
+- `examples/example_logit_otlp_http.cpp` - OTLP/HTTP export with batching, retries, optional compression, and contextual trace/span fields.
+- `examples/example_logit_prometheus_payload.cpp` - callback-based Prometheus payload emission with custom registry metrics.
+- `examples/example_logit_prometheus_server.cpp` - embedded `/metrics` endpoint with built-in and application metrics.
+- `examples/example_logit_mdc_ndc.cpp` - mapped and nested diagnostic context across scopes and threads.
+
 ## Macro Examples
 
 ### Long-form macros
@@ -100,6 +107,41 @@ void short_names_demo() {
 
 For a standalone program that brings everything together and intentionally aborts after logging a fatal message, check `examples/example_logit_minimal_crash.cpp`.
 
+### Diagnostic context
+
+Mapped diagnostic context (MDC) stores thread-local key-value pairs, while nested
+diagnostic context (NDC) stores a thread-local stack of scope names. The context
+is available when the library is configured with `-DLOGIT_WITH_CONTEXT=ON`.
+When this option is off, `LogRecord` keeps the original hot-path shape and the
+context macros become no-ops.
+
+With context enabled, a `LogRecord` captures a shared snapshot only when the
+current thread has non-empty MDC or NDC values.
+
+```cpp
+#include <logit.hpp>
+
+int main() {
+    LOGIT_ADD_LOGGER(
+        logit::ConsoleLogger, (),
+        logit::SimpleLogFormatter,
+        ("[%T] request=%K{request_id} ndc=[%J] %v")
+    );
+
+    LOGIT_MDC_PUT("request_id", "req-42");
+    LOGIT_NDC_PUSH("checkout");
+
+    {
+        LOGIT_NDC_GUARD("payment");
+        LOGIT_INFO("charge started");
+    }
+
+    LOGIT_MDC_CLEAR();
+    LOGIT_NDC_CLEAR();
+    LOGIT_WAIT();
+}
+```
+
 ### System error helpers
 
 `LOGIT_SYSERR_<LEVEL>` captures the current `errno` (or `GetLastError()` on Windows) and appends the decoded information to the message, so failure details stay attached to the original context. The lower-level `LOGIT_PERROR_<LEVEL>` and `LOGIT_WINERR_<LEVEL>` families are also available if you want to explicitly choose the platform macro.
@@ -521,6 +563,12 @@ Below is a list of supported formatting flags:
 - *Thread Flags*:
 
     - `%t`: Thread identifier
+
+- *Diagnostic Context Flags*:
+
+    - `%K`: All mapped diagnostic context values as `key=value` pairs
+    - `%K{key}`: One mapped diagnostic context value by key
+    - `%J`: Nested diagnostic context stack
 
 - *Color Flags*:
 
@@ -775,6 +823,8 @@ public:
 | `LOGIT_<LEVEL>_EVERY_N(n, ...)` | Log on every `n`th invocation. |
 | `LOGIT_<LEVEL>_THROTTLE(period_ms, ...)` | Log at most once per `period_ms` milliseconds. |
 | `LOGIT_<LEVEL>_TAG(({{"k", "v"}}), msg)` | Attach key-value tags to a message. |
+| `LOGIT_MDC_PUT(key, value)`, `LOGIT_MDC_REMOVE(key)`, `LOGIT_MDC_CLEAR()` | Manage thread-local mapped diagnostic context when `LOGIT_WITH_CONTEXT` is enabled. |
+| `LOGIT_NDC_PUSH(value)`, `LOGIT_NDC_POP()`, `LOGIT_NDC_CLEAR()`, `LOGIT_NDC_GUARD(value)` | Manage thread-local nested diagnostic context when `LOGIT_WITH_CONTEXT` is enabled. |
 | `LOGIT_RAW(msg)`, `LOGIT_RAW_TO(index, msg)`, `LOGIT_RAW_IF(condition, msg)` | Write already formatted text without applying level filters or formatter patterns. |
 | `LOGIT_SECTION(name)`, `LOGIT_SECTION_TO(index, name)`, `LOGIT_SECTION_IF(condition, name)` | Write raw section headers such as `[Proxy]`. |
 | `LOGIT_<LEVEL>_TO(index, ...)` | Target a specific logger index, including single-mode backends. |
@@ -879,7 +929,9 @@ The following toggles cover all build-time features:
 - `LOGIT_CPP_BUILD_EXAMPLES` (default: OFF) — build the example programs.
 - `LOGIT_BENCH_ENABLE` (default: OFF) — build benchmarks; `LOGIT_BENCH_WITH_SPDLOG` (default: OFF) also builds the spdlog comparisons.
 - `LOGIT_WITH_GZIP` / `LOGIT_WITH_ZSTD` (defaults: OFF) — enable gzip or zstd support for rotated files.
-- `LOGIT_WITH_FMT` (default: OFF) — include the `{}`-style formatting macros; `LOGIT_USE_SUBMODULES` (default: OFF) allows bundled optional dependency fallbacks such as fmt, zlib, and zstd when system packages are missing.
+- `LOGIT_WITH_FMT` (default: OFF) — include the `{}`-style formatting macros.
+- `LOGIT_WITH_CONTEXT` (default: OFF) — enable MDC/NDC helpers and `%K`, `%K{key}`, `%J` formatter tokens.
+- `LOGIT_USE_SUBMODULES` (default: OFF) allows bundled optional dependency fallbacks such as fmt, zlib, and zstd when system packages are missing.
 - `LOGIT_WITH_SYSLOG` (default: ON on Unix-like targets) — build the syslog backend.
 - `LOGIT_WITH_WIN_EVENT_LOG` (default: ON on Windows) — build the Windows Event Log backend.
 - `LOGIT_FORCE_ASYNC_OFF` (default: OFF) — force synchronous logging even in multi-threaded builds.

docs/OtlpHttpLogger.md

@@ -18,6 +18,11 @@ For Windows MinGW builds, the CMake integration enables kurlyk fallback options
 
 ## Usage
 
+For a runnable version with environment overrides, graceful shutdown, optional
+compression, and optional MDC trace/span fields, see
+`examples/example_logit_otlp_http.cpp`. Build with `-DLOGIT_WITH_CONTEXT=ON`
+if you want the MDC trace/span fields to be populated.
+
 ```cpp
 #include <logit.hpp>
 

docs/PrometheusLogger.md

@@ -24,7 +24,16 @@ LogIt++ provides two Prometheus backends for exposing internal log metrics in th
 | `logit_time_since_last_log_ms` | gauge | Time since last log (ms) |
 | `logit_build_info` | gauge | Build info (value=1, labels: version, compiler) |
 
-The `metric_prefix` config option (default: `logit_`) is applied to all metric names.
+The `metric_prefix` config option (default: `logit_`) is applied to built-in logger metric names.
+
+`PrometheusHttpServerLogger` also exposes scrape diagnostics:
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `logit_prometheus_scrapes_total` | counter | Total `/metrics` scrape requests |
+| `logit_prometheus_scrape_errors_total` | counter | Failed scrape requests |
+| `logit_prometheus_last_scrape_timestamp_ms` | gauge | Timestamp of the last scrape request |
+| `logit_prometheus_collect_duration_seconds` | gauge | Duration of the last metrics collection |
 
 ## CMake Options
 
@@ -38,6 +47,9 @@ option(LOGIT_WITH_PROMETHEUS_SERVER "Enable Prometheus HTTP server backend" OFF)
 
 ## Usage: PrometheusPayloadLogger
 
+For a runnable callback example with custom application metrics, see
+`examples/example_logit_prometheus_payload.cpp`.
+
 ```cpp
 #include <logit.hpp>
 
@@ -61,6 +73,9 @@ LOGIT_WAIT(); // triggers on_payload with current metrics
 
 ## Usage: PrometheusHttpServerLogger
 
+For a runnable embedded `/metrics` server example, see
+`examples/example_logit_prometheus_server.cpp`.
+
 ```cpp
 #include <logit.hpp>
 
@@ -81,22 +96,36 @@ LOGIT_INFO("Server started");
 
 ## Custom Metrics
 
-Use the `on_collect` callback to add application-specific metrics on each scrape:
+Use `PrometheusRegistry` with the `on_collect` callback to register
+application-specific metrics once and collect them on each scrape:
 
 ```cpp
-config.on_collect = [](std::vector<logit::PrometheusMetricFamily>& families) {
-    logit::PrometheusMetricFamily mf;
-    mf.name = "myapp_queue_size";
-    mf.help = "Current queue depth";
-    mf.type = logit::PrometheusMetricType::Gauge;
-    logit::PrometheusSample s;
-    s.name = "myapp_queue_size";
-    s.value = get_queue_depth();
-    mf.samples.push_back(s);
-    families.push_back(mf);
+#include <logit/loggers/prometheus/PrometheusRegistry.hpp>
+
+logit::PrometheusRegistry registry("myapp_");
+
+registry.set_gauge(
+    "queue_size",
+    "Current queue depth",
+    []() { return get_queue_depth(); });
+
+config.on_collect = [&registry](std::vector<logit::PrometheusMetricFamily>& families) {
+    registry.collect(families);
 };
 ```
 
+`PrometheusTextFormatConfig::metric_prefix` applies only to LogIt++ built-in
+metrics. Custom metric names are written as supplied by the registry or manual
+builders, so use the registry prefix for application metric namespaces.
+
+If a registry value callback throws, the registry skips that sample, keeps
+collecting later metrics, appends the healthy samples, and then rethrows the
+first exception. The Prometheus loggers catch that exception from `on_collect`
+and increment their failed export counter.
+
+For low-level control, `on_collect` can still append `PrometheusMetricFamily`
+objects directly or use helpers such as `add_prometheus_gauge()`.
+
 ## Prometheus Scrape Config
 
 ```yaml

examples/example_logit_mdc_ndc.cpp

@@ -0,0 +1,51 @@
+#include <logit.hpp>
+
+#include <thread>
+
+int main() {
+#ifndef LOGIT_WITH_CONTEXT
+    LOGIT_ADD_CONSOLE_DEFAULT();
+    LOGIT_WARN("MDC/NDC example requires LOGIT_WITH_CONTEXT=ON");
+    LOGIT_WAIT();
+    return 0;
+#else
+    LOGIT_ADD_CONSOLE(
+        "[%T] [%l] request=%K{request_id} user=%K{user_id} ndc=[%J] %v",
+        false);
+
+    LOGIT_MDC_PUT("request_id", "req-42");
+    LOGIT_MDC_PUT("user_id", "alice");
+    LOGIT_NDC_PUSH("http");
+    LOGIT_NDC_PUSH("POST /checkout");
+
+    LOGIT_INFO("request accepted");
+
+    {
+        LOGIT_NDC_GUARD("payment");
+        LOGIT_WARN("payment provider latency is high");
+    }
+
+    LOGIT_INFO("payment scope has ended");
+
+    std::thread worker([]() {
+        LOGIT_MDC_PUT("request_id", "worker-7");
+        LOGIT_MDC_PUT("user_id", "background");
+        LOGIT_NDC_PUSH("worker");
+        LOGIT_INFO("background task has its own thread-local context");
+        LOGIT_MDC_CLEAR();
+        LOGIT_NDC_CLEAR();
+    });
+    worker.join();
+
+    LOGIT_INFO("main thread context is unchanged");
+
+    LOGIT_MDC_REMOVE("user_id");
+    LOGIT_NDC_POP();
+    LOGIT_INFO("specific MDC keys can be removed and NDC can be popped");
+
+    LOGIT_MDC_CLEAR();
+    LOGIT_NDC_CLEAR();
+    LOGIT_SHUTDOWN();
+    return 0;
+#endif
+}

examples/example_logit_otlp_http.cpp

@@ -1,5 +1,17 @@
 #include <logit.hpp>
 
+#include <cstdlib>
+#include <string>
+
+namespace {
+
+std::string env_or(const char* name, const char* fallback) {
+    const char* value = std::getenv(name);
+    return (value && *value) ? std::string(value) : std::string(fallback);
+}
+
+} // namespace
+
 int main() {
 #ifndef LOGIT_WITH_OTLP
     LOGIT_ADD_CONSOLE_DEFAULT();
@@ -8,25 +20,62 @@ int main() {
     return 0;
 #else
     logit::OtlpHttpLogger::Config config;
-    config.host = "http://localhost:4318";
-    config.path = "/v1/logs";
-    config.format.service_name = "logit-otlp-example";
-    config.format.deployment_environment = "dev";
-    config.max_batch_size = 32;
-    config.export_interval_ms = 500;
+    config.host = env_or("LOGIT_OTLP_ENDPOINT", "http://localhost:4318");
+    config.path = env_or("LOGIT_OTLP_PATH", "/v1/logs");
+    config.format.service_name = env_or("LOGIT_SERVICE_NAME", "checkout-service");
+    config.format.service_namespace = "examples";
+    config.format.service_instance_id = "local-dev";
+    config.format.deployment_environment = env_or("LOGIT_ENVIRONMENT", "dev");
+    config.max_queue_size = 1024;
+    config.max_batch_size = 64;
+    config.max_in_flight_requests = 2;
+    config.export_interval_ms = 250;
+    config.request_timeout_sec = 3;
+    config.retry_attempts = 1;
+    config.retry_delay_ms = 100;
+    config.cancel_on_shutdown = false;
+
+#if defined(LOGIT_HAS_ZSTD)
+    config.compression = logit::OtlpCompression::Zstd;
+    config.compression_level = 3;
+#elif defined(LOGIT_HAS_ZLIB)
+    config.compression = logit::OtlpCompression::Gzip;
+    config.compression_level = 6;
+#endif
 
     LOGIT_ADD_LOGGER(
         logit::OtlpHttpLogger,
         (config),
         logit::SimpleLogFormatter,
-        ("%v")
+#ifdef LOGIT_WITH_CONTEXT
+        ("[%l] trace=%K{trace_id} span=%K{span_id} %v")
+#else
+        ("[%l] %v")
+#endif
     );
 
+#ifdef LOGIT_WITH_CONTEXT
+    LOGIT_MDC_PUT("trace_id", "7b3f1c8a2e914a99");
+    LOGIT_MDC_PUT("span_id", "checkout-001");
+    LOGIT_NDC_PUSH("checkout");
+#endif
+
     LOGIT_INFO("OTLP logger started");
-    LOGIT_WARN("Example warning message");
-    LOGIT_ERROR("Example error message");
+    LOGIT_WARN("payment provider latency is above threshold");
+
+    {
+#ifdef LOGIT_WITH_CONTEXT
+        LOGIT_NDC_GUARD("submit-order");
+#endif
+        LOGIT_ERROR("order export failed; collector will count failed exports if HTTP fails");
+    }
 
     LOGIT_WAIT();
+#ifdef LOGIT_WITH_CONTEXT
+    LOGIT_MDC_CLEAR();
+    LOGIT_NDC_CLEAR();
+#endif
+    LOGIT_SHUTDOWN();
     return 0;
 #endif
 }