Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -103,8 +103,8 @@ For handler, client, and library boundary patterns, see the tracing how-to pages
[Use the otel4s Java agent](use-the-otel4s-java-agent.md)
- for background on why that agent needs its own setup path, see
[How otel4s context works with the otel4s Java agent](../explanations/how-otel4s-context-works-with-the-otel4s-java-agent.md)
- if you use the testkit and rely on `IOLocalContextStorage`, see the existing
[Testkit](../oteljava/testkit.md#iolocalcontextstorage) page
- if you use the testkit and rely on `IOLocalContextStorage`, see
[Use IOLocalContextStorage with the testkit](../how-to-testkit/use-iolocal-context-storage-with-the-testkit.md)

## What's next

Expand Down
1 change: 1 addition & 0 deletions docs/how-to-testkit/directory.conf
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@ laika.navigationOrder = [
test-metrics-emitted-by-your-code.md
test-traces-emitted-by-your-code.md
test-logs-emitted-by-your-code.md
use-iolocal-context-storage-with-the-testkit.md
]
1 change: 1 addition & 0 deletions docs/how-to-testkit/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ telemetry it emits.
- [Test metrics emitted by your code](test-metrics-emitted-by-your-code.md)
- [Test traces emitted by your code](test-traces-emitted-by-your-code.md)
- [Test logs emitted by your code](test-logs-emitted-by-your-code.md)
- [Use IOLocalContextStorage with the testkit](use-iolocal-context-storage-with-the-testkit.md)

## Related material

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Use IOLocalContextStorage with the testkit

Use `IOLocalTestContextStorage` when application code uses `IOLocalContextStorage` and a test runs that code with an
OpenTelemetry Java testkit.

The testkit depends on the OpenTelemetry Java SDK testing module, which installs a test-specific `ContextStorageProvider`.
Using `IOLocalContextStorage.localProvider` directly with that provider produces an error like:

```text
java.lang.IllegalStateException: IOLocalContextStorage is not configured for use as the ContextStorageProvider.
The current ContextStorage is: io.opentelemetry.sdk.testing.context.SettableContextStorageProvider$SettableContextStorage
```

## 1. Add the context-storage testkit

This page assumes the test already depends on `otel4s-oteljava-testkit` and the application is configured to use
`IOLocalContextStorage`. See
[Keep otel4s context in sync with OpenTelemetry Java](../how-to-jvm-setup/keep-otel4s-context-in-sync-with-opentelemetry-java.md)
for the application setup.

@:select(build-tool)

@:choice(sbt)

Add the context-storage testkit to the test scope in `build.sbt`:

```scala
libraryDependencies +=
"org.typelevel" %% "otel4s-oteljava-context-storage-testkit" % "@VERSION@" % Test
```

@:choice(scala-cli)

Add the test dependency to the test source file:

```scala
//> using test.dep "org.typelevel::otel4s-oteljava-context-storage-testkit:@VERSION@"
```

@:@

## 2. Provide the test local context

Define `IOLocalTestContextStorage.localProvider[IO]` as the `LocalContextProvider` used to create the testkit.

```scala mdoc:silent
import cats.effect.IO
import org.typelevel.otel4s.oteljava.context.LocalContextProvider
import org.typelevel.otel4s.oteljava.testkit.OtelJavaTestkit
import org.typelevel.otel4s.oteljava.testkit.context.IOLocalTestContextStorage

def test: IO[Unit] = {
implicit val localContextProvider: LocalContextProvider[IO] =
IOLocalTestContextStorage.localProvider[IO]

OtelJavaTestkit.inMemory[IO]().use { testkit =>
testkit.tracerProvider.get("service").flatMap { tracer =>
tracer.span("test").surround(IO.unit)
}
}
}
```

The same provider works with `MetricsTestkit`, `TracesTestkit`, and `LogsTestkit`.

## What's next

- [Test metrics emitted by your code](test-metrics-emitted-by-your-code.md)
- [Test traces emitted by your code](test-traces-emitted-by-your-code.md)
- [Test logs emitted by your code](test-logs-emitted-by-your-code.md)
266 changes: 35 additions & 231 deletions docs/oteljava/testkit.md
Original file line number Diff line number Diff line change
@@ -1,256 +1,60 @@
# Testkit

The `otel4s-oteljava-testkit` module provides in-memory metric, trace, and log exporters for the OpenTelemetry Java
backend.
backend. It runs instrumentation against the real backend and exposes exported telemetry as OpenTelemetry Java SDK
models.

Use it when you want to:
The testkit is framework-independent, so it can be used with munit, weaver, ScalaTest, or another test framework.

- run instrumentation against a real `otel4s-oteljava` implementation
- collect exported telemetry as OpenTelemetry Java SDK models
- assert telemetry with the expectation APIs instead of building local test ADTs
## Choose a testkit

The testkit is framework-agnostic, so it can be used with munit, weaver, ScalaTest, or any other test framework.
Use a signal-specific testkit when a test covers one signal. Use `OtelJavaTestkit` when a test needs two or more signals.

## Getting started
| Testkit | Providers | Exported telemetry |
|---------------------|--------------------------------------------------|-----------------------------------------------------|
| `MetricsTestkit` | `meterProvider` | `collectMetrics`: `List[MetricData]` |
| `TracesTestkit` | `tracerProvider` | `finishedSpans`: `List[SpanData]` |
| `LogsTestkit` | `loggerProvider` | `finishedLogs`: `List[LogRecordData]` |
| `OtelJavaTestkit` | `meterProvider`, `tracerProvider`, `loggerProvider` | Metrics, spans, and logs from the methods above |

@:select(build-tool)
## How-to guides

@:choice(sbt)

Add settings to `build.sbt`:

```scala
libraryDependencies ++= Seq(
"org.typelevel" %% "otel4s-oteljava-testkit" % "@VERSION@" % Test, // <1>
)
```

@:choice(scala-cli)

Add directives to the `*.scala` file:

```scala
//> using test.dep "org.typelevel::otel4s-oteljava-testkit:@VERSION@" // <1>
```

@:@

1. Add the `otel4s-oteljava-testkit` module to the test scope

## Choosing a testkit

You can use either the domain-specific testkits or the combined testkit:

- `MetricsTestkit` for metrics only
- `TracesTestkit` for traces only
- `LogsTestkit` for logs only
- `OtelJavaTestkit` when one test needs multiple signal types

The combined testkit exposes:

- `meterProvider`
- `tracerProvider`
- `loggerProvider`
- `collectMetrics`
- `finishedSpans`
- `finishedLogs`

```scala mdoc:silent
import cats.effect.IO
import org.typelevel.otel4s.oteljava.testkit.OtelJavaTestkit

def test: IO[Unit] =
OtelJavaTestkit.inMemory[IO]().use { testkit =>
for {
metrics <- testkit.collectMetrics
spans <- testkit.finishedSpans
logs <- testkit.finishedLogs
} yield {
val _ = (metrics, spans, logs)
}
}
```

If a test exercises only one signal, the signal-specific testkit usually reads a bit more clearly.
- [Test metrics emitted by your code](../how-to-testkit/test-metrics-emitted-by-your-code.md)
- [Test traces emitted by your code](../how-to-testkit/test-traces-emitted-by-your-code.md)
- [Test logs emitted by your code](../how-to-testkit/test-logs-emitted-by-your-code.md)
- [Use IOLocalContextStorage with the testkit](../how-to-testkit/use-iolocal-context-storage-with-the-testkit.md)

## Expectation APIs

The recommended testing style is:

1. run the program against the in-memory testkit
2. collect raw OpenTelemetry Java SDK models
3. assert them with the expectation API

The expectation APIs are partial by default. This means:

- unspecified fields are ignored
- you can match only the properties that matter for the current test
- you can still add detail for timestamps, attributes, scope, resource, and structure when needed

Dedicated guides:

- [Testkit | Metrics](testkit-metrics.md)
- [Testkit | Traces](testkit-traces.md)
- [Testkit | Logs](testkit-logs.md)

### Metrics

Metrics are matched against OpenTelemetry Java `MetricData` using:

- `MetricExpectation`
- `PointExpectation`
- `PointSetExpectation`
- `MetricExpectations`

Use this when you want to assert metric name, metric type, values, point attributes, scope, resource, summaries,
histograms, exemplars, and collection-wide point constraints.

### Traces

Traces are matched against OpenTelemetry Java `SpanData` using:

- `SpanExpectation`
- `EventExpectation`
- `LinkExpectation`
- `TraceExpectation`
- `TraceForestExpectation`
- `SpanExpectations`
- `TraceExpectations`

Use `SpanExpectations` for flat exported-span checks and `TraceExpectations` when exact parent/child topology matters.

### Logs

Logs are matched against OpenTelemetry Java `LogRecordData` using:
The expectation APIs match OpenTelemetry Java SDK models. Expectations are partial by default: fields that are not
specified are ignored.

- `LogRecordExpectation`
- `LogRecordExpectations`
| Signal | SDK model | Main expectation APIs | API reference |
|---------|-----------------|-----------------------------------------------------------------------------------------|---------------------------------------|
| Metrics | `MetricData` | `MetricExpectation`, `PointExpectation`, `PointSetExpectation`, `MetricExpectations` | [Testkit \| Metrics](testkit-metrics.md) |
| Traces | `SpanData` | `SpanExpectation`, `TraceExpectation`, `TraceForestExpectation`, `SpanExpectations`, `TraceExpectations` | [Testkit \| Traces](testkit-traces.md) |
| Logs | `LogRecordData` | `LogRecordExpectation`, `LogRecordExpectations` | [Testkit \| Logs](testkit-logs.md) |

Use this when you want to assert body/message, severity, trace/span correlation, attributes, scope, resource, and
timestamps directly on exported log records.

## Mismatch rendering

Each top-level expectation API has a formatting helper that turns structured mismatches into a readable failure message:
Each top-level expectation API has a `format` method for rendering structured mismatches:

- `MetricExpectations.format(...)`
- `SpanExpectations.format(...)`
- `TraceExpectations.format(...)`
- `LogRecordExpectations.format(...)`

This is the easiest way to integrate expectations with a testing framework:

```scala mdoc:silent
import cats.data.NonEmptyList

def failWith[A](mismatches: NonEmptyList[A], render: NonEmptyList[A] => String): Nothing =
sys.error(render(mismatches))
```

In practice:

```scala mdoc:silent
import cats.effect.IO
import org.typelevel.otel4s.metrics.MeterProvider
import org.typelevel.otel4s.oteljava.testkit.metrics._

def program(meterProvider: MeterProvider[IO]): IO[Unit] = {
val _ = meterProvider
IO.unit
}

def assertMetrics(testkit: MetricsTestkit[IO]): IO[Unit] =
for {
_ <- program(testkit.meterProvider)
metrics <- testkit.collectMetrics
} yield {
MetricExpectations.checkAllDistinct(
metrics,
MetricExpectation.sum[Long]("service.requests")
) match {
case Right(_) =>
()
case Left(mismatches) =>
sys.error(MetricExpectations.format(mismatches))
}
}
```

## When to use raw SDK models

The expectation APIs should be the default path, but raw model assertions still make sense when:

- you are investigating a failing test and want to inspect the full exported payload
- you need to compare a field that is not exposed by the current expectation DSL
- you want to prototype a new matcher before turning it into a reusable expectation

The in-memory testkits always expose the underlying OpenTelemetry Java SDK models:

- `MetricData`
- `SpanData`
- `LogRecordData`

That means you can mix both styles in the same test:

- use expectation APIs for the stable high-value assertions
- drop down to raw SDK models for one-off or low-level checks

## IOLocalContextStorage

The `otel4s-oteljava-testkit` module depends on `io.opentelemetry:opentelemetry-sdk-testing`, which sets the
`ContextStorageProvider` to `io.opentelemetry.sdk.testing.context.SettableContextStorageProvider`.

If you rely on `IOLocalContextStorage` in tests, you will see an error like:

```scala
java.lang.IllegalStateException: IOLocalContextStorage is not configured for use as the ContextStorageProvider.
The current ContextStorage is: io.opentelemetry.sdk.testing.context.SettableContextStorageProvider$SettableContextStorage
```

To solve this, use `IOLocalTestContextStorage` from the `otel4s-oteljava-context-storage-testkit` module.

@:select(build-tool)

@:choice(sbt)

Add settings to `build.sbt`:

```scala
libraryDependencies ++= Seq(
"org.typelevel" %% "otel4s-oteljava-context-storage-testkit" % "@VERSION@" % Test,
)
```

@:choice(scala-cli)

Add directives to the `*.scala` file:

```scala
//> using test.dep "org.typelevel::otel4s-oteljava-context-storage-testkit:@VERSION@"
```

@:@

Parametrize your code to make `LocalContextProvider` overrideable:

```scala
import cats.effect.IO
import org.typelevel.otel4s.oteljava.context.LocalContextProvider
## Use raw SDK models when needed

def program(implicit provider: LocalContextProvider[IO]): IO[Unit] = ???
```
The in-memory testkits always expose the underlying `MetricData`, `SpanData`, and `LogRecordData` values. Direct model
assertions are useful when:

And override it in tests:
- diagnosing a failing test by inspecting the complete exported payload
- checking a field that the expectation API does not expose
- prototyping a matcher before adding a reusable expectation

```scala
import cats.effect.IO
import org.typelevel.otel4s.oteljava.context.LocalContextProvider
import org.typelevel.otel4s.oteljava.testkit.context.IOLocalTestContextStorage
Expectation-based and direct model assertions can be used in the same test.

def test: IO[Unit] = {
implicit val provider: LocalContextProvider[IO] =
IOLocalTestContextStorage.localProvider[IO]
## Context storage in tests

program
}
```
The OpenTelemetry Java SDK testing dependency installs a test-specific `ContextStorageProvider`. When production code
uses `IOLocalContextStorage`, provide `IOLocalTestContextStorage.localProvider` in tests instead. See
[Use IOLocalContextStorage with the testkit](../how-to-testkit/use-iolocal-context-storage-with-the-testkit.md).
Loading
Loading