Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .trailblaze-sync
Original file line number Diff line number Diff line change
@@ -1 +1 @@
593c43b070fc9636da56b078c7beb062d44bc242
1582142180765d5b1686d984deea619218ab4343
4 changes: 4 additions & 0 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@ subprojects {
// Apply shared git version computation
apply(from = "gradle/git-version.gradle.kts")

// Apply shared dependency-resolution security overrides (single source of truth,
// shared with the internal root build).
apply(from = "gradle/dependency-security.gradle.kts")

subprojects
.forEach {
it.plugins.withId("com.android.library") {
Expand Down
10 changes: 5 additions & 5 deletions docs/CLI.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ It does not reap device-scoped per-device sessions; use `app --stop` for those.
| `session` | Manage the current device session — save it as a replayable trail, inspect steps, end it |
| `report` | Generate an HTML report for session recordings, plus a best-effort JSON summary, and optionally MP4/GIF/WebP exports for a single session. JSON-only failures log a warning and still exit 0 — HTML is the primary artifact and is what gates the exit code. |
| `waypoint` | Match named app locations (waypoints) against captured screen state. |
| `results` | Query the persisted test-result index for a TestRail case. Passing a positional `<case-id>` (e.g. `trailblaze results C12345 --device android-phone`) is equivalent to the explicit `trailblaze results show <case-id>` form — picocli routes the bare case-id straight to the `show` subcommand. |
| `results` | Query the persisted test-result index for a test case. Passing a positional `<case-id>` (e.g. `trailblaze results C12345 --device android-phone`) is equivalent to the explicit `trailblaze results show <case-id>` form — picocli routes the bare case-id straight to the `show` subcommand. |
| `config` | View and set configuration (target app, device defaults, AI provider) |
| `device` | List and connect devices (Android, iOS, Web) |
| `show` | Open the multi-device live grid (/devices/all) in your default browser |
Expand Down Expand Up @@ -997,7 +997,7 @@ trailblaze waypoint shortcut verify [OPTIONS]

### `trailblaze results`

Query the persisted test-result index for a TestRail case. Passing a positional `<case-id>` (e.g. `trailblaze results C12345 --device android-phone`) is equivalent to the explicit `trailblaze results show <case-id>` form — picocli routes the bare case-id straight to the `show` subcommand.
Query the persisted test-result index for a test case. Passing a positional `<case-id>` (e.g. `trailblaze results C12345 --device android-phone`) is equivalent to the explicit `trailblaze results show <case-id>` form — picocli routes the bare case-id straight to the `show` subcommand.

**Synopsis:**

Expand All @@ -1010,7 +1010,7 @@ trailblaze results show

| Argument | Description | Required |
|----------|-------------|----------|
| `<<case-id>>` | TestRail case ID. When supplied without a subcommand, this routes to `show <case-id>`. Case-insensitive; the leading `C` is required (e.g. C12345). | No |
| `<<case-id>>` | Test-case ID. When supplied without a subcommand, this routes to `show <case-id>`. Case-insensitive; the leading `C` is required (e.g. C12345). | No |

**Options:**

Expand All @@ -1028,7 +1028,7 @@ trailblaze results show

### `trailblaze results show`

Show the recorded result for a TestRail case ID
Show the recorded result for a test case ID

**Synopsis:**

Expand All @@ -1040,7 +1040,7 @@ trailblaze results show [OPTIONS] <<case-id>>

| Argument | Description | Required |
|----------|-------------|----------|
| `<<case-id>>` | TestRail case ID, e.g. C12345. Case-insensitive; the leading `C` is required. | Yes |
| `<<case-id>>` | Test-case ID, e.g. C12345. Case-insensitive; the leading `C` is required. | Yes |

**Options:**

Expand Down
10 changes: 5 additions & 5 deletions docs/devlog/2026-03-06-trail-yaml-v2-syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ The `trailhead` is everything about the starting point: what this trail is, how
```yaml
# ── Trailhead: identity, configuration, and setup ──────────────────
trailhead:
id: testrail/suite_71172/section_838052/case_4837714
id: regression/suite_71172/section_838052/case_4837714
title: Verify user cannot load more than $2,000 onto a Gift Card within 24 hours
priority: P0

Expand All @@ -79,7 +79,7 @@ trailhead:
metadata:
caseId: "4837714"
sectionId: "838052"
testRailUrl: https://testrail.example.com/index.php?/cases/view/12345
sourceUrl: https://tracker.example.com/cases/view/12345

# Setup objectives (checkpoint for recording iteration)
setup:
Expand Down Expand Up @@ -375,7 +375,7 @@ This part of the original v2 design (CLI flows that walk authors through picking
- Optional `--target=<T> --platform=<P>` to narrow.
- The implementation noun stays `tool` everywhere: `TrailblazeTool` (Kotlin class), `./trailblaze toolbox` (CLI), `./trailblaze tool <name>` (invocation), `tools:` (YAML key). The TOOLS section in the listing is the same word as the implementation type — the overload is harmless because section descriptions disambiguate the role.
- Plain-text output for now; structured output (YAML / JSON) deferred until a concrete need surfaces.
- Specific funnel surfaces (interactive flows in `./trailblaze blaze`, scaffolding defaults, the on-demand pipeline's TestRail → trail generator) are NOT settled here. They will be designed against a working discovery surface first.
- Specific funnel surfaces (interactive flows in `./trailblaze blaze`, scaffolding defaults, the on-demand pipeline's external-test-case → trail generator) are NOT settled here. They will be designed against a working discovery surface first.

The principle the funnel work will hold to: **make it easy to do things the right way.** Discovery comes first so authors can SEE what's available; funneling them through workflows is a follow-on once discovery is solid.

Expand Down Expand Up @@ -412,7 +412,7 @@ trailhead:
**NL-only `blaze.yaml` (recorder materializes `tools:` on first run):**
```yaml
config:
id: testrail/suite_71172/section_946176/case_5552497
id: regression/suite_71172/section_946176/case_5552497
target: myapp
platform: android
trailhead:
Expand All @@ -439,7 +439,7 @@ Parses and runs. Soft warning at lint time; no failure.

### Open questions for follow-up devlogs

1. **Funnel design** — how the authoring surfaces (`./trailblaze blaze`, scaffolding, the TestRail generator) walk authors through trailhead selection. Discovery via `./trailblaze toolbox trailheads` is the foundation; the workflow design happens after that lands.
1. **Funnel design** — how the authoring surfaces (`./trailblaze blaze`, scaffolding, the test-case generator) walk authors through trailhead selection. Discovery via `./trailblaze toolbox trailheads` is the foundation; the workflow design happens after that lands.
2. **Default trailhead per target** — each registered target needs a designated default trailhead tool (e.g. `myapp` defaults to `myapp_launchAppSignedInWithAccount(key: defaults/standard-account)`). Lives in a registry of some kind; format TBD.
3. **`@LLMDescription` audit** — for the existing trailhead-tagged tool classes, ensure each description is decision-grade. Some are; others need a pass.
4. **Validation lint behavior** — `./trailblaze toolbox trailheads`-based lint warnings vs CI-gating. Default warn-only; opt-in `--strict` for teams that want PR-blocking.
Expand Down
36 changes: 36 additions & 0 deletions gradle/dependency-security.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import org.gradle.api.artifacts.VersionCatalogsExtension

// Single source of truth for dependency-resolution security overrides.
//
// Applied from BOTH root builds so the internal and open-source project
// configurations resolve identical versions:
// - internal root: apply(from = "opensource/gradle/dependency-security.gradle.kts")
// - open-source root: apply(from = "gradle/dependency-security.gradle.kts")
//
// Keeping the force here (in the mirrored opensource/ tree) rather than inline in a
// single root means the open-source mirror gets the same resolution the internal
// build does. Otherwise the dependency-guard baselines — generated against the
// internal resolution and then synced — fail in open-source CI because the mirror
// resolves the un-forced transitive version.

val micrometerVersion =
extensions.getByType<VersionCatalogsExtension>()
.named("libs")
.findVersion("micrometer")
.get()
.requiredVersion

// micrometer is a transitive dependency of dev.mobile:maestro-utils, which pins the
// vulnerable 1.13.4; Trailblaze does not depend on it directly. Force the
// catalog-pinned version (CVE-2026-40984, CWE-770: Allocation of Resources Without
// Limits or Throttling) across every configuration in every subproject.
subprojects {
configurations.all {
resolutionStrategy.eachDependency {
if (requested.group == "io.micrometer") {
useVersion(micrometerVersion)
because("CVE-2026-40984: micrometer-core < 1.15.12 is vulnerable to resource exhaustion")
}
}
}
}
2 changes: 1 addition & 1 deletion skills/trailblaze/references/session-logs-inspection.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ trailblaze-models/src/commonMain/kotlin/xyz/block/trailblaze/logs/client/Trailbl
```

When a new log type lands or fields change, that file is the source of truth — update this
reference and `scripts/testrail-build-analysis/analyze.py` accordingly.
reference accordingly.

## Related references

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,7 @@ class MultiAgentV3Runner private constructor(
sessionId: SessionId = TrailblazeSessionManager.generateSessionId("trail"),
initialActionHistory: List<String> = emptyList(),
/**
* The overall test case title (e.g. TestRail case name) that encompasses all steps.
* The overall test case title (e.g. an external test-case name) that encompasses all steps.
*
* Passed as [RecommendationContext.overallObjective] so the inner agent can reason
* about each step in the context of the broader test goal and detect impossible
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import xyz.block.trailblaze.yaml.PromptStep
* @param v3Runner The underlying V3 runner
* @param screenStateProvider Provider for current screen state
* @param sessionIdProvider Provider for the current session ID
* @param caseTitleProvider Returns the current test case title (e.g. TestRail case name).
* @param caseTitleProvider Returns the current test case title (e.g. an external test-case name).
* Invoked per step so the title can change between trails when the runner is reused.
* The value is forwarded as [RecommendationContext.overallObjective] so the inner agent
* can detect impossible steps early instead of exhausting all retries.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ class TrailStepPlanner(
private val availableToolsProvider: () -> List<TrailblazeToolDescriptor> = { emptyList() },
private val initialActionHistory: List<String> = emptyList(),
/**
* The overall test case title (e.g. TestRail case name) that encompasses all steps.
* The overall test case title (e.g. an external test-case name) that encompasses all steps.
*
* When non-null, this is passed as [RecommendationContext.overallObjective] so the inner
* agent can reason about each step in the context of the broader test goal. This enables
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -420,7 +420,7 @@ open class AndroidTrailblazeRule(
private val agentImplementation: AgentImplementation = InstrumentationArgUtil.agentImplementation()

/**
* Title of the trail currently executing (e.g. TestRail case name).
* Title of the trail currently executing (e.g. an external test-case name).
*
* Set at [runSuspend] entry before any step runs, then forwarded via
* [caseTitleProvider] in the V3 runner so the inner agent receives the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,7 @@ class TrailblazeSessionManager(
*
* Format: `YYYY_MM_DD_HH_MM_SS_<seed>_<random>`, sanitized via [SessionId.sanitized].
*
* The full test name (including TestRail suite/section/case suffixes) is
* The full test name (including external suite/section/case suffixes) is
* preserved so downstream tooling can reliably map session IDs back to test
* identifiers. Both host-generated and externally-provided IDs flow through
* [SessionId.sanitized] so a host-generated ID passed to the on-device handler
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ fun toPascalCaseIdentifier(input: String, prefixIfStartsWithDigit: String = "C")
*
* Not for session IDs. For session-ID sanitization use
* `SessionId.sanitized` — it is idempotent and preserves long suffixes
* (e.g., TestRail `__suite__section__case`), which this helper would
* (e.g., an external `__suite__section__case` identifier), which this helper would
* collapse into a single underscore.
*/
fun toSnakeCaseIdentifier(input: String, prefixIfStartsWithDigit: String = "c_"): String {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ class TrailComparator {
/**
* Compare two trails and return a detailed comparison result.
*
* @param expected The expected/reference trail items (e.g., from TestRail)
* @param expected The expected/reference trail items (e.g., from an external test-management system)
* @param actual The actual trail items (e.g., from a recorded trail file)
* @param compareConfig Whether to compare config blocks (default: true)
* @return A [TrailComparisonResult] containing any differences found
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,17 @@ class TrailblazeSessionManagerTest {
val sessionId = generateSessionId(longSeed)

assert(sessionId.value.contains("__suite_1__section_2__case_3")) {
"TestRail-style suffix must survive generateSessionId; got: ${sessionId.value}"
"Long suite/section/case suffix must survive generateSessionId; got: ${sessionId.value}"
}
assert(sessionId.value.length > 200) {
"Expected un-truncated session id; got length ${sessionId.value.length}"
}
}

@Test
fun `createSessionWithId preserves long TestRail IDs without truncation`() {
fun `createSessionWithId preserves long external IDs without truncation`() {
// Regression pin: the previous 100-char cap on externally-provided override
// IDs also dropped TestRail suffixes. Now both paths go through the single
// IDs also dropped long suite/section/case suffixes. Now both paths go through the single
// SessionId.sanitized source of truth.
val longOverride = SessionId(
"2026_04_20_11_16_18_example_suite_long_test_name_" +
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -989,11 +989,11 @@ class TrailblazeRecordingGeneratorTest {
@Test
fun roundTripFullTrailFile() {
val config = TrailConfig(
id = "testrail/suite_123/case_456",
id = "regression/suite_123/case_456",
title = "User can log in",
priority = "P0",
context = "Account email: [email protected]",
metadata = mapOf("testRailCaseId" to "456"),
metadata = mapOf("caseId" to "456"),
)
val promptsYaml = """
|- prompts:
Expand Down Expand Up @@ -1045,11 +1045,11 @@ class TrailblazeRecordingGeneratorTest {
assertThat(decoded.size).isEqualTo(3) // config + tools + prompts

val decodedConfig = (decoded[0] as TrailYamlItem.ConfigTrailItem).config
assertThat(decodedConfig.id).isEqualTo("testrail/suite_123/case_456")
assertThat(decodedConfig.id).isEqualTo("regression/suite_123/case_456")
assertThat(decodedConfig.title).isEqualTo("User can log in")
assertThat(decodedConfig.priority).isEqualTo("P0")
assertThat(decodedConfig.context).isEqualTo("Account email: [email protected]")
assertThat(decodedConfig.metadata).isEqualTo(mapOf("testRailCaseId" to "456"))
assertThat(decodedConfig.metadata).isEqualTo(mapOf("caseId" to "456"))

val decodedTools = (decoded[1] as TrailYamlItem.ToolTrailItem).tools
assertThat(decodedTools.size).isEqualTo(1)
Expand Down
6 changes: 3 additions & 3 deletions trailblaze-desktop/dependencies/runtimeClasspath.txt
Original file line number Diff line number Diff line change
Expand Up @@ -255,9 +255,9 @@ io.ktor:ktor-websocket-serialization-jvm:3.3.3
io.ktor:ktor-websocket-serialization:3.3.3
io.ktor:ktor-websockets-jvm:3.3.3
io.ktor:ktor-websockets:3.3.3
io.micrometer:micrometer-commons:1.13.4
io.micrometer:micrometer-core:1.13.4
io.micrometer:micrometer-observation:1.13.4
io.micrometer:micrometer-commons:1.15.12
io.micrometer:micrometer-core:1.15.12
io.micrometer:micrometer-observation:1.15.12
io.modelcontextprotocol:kotlin-sdk-client-jvm:0.11.1
io.modelcontextprotocol:kotlin-sdk-client:0.11.1
io.modelcontextprotocol:kotlin-sdk-core-jvm:0.11.1
Expand Down
6 changes: 3 additions & 3 deletions trailblaze-host/dependencies/runtimeClasspath.txt
Original file line number Diff line number Diff line change
Expand Up @@ -255,9 +255,9 @@ io.ktor:ktor-websocket-serialization-jvm:3.3.3
io.ktor:ktor-websocket-serialization:3.3.3
io.ktor:ktor-websockets-jvm:3.3.3
io.ktor:ktor-websockets:3.3.3
io.micrometer:micrometer-commons:1.13.4
io.micrometer:micrometer-core:1.13.4
io.micrometer:micrometer-observation:1.13.4
io.micrometer:micrometer-commons:1.15.12
io.micrometer:micrometer-core:1.15.12
io.micrometer:micrometer-observation:1.15.12
io.modelcontextprotocol:kotlin-sdk-client-jvm:0.11.1
io.modelcontextprotocol:kotlin-sdk-client:0.11.1
io.modelcontextprotocol:kotlin-sdk-core-jvm:0.11.1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import java.util.concurrent.Callable
/**
* Query persisted test results from a flat-file index repo.
*
* The index repo is a key-value store on GitHub: two JSON files per (TestRail case, device
* The index repo is a key-value store on GitHub: two JSON files per (test case, device
* profile) cell at
* `results/testrail/<caseId>/<device>/latest.json` (every terminal run)
* `results/testrail/<caseId>/<device>/latest_success.json` (passing runs only)
Expand All @@ -52,7 +52,7 @@ import java.util.concurrent.Callable
name = "results",
mixinStandardHelpOptions = true,
description = [
"Query the persisted test-result index for a TestRail case. " +
"Query the persisted test-result index for a test case. " +
"Passing a positional `<case-id>` (e.g. `trailblaze results C12345 --device android-phone`) " +
"is equivalent to the explicit `trailblaze results show <case-id>` form — picocli routes " +
"the bare case-id straight to the `show` subcommand.",
Expand All @@ -76,7 +76,7 @@ class ResultsCommand : Callable<Int> {
arity = "0..1",
paramLabel = "<case-id>",
description = [
"TestRail case ID. When supplied without a subcommand, this routes to `show <case-id>`. " +
"Test-case ID. When supplied without a subcommand, this routes to `show <case-id>`. " +
"Case-insensitive; the leading `C` is required (e.g. C12345).",
],
)
Expand Down Expand Up @@ -153,7 +153,7 @@ class ResultsCommand : Callable<Int> {
@Command(
name = "show",
mixinStandardHelpOptions = true,
description = ["Show the recorded result for a TestRail case ID"],
description = ["Show the recorded result for a test case ID"],
)
class ResultsShowCommand : Callable<Int> {

Expand All @@ -177,7 +177,7 @@ class ResultsShowCommand : Callable<Int> {
arity = "1",
paramLabel = "<case-id>",
description = [
"TestRail case ID, e.g. C12345. Case-insensitive; the leading `C` is required.",
"Test-case ID, e.g. C12345. Case-insensitive; the leading `C` is required.",
],
)
lateinit var caseId: String
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1593,7 +1593,7 @@ object TrailblazeHostYamlRunner {
if (promptSteps.isEmpty()) {
throw TrailblazeException(
"Trail has no executable prompt steps — this would be a false positive pass. " +
"Add steps to this trail file or the TestRail case.",
"Add steps to this trail file or the source test case.",
)
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,7 @@ abstract class BaseHostTrailblazeTest(
}

/**
* The title of the trail currently being executed (e.g. TestRail case name).
* The title of the trail currently being executed (e.g. an external test-case name).
*
* Updated at the start of each [runTrail] call so [MultiAgentV3TestAgentRunner] can
* forward it as [RecommendationContext.overallObjective] for every step. This lets the
Expand Down
Loading
Loading