Skip to content

Add regression tests for concat strings through println and shell#3713

Draft
ATX24 wants to merge 1 commit into
canaryfrom
cursor/string-concat-println-shell-15fa
Draft

Add regression tests for concat strings through println and shell#3713
ATX24 wants to merge 1 commit into
canaryfrom
cursor/string-concat-println-shell-15fa

Conversation

@ATX24

@ATX24 ATX24 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

The error

Claimed regression shape:

function main() -> string {
  let cmd = "echo " + "hello_world";
  baml.io.println("-> " + cmd);
  baml.sys.shell(cmd, null).stdout.to_string()
}

The reported incorrect behavior was silent empty shell output after the println consumption step:

exit_code: 0
stdout: ""
stderr: ""

The expected shell stdout is non-empty and exactly:

hello_world

Root cause

No current compiler/runtime bug was reproduced in this branch. The gap was missing CI coverage in baml_language/crates/baml_tests/tests/shell.rs for this specific interaction:

  • string concatenation produces a command string,
  • baml.io.println consumes that same value first,
  • baml.sys.shell later receives the command string,
  • stdout must remain non-empty and exact.

That left the compiler2 VM/sysop path exercised by baml_tests::baml_test! without a regression guard for this silent-failure shape.

The fix

Added three Unix shell regression tests in baml_language/crates/baml_tests/tests/shell.rs:

  1. concat-built command passed through one baml.io.println before baml.sys.shell,
  2. the same command passed through two println calls before shell,
  3. a multi-step concat from two let bindings with the println result assigned to _ before shell.

Each test asserts stdout is not empty and equals "hello_world\n".

Verification

Ran from baml_language/ unless noted:

cargo fmt
cargo test --package baml_tests shell_concat --test shell

Result:

running 2 tests
... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 12 filtered out
cargo test --package baml_tests --test shell

Result:

running 14 tests
... ok

test result: ok. 14 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
cargo test --package baml_tests --lib

Result:

test result: ok. 1589 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out
cargo nextest run

Result: completed with exit code 0 after running nextest setup scripts for the SDK fixtures.

Note: plain workspace cargo test reached sdk_test_typescript_node and failed because the SDK test docs require cargo nextest run for setup-script-driven fixture installs; the nextest full-suite run passed.

Issue Reference

  • This PR fixes/closes #[issue number]

Changes

  • Add focused compiler2 regression coverage for concat strings consumed by baml.io.println and then passed to baml.sys.shell.

Testing

  • Unit tests added/updated
  • Manual testing performed
  • Tested in Linux cloud agent environment

PR Checklist

  • I have read and followed the contributing guidelines
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings

Additional Notes

No legacy engine/ files were modified; all changes are scoped to baml_language/.

Open in Web Open in Cursor 

@vercel

vercel Bot commented Jun 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
beps Ready Ready Preview, Comment Jun 8, 2026 6:36pm
promptfiddle Ready Ready Preview, Comment Jun 8, 2026 6:36pm
promptfiddle2 Ready Ready Preview, Comment Jun 8, 2026 6:36pm

Request Review

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown

No description provided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants