Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,26 @@ else)` evaluates only the branch the condition selects, so `if(n != 0, total/n,
needed), but you can shadow any of them by registering a function or env value
of the same name.

## Pipelines

The pipe operator `a | f(x)` compiles as `f(a, x)`, so chained
transformations read left to right instead of inside out:

```go
p, err := expr.Compile(
`checks | filter(!it.ok) | map(it.name) | join(", ")`,
expr.WithBuiltins(),
expr.WithFunctions(expr.StringFuncs()),
)
```

The pipe is compile-time sugar over ordinary calls, so it composes with
everything above: builtins, your registered functions, and the
higher-order forms in both shapes. In Go this token means bitwise or,
which expr has always rejected, so the pipe changed the meaning of no
existing expression. Design rationale lives in
[RFC 0001](docs/rfcs/0001-pipe-operator.md).

## What it isn't

`expr` evaluates a **single expression**. No statements, no `:=`, no
Expand Down
2 changes: 1 addition & 1 deletion boundaries1_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1244,7 +1244,7 @@ func TestSyntax_UnsupportedNodesAllReject(t *testing.T) {
"*p", // unary * (unsupported token)
"&x", // address-of
"1 & 2", // bitwise AND
"1 | 2", // bitwise OR
"1 | 2", // pipe with non-call right side
"1 ^ 2", // bitwise XOR
"1 << 2", // shift left
"1 >> 2", // shift right
Expand Down
1 change: 1 addition & 0 deletions cmd/expr/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ func main() {
fmt.Fprintf(os.Stderr, " expr -i '{\"user\":{\"age\":36}}' 'user.age >= 18'\n")
fmt.Fprintf(os.Stderr, " expr 'user.age >= 18' -i @user.json\n")
fmt.Fprintf(os.Stderr, " echo '{\"x\":41}' | expr -i - 'x + 1'\n")
fmt.Fprintf(os.Stderr, " expr -i '{\"xs\":[3,1,2]}' 'xs | filter(it > 1) | len()'\n")
}

// Reorder arguments so that flags may appear before or after the
Expand Down
47 changes: 47 additions & 0 deletions docs/guides/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -554,3 +554,50 @@ sort(["banana", "apple"]) // → ["apple", "banana"]
`sort` accepts all-numbers or all-strings; mixed types produce
`ErrEvaluate`. It never mutates the input and returns a fresh `[]any`.
`reverse` works on any list type and also returns a fresh copy.

---

## 13. Pipelines

`a | f(x)` compiles as `f(a, x)`, so a transformation chain reads
left to right instead of inside out. Rendering an error report:

```go
events |
filter(it.kind == "error") |
map(sprintf("[%s] %s", it.source, it.message)) |
join("\n")
```

(As with every operator, the `|` goes at the end of a continuation
line, never at the start, because Go's semicolon insertion terminates
a line that ends in an identifier or `)`.)

Env (compile with `WithBuiltins` for `sprintf` and `StringFuncs` for
`join`):

```go
map[string]any{
"events": []any{
map[string]any{"kind": "error", "source": "api", "message": "timeout"},
map[string]any{"kind": "info", "source": "api", "message": "ok"},
map[string]any{"kind": "error", "source": "db", "message": "deadlock"},
},
}
```

Result:

```
[api] timeout
[db] deadlock
```

The pipe is sugar only: the chain above is exactly
`join(map(filter(events, ...), ...), "\n")` after compilation, and the
named-binding forms compose the same way
(`orders | filter(o, o.paid) | sortBy(o, o.total)`). The right side of
each `|` must be written as a call (`xs | len()`, not `xs | len`). See
the [spec](../reference/spec.md#pipeline-) for precedence rules. In
short: pipe first, compare after (`xs | count(it.ok) == 2` works;
`a == b | f()` is a compile error asking for parentheses).
23 changes: 23 additions & 0 deletions docs/guides/higher-order-patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,29 @@ map(
)
```

## Pipelines: the same chains, left to right

`a | f(x)` compiles as `f(a, x)`, so a filter-then-map chain can read
in execution order instead of inside out:

```
orders | filter(it.status == "paid") | map(it.id)
```

This is purely compile-time sugar over the nested form above. The
forms, bindings, and laziness rules are identical, and the three-arg
named-binding shape composes the same way:

```
orders | filter(o, o.status == "paid") | map(o, {o.id: o.total})
```

The right side of each `|` must be written as a call (`xs | trim()`,
not `xs | trim`), and a pipe on the right-hand side of a comparison
must be parenthesized. Precedence details live in the
[spec](../reference/spec.md#pipeline-); a worked example is in
[examples.md](examples.md#13-pipelines).

## Nested forms and the `it` rebinding rule

Inside a nested two-arg higher-order form, `it` and `index` always refer
Expand Down
93 changes: 88 additions & 5 deletions docs/reference/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,14 +64,16 @@ Precedence and associativity come from `go/parser`. They match Go:
| Precedence | Operators | Associativity |
| ---------- | ------------------------------ | ------------- |
| 5 (high) | `* / %` | left |
| 4 | `+ -` | left |
| 4 | `+ - \|` | left |
| 3 | `== != < <= > >=` | left |
| 2 | `&&` | left |
| 1 (low) | `\|\|` | left |

Unary `!`, `-`, `+` bind tighter than any binary operator. Parentheses
group expressions as usual. Go's bitwise operators (`&`, `|`, `^`, `<<`,
`>>`, `&^`) parse but are rejected at Compile time with `ErrCompile`.
group expressions as usual. The `|` token is the [pipeline
operator](#pipeline-), not bitwise or. Go's remaining bitwise
operators (`&`, `^`, `<<`, `>>`, `&^`) parse but are rejected at
Compile time with `ErrCompile`.

### Arithmetic (`+ - * / %`)

Expand Down Expand Up @@ -110,6 +112,84 @@ group expressions as usual. Go's bitwise operators (`&`, `|`, `^`, `<<`,
and `count || 0` falls back to `0` only when `count` is falsey.
Where a strict bool is required, wrap with `bool(...)`.

### Pipeline (`|`)

`a | f(x, y)` evaluates exactly as `f(a, x, y)`: the left side becomes
the first argument of the call on the right side. The rewrite happens once at Compile time,
so the pipeline has no runtime semantics of its own: a piped call is
the call, including for the higher-order special forms and their lazy
evaluation rules. Pipes chain left to right:

```
checks | filter(!it.ok) | map(sprintf("- %s: %s", it.name, it.msg)) | join("\n")
// identical to:
join(map(filter(checks, !it.ok), sprintf("- %s: %s", it.name, it.msg)), "\n")
```

The pipe is a deliberate deviation from expr's strict-Go-subset
identity: in Go this token means bitwise or. Before v1.2.0, expr
rejected `|` at Compile time as an unsupported bitwise operator, so no
previously-compilable expression changes meaning under the pipeline
reading. The full design rationale lives in
[RFC 0001](../rfcs/0001-pipe-operator.md).

The right-hand side must be written as a call. Anything else (an
identifier, a selector, an index, an optional access, a parenthesized
expression, a literal) is rejected at Compile time with `ErrCompile`:

```
xs | len() // ok: len(xs)
xs | f // ErrCompile: "f" is not a call (did you mean to write f(...)?)
xs | filter // ErrCompile: "filter" is a special form, did you mean
// to write filter(predicate)?
xs | f()[0] // ErrCompile: the right side parses as the index f()[0]
(xs | f())[0] // ok: index the piped result
xs | a?.b // ErrCompile: optional access is not a call
```

Because the rewrite is purely syntactic, it composes with every call
shape: the iterating forms (`xs | filter(it > 0)` is
`filter(xs, it > 0)`), the three-arg named-binding forms
(`orders | filter(o, o.paid)`), the lazy forms (`v | try(fallback)` is
`try(v, fallback)`, with `v` still evaluated under try's error
handling), env-provided callables, and selector calls on env values.
`map` and `if` work even though they are Go keywords; the keyword
rewrite runs before parsing.

Pipe targets follow the same resolution rules as any other call: an
unregistered name compiles (the env may provide the callable at Run)
and fails at evaluation with the usual unknown-function error.

**Precedence.** `|` keeps Go's precedence: level 4, the same as `+`
and `-`. That is tighter than comparisons, `&&`, and `||`, and looser
than `*`, `/`, `%`, and unary operators. Postfix syntax (calls, `.field`,
`[idx]`, `?.`, `?[`) binds tighter still. Consequences:

```
xs | count(it > 1) == 2 // (xs | count(it > 1)) == 2: pipe, then compare
n + 1 | double() // (n + 1) | double(): same level, left-assoc
n | double() + 1 // (n | double()) + 1
ok && xs | any(it > 0) // ok && (xs | any(it > 0))
x > 2 | if("big", "small") // ErrCompile: ambiguous, see below
(x > 2) | if("big", "small") // parenthesize to pipe a comparison
```

One shape is rejected outright rather than silently mis-grouping: a
bare pipe as the **right** operand of a comparison. `a == b | f()`
parses as `a == (b | f())`, with the pipe consuming the comparison's
right operand, so expr fails compilation with an "ambiguous expression"
error demanding parentheses; write `(a | f()) == b` or `a == (b | f())`
to state which one you meant. A pipe on the *left* of a comparison
(`xs | count(it > 1) == 2`) is the useful, unambiguous order and needs
no parentheses.

**Optional access.** An optional-access result pipes normally
(`user?.name | upper()` is `upper(user?.name)`; a nil receiver pipes
`nil` into the call, which the iterating forms treat as an empty
list). The reverse needs parentheses: `?.` binds tighter than `|`, so
accessing a field on a piped result is written
`(xs | find(it.ok))?.name`.

### Unary

- `!x` is logical negation using truthiness (so `!0` is `true`).
Expand Down Expand Up @@ -743,7 +823,8 @@ Only these `ast.Expr` node kinds are accepted; everything else returns
- `*ast.Ident` — identifiers
- `*ast.ParenExpr` — `( x )`
- `*ast.UnaryExpr` — `!x`, `-x`, `+x`
- `*ast.BinaryExpr` — arithmetic, comparison, logical
- `*ast.BinaryExpr` — arithmetic, comparison, logical, pipeline
(`a | f(x)`, desugared to `f(a, x)` at Compile time)
- `*ast.SelectorExpr` — `x.y`
- `*ast.IndexExpr` — `x[i]`
- `*ast.CallExpr` — `f(a, b, ...)`
Expand All @@ -760,7 +841,9 @@ with `ErrCompile`):
- Function literals (`func() {}`)
- Channel ops (`<-ch`, `ch <- v`)
- Pointer/address ops (`*x`, `&x`)
- Bitwise operators (`& | ^ << >> &^`)
- Bitwise operators (`& ^ << >> &^`). The `|` token is the
[pipeline operator](#pipeline-); a `|` whose right side is not a
call is rejected at Compile time.
- Imaginary number literals (`1i`)
- Spread call arguments (`f(xs...)`)
- Label and selector type names (`pkg.Type`)
Expand Down
11 changes: 9 additions & 2 deletions docs/rfcs/0001-pipe-operator.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
# RFC 0001: Pipe Operator (`|`)

**Status:** Draft
**Status:** Implemented (always on)
**Date:** 2026-06-12
**No implementation commitment has been made. This document exists to think the design through.**
**Implementation notes:** The desugar, the non-call right-side errors,
and the ambiguous-comparison diagnostic (§3.3) shipped as recommended.
The opt-in recommendation (§7.3) was overridden: the default-on
question (§9.2 step 5) was resolved in favor of enabling the pipe
unconditionally, since `|` never compiled before and the token reuse
therefore breaks no existing expression. Normative language
documentation lives in [the spec](../reference/spec.md); this document
records the design rationale.

---

Expand Down
37 changes: 37 additions & 0 deletions docs_examples_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -479,3 +479,40 @@ func TestDocsExample10_GuardedMathAndGroups(t *testing.T) {
}
assertDeepEqual(t, got, want)
}

// Example 13: Pipelines.
func TestDocsExample13_Pipelines(t *testing.T) {
src := `events |
filter(it.kind == "error") |
map(sprintf("[%s] %s", it.source, it.message)) |
join("\n")`
env := map[string]any{
"events": []any{
map[string]any{"kind": "error", "source": "api", "message": "timeout"},
map[string]any{"kind": "info", "source": "api", "message": "ok"},
map[string]any{"kind": "error", "source": "db", "message": "deadlock"},
},
}
opts := []Option{WithBuiltins(), WithFunctions(StringFuncs())}
got := runDocExample(t, src, env, opts...)
assertDeepEqual(t, got, "[api] timeout\n[db] deadlock")

// The doc claims the pipe is sugar for the nested call form.
nested := runDocExample(t,
`join(map(filter(events, it.kind == "error"), sprintf("[%s] %s", it.source, it.message)), "\n")`,
env, opts...)
assertDeepEqual(t, got, nested)

// The doc claims a bare (non-call) right side is a compile error.
if _, err := Compile(`xs | len`, WithBuiltins()); err == nil {
t.Fatal("expected non-call pipe right side to fail compilation")
}

// The doc claims "pipe first, compare after": the left-of-comparison
// order works, the right-of-comparison order is a compile error.
got = runDocExample(t, `events | count(it.kind == "error") == 2`, env, opts...)
assertDeepEqual(t, got, true)
if _, err := Compile(`a == b | upper()`, opts...); err == nil {
t.Fatal("expected ambiguous pipe-right-of-comparison to fail compilation")
}
}
30 changes: 30 additions & 0 deletions docs_guides_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -701,3 +701,33 @@ func TestGuide_Templates_ErrorsReportLineColumn(t *testing.T) {
t.Fatalf("expected line:col in error, got %v", rerr)
}
}

// --- higher-order-patterns.md: pipelines --------------------------------------

func TestGuide_HigherOrder_Pipelines(t *testing.T) {
// higher-order-patterns.md: `a | f(x)` is exactly `f(a, x)`, for
// both the two-arg and named-binding shapes.
orders := []any{
map[string]any{"id": "a-1", "status": "paid", "total": int64(50)},
map[string]any{"id": "b-2", "status": "open", "total": int64(10)},
map[string]any{"id": "c-3", "status": "paid", "total": int64(70)},
}
env := map[string]any{"orders": orders}
opts := []Option{WithBuiltins()}

piped := runGuide(t, `orders | filter(it.status == "paid") | map(it.id)`, env, opts...)
nested := runGuide(t, `map(filter(orders, it.status == "paid"), it.id)`, env, opts...)
assertDeepEqual(t, piped, nested)
assertDeepEqual(t, piped, []any{"a-1", "c-3"})

named := runGuide(t, `orders | filter(o, o.status == "paid") | map(o, {o.id: o.total})`, env, opts...)
assertDeepEqual(t, named, []any{
map[string]any{"a-1": int64(50)},
map[string]any{"c-3": int64(70)},
})

// The guide claims a bare (non-call) right side is a compile error.
if _, err := Compile(`xs | trim`, opts...); err == nil {
t.Fatal("expected non-call pipe right side to fail compilation")
}
}
10 changes: 9 additions & 1 deletion engine.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
// go/parser. It accepts the subset of Go expression syntax useful for
// conditions, templates, and parameter interpolation: identifiers,
// selectors, index expressions, arithmetic, comparisons, logical
// operators, and calls to registered functions.
// operators, calls to registered functions, and pipelines
// (`a | f(x)` compiles as `f(a, x)`).
//
// expr is intentionally small and adds no external dependencies.
//
Expand Down Expand Up @@ -216,6 +217,13 @@ func compileWithConfig(code string, cfg *compileConfig) (*Program, error) {
if err != nil {
return nil, fmt.Errorf("%w: %v", ErrCompile, err)
}
// Desugar pipe expressions (`a | f(x)` → `f(a, x)`) before
// validation so validate and the evaluator only ever see ordinary
// call nodes. See pipe.go.
node, err = desugarPipes(fset, node)
if err != nil {
return nil, err
}
if err := validate(fset, node); err != nil {
return nil, err
}
Expand Down
12 changes: 12 additions & 0 deletions fuzz_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,18 @@ var fuzzCorpus = []string{
"?.x",
"a?.",
"a?[",
// pipe operator
"state.items | len()",
`state.name | upper() | lower()`,
"state.items | filter(it > 1) | map(it * 2)",
"state.user?.name | upper()",
`"a|b" | upper()`,
"1 | 2",
"state.items | foo",
"state.items | a?.b",
"state.items | (len())",
"state.items | len() > 2",
"1 + 2 | len()",
}

// fuzzEnv is the environment FuzzEval runs every mutated expression
Expand Down
Loading
Loading