perf(runtime): add typed columnar aggregate reducers#57
Conversation
d99a12d to
daa040a
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d99a12df4a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| out.selected[i + j] = 1; | ||
| out.indices.push_back(i + j); | ||
| ++out.selected_count; | ||
| if (bounded && out.selected_count >= max_selected) goto scalar_tail; |
There was a problem hiding this comment.
Advance SIMD tail after bounded vector hit
On AVX2 builds, if a bounded selection (for example WHERE ... LIMIT n via combinedFilterSelection(..., max_selected)) reaches max_selected inside this vector loop, goto scalar_tail leaves i at the start of the same vector block. The scalar tail then reprocesses lanes that were already emitted and appends duplicate indices (e.g. LIMIT 1 can produce [0,0]), so filterTable can duplicate rows and exceed the requested limit; return immediately or advance the tail past the processed lane/block.
Useful? React with 👍 / 👎.
| auto& col = cache->columns[ci]; | ||
| for (auto& v : col.values) { | ||
| if (v.isNull()) continue; | ||
| v = Value(v.asFloat()); |
There was a problem hiding this comment.
Keep Float32 values serializable
When a SQL FLOAT/REAL table receives rows, this rewrites cells to the new DataType::Float32, but the existing table codecs were not updated for that type: BinaryRowBatchCodec falls through and deserializes it as a string, ProtoLikeSerializer decodes type 6 as null, and nanoarrow materialization infers string then calls asString(). Any inserted float column crossing RPC/plan serialization/export boundaries is therefore corrupted or throws, so either handle Float32 in those codecs or avoid emitting it here.
Useful? React with 👍 / 👎.
daa040a to
c40379d
Compare
Use state-columnar typed reducer paths for single- and two-INT64 SUM/COUNT/AVG aggregate shapes, and classify single-key predicate source COUNT/SUM/AVG pushdown onto typed shapes with CSV predicate evaluation support. Add focused optimizer/file-source coverage, benchmark scenarios, columnar-first design notes, and Mobius delivery evidence. Validation: bazel test //:core_regression --test_output=errors; bazel test //:experimental_regression --test_output=errors; bazel run //:batch_aggregate_benchmark -- 1048576 5; bazel run //:file_source_benchmark -- 200000 3; bazel run //:string_builtin_benchmark -- 100000 5; bazel build //:sql_demo //:df_demo //:stream_demo; bazel run //:actor_rpc_smoke; git diff --check; targeted secret scan.
c40379d to
d65e3c3
Compare
Summary
ColumnarExecBatch,ColumnarExecColumn, andColumnarExecViewPerformance Evidence
mixed-string-int64regressed to 634 msValidation