Skip to content

GH-31464: [C++][Python] Fix string filters with empty value sets - #50882

Draft
fallenmi wants to merge 1 commit into
apache:mainfrom
fallenmi:agent/fix-empty-string-set-filter
Draft

GH-31464: [C++][Python] Fix string filters with empty value sets#50882
fallenmi wants to merge 1 commit into
apache:mainfrom
fallenmi:agent/fix-empty-string-set-filter

Conversation

@fallenmi

Copy link
Copy Markdown

Rationale for this change

field.isin([]) creates an empty null-typed value set. String inputs currently fail the string-specific type check before that empty set can be safely cast, so filtering string and dictionary-encoded string columns raises ArrowTypeError instead of returning no rows.

What changes are included in this PR?

Allow only zero-length null-typed value sets through the string type check so the existing safe-cast path can handle them. The existing rejection of non-empty non-binary value sets is preserved.

Add C++ coverage for is_in and index_in with string and large_string, and Python coverage for string and dictionary-encoded string table filters.

Are these changes tested?

Yes:

  • Reproduced on pyarrow 26.0.0.dev131: string and dictionary-string filters raised the reported type error.
  • Full arrow-compute-scalar-utility-test: 95 passed.
  • Focused PyArrow compute tests: 4 passed.
  • C++ format/lint and Python format/lint hooks for the changed files: passed.
  • git diff --check: passed.

Are there any user-facing changes?

Yes. Filtering string or dictionary-encoded string data with an empty set now returns an empty result instead of raising ArrowTypeError. There is no public API change.

AI usage

OpenAI Codex assisted with reproducing the issue, tracing the type check, and drafting the patch, tests, and PR description. This draft remains pending my line-by-line review; I will only mark it ready after I understand and verify every change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant