GH-31464: [C++][Python] Fix string filters with empty value sets - #50882
Draft
fallenmi wants to merge 1 commit into
Draft
GH-31464: [C++][Python] Fix string filters with empty value sets#50882fallenmi wants to merge 1 commit into
fallenmi wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
field.isin([])creates an empty null-typed value set. String inputs currently fail the string-specific type check before that empty set can be safely cast, so filtering string and dictionary-encoded string columns raisesArrowTypeErrorinstead of returning no rows.What changes are included in this PR?
Allow only zero-length null-typed value sets through the string type check so the existing safe-cast path can handle them. The existing rejection of non-empty non-binary value sets is preserved.
Add C++ coverage for
is_inandindex_inwithstringandlarge_string, and Python coverage for string and dictionary-encoded string table filters.Are these changes tested?
Yes:
pyarrow 26.0.0.dev131: string and dictionary-string filters raised the reported type error.arrow-compute-scalar-utility-test: 95 passed.git diff --check: passed.Are there any user-facing changes?
Yes. Filtering string or dictionary-encoded string data with an empty set now returns an empty result instead of raising
ArrowTypeError. There is no public API change.AI usage
OpenAI Codex assisted with reproducing the issue, tracing the type check, and drafting the patch, tests, and PR description. This draft remains pending my line-by-line review; I will only mark it ready after I understand and verify every change.