fix: treat Compression.NONE sentinel as uncompressed in COPY INTO formats#1
Merged
Merged
Conversation
rad-pat
requested changes
Jun 18, 2026
rad-pat
left a comment
Member
There was a problem hiding this comment.
Ok apart from the test which does not follow the Sqlalchemy testing suite pattern
3f5695b to
ada0c50
Compare
Member
Author
|
@rad-pat Reworked the test to follow the SQLAlchemy testing-suite pattern — |
…mats
Compression.NONE is a truthy Enum member, so the `if compression:` guard
in CSVFormat/TSVFormat/NDJSONFormat/ParquetFormat treated the
no-compression sentinel as 'a codec was specified'. For ParquetFormat this
raised TypeError('Compression should be None, ZStd, or Snappy.') outright;
for the others it emitted a spurious COMPRESSION = NONE option.
Guard now skips both None and Compression.NONE, so the uncompressed case
renders no COMPRESSION option. Explicit codecs are unchanged, and
ParquetFormat still rejects codecs its writer can't use (gzip/bz2/zip).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
ada0c50 to
50e7206
Compare
Member
Author
|
@rad-pat ticket logged with databend/sqlaclhemy databendlabs#75 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Compression.NONEis a truthyEnummember, so theif compression:guardin
CSVFormat,TSVFormat,NDJSONFormat, andParquetFormattreats theno-compression sentinel as "a codec was specified":
ParquetFormat(compression=Compression.NONE)raisesTypeError: Compression should be None, ZStd, or Snappy.— even thoughNONEis the "no compression" case.COMPRESSION = NONEoption for the sentinel.Fix
Skip the block for both
Noneand theCompression.NONEsentinel, so theuncompressed case renders no
COMPRESSIONoption:Applied to all four format classes. Explicit codecs are unchanged, and
ParquetFormatstill rejects codecs its writer can't use (gzip/bz2/zip).Tests
tests/test_copy_format.py— constructor-level (no DB):NoneandCompression.NONEemit noCOMPRESSIONoption, across all four classes.ParquetFormatstill raisesTypeErrorfor unsupported codecs.Context / downstream
Found in PlaidCloud's data-export path: every Parquet export with the default
(no) compression failed with this
TypeError. Tracked there as:workaround; this is the root-cause fix.
Note: this fork's
mainis at0.5.1, while PlaidCloud pins PyPIdatabend-sqlalchemy==0.5.5. To consume this fix, plaid will either re-pin atthe fork or this should be upstreamed to
databendlabs/databend-sqlalchemy.🤖 Generated with Claude Code