Skip to content

quote handling#19

Merged
gkennos merged 4 commits into
mainfrom
quote_handler
Jul 5, 2026
Merged

quote handling#19
gkennos merged 4 commits into
mainfrom
quote_handler

Conversation

@gkennos

@gkennos gkennos commented Jul 5, 2026

Copy link
Copy Markdown
Member

to fix #16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to address issue #16 (“improper quotation handling for athena files”) by making quote handling explicit/consistent across the PostgreSQL COPY fast-path and the pandas fallback, and by adding tests that exercise literal vs CSV quoting behavior.

Changes:

  • Added resolve_quote_mode() and used it in both quick_load_pg() and the pandas fallback loader to ensure consistent parsing behavior.
  • Expanded test coverage for quote-mode propagation and delimiter-derived quote-mode selection.
  • Bumped project version to 0.5.2 and updated the pyarrow minimum requirement / lockfile.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/orm_loader/loaders/loading_helpers.py Adds resolve_quote_mode() and routes COPY quote behavior through it; increases infer sampling size.
src/orm_loader/loaders/loader_interface.py Ensures pandas fallback honors resolved quote mode via quoting=.
tests/loaders/test_helpers.py Adds unit tests for resolve_quote_mode() and existing sniffing behavior.
tests/loaders/test_pandas_loader.py Adds integration tests proving quote_mode affects pandas parsing and delimiter-derived behavior.
pyproject.toml Version bump and dependency floor update for pyarrow.
uv.lock Lockfile updates reflecting version/dependency changes (including pyarrow resolution).
CHANGELOG.md Adds a 0.5.2 entry for quote-handling changes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/orm_loader/loaders/loading_helpers.py
Comment thread src/orm_loader/loaders/loading_helpers.py Outdated
Comment thread CHANGELOG.md
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
@gkennos gkennos merged commit fe978a0 into main Jul 5, 2026
3 checks passed
@gkennos gkennos deleted the quote_handler branch July 5, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

improper quotation handling for athena files

2 participants