fix: keep timezone-aware datetimes aware and UTC-normalise naive coercion#76
Open
simozzy wants to merge 1 commit into
Open
fix: keep timezone-aware datetimes aware and UTC-normalise naive coercion#76simozzy wants to merge 1 commit into
simozzy wants to merge 1 commit into
Conversation
…cion DatabendDateTime.result_processor unconditionally did value.replace(tzinfo=None), which has two problems: - It strips tzinfo even for timezone=True columns, so a DateTime(timezone=True) / TIMESTAMP WITH TIME ZONE column loses its awareness. - It drops tzinfo without converting to UTC first, so a tz-aware value carrying a non-UTC offset yields the wrong naive wall-clock. Only coerce to naive for timezone=False columns, and convert to UTC before dropping tzinfo so the wall-clock is correct regardless of the driver's offset; timezone=True columns are returned unchanged. None and string parsing are preserved. Adds a server-less unit test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
DatabendDateTime.result_processorcurrently coerces every non-string value with an unconditionalvalue.replace(tzinfo=None):The driver returns tz-aware (UTC) datetimes, so this does fix the naive-column case — but it has two side effects:
timezone=Truecolumns. ADateTime(timezone=True)/TIMESTAMP WITH TIME ZONEcolumn loses its tzinfo, even thoughself.timezoneis available (inherited fromsqltypes.DATETIME) to distinguish the two cases.replace(tzinfo=None)truncates the offset instead of converting it — yielding the wrong naive wall-clock (e.g. a+02:00value comes back two hours off).Fix
timezone=Falsecolumns still yield naive values (so the compliance suite's naive round-trips hold), but the value is normalised to UTC before tzinfo is dropped, so the wall-clock is correct regardless of the driver's offset.timezone=Truecolumns are returned unchanged, keeping their awareness.Noneand string inputs are handled exactly as before.Tests
tests/unit/test_databend_dialect.py— server-less, callsresult_processordirectly:timezone=Falsecolumn;+02:00) input is normalised to UTC, not merely truncated;Noneis preserved, strings still parse;timezone=Truecolumn keeps its tzinfo.