fix(e6): gate TO_UNIX_TIMESTAMP /1000 on executor type#253
Merged
gauravdawar-e6 merged 1 commit intoe6data:mainfrom Apr 23, 2026
Merged
Conversation
The native executor already returns seconds (matching Databricks), while the Java executor returns milliseconds. The transpiler unconditionally divided by 1000, producing wrong results on NE. Introduce E6_EXECUTOR_TYPE env flag (default: "java"): - "java" -> TO_UNIX_TIMESTAMP(...) / 1000 - "native" -> TO_UNIX_TIMESTAMP(...) Applied to both the no-format and PARSE_DATETIME code paths. Flag is also mirrored at module scope in converter_api.py for startup logging, consistent with FIX_QUOTE_ESCAPES.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TO_UNIX_TIMESTAMP(...)by 1000; native executor already returns seconds (matches Databricks) so the division was producing wrong results on NE.E6_EXECUTOR_TYPEenv flag (defaultjava) to pick the right behavior per deployment:java→TO_UNIX_TIMESTAMP(...) / 1000native→TO_UNIX_TIMESTAMP(...)PARSE_DATETIME(format) branch ofto_unix_timestamp_sql.converter_api.pyand adds it to the startup env-flag log line, matching theFIX_QUOTE_ESCAPESpattern.Context
Flagged in Slack from planner logs — the transpiled query was emitting
TO_UNIX_TIMESTAMP(CAST(... AS TIMESTAMP)) / 1000 + CAST(hour24 AS BIGINT) * 3600against NE, where the first term ended up in µs-scale because NE already returns seconds. Rohit confirmed the original/1000was added for the Java executor and a flag was needed to toggle per executor.Test plan
python -m unittest tests.dialects.test_e6— all 49 tests passtest_unix_timestamp_native_executorcovers native (no-format + format branches) and java defaultnative/java/NATIVE/garbage — flag is read live inside the generator method