Repository navigation
all(feat): support custom time formats on CSV and Parquet imports - #884
Merged
Merged
Conversation
Contributor
|
Python docs preview: https://sift-stack.github.io/sift/python/pr-884/ Deployed from |
omar-sift
force-pushed
the
sift/support-custom-date-formats
branch
5 times, most recently
from
November 12, 2026 19:48
75c960c to
a2c6a8d
Compare
omar-sift
marked this pull request as ready for review
November 12, 2026 21:06
omar-sift
force-pushed
the
sift/support-custom-date-formats
branch
from
November 12, 2026 21:47
a2c6a8d to
8e3b491
Compare
omar-sift
marked this pull request as draft
November 13, 2026 16:53
omar-sift
force-pushed
the
sift/support-custom-date-formats
branch
from
November 14, 2026 01:10
8e3b491 to
968af4a
Compare
omar-sift
marked this pull request as ready for review
November 14, 2026 01:31
omar-sift
force-pushed
the
sift/support-custom-date-formats
branch
from
November 16, 2026 23:47
968af4a to
86b6ceb
Compare
wei-sift
requested changes
Nov 17, 2026
omar-sift
force-pushed
the
sift/support-custom-date-formats
branch
from
November 17, 2026 21:45
dd7ebd3 to
89aaf3e
Compare
wei-sift
approved these changes
Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CSV and Parquet imports can now read timestamps in a custom Unix date format. The built-in formats only cover fixed layouts, so a file with month-first timestamps like
10-29-2026 16:53:45could not be imported.TimeFormat.CUSTOMandcustom_format: new onCsvTimeColumnandParquetTimeColumn.custom_formatdefaults to"", so existing configs are unchanged, and it is kept when a config is read back from the server. Importing withTimeFormat.CUSTOMand an emptycustom_formatraisesValueErrorbefore upload. Times without an offset are read as UTC; zone names (%Z) are rejected, so use%zor%:z.sift-cli import csvandsift-cli import parquet(flat-dataset,cpr single,cpr multi):--time-format customwith--time-format-str "%m-%d-%Y %H:%M:%S".customwithout--time-format-strfails before upload.TimeFormat.CUSTOMand--time-format customare rejected, since those configs have no format string field.TIME_FORMAT_CUSTOMandcustom_formatcome from the public protos update in sift(chore): update public protos #890. This PR does not change protos or generated code.Merge after the server side of this change is deployed. Older servers ignore
custom_format.Merge after the server side (azimuth #16601, merged) is deployed. Older servers ignore
custom_format.Verification
sift_client:test_data_imports.py, 167 passed against a local server running azimuth main, including CSV and Parquet round trips, the empty default, CUSTOM withoutcustom_formatrejected at import, and TDMS and HDF5 rejecting CUSTOM. ruff and pyright pass.sift-cli:cargo test -p sift_cli, 289 passed; clippy clean. Ran the built binary with--preview: CSV, Parquetflat-datasetandcpr singleacceptcustomwith--time-format-strand error without it;import tdmsandimport hdf5rejectcustom.custom_format="%m-%d-%Y %H:%M:%S"read back 120 points starting at2026-10-29T16:53:45Z. Parquet imports (flat dataset and single channel per row) with the same format were verified before the field was renamed tocustom_format; the field number is unchanged, so the wire format is the same. A mismatchedcustom_formatfails with an error naming the value and format, and existing formats import unchanged.