Skip to content

python(feat): ENG-16878: processed channels - #881

Closed
David-sift wants to merge 1 commit into
mainfrom
eng-16878-python-client-work-for-uploading-processed
Closed

David-sift wants to merge 1 commit into
mainfrom
eng-16878-python-client-work-for-uploading-processed

Conversation

@David-sift

@David-sift David-sift commented Nov 11, 2026 •

Copy link
Copy Markdown

🎫 ENG-16878 🎫

Description

Adds client.processed_channel, which uploads a DataFrame to a run as processed channels. Each column other than the time becomes one channel on the asset.

client.processed_channel.upload(df, asset="satellite-01", run=run, units={"battery_health": "pct"})
  • By default the call blocks until the data is ingested. wait=False returns a handle with wait_until_complete().
  • Times come from the DataFrame's DatetimeIndex, or from a column named with time_column=.
  • code defaults to the calling script's path, its git commit and a hash of the file.
  • list_writes() and get_write() read the write history.
  • Server refusals raise ProcessedChannelRequestError, which carries the gRPC code and the server's message. Writes that fail ingestion raise ProcessedChannelUploadError.

The API is experimental and raises a SiftExperimentalWarning. The service's proto is still marked unstable and isn't published to this repo, so the client calls its REST endpoints. This is the first sift_client API backed by an unstable proto. Once the proto is published, moving to gRPC stubs only touches ProcessedChannelsLowLevelClient.

Requirements:

  • The parquet extra (sift-stack-py[parquet]).
  • The organization's processed-channels flag is enabled

Verification

  • just test: 1716 passed. The one failure, sift_py/data_import/_csv_test.py::test_simple_upload_invalid_csv, also fails on main.
  • New unit tests cover the Parquet writer, the code defaults (including worktrees and packed refs), and the low-level REST mapping and errors. They also cover blocking and non-blocking uploads through the sync client.
  • ruff check, ruff format --check, mypy (including --platform=win32) and pyright are clean, and the stubs are regenerated with just gen-stubs.
  • Against dev with the flag off, the upload is refused with "processed channels are not enabled for this organization", and nothing is created.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Python docs preview: https://sift-stack.github.io/sift/python/pr-881/

Deployed from d1814dd. The link may take up to a minute to become live as GitHub Pages propagates.

@David-sift
David-sift force-pushed the eng-16878-python-client-work-for-uploading-processed branch from 0bfce2f to 448283a Compare November 11, 2026 19:11
@David-sift David-sift closed this Nov 11, 2026
@David-sift
David-sift deleted the eng-16878-python-client-work-for-uploading-processed branch November 11, 2026 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant