Repository navigation
Avoid transporting unused worker artifacts into plate exports - #193
Merged
Merged
Conversation
Derive contract-owned export paths before worker cleanup, retain those lightweight paths with execution results, and let the ZMQ server observe them even when outcome-only retention releases full parent values. Refs #145.
trissim
marked this pull request as ready for review
September 29, 2026 13:35
Keep the typed observation retention policy while reconciling worker execution with current main. Local focused execution and ZMQ tests: 40 passed; Black and PR diff checks passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A 16-well plate export waits after all four workers finish because each lane serializes about 1.54 GB of runtime records, mostly images and object labels that the plate spreadsheet does not consume. The new
MERGE_PLATE_INPUTSmode retains artifact types declared by compiled plate invocations. Explicit full-value observation and analysis consolidation keep full records, and output paths are captured before filtering. Fixes #192.This PR is stacked on #191. It includes the outcome-path preservation commit from #157 as a prerequisite because #191's branch does not contain it; that commit can be dropped from this stack after #157 lands. Benchmarks used the integrated checkout documented in the report, which also contains separate pending performance work from #170 and #175.
All 30 one-well cases succeeded and all 110 generated files matched the control by path and bytes. Both 16-well plate exports matched byte for byte. The exact review branch passed 76 focused unit tests, a real ZMQ full-value export test, and an analysis-consolidation integration test. Its warmed 16-well Colocalization run measured 33.217 s execution and produced the same plate CSV. The report now links the measured four-job native CP 16-well results in #197; its earlier 16× cold-launch projection remains separately labeled.