diff --git a/benchmark/results/perf_coloc_step_cache_20260929/README.md b/benchmark/results/perf_coloc_step_cache_20260929/README.md new file mode 100644 index 0000000000..882b776ece --- /dev/null +++ b/benchmark/results/perf_coloc_step_cache_20260929/README.md @@ -0,0 +1,44 @@ +# Step-scoped Costes threshold reuse: official30 1w_1t (2026-09-29) + +This report accompanies [issue #185](https://github.com/OpenHCSDev/openhcs/issues/185). It compares the same CPU-only integration code with and without a step-scoped Costes threshold cache. The control is commit `30e1e905c`, including the faster completion polling from [PR #183](https://github.com/OpenHCSDev/openhcs/pull/183). Each benchmark observation owns a fresh ZMQ execution server and uses one well and one OpenHCS thread with the `fork` start method. `total_seconds` includes server startup, compilation, execution, and shutdown; `execute_seconds` is the server execution job and can include first-use JIT work. The native CellProfiler 4.2.8.1 reference is retained in the [native summary](../perf_lazy_catalog_20260929/native_cp_summary.csv). + +An instrumentation-only probe on the two segmentation tutorials found **180 object-colocalization calls but only 40 distinct selected image-pair pixel inputs**: 20 pairs repeated five times and 20 repeated four times across object sets. The image payloads are reconstructed between sets, so object identity cannot establish reuse. The candidate fingerprints the exact selected pixels and keeps method, scale, and backend selection in its key. The cache lives only for one FunctionStep invocation. + +An interleaved control/candidate/candidate/control experiment measured the two affected pipelines. The first control had a cold compilation run and is retained as raw data but excluded from the warmed comparison. The later control and the mean of both candidates give: + +| Pipeline | Control execution (s) | Candidate execution (s) | Execution saved (s) | Control total (s) | Candidate total (s) | +| --- | ---: | ---: | ---: | ---: | ---: | +| Advanced segmentation | 10.815 | 9.384 | 1.431 | 18.642 | 17.286 | +| Beginner segmentation | 6.778 | 5.607 | 1.171 | 13.362 | 12.060 | +| **Both** | **17.593** | **14.991** | **2.602** | **32.004** | **29.347** | + +All 15 produced result files in both candidate runs matched the control byte-for-byte. A separate instrumented run confirmed the mechanism: `coloc_object_costes_thresholds` fell from **3.999 to 1.400 seconds** over 180 calls, and `coloc_object_total` fell from **5.217 to 2.620 seconds**. These are phase sums from separate profiled runs, not additional time to subtract from the benchmark. + +![Native CellProfiler and OpenHCS execution time for the affected tutorials](affected_case_execution.png) + +The full official30 sweeps showed material run-order variation, including in cases this cache cannot affect. All 30 cases succeeded in every sweep; the two warmed sweeps produced the same 110 result paths with byte-identical contents. + +| Sweep order | Compile sum (s) | Execution sum (s) | Total sum (s) | +| --- | ---: | ---: | ---: | +| First candidate | 84.079 | 97.248 | 270.468 | +| First control, cold checkout | 99.153 | 100.178 | 288.514 | +| Warmed control | 83.701 | 89.286 | 261.691 | +| Warmed candidate | 84.095 | 87.857 | 260.928 | + +The warmed candidate saved **1.429 seconds in summed execution** and **0.763 seconds in summed total** against the immediately preceding warmed control. The two affected tutorials saved 2.654 seconds in that full-suite comparison; unrelated cases varied in the other direction. The pair-specific improvement and its profiled mechanism are reproducible. The small full-suite net difference is not a stable estimate of a global speedup. It is also much smaller than the remaining end-to-end startup and compilation terms, so this cache is a targeted improvement, not the dominant route to faster official30 total time. + +![Per-case execution changes in the warmed full sweep](full_suite_execution_delta.png) + +The warmed full-sweep native execution ratios for the affected tutorials are 3.34× to 3.84× for advanced segmentation and 2.37× to 2.90× for beginner segmentation. These use the saved native execution reference and OpenHCS's `execute_seconds`; they do not include either system's total startup boundary. + +The retained CSVs are `abba_*.csv`, `full_first_*.csv`, and `full_warm_*.csv`. `plot.py` regenerates both figures. Raw result trees and profiling events are under `/home/ts/code/projects/openhcs-benchmark-runs/perf-coloc-step-cache-*20260929`. Validation also included 624 focused unit tests and byte parity in the full sweep. The separate [batch executor metadata issue #176](https://github.com/OpenHCSDev/openhcs/issues/176) remains open; this cache does not restore those declarations. + +Reproduce the fresh-server sweep with the native reference: + +```bash +OPENHCS_CPU_ONLY=true .venv/bin/python scripts/benchmark_cppipe_well_throughput.py \ + --manifest benchmark/manifests/official30_portable_axis1.json \ + --output-dir /path/to/output \ + --native-summary-csv benchmark/results/perf_lazy_catalog_20260929/native_cp_summary.csv \ + --mode 1w_1t +``` diff --git a/benchmark/results/perf_coloc_step_cache_20260929/abba_1_control.csv b/benchmark/results/perf_coloc_step_cache_20260929/abba_1_control.csv new file mode 100644 index 0000000000..98b1247997 --- /dev/null +++ b/benchmark/results/perf_coloc_step_cache_20260929/abba_1_control.csv @@ -0,0 +1,3 @@ +case_name,mode_name,worker_count,well_count,compile_seconds,prepare_seconds,execute_seconds,total_seconds,wells_per_second,successful_wells,native_single_sample_execution_seconds,projected_native_execution_seconds,projected_execution_speedup,peak_memory_mb,status,memory_limit_mb,error_message,execution_route,server_lifecycle,run_input_sha256 +cp_tutorial_advanced_segmentation_final,1w_1t,1,1,50.092376708984375,0.0,12.277675151824951,66.6460295310244,0.08144864460364552,1,36.253466514870524,36.253466514870524,2.9527957098198527,1523.5390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,45578cef16d0f9006b11f3172f9a33471622f9c1ab3a461e2dd3316561930482 +cp_tutorial_beginner_segmentation_final,1w_1t,1,1,3.277191638946533,0.0,7.9185473918914795,14.549100143834949,0.1262857883535547,1,16.074153264053166,16.074153264053166,2.029937117066819,1014.4140625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,45578cef16d0f9006b11f3172f9a33471622f9c1ab3a461e2dd3316561930482 diff --git a/benchmark/results/perf_coloc_step_cache_20260929/abba_2_candidate.csv b/benchmark/results/perf_coloc_step_cache_20260929/abba_2_candidate.csv new file mode 100644 index 0000000000..a0b3214e5f --- /dev/null +++ b/benchmark/results/perf_coloc_step_cache_20260929/abba_2_candidate.csv @@ -0,0 +1,3 @@ +case_name,mode_name,worker_count,well_count,compile_seconds,prepare_seconds,execute_seconds,total_seconds,wells_per_second,successful_wells,native_single_sample_execution_seconds,projected_native_execution_seconds,projected_execution_speedup,peak_memory_mb,status,memory_limit_mb,error_message,execution_route,server_lifecycle,run_input_sha256 +cp_tutorial_advanced_segmentation_final,1w_1t,1,1,4.238177299499512,0.0,9.434734344482422,17.30373717378825,0.10599132561531162,1,36.253466514870524,36.253466514870524,3.842552974061438,1089.796875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,56a334216fae7002dfa87fe7431ab12e1df13742d256f30887eb2a2446bd4041 +cp_tutorial_beginner_segmentation_final,1w_1t,1,1,3.1875345706939697,0.0,5.5730133056640625,12.019987728912383,0.17943614076493636,1,16.074153264053166,16.074153264053166,2.8842840277658053,960.5625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,56a334216fae7002dfa87fe7431ab12e1df13742d256f30887eb2a2446bd4041 diff --git a/benchmark/results/perf_coloc_step_cache_20260929/abba_3_candidate.csv b/benchmark/results/perf_coloc_step_cache_20260929/abba_3_candidate.csv new file mode 100644 index 0000000000..4450c36292 --- /dev/null +++ b/benchmark/results/perf_coloc_step_cache_20260929/abba_3_candidate.csv @@ -0,0 +1,3 @@ +case_name,mode_name,worker_count,well_count,compile_seconds,prepare_seconds,execute_seconds,total_seconds,wells_per_second,successful_wells,native_single_sample_execution_seconds,projected_native_execution_seconds,projected_execution_speedup,peak_memory_mb,status,memory_limit_mb,error_message,execution_route,server_lifecycle,run_input_sha256 +cp_tutorial_advanced_segmentation_final,1w_1t,1,1,4.3331286907196045,0.0,9.333743572235107,17.268264922779053,0.1071381479746968,1,36.253466514870524,36.253466514870524,3.8841292600659134,1090.54296875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,56a334216fae7002dfa87fe7431ab12e1df13742d256f30887eb2a2446bd4041 +cp_tutorial_beginner_segmentation_final,1w_1t,1,1,3.1799824237823486,0.0,5.640503883361816,12.100588019937277,0.17728912534743022,1,16.074153264053166,16.074153264053166,2.8497725728845262,977.95703125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,56a334216fae7002dfa87fe7431ab12e1df13742d256f30887eb2a2446bd4041 diff --git a/benchmark/results/perf_coloc_step_cache_20260929/abba_4_control.csv b/benchmark/results/perf_coloc_step_cache_20260929/abba_4_control.csv new file mode 100644 index 0000000000..b7a0888067 --- /dev/null +++ b/benchmark/results/perf_coloc_step_cache_20260929/abba_4_control.csv @@ -0,0 +1,3 @@ +case_name,mode_name,worker_count,well_count,compile_seconds,prepare_seconds,execute_seconds,total_seconds,wells_per_second,successful_wells,native_single_sample_execution_seconds,projected_native_execution_seconds,projected_execution_speedup,peak_memory_mb,status,memory_limit_mb,error_message,execution_route,server_lifecycle,run_input_sha256 +cp_tutorial_advanced_segmentation_final,1w_1t,1,1,4.226243019104004,0.0,10.815070152282715,18.641795020084828,0.09246357036241064,1,36.253466514870524,36.253466514870524,3.352124951979029,1088.5390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,45578cef16d0f9006b11f3172f9a33471622f9c1ab3a461e2dd3316561930482 +cp_tutorial_beginner_segmentation_final,1w_1t,1,1,3.3164355754852295,0.0,6.7775774002075195,13.362209782004356,0.1475453456229628,1,16.074153264053166,16.074153264053166,2.3716664989411997,957.03125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,45578cef16d0f9006b11f3172f9a33471622f9c1ab3a461e2dd3316561930482 diff --git a/benchmark/results/perf_coloc_step_cache_20260929/affected_case_execution.png b/benchmark/results/perf_coloc_step_cache_20260929/affected_case_execution.png new file mode 100644 index 0000000000..4f2c929e6a Binary files /dev/null and b/benchmark/results/perf_coloc_step_cache_20260929/affected_case_execution.png differ diff --git a/benchmark/results/perf_coloc_step_cache_20260929/full_first_candidate.csv b/benchmark/results/perf_coloc_step_cache_20260929/full_first_candidate.csv new file mode 100644 index 0000000000..e42c59ac6d --- /dev/null +++ b/benchmark/results/perf_coloc_step_cache_20260929/full_first_candidate.csv @@ -0,0 +1,31 @@ +case_name,mode_name,worker_count,well_count,compile_seconds,prepare_seconds,execute_seconds,total_seconds,wells_per_second,successful_wells,native_single_sample_execution_seconds,projected_native_execution_seconds,projected_execution_speedup,peak_memory_mb,status,memory_limit_mb,error_message,execution_route,server_lifecycle,run_input_sha256 +ExampleColocalization,1w_1t,1,1,3.3314623832702637,0.0,4.047102689743042,10.49303712695837,0.2470903450348308,1,37.035618890076876,37.035618890076876,9.151143850127593,1313.76171875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleCometAssay,1w_1t,1,1,3.0678517818450928,0.0,3.382840394973755,9.415838974993676,0.29560957161496776,1,13.304211451206356,13.304211451206356,3.9328522477660592,1233.125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleFly,1w_1t,1,1,2.9995768070220947,0.0,2.528999090194702,8.652359592262655,0.3954133490506761,1,8.094948755111545,8.094948755111545,3.200850797652257,911.796875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleFlyURL,1w_1t,1,1,2.6812729835510254,0.0,1.1671361923217773,6.791301331017166,0.8567980382912347,1,4.900778113398701,4.900778113398701,4.198977073660625,824.68359375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleHuman,1w_1t,1,1,2.620032548904419,0.0,1.2181837558746338,6.832155702169985,0.8208942166381279,1,4.420192994643003,4.420192994643003,3.628510865726809,795.67578125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleIlluminationCorrection_Example1_AllMethod,1w_1t,1,1,2.5418341159820557,0.0,4.662762403488159,9.888185085728765,0.2144651417906071,1,946.6214662808925,946.6214662808925,203.017306987964,1312.375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleIlluminationCorrection_Example1_EachMethod,1w_1t,1,1,2.4701032638549805,0.0,0.16818952560424805,5.320872409734875,5.945673468114845,1,4.54475614009425,4.54475614009425,27.021636001210418,757.953125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleIlluminationCorrection_Example2,1w_1t,1,1,2.85026478767395,0.0,2.3008546829223633,8.016120202839375,0.4346211029415727,1,48.04419008316472,48.04419008316472,20.88101888387962,973.43359375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleIlluminationCorrection_Example3,1w_1t,1,1,2.956389904022217,0.0,0.8741958141326904,6.6492008520290256,1.1439084743183343,1,2.1520505752414465,2.1520505752414465,2.4617488901803366,796.0390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleImagingFlowCytometryObjectsInGrid,1w_1t,1,1,3.4203906059265137,0.0,23.51734471321106,30.361198461148888,0.042521807295627295,1,61.60158259188756,61.60158259188756,2.619410624077912,2453.83984375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleNeighbors,1w_1t,1,1,2.4734487533569336,0.0,1.379551649093628,6.744648599997163,0.724873186630602,1,4.5466188420541584,4.5466188420541584,3.2957220884345353,833.3671875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExamplePercentPositive,1w_1t,1,1,2.445389747619629,0.0,0.5640556812286377,5.9112587329000235,1.772874617310439,1,2.5050158216618,2.5050158216618,4.441078966185259,798.546875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleSpeckles,1w_1t,1,1,2.3372161388397217,0.0,1.4638993740081787,6.56630947580561,0.6831070616978165,1,3.179383432958275,3.179383432958275,2.171859274898844,916.71875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleTrackObjects,1w_1t,1,1,2.4260778427124023,0.0,5.029005765914917,10.31930301617831,0.19884646121857688,1,9.587221414782107,9.587221414782107,1.9063850512483798,1578.8359375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleTumor,1w_1t,1,1,2.303354501724243,0.0,1.0990476608276367,6.154544240795076,0.9098786482534806,1,4.360487373080105,4.360487373080105,3.9675143567444966,1030.734375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleUntangleAndStraightenWorms,1w_1t,1,1,3.006577253341675,0.0,1.2314109802246094,7.1993993683718145,0.8120765658737265,1,3.4416790287941694,3.4416790287941694,2.794906886542791,865.18359375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleUntangleWorms,1w_1t,1,1,3.2606966495513916,0.0,2.211594343185425,8.507513043936342,0.4521624877009182,1,4.618807300925255,4.618807300925255,2.088451399397527,924.35546875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleUntangleWormsBrightField,1w_1t,1,1,3.3257691860198975,0.0,3.895765781402588,10.349423209670931,0.25668894284501137,1,7.377188352867961,7.377188352867961,1.8936426794662078,1020.09765625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleVitra,1w_1t,1,1,2.9028685092926025,0.0,1.5386874675750732,7.410238386131823,0.649904558965422,1,4.122247557155788,4.122247557155788,2.6790674805796204,996.34765625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleWoundHealing,1w_1t,1,1,2.023697853088379,0.0,2.044299364089966,6.776258641853929,0.4891651475150544,1,4.702593941241503,4.702593941241503,2.3003450589708008,1167.01953125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleYeastColonies,1w_1t,1,1,3.223799705505371,0.0,2.2835490703582764,8.649375961627811,0.43791482871139065,1,12.096297012176365,12.096297012176365,5.297147834129319,1149.1171875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +ExampleYeastPatches,1w_1t,1,1,3.211822986602783,0.0,2.7104239463806152,8.996431880164891,0.368945972948386,1,5.990533472970128,5.990533472970128,2.2101832006648374,1113.39453125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +cp_tutorial_3d_monolayer,1w_1t,1,1,3.5883374214172363,0.0,10.085208892822266,16.968512172345072,0.09915511028350726,1,14.058660756796598,14.058660756796598,1.3939880577785826,1430.68359375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +cp_tutorial_advanced_segmentation_final,1w_1t,1,1,4.423793315887451,0.0,9.268096923828125,17.375662088859826,0.10789701577559212,1,36.253466514870524,36.253466514870524,3.9116408484748857,1100.3125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +cp_tutorial_quality_control,1w_1t,1,1,2.0238704681396484,0.0,0.6121540069580078,5.509663320146501,1.6335758463288101,1,4.787729676347226,4.787729676347226,7.82111955803248,807.65625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +cp_tutorial_beginner_segmentation_final,1w_1t,1,1,3.170426845550537,0.0,5.66499924659729,12.072373040020466,0.17652253009577273,1,16.074153264053166,16.074153264053166,2.8374502033178883,970.3984375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +cp_tutorial_pixel_based_classification,1w_1t,1,1,2.3951919078826904,0.0,0.9567317962646484,6.207462176214904,1.0452250086223567,1,4.208680049050599,4.208680049050599,4.399017640557654,1020.7890625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +cp_tutorial_translocation_final,1w_1t,1,1,2.5509088039398193,0.0,0.9266119003295898,6.385702965315431,1.079200471787926,1,3.004684013314545,3.004684013314545,3.242656404742696,879.53515625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +cp4_supplement_combine_objects,1w_1t,1,1,1.9912192821502686,0.0,0.15514206886291504,4.98156897816807,6.445704942117339,1,1.8334413380362093,1.8334413380362093,11.817821893662222,777.11328125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 +cp_tutorial_translocation_start,1w_1t,1,1,2.0552892684936523,0.0,0.2602684497833252,4.96244603022933,3.8421867914935715,1,2.1760300607420504,2.1760300607420504,8.36071395727606,816.0546875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2712686a312b6cd048a8c3f2c24844a05fd8dc668390809299286250a672ceb2 diff --git a/benchmark/results/perf_coloc_step_cache_20260929/full_first_control.csv b/benchmark/results/perf_coloc_step_cache_20260929/full_first_control.csv new file mode 100644 index 0000000000..51dfb7fe0c --- /dev/null +++ b/benchmark/results/perf_coloc_step_cache_20260929/full_first_control.csv @@ -0,0 +1,31 @@ +case_name,mode_name,worker_count,well_count,compile_seconds,prepare_seconds,execute_seconds,total_seconds,wells_per_second,successful_wells,native_single_sample_execution_seconds,projected_native_execution_seconds,projected_execution_speedup,peak_memory_mb,status,memory_limit_mb,error_message,execution_route,server_lifecycle,run_input_sha256 +ExampleColocalization,1w_1t,1,1,8.959827661514282,0.0,5.9743382930755615,18.10615696478635,0.1673825536727691,1,37.035618890076876,37.035618890076876,6.199116466672514,1399.75390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleCometAssay,1w_1t,1,1,3.8960726261138916,0.0,4.9796857833862305,11.971674900036305,0.20081588347126414,1,13.304211451206356,13.304211451206356,2.671696976462514,1236.02734375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleFly,1w_1t,1,1,5.515779972076416,0.0,2.245816469192505,10.944007717072964,0.44527236028309797,1,8.094948755111545,8.094948755111545,3.604456938559243,923.53125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleFlyURL,1w_1t,1,1,2.6587178707122803,0.0,1.0173213481903076,6.621140057686716,0.9829735724890466,1,4.900778113398701,4.900778113398701,4.817335370103651,829.79296875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleHuman,1w_1t,1,1,2.6428587436676025,0.0,1.229111671447754,6.849782811943442,0.8135957238304585,1,4.420192994643003,4.420192994643003,3.596250118946896,800.58984375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleIlluminationCorrection_Example1_AllMethod,1w_1t,1,1,2.551304340362549,0.0,3.659698963165283,8.897983701899648,0.2732465183789591,1,946.6214662808925,946.6214662808925,258.6610198840391,1305.21875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleIlluminationCorrection_Example1_EachMethod,1w_1t,1,1,2.470106840133667,0.0,0.17411589622497559,5.331762987188995,5.7433009948034695,1,4.54475614009425,4.54475614009425,26.101902460542487,760.90234375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleIlluminationCorrection_Example2,1w_1t,1,1,2.87899112701416,0.0,1.8799498081207275,7.5637625036761165,0.5319290949579338,1,48.04419008316472,48.04419008316472,25.556102548924752,964.0703125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleIlluminationCorrection_Example3,1w_1t,1,1,2.9665210247039795,0.0,1.7319183349609375,7.572056937031448,0.577394430103169,1,2.1520505752414465,2.1520505752414465,1.2425820154447322,816.33984375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleImagingFlowCytometryObjectsInGrid,1w_1t,1,1,6.606513261795044,0.0,22.775487184524536,32.80469819577411,0.04390685441317273,1,61.60158259188756,61.60158259188756,2.7047317184830426,2531.44921875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleNeighbors,1w_1t,1,1,2.4554996490478516,0.0,2.6658759117126465,8.062196737155318,0.3751112328996465,1,4.5466188420541584,4.5466188420541584,1.7054877993676985,850.078125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExamplePercentPositive,1w_1t,1,1,2.46867299079895,0.0,0.440265417098999,5.811366650741547,2.2713571431279096,1,2.5050158216618,2.5050158216618,5.68978558017996,799.58984375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleSpeckles,1w_1t,1,1,2.3365750312805176,0.0,0.8261454105377197,5.933030019979924,1.2104406648571977,1,3.179383432958275,3.179383432958275,3.848454996425974,896.88671875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleTrackObjects,1w_1t,1,1,3.293782949447632,0.0,3.3337652683258057,9.467654252890497,0.29996113088735643,1,9.587221414782107,9.587221414782107,2.8757937776455225,1563.41015625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleTumor,1w_1t,1,1,2.2969553470611572,0.0,0.8088836669921875,5.8879763642326,1.2362717171907718,1,4.360487373080105,4.360487373080105,5.390747212506418,956.4375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleUntangleAndStraightenWorms,1w_1t,1,1,4.349471092224121,0.0,1.4747226238250732,8.752413948997855,0.6780936183146375,1,3.4416790287941694,3.4416790287941694,2.333780585712646,873.00390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleUntangleWorms,1w_1t,1,1,3.2605271339416504,0.0,2.2592837810516357,8.497339265886694,0.44261814668298416,1,4.618807300925255,4.618807300925255,2.044367927421373,912.14453125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleUntangleWormsBrightField,1w_1t,1,1,3.2337803840637207,0.0,3.754312753677368,10.117098330985755,0.2663603342637065,1,7.377188352867961,7.377188352867961,1.964990355596232,1021.12109375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleVitra,1w_1t,1,1,2.9137535095214844,0.0,1.2614099979400635,7.0860986933112144,0.7927636546666372,1,4.122247557155788,4.122247557155788,3.2679680388514396,934.73046875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleWoundHealing,1w_1t,1,1,2.0257351398468018,0.0,1.4442038536071777,6.18752931104973,0.692423024285877,1,4.702593941241503,4.702593941241503,3.2561843187828834,1201.890625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleYeastColonies,1w_1t,1,1,3.2377517223358154,0.0,2.0723824501037598,8.382851363159716,0.4825364159737659,1,12.096297012176365,12.096297012176365,5.836903806809756,1147.84375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleYeastPatches,1w_1t,1,1,3.2135751247406006,0.0,3.312673330307007,9.61351311719045,0.3018709967116871,1,5.990533472970128,5.990533472970128,1.808368310320217,1125.64453125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_3d_monolayer,1w_1t,1,1,4.531014919281006,0.0,10.493285179138184,18.33756344905123,0.09529903961707922,1,14.058660756796598,14.058660756796598,1.339776868425036,1422.68359375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_advanced_segmentation_final,1w_1t,1,1,4.264570951461792,0.0,10.793980121612549,18.682571393903345,0.0926442321306227,1,36.253466514870524,36.253466514870524,3.3586745673434217,1098.76953125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_quality_control,1w_1t,1,1,1.9164009094238281,0.0,0.6174421310424805,5.479508075863123,1.6195849776425433,1,4.787729676347226,4.787729676347226,7.754135060825363,804.96484375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_beginner_segmentation_final,1w_1t,1,1,3.2973175048828125,0.0,6.770091772079468,13.314955311827362,0.14770848515290436,1,16.074153264053166,16.074153264053166,2.3742888287489063,966.33984375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_pixel_based_classification,1w_1t,1,1,2.2479281425476074,0.0,0.9247426986694336,6.006233660969883,1.0813818821590593,1,4.208680049050599,4.208680049050599,4.55119035284762,990.296875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_translocation_final,1w_1t,1,1,2.5995290279388428,0.0,0.8435826301574707,6.301024904940277,1.185420330209183,1,3.004684013314545,3.004684013314545,3.5618135152375814,878.30859375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp4_supplement_combine_objects,1w_1t,1,1,1.9942057132720947,0.0,0.15790820121765137,4.98315173573792,6.332793308320059,1,1.8334413380362093,1.8334413380362093,11.610805036713082,775.48828125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_translocation_start,1w_1t,1,1,2.0697431564331055,0.0,0.25570130348205566,4.9467200338840485,3.9108130712762557,1,2.1760300607420504,2.1760300607420504,8.510046805040075,802.16015625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 diff --git a/benchmark/results/perf_coloc_step_cache_20260929/full_suite_execution_delta.png b/benchmark/results/perf_coloc_step_cache_20260929/full_suite_execution_delta.png new file mode 100644 index 0000000000..697d56bae3 Binary files /dev/null and b/benchmark/results/perf_coloc_step_cache_20260929/full_suite_execution_delta.png differ diff --git a/benchmark/results/perf_coloc_step_cache_20260929/full_warm_candidate.csv b/benchmark/results/perf_coloc_step_cache_20260929/full_warm_candidate.csv new file mode 100644 index 0000000000..9304dbb288 --- /dev/null +++ b/benchmark/results/perf_coloc_step_cache_20260929/full_warm_candidate.csv @@ -0,0 +1,31 @@ +case_name,mode_name,worker_count,well_count,compile_seconds,prepare_seconds,execute_seconds,total_seconds,wells_per_second,successful_wells,native_single_sample_execution_seconds,projected_native_execution_seconds,projected_execution_speedup,peak_memory_mb,status,memory_limit_mb,error_message,execution_route,server_lifecycle,run_input_sha256 +ExampleColocalization,1w_1t,1,1,3.302696704864502,0.0,3.718935251235962,10.096186046022922,0.2688941679389705,1,37.035618890076876,37.035618890076876,9.958661925552038,1310.671875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleCometAssay,1w_1t,1,1,3.055711030960083,0.0,2.591752529144287,8.60999168176204,0.38583930709239733,1,13.304211451206356,13.304211451206356,5.133287727744198,1217.671875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleFly,1w_1t,1,1,2.9811851978302,0.0,2.148517608642578,8.231432453263551,0.4654371907297491,1,8.094948755111545,8.094948755111545,3.7676902076803973,919.47265625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleFlyURL,1w_1t,1,1,2.655057668685913,0.0,1.139488697052002,6.735906984191388,0.8775865900092942,1,4.900778113398701,4.900778113398701,4.300857152929748,829.60546875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleHuman,1w_1t,1,1,2.6445040702819824,0.0,1.2340328693389893,6.7614734950475395,0.8103511866224851,1,4.420192994643003,4.420192994643003,3.581908638309353,798.98828125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleIlluminationCorrection_Example1_AllMethod,1w_1t,1,1,2.55765700340271,0.0,2.846787214279175,8.120627946220338,0.35127318086301246,1,946.6214662808925,946.6214662808925,332.522733533698,1274.73046875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleIlluminationCorrection_Example1_EachMethod,1w_1t,1,1,2.516428232192993,0.0,0.16867399215698242,5.414463059045374,5.928596265566031,1,4.54475614009425,4.54475614009425,26.944024280071062,761.5625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleIlluminationCorrection_Example2,1w_1t,1,1,2.8671979904174805,0.0,1.772667646408081,7.5290824440307915,0.5641215385333392,1,48.04419008316472,48.04419008316472,27.10276242730308,1004.50390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleIlluminationCorrection_Example3,1w_1t,1,1,2.940793514251709,0.0,0.8718390464782715,6.647682448849082,1.1470007039021997,1,2.1520505752414465,2.1520505752414465,2.468403524635073,799.00390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleImagingFlowCytometryObjectsInGrid,1w_1t,1,1,3.4476630687713623,0.0,21.533740043640137,28.337062722072005,0.046438751372191105,1,61.60158259188756,61.60158259188756,2.8607005781181623,2426.86328125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleNeighbors,1w_1t,1,1,2.472842216491699,0.0,1.471928358078003,6.772686000913382,0.6793808914081716,1,4.5466188420541584,4.5466188420541584,3.088885961807943,837.9375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExamplePercentPositive,1w_1t,1,1,2.444183111190796,0.0,0.5557022094726562,5.940390051808208,1.7995249667064817,1,2.5050158216618,2.5050158216618,4.507838513075161,801.6953125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleSpeckles,1w_1t,1,1,2.3182339668273926,0.0,0.8259930610656738,5.874337824061513,1.21066392338675,1,3.179383432958275,3.179383432958275,3.849164820896099,902.00390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleTrackObjects,1w_1t,1,1,2.423386573791504,0.0,3.357529878616333,8.586916312109679,0.29783800476918126,1,9.587221414782107,9.587221414782107,2.85543889745907,1571.36328125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleTumor,1w_1t,1,1,2.3061318397521973,0.0,0.8246448040008545,5.943696439266205,1.2126433043031262,1,4.360487373080105,4.360487373080105,5.287715816463916,955.89453125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleUntangleAndStraightenWorms,1w_1t,1,1,3.0381131172180176,0.0,1.2356970310211182,7.267080613877624,0.8092598548801643,1,3.4416790287941694,3.4416790287941694,2.7852126713860743,863.01953125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleUntangleWorms,1w_1t,1,1,3.2455945014953613,0.0,2.2246034145355225,8.542148170992732,0.44951832468925307,1,4.618807300925255,4.618807300925255,2.076238519974411,913.421875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleUntangleWormsBrightField,1w_1t,1,1,3.3344175815582275,0.0,3.7285454273223877,10.139678824227303,0.26820110402091535,1,7.377188352867961,7.377188352867961,1.9785700608094252,1025.49609375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleVitra,1w_1t,1,1,2.9031996726989746,0.0,1.3693935871124268,7.169969641137868,0.7302502431814736,1,4.122247557155788,4.122247557155788,3.0102722810672495,936.21875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleWoundHealing,1w_1t,1,1,2.035460948944092,0.0,1.54510498046875,6.293716668151319,0.6472052142998223,1,4.702593941241503,4.702593941241503,3.0435433195062527,1194.7890625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleYeastColonies,1w_1t,1,1,3.2516937255859375,0.0,2.1811540126800537,8.476604882627726,0.45847289746003217,1,12.096297012176365,12.096297012176365,5.545824339709628,1150.05078125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +ExampleYeastPatches,1w_1t,1,1,3.356388568878174,0.0,2.5256619453430176,9.025991102680564,0.3959358067867579,1,5.990533472970128,5.990533472970128,2.3718667037035064,1094.1796875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +cp_tutorial_3d_monolayer,1w_1t,1,1,3.613546371459961,0.0,10.241461515426636,17.157875870820135,0.09764231389179245,1,14.058660756796598,14.058660756796598,1.372720166513358,1420.94140625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +cp_tutorial_advanced_segmentation_final,1w_1t,1,1,4.264207124710083,0.0,9.438591003417969,17.339269490912557,0.10594801699087003,1,36.253466514870524,36.253466514870524,3.84098288629544,1102.30078125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +cp_tutorial_quality_control,1w_1t,1,1,1.9192872047424316,0.0,0.6170415878295898,5.4689456769265234,1.6206363067316831,1,4.787729676347226,4.787729676347226,7.759168540305045,807.9375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +cp_tutorial_beginner_segmentation_final,1w_1t,1,1,3.207782506942749,0.0,5.538364887237549,12.039867510087788,0.18055870647027458,1,16.074153264053166,16.074153264053166,2.902328320962382,996.01953125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +cp_tutorial_pixel_based_classification,1w_1t,1,1,2.3737740516662598,0.0,0.8111603260040283,6.017411075998098,1.232801910968996,1,4.208680049050599,4.208680049050599,5.188468807126667,1021.8046875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +cp_tutorial_translocation_final,1w_1t,1,1,2.562145948410034,0.0,0.9290289878845215,6.426769196987152,1.0763926777754111,1,3.004684013314545,3.004684013314545,3.234219870960612,880.37109375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +cp4_supplement_combine_objects,1w_1t,1,1,1.9982993602752686,0.0,0.15286612510681152,4.984665384981781,6.541671670563208,1,1.8334413380362093,1.8334413380362093,11.993771260670972,778.32421875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 +cp_tutorial_translocation_start,1w_1t,1,1,2.0574376583099365,0.0,0.2561960220336914,4.975822530221194,3.9032612296708247,1,2.1760300607420504,2.1760300607420504,8.493613770692695,835.33203125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,96d7bea1c83b1b08e752ca5c6ed1f7e19a221bf2965d6aafb8b96c6061ce7c98 diff --git a/benchmark/results/perf_coloc_step_cache_20260929/full_warm_control.csv b/benchmark/results/perf_coloc_step_cache_20260929/full_warm_control.csv new file mode 100644 index 0000000000..3170876390 --- /dev/null +++ b/benchmark/results/perf_coloc_step_cache_20260929/full_warm_control.csv @@ -0,0 +1,31 @@ +case_name,mode_name,worker_count,well_count,compile_seconds,prepare_seconds,execute_seconds,total_seconds,wells_per_second,successful_wells,native_single_sample_execution_seconds,projected_native_execution_seconds,projected_execution_speedup,peak_memory_mb,status,memory_limit_mb,error_message,execution_route,server_lifecycle,run_input_sha256 +ExampleColocalization,1w_1t,1,1,3.323961019515991,0.0,3.617356538772583,9.969379255082458,0.2764449645152516,1,37.035618890076876,37.035618890076876,10.238310349867684,1288.4375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleCometAssay,1w_1t,1,1,3.02347731590271,0.0,2.520958185195923,8.462881289888173,0.396674568373407,1,13.304211451206356,13.304211451206356,5.277442334955819,1218.84765625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleFly,1w_1t,1,1,3.111642837524414,0.0,2.0385589599609375,8.29406239464879,0.49054259388169075,1,8.094948755111545,8.094948755111545,3.9709171596717807,920.3828125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleFlyURL,1w_1t,1,1,2.667515993118286,0.0,1.0166563987731934,6.562555972021073,0.9836164914780522,1,4.900778113398701,4.900778113398701,4.820486173413658,829.453125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleHuman,1w_1t,1,1,2.646012544631958,0.0,1.2079601287841797,6.767570375930518,0.8278418932639002,1,4.420192994643003,4.420192994643003,3.659220937277092,798.5546875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleIlluminationCorrection_Example1_AllMethod,1w_1t,1,1,2.5429792404174805,0.0,2.791534900665283,8.050101389177144,0.3582258633992641,1,946.6214662808925,946.6214662808925,339.1042920707501,1281.109375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleIlluminationCorrection_Example1_EachMethod,1w_1t,1,1,2.4414167404174805,0.0,0.1653294563293457,5.297076327726245,6.04852893248462,1,4.54475614009425,4.54475614009425,27.489089004447198,760.359375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleIlluminationCorrection_Example2,1w_1t,1,1,2.869089126586914,0.0,1.7775242328643799,7.4743453389965,0.5625802346382398,1,48.04419008316472,48.04419008316472,27.028711729991002,981.32421875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleIlluminationCorrection_Example3,1w_1t,1,1,2.920943260192871,0.0,0.8762273788452148,6.648663740139455,1.1412562813523424,1,2.1520505752414465,2.1520505752414465,2.4560412367822226,798.50390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleImagingFlowCytometryObjectsInGrid,1w_1t,1,1,3.4163036346435547,0.0,21.254642009735107,27.948565289843827,0.04704854589138586,1,61.60158259188756,61.60158259188756,2.8982648855564186,2424.671875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleNeighbors,1w_1t,1,1,2.455495595932007,0.0,1.4713807106018066,6.794630188029259,0.6796337567800463,1,4.5466188420541584,4.5466188420541584,3.090035644272212,836.67578125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExamplePercentPositive,1w_1t,1,1,2.4268131256103516,0.0,0.4376182556152344,5.770994539372623,2.2850966274113267,1,2.5050158216618,2.5050158216618,5.724203205691393,798.9375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleSpeckles,1w_1t,1,1,2.327207565307617,0.0,0.8347718715667725,5.954534489661455,1.197932074691392,1,3.179383432958275,3.179383432958275,3.808685392083147,889.80078125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleTrackObjects,1w_1t,1,1,2.4236276149749756,0.0,3.143188238143921,8.357484063133597,0.31814830173534514,1,9.587221414782107,9.587221414782107,3.05015821147366,1567.6953125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleTumor,1w_1t,1,1,2.338942766189575,0.0,0.8002221584320068,6.020175698213279,1.2496529738184798,1,4.360487373080105,4.360487373080105,5.449096013067484,947.01953125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleUntangleAndStraightenWorms,1w_1t,1,1,2.9941883087158203,0.0,1.2451016902923584,7.191384637262672,0.8031472511817032,1,3.4416790287941694,3.4416790287941694,2.764175051425751,857.95703125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleUntangleWorms,1w_1t,1,1,3.2426037788391113,0.0,2.2027556896209717,8.482035635970533,0.45397680946272806,1,4.618807300925255,4.618807300925255,2.0968314019972016,906.1484375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleUntangleWormsBrightField,1w_1t,1,1,3.2127177715301514,0.0,3.745802640914917,10.062537916004658,0.26696547999542997,1,7.377188352867961,7.377188352867961,1.9694546296400905,1020.91796875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleVitra,1w_1t,1,1,2.8912148475646973,0.0,1.3636698722839355,7.20144598511979,0.7333153135701056,1,4.122247557155788,4.122247557155788,3.022907259989298,932.11328125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleWoundHealing,1w_1t,1,1,2.0138261318206787,0.0,1.3847479820251465,6.037405799143016,0.7221530653812792,1,4.702593941241503,4.702593941241503,3.395992629910982,1170.62890625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleYeastColonies,1w_1t,1,1,3.2719123363494873,0.0,2.0856516361236572,8.444489850196987,0.47946645675620897,1,12.096297012176365,12.096297012176365,5.799768668298919,1148.171875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +ExampleYeastPatches,1w_1t,1,1,3.2333009243011475,0.0,2.6243515014648438,8.92510417709127,0.38104651737460715,1,5.990533472970128,5.990533472970128,2.2826719170912777,1083.1640625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_3d_monolayer,1w_1t,1,1,3.5835182666778564,0.0,10.267441272735596,17.229198317974806,0.09739524906321338,1,14.058660756796598,14.058660756796598,1.3692467659034286,1430.11328125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_advanced_segmentation_final,1w_1t,1,1,4.228123188018799,0.0,10.850493907928467,18.6846966301091,0.0921617032814791,1,36.253466514870524,36.253466514870524,3.3411812238685354,1100.0546875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_quality_control,1w_1t,1,1,1.907684326171875,0.0,0.6087887287139893,5.400636968202889,1.6426059695822701,1,4.787729676347226,4.787729676347226,7.864353347114144,807.28125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_beginner_segmentation_final,1w_1t,1,1,3.2727670669555664,0.0,6.780334949493408,13.271101823076606,0.14748533921244036,1,16.074153264053166,16.074153264053166,2.3707019467016366,992.53515625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_pixel_based_classification,1w_1t,1,1,2.2634620666503906,0.0,0.9301130771636963,6.036275522317737,1.0751380929396437,1,4.208680049050599,4.208680049050599,4.524912241729387,992.42578125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_translocation_final,1w_1t,1,1,2.560678720474243,0.0,0.825181245803833,6.3381188018247485,1.2118549774187741,1,3.004684013314545,3.004684013314545,3.6412412771058498,880.296875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp4_supplement_combine_objects,1w_1t,1,1,1.999769926071167,0.0,0.15795016288757324,5.05013538338244,6.331110913204858,1,1.8334413380362093,1.8334413380362093,11.607720463961963,777.8984375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 +cp_tutorial_translocation_start,1w_1t,1,1,2.0898513793945312,0.0,0.25942063331604004,4.963807687163353,3.8547434998423844,1,2.1760300607420504,2.1760300607420504,8.388037732107048,817.73828125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,50c23a3811b9352d3843d7afb36b4502b39da167b48378dbe6c7918345ec4ca5 diff --git a/benchmark/results/perf_coloc_step_cache_20260929/plot.py b/benchmark/results/perf_coloc_step_cache_20260929/plot.py new file mode 100644 index 0000000000..64ddbfd465 --- /dev/null +++ b/benchmark/results/perf_coloc_step_cache_20260929/plot.py @@ -0,0 +1,78 @@ +"""Regenerate Costes cache execution figures from retained benchmark CSVs.""" + +from __future__ import annotations + +import csv +from pathlib import Path + +import matplotlib.pyplot as plt +import numpy as np + +HERE = Path(__file__).resolve().parent +TARGETS = ( + "cp_tutorial_advanced_segmentation_final", + "cp_tutorial_beginner_segmentation_final", +) + + +def rows(path: Path) -> dict[str, dict[str, str]]: + with path.open(newline="", encoding="utf-8") as handle: + return {row["case_name"]: row for row in csv.DictReader(handle)} + + +control = rows(HERE / "full_warm_control.csv") +candidate = rows(HERE / "full_warm_candidate.csv") +native = rows(HERE.parent / "perf_lazy_catalog_20260929" / "native_cp_summary.csv") + +fig, ax = plt.subplots(figsize=(9.5, 4.5), layout="constrained") +x = np.arange(len(TARGETS)) +width = 0.23 +for offset, label, values, color in ( + ( + -width, + "Native CellProfiler", + [float(native[name]["median_native_execution_seconds"]) for name in TARGETS], + "#697b8c", + ), + ( + 0, + "OpenHCS control", + [float(control[name]["execute_seconds"]) for name in TARGETS], + "#d87a57", + ), + ( + width, + "OpenHCS step cache", + [float(candidate[name]["execute_seconds"]) for name in TARGETS], + "#3a8c83", + ), +): + bars = ax.bar(x + offset, values, width, label=label, color=color) + ax.bar_label(bars, fmt="%.2f", padding=3, fontsize=9) +ax.set_xticks(x, ("Advanced segmentation", "Beginner segmentation")) +ax.set_ylabel("Execution seconds, one well / one thread") +ax.set_ylim(0, 42) +ax.legend(frameon=False, ncol=3, loc="upper right") +ax.set_title("Repeated Costes inputs: exact step-local threshold reuse") +fig.savefig(HERE / "affected_case_execution.png", dpi=180) +plt.close(fig) + +ordered = sorted( + control, + key=lambda name: float(candidate[name]["execute_seconds"]) + - float(control[name]["execute_seconds"]), +) +deltas = [ + float(candidate[name]["execute_seconds"]) - float(control[name]["execute_seconds"]) + for name in ordered +] +colors = ["#3a8c83" if name in TARGETS else "#aeb7bf" for name in ordered] +fig, ax = plt.subplots(figsize=(10, 9), layout="constrained") +ax.barh(range(len(ordered)), deltas, color=colors) +ax.set_yticks(range(len(ordered)), ordered, fontsize=7.5) +ax.invert_yaxis() +ax.axvline(0, color="#2f3640", linewidth=0.8) +ax.set_xlabel("Candidate minus control execution seconds (lower is faster)") +ax.set_title("Warmed official30 paired execution differences") +fig.savefig(HERE / "full_suite_execution_delta.png", dpi=180) +plt.close(fig) diff --git a/openhcs/core/context/processing_context.py b/openhcs/core/context/processing_context.py index e9a6f47172..187565b391 100644 --- a/openhcs/core/context/processing_context.py +++ b/openhcs/core/context/processing_context.py @@ -6,8 +6,10 @@ from __future__ import annotations +from collections.abc import Iterator +from contextlib import contextmanager from dataclasses import dataclass -from typing import TYPE_CHECKING +from typing import TYPE_CHECKING, TypeVar, cast if TYPE_CHECKING: from openhcs.core.orchestrator.worker_lanes import WorkerLaneExecutionContext @@ -40,6 +42,8 @@ from openhcs.core.steps.function_output_identity import FunctionOutputIdentityCache from openhcs.runtime.zmq_config import OPENHCS_ZMQ_CONFIG +_RuntimeStepValue = TypeVar("_RuntimeStepValue") + @dataclass(frozen=True, slots=True) class RequiredVisualizer: @@ -126,6 +130,7 @@ def __init__( self.runtime_function_output_identity_cache = FunctionOutputIdentityCache() self.runtime_pattern_discovery_cache = RuntimePatternDiscoveryCache() self.runtime_source_binding_context_cache = RuntimeSourceBindingContextCache() + self._runtime_step_values: dict[type[object], object] | None = None self.runtime_source_workspace_projection_cache = ( VirtualWorkspaceSourceProjectionCache() ) @@ -175,6 +180,29 @@ def release_execution_image_cache(self) -> None: """Release image reuse storage without discarding runtime observations.""" self.runtime_image_stack_cache.clear() + @contextmanager + def runtime_step_scope(self) -> Iterator[None]: + """Own runtime-only values for exactly one FunctionStep invocation.""" + previous = self._runtime_step_values + self._runtime_step_values = {} + try: + yield + finally: + self._runtime_step_values = previous + + def runtime_step_value( + self, value_type: type[_RuntimeStepValue] + ) -> _RuntimeStepValue: + """Resolve one declared runtime value shared within the active step.""" + values = self._runtime_step_values + if values is None: + raise RuntimeError("Runtime step values require an active FunctionStep.") + value = values.get(value_type) + if value is None: + value = value_type() + values[value_type] = value + return cast(_RuntimeStepValue, value) + def install_debug_event_sink(self, debug_event_sink: DebugEventSink) -> None: """Install the debug sink selected for this execution context.""" diff --git a/openhcs/core/steps/function_execution.py b/openhcs/core/steps/function_execution.py index 25257bb69b..e376923be2 100644 --- a/openhcs/core/steps/function_execution.py +++ b/openhcs/core/steps/function_execution.py @@ -691,7 +691,8 @@ def execute( try: executor = cls(context, step_index) step_name = executor.plan.step_name or step_name - return executor.run() + with context.runtime_step_scope(): + return executor.run() except Exception as error: full_traceback = traceback.format_exc() logger.error( diff --git a/openhcs/interop/cellprofiler/module_callable_abi.py b/openhcs/interop/cellprofiler/module_callable_abi.py index 595b711da5..78a23086ea 100644 --- a/openhcs/interop/cellprofiler/module_callable_abi.py +++ b/openhcs/interop/cellprofiler/module_callable_abi.py @@ -39,6 +39,7 @@ from openhcs.interop.cellprofiler.runtime.invocation import ( CellProfilerImageRequest, ) + from openhcs.interop.cellprofiler.runtime.adapter import CellProfilerRuntimeAdapter from openhcs.interop.cellprofiler.runtime.output_contexts import ( CellProfilerObjectLabelOutputSourceContext, ) @@ -253,6 +254,14 @@ def object_measurement_invocation_kwargs( del cls, include_image_measurements return runtime_kwargs + @classmethod + def shared_object_measurement_runtime_kwargs( + cls, adapter: "CellProfilerRuntimeAdapter" + ) -> "RuntimeCallableKwargs": + """Project runtime-owned values shared by this module's object calls.""" + del cls, adapter + return {} + @classmethod def validate_callable_artifact_abi( cls, diff --git a/openhcs/interop/cellprofiler/runtime/module_execution.py b/openhcs/interop/cellprofiler/runtime/module_execution.py index e9b9898234..f076bfb092 100644 --- a/openhcs/interop/cellprofiler/runtime/module_execution.py +++ b/openhcs/interop/cellprofiler/runtime/module_execution.py @@ -854,6 +854,9 @@ def _run_per_object_measurement( ) ) measurement_row_policy = module_type.runtime_object_measurement_row_policy() + shared_runtime_kwargs = module_type.shared_object_measurement_runtime_kwargs( + cellprofiler_runtime + ) label_payload_seconds = 0.0 label_align_seconds = 0.0 contract_execute_seconds = 0.0 @@ -934,6 +937,7 @@ def _run_per_object_measurement( image=aligned_measurement_image.payload, kwargs={ **invocation_kwargs, + **shared_runtime_kwargs, **_execution_mode_semantic_control_kwargs( processing_contract, execution_mode, diff --git a/openhcs/processing/backends/cellprofiler/colocalization.py b/openhcs/processing/backends/cellprofiler/colocalization.py index 40b6cd78fe..14a5131593 100644 --- a/openhcs/processing/backends/cellprofiler/colocalization.py +++ b/openhcs/processing/backends/cellprofiler/colocalization.py @@ -6,10 +6,11 @@ from collections.abc import Callable, Hashable, Mapping from dataclasses import dataclass, field, fields, make_dataclass, replace from enum import Enum +import hashlib import logging from types import MappingProxyType import time -from typing import Annotated, ClassVar, Tuple +from typing import TYPE_CHECKING, Annotated, ClassVar, Tuple, cast from metaclass_registry import AutoRegisterMeta import numpy as np @@ -167,6 +168,9 @@ RuntimeInputBindingRequest, ) +if TYPE_CHECKING: + from openhcs.interop.cellprofiler.runtime.adapter import CellProfilerRuntimeAdapter + @dataclass(frozen=True, slots=True) class ColocalizationThresholdMaskGroup: @@ -440,6 +444,21 @@ class MeasureColocalizationModule( object_gate_setting = SettingNameFamily("Select an object to measure") ignored_settings = ("Hidden",) + @classmethod + def shared_object_measurement_runtime_kwargs( + cls, adapter: "CellProfilerRuntimeAdapter" + ) -> RuntimeCallableKwargs: + """Share exact Costes image-pair thresholds across object sets in one step.""" + del cls + return cast( + RuntimeCallableKwargs, + { + "costes_threshold_batch": adapter.request.context.runtime_step_value( + ColocalizationCostesThresholdBatch + ) + }, + ) + class MeasurementFeature(RuntimeMeasurementFeature): """Exact source-pair feature identities emitted by MeasureColocalization.""" @@ -1582,15 +1601,12 @@ def from_dense_label_payload( @dataclass(frozen=True, slots=True) class ColocalizationCostesThresholdCacheKey: - """Batch-local identity for Costes thresholds over one image pair.""" + """Exact selected-pixel content and algorithm identity for Costes thresholds.""" - image_payload_id: int - image_data_id: int - channel_1: int - channel_2: int + pair_digest: bytes method: CostesMethod scale_max: int - backend_provider: object + backend_provider: CellProfilerBackendProvider | None @dataclass(frozen=True, slots=True) @@ -1892,6 +1908,14 @@ class _ObjectColocalizationRankProviderRuntimeParameter(KeywordRuntimeParameter) parameter_default = _DIRECT_OBJECT_COLOCALIZATION_RANK_PROVIDER +class _ColocalizationCostesThresholdBatchRuntimeParameter(KeywordRuntimeParameter): + """Step-owned Costes cache supplied by the CellProfiler module executor.""" + + parameter_name = "costes_threshold_batch" + annotation_type = RuntimeSliceInvariantValue + parameter_default = None + + @dataclass(slots=True) class CachedObjectColocalizationRankProvider(ObjectColocalizationRankProvider): """Share RWC ranks across source-pair calls in one measurement-image batch.""" @@ -2221,32 +2245,19 @@ def _resolve_object_costes_thresholds( base: ObjectColocalizationBaseStage, provided: ColocalizationCostesThresholds | None, metrics: ObjectColocalizationMetricArrays, + threshold_batch: ColocalizationCostesThresholdBatch | None = None, ) -> ColocalizationCostesThresholds | None: options = context.options if not (options.do_costes and base.full_first_pixels.size): return None if provided is not None: resolved = provided - elif options.costes_method == CostesMethod.FASTER: - threshold_c1, threshold_c2 = costes_backend( - backend_provider=options.costes_backend_provider - ).scaled_second_channel_costes( - base.full_first_pixels, base.full_second_pixels, options.scale_max - ) - resolved = ColocalizationCostesThresholds.from_thresholds( - threshold_c1, threshold_c2 - ) else: - threshold_c1, threshold_c2 = costes_backend( - backend_provider=options.costes_backend_provider - ).linear_costes( - base.full_first_pixels, - base.full_second_pixels, - options.scale_max, - options.costes_method == CostesMethod.FAST, - ) - resolved = ColocalizationCostesThresholds.from_thresholds( - threshold_c1, threshold_c2 + request = ColocalizationCostesThresholdRequest.from_object_context(context) + resolved = ( + threshold_batch.resolve(request) + if threshold_batch is not None + else request.thresholds() ) metrics.costes_threshold_1.fill(resolved.first) metrics.costes_threshold_2.fill(resolved.second) @@ -2626,6 +2637,7 @@ def _measure_colocalization_objects_core( *, costes_thresholds: ColocalizationCostesThresholds | None = None, rank_provider: ObjectColocalizationRankProvider = DirectObjectColocalizationRankProvider(), + threshold_batch: ColocalizationCostesThresholdBatch | None = None, ) -> Tuple[RuntimeArrayData, ObjectColocalizationColumnarMeasurements]: """Measure colocalization between two channels within labeled objects.""" total_started_at = time.perf_counter() @@ -2680,7 +2692,7 @@ def _measure_colocalization_objects_core( ) phase_started_at = time.perf_counter() resolved_costes_thresholds = _resolve_object_costes_thresholds( - context, base, costes_thresholds, metrics + context, base, costes_thresholds, metrics, threshold_batch ) runtime_profiler.log( "coloc_object_costes_thresholds", @@ -2755,6 +2767,7 @@ def _measure_colocalization_objects_core( @runtime_bound_parameters( _ObjectColocalizationRankProviderRuntimeParameter, _ColocalizationThresholdMaskOutputsRuntimeParameter, + _ColocalizationCostesThresholdBatchRuntimeParameter, ) def measure_colocalization_objects( image: np.ndarray, @@ -2778,6 +2791,7 @@ def measure_colocalization_objects( threshold_mask_outputs: tuple[ColocalizationThresholdMaskRuntimeOutput, ...] = (), *, rank_provider: ObjectColocalizationRankProvider = _DIRECT_OBJECT_COLOCALIZATION_RANK_PROVIDER, + costes_threshold_batch: RuntimeSliceInvariantValue | None = None, ) -> Tuple[RuntimeArrayData | AlignedImageStack, ColumnarRows]: """Measure image and/or object colocalization through one declared callable. @@ -2799,6 +2813,10 @@ def measure_colocalization_objects( measurement_scope, CellProfilerMeasurementTargetScope.OBJECT, ).measurement_scope_selection + if costes_threshold_batch is not None and not isinstance( + costes_threshold_batch, ColocalizationCostesThresholdBatch + ): + raise TypeError("Costes threshold batch must be a runtime cache instance.") context = _prepare_object_colocalization_context( image, labels, @@ -2820,6 +2838,7 @@ def measure_colocalization_objects( context, costes_thresholds=costes_thresholds, rank_provider=rank_provider, + threshold_batch=costes_threshold_batch, ) output = _colocalization_threshold_mask_canonical_output( image, @@ -2972,15 +2991,39 @@ class ColocalizationCostesThresholdRequest: @property def cache_key(self) -> ColocalizationCostesThresholdCacheKey: - """Return the batch-local identity for this resolved source pair.""" + """Fingerprint the exact selected pixels reused across object sets.""" + pair = self.image_pair_context + if pair is None: + pair = ColocalizationImagePairContext.from_request( + self.image, channel_1=self.channel_1, channel_2=self.channel_2 + ) + digest = hashlib.blake2b(digest_size=32) + for pixels in (pair.full_first_pixels, pair.full_second_pixels): + array = np.ascontiguousarray(pixels) + digest.update(array.dtype.str.encode("ascii")) + digest.update(repr(array.shape).encode("ascii")) + digest.update(array.tobytes()) return ColocalizationCostesThresholdCacheKey( - id(self.image), - id(self.image_data), - self.channel_1, - self.channel_2, - self.method, - self.scale_max, - self.backend_provider, + pair_digest=digest.digest(), + method=self.method, + scale_max=self.scale_max, + backend_provider=self.backend_provider, + ) + + @classmethod + def from_object_context( + cls, context: ObjectColocalizationRequestContext + ) -> "ColocalizationCostesThresholdRequest": + """Use the object call's already-selected full image-pair pixels.""" + return cls( + image=context.image, + image_data=context.image_data, + channel_1=context.channel_1, + channel_2=context.channel_2, + method=context.options.costes_method, + scale_max=context.options.scale_max, + backend_provider=context.options.costes_backend_provider, + image_pair_context=context.image_pair, ) @staticmethod @@ -3070,8 +3113,8 @@ def thresholds(self) -> ColocalizationCostesThresholds: return ColocalizationCostesThresholds.from_thresholds(first, second) -class ColocalizationCostesThresholdBatch: - """Batch-local Costes threshold cache keyed by resolved image-pair identity.""" +class ColocalizationCostesThresholdBatch(RuntimeSliceInvariantValue): + """Costes threshold cache scoped to one runtime FunctionStep.""" def __init__(self) -> None: self._thresholds: dict[ @@ -3084,6 +3127,17 @@ def __init__(self) -> None: ColocalizationObjectLabelCacheKey, ColocalizationObjectLabelContext ] = {} + def resolve( + self, request: ColocalizationCostesThresholdRequest + ) -> ColocalizationCostesThresholds: + """Compute once for each exact pixel pair and algorithm selection.""" + key = request.cache_key + thresholds = self._thresholds.get(key) + if thresholds is None: + thresholds = request.thresholds() + self._thresholds[key] = thresholds + return thresholds + def image_pair_context( self, request: RuntimeBatchInvocationRequest ) -> ColocalizationImagePairContext: @@ -3156,11 +3210,7 @@ def request_kwargs( ) thresholds = None if threshold_request is not None: - key = threshold_request.cache_key - thresholds = self._thresholds.get(key) - if thresholds is None: - thresholds = threshold_request.thresholds() - self._thresholds[key] = thresholds + thresholds = self.resolve(threshold_request) kwargs = { **request.kwargs, "image_pair_context": image_pair_context, diff --git a/tests/unit/cellprofiler_runtime_test_support.py b/tests/unit/cellprofiler_runtime_test_support.py index e13afbffd3..c387ce86f4 100644 --- a/tests/unit/cellprofiler_runtime_test_support.py +++ b/tests/unit/cellprofiler_runtime_test_support.py @@ -46,6 +46,13 @@ class CellProfilerRuntimeTestContext: runtime_function_output_identity_cache: FunctionOutputIdentityCache = field( default_factory=FunctionOutputIdentityCache ) + _runtime_step_values: dict[type[object], object] = field(default_factory=dict) + + def runtime_step_value(self, value_type: type[Any]) -> Any: + """Provide one runtime-only value per type for a test step.""" + if value_type not in self._runtime_step_values: + self._runtime_step_values[value_type] = value_type() + return self._runtime_step_values[value_type] def cellprofiler_runtime_input_edge_for_test( diff --git a/tests/unit/test_cellprofiler_library_loading.py b/tests/unit/test_cellprofiler_library_loading.py index 44959f5c0d..bf774ae583 100644 --- a/tests/unit/test_cellprofiler_library_loading.py +++ b/tests/unit/test_cellprofiler_library_loading.py @@ -1,4 +1,5 @@ import importlib +from dataclasses import replace import sys import types from typing import get_type_hints @@ -74,9 +75,11 @@ ) from openhcs.processing.backends.cellprofiler.colocalization import ( ColocalizationCostesThresholdBatch, + ColocalizationCostesThresholdRequest, ColocalizationCostesThresholds, ColocalizationImagePairContext, ColocalizationObjectLabelContext, + CostesMethod, MeasureColocalizationModule, ObjectColocalizationMetricArrays, costes_backend, @@ -1323,6 +1326,38 @@ def test_colocalization_threshold_batch_caches_semantic_label_context() -> None: assert second is first +def test_costes_thresholds_reuse_equal_pixels_across_distinct_image_payloads() -> None: + image = np.stack( + ( + np.array(((0.1, 0.2), (0.3, 0.4)), dtype=np.float32), + np.array(((0.4, 0.3), (0.2, 0.1)), dtype=np.float32), + ) + ) + + def request_for(pixels: np.ndarray) -> ColocalizationCostesThresholdRequest: + return ColocalizationCostesThresholdRequest( + image=pixels, + image_data=pixels, + channel_1=0, + channel_2=1, + method=CostesMethod.FASTER, + scale_max=255, + backend_provider=None, + image_pair_context=ColocalizationImagePairContext.from_request( + pixels, channel_1=0, channel_2=1 + ), + ) + + batch = ColocalizationCostesThresholdBatch() + first = batch.resolve(request_for(image)) + assert batch.resolve(request_for(image.copy())) is first + + changed = image.copy() + changed[0, 0, 0] += np.float32(0.1) + assert batch.resolve(request_for(changed)) is not first + assert batch.resolve(replace(request_for(image), scale_max=65535)) is not first + + def test_measure_colocalization_objects_batch_uses_contract_execution() -> None: image = np.stack( ( diff --git a/tests/unit/test_function_step_execution_scope.py b/tests/unit/test_function_step_execution_scope.py index af672642db..9836d217d2 100644 --- a/tests/unit/test_function_step_execution_scope.py +++ b/tests/unit/test_function_step_execution_scope.py @@ -83,6 +83,25 @@ from openhcs.core.steps.function_step import FunctionStep +def test_runtime_step_values_share_within_step_and_release_between_steps() -> None: + class Counter: + pass + + context = ProcessingContext() + with pytest.raises(RuntimeError, match="active FunctionStep"): + context.runtime_step_value(Counter) + + with context.runtime_step_scope(): + first = context.runtime_step_value(Counter) + assert context.runtime_step_value(Counter) is first + with context.runtime_step_scope(): + assert context.runtime_step_value(Counter) is not first + assert context.runtime_step_value(Counter) is first + + with context.runtime_step_scope(): + assert context.runtime_step_value(Counter) is not first + + def test_runtime_axis_scope_returns_exact_complete_plane_selection() -> None: scope = RuntimeExecutionAxisScope.from_raw( "A01",