Reduce pipeline compile time with direct docstring parsing

benchmark/results/perf_direct_docstring_20260929/README.md

@@ -0,0 +1,28 @@
+# Direct docstring parser: official30 one-well benchmark (2026-09-29)
+
+The changed dependency is python-introspect `5bb79d5` → `4e8f8bd` ([upstream PR #4](https://github.com/OpenHCSDev/python-introspect/pull/4)). The two fresh-server runs used the same OpenHCS integration checkout, containing the separately proposed single-thread performance changes through `38ec4fbc1`; only the python-introspect checkout changed between control and direct-parser runs. The direct-parser checkout was then committed as `8ace32939` for the reused-server run. This report accompanies the isolated submodule bump in [OpenHCS PR #175](https://github.com/OpenHCSDev/openhcs/pull/175).
+
+All 30 official portable cases succeeded in each run. `control_fresh.csv` and `direct_fresh.csv` use a fresh client-owned ZMQ server per observation. `direct_reused.csv` uses one client-owned server across the sweep. `native_cp_summary.csv` is the fresh native CellProfiler 4.2.8.1 baseline used to project native execution time by well count. Each CSV carries `server_lifecycle` where applicable.
+
+| 30-case sum (s) | Control fresh | Direct fresh | Direct reused |
+| --- | ---: | ---: | ---: |
+| Server compilation jobs | 100.065 | 58.632 | 22.607 |
+| Server execution jobs | 103.767 | 93.115 | 93.564 |
+| Non-job (`total - compile - execute`) | 171.189 | 160.110 | 25.773 |
+| Per-observation `total_seconds` | 375.020 | 311.858 | 141.944 |
+
+All 30 cases compiled faster after the parser change; the median case saved 1.342 s in compilation. The direct-parser patch changes compilation metadata extraction, not runtime execution. The execution and fresh-server non-job differences are observed variation and should not be attributed to the patch. The reused-server total excludes its once-per-sweep startup/shutdown, while the fresh-server total includes startup/shutdown for every observation. Reuse also benefits from warm in-process compilation and cache state, so the 169.914 s fresh-to-reused difference is not a measured whole-sweep wall-time speedup.
+
+A targeted A/B/A check gave ExampleFly compile times 3.989 → 2.260 → 3.976 s and advanced segmentation 6.259 → 3.176 → 6.303 s (old/direct/old). Direct parsing produced identical `DocstringInfo` for 114 real CellProfiler callables. The upstream test suite passed 127 tests.
+
+`direct_fresh_native_relative.png` and `direct_reused_native_relative.png` show **execution** speedup relative to the native CP baseline. They do not depict total time or multiple-well scaling. The completed four-mode native-relative execution sweep predating this compile-only change is stored separately at `official30-four-modes-native-relative-fresh-v2-20260928` in the benchmark run workspace; a new 120-observation sweep was intentionally not used as evidence for this patch because its execution path did not change.
+
+Fresh command (repeat with `--reuse-execution-server` for the reused run):
+
+```bash
+OPENHCS_CPU_ONLY=true .venv/bin/python scripts/benchmark_cppipe_well_throughput.py \
+  --manifest benchmark/manifests/official30_portable_axis1.json \
+  --output-dir /path/to/output \
+  --native-summary-csv benchmark/results/perf_direct_docstring_20260929/native_cp_summary.csv \
+  --mode 1w_1t --figures
+```

benchmark/results/perf_direct_docstring_20260929/control_fresh.csv

@@ -0,0 +1,31 @@
+case_name,mode_name,worker_count,well_count,compile_seconds,prepare_seconds,execute_seconds,total_seconds,wells_per_second,successful_wells,native_single_sample_execution_seconds,projected_native_execution_seconds,projected_execution_speedup,peak_memory_mb,status,memory_limit_mb,error_message,execution_route,server_lifecycle,run_input_sha256
+ExampleColocalization,1w_1t,1,1,3.7952799797058105,0.0,4.710984706878662,14.525511140935123,0.212269846374128,1,37.035618890076876,37.035618890076876,7.861545132167371,1356.85546875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleCometAssay,1w_1t,1,1,3.530890703201294,0.0,2.856804847717285,11.883519758936018,0.35004141105369685,1,13.304211451206356,13.304211451206356,4.657024949337025,1280.359375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleFly,1w_1t,1,1,3.9887683391571045,0.0,2.0932228565216064,12.123057164251804,0.47773221894860285,1,8.094948755111545,8.094948755111545,3.8672178310546688,959.3984375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleFlyURL,1w_1t,1,1,3.3167309761047363,0.0,1.085144281387329,10.569152536336333,0.9215364418835859,1,4.900778113398701,4.900778113398701,4.516245625082392,896.078125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleHuman,1w_1t,1,1,3.4526987075805664,0.0,1.19232177734375,10.470840559806675,0.8386997696442283,1,4.420192994643003,4.420192994643003,3.707214846390118,876.77734375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleIlluminationCorrection_Example1_AllMethod,1w_1t,1,1,2.596431016921997,0.0,3.8865044116973877,12.078059430234134,0.25730062134761894,1,946.6214662808925,946.6214662808925,243.56629145506773,1299.75390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleIlluminationCorrection_Example1_EachMethod,1w_1t,1,1,2.450819730758667,0.0,0.17623472213745117,7.973976640030742,5.674250725802306,1,4.54475614009425,4.54475614009425,25.788085826524288,800.22265625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleIlluminationCorrection_Example2,1w_1t,1,1,3.406686782836914,0.0,2.164682388305664,11.383527447935194,0.4619615355131697,1,48.04419008316472,48.04419008316472,22.194567823305373,1049.32421875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleIlluminationCorrection_Example3,1w_1t,1,1,3.1361730098724365,0.0,0.7579882144927979,9.801032058894634,1.3192817261270249,1,2.1520505752414465,2.1520505752414465,2.8391609976171925,836.4140625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleImagingFlowCytometryObjectsInGrid,1w_1t,1,1,4.411989450454712,0.0,23.148673057556152,33.60791293112561,0.04319902041527955,1,61.60158259188756,61.60158259188756,2.6611280240004804,2599.453125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleNeighbors,1w_1t,1,1,3.132838726043701,0.0,1.3849973678588867,10.27791321510449,0.7220230328278047,1,4.5466188420541584,4.5466188420541584,3.2827635254519847,912.40625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExamplePercentPositive,1w_1t,1,1,3.2013676166534424,0.0,0.48795175552368164,9.249851203057915,2.0493829332098126,1,2.5050158216618,2.5050158216618,5.133736672334249,853.01171875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleSpeckles,1w_1t,1,1,2.9727139472961426,0.0,0.9613838195800781,9.074604033026844,1.0401672876466643,1,3.179383432958275,3.179383432958275,3.307090641848949,985.56640625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleTrackObjects,1w_1t,1,1,2.8912010192871094,0.0,3.975447654724121,12.278937883209437,0.2515439987775152,1,9.587221414782107,9.587221414782107,2.411608011839718,1643.51171875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleTumor,1w_1t,1,1,2.7188608646392822,0.0,1.5136144161224365,9.700594955123961,0.660670240286024,1,4.360487373080105,4.360487373080105,2.880844240537006,1095.109375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleUntangleAndStraightenWorms,1w_1t,1,1,3.427931070327759,0.0,1.4363837242126465,10.562146157957613,0.6961927952421975,1,3.4416790287941694,3.4416790287941694,2.396072143382664,893.65625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleUntangleWorms,1w_1t,1,1,3.342055559158325,0.0,2.3640949726104736,11.418135286774486,0.42299485070846504,1,4.618807300925255,4.618807300925255,1.9537317047060465,944.69921875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleUntangleWormsBrightField,1w_1t,1,1,3.7159950733184814,0.0,3.9000895023345947,13.530524459667504,0.2564043721051529,1,7.377188352867961,7.377188352867961,1.8915433475185566,1056.5,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleVitra,1w_1t,1,1,3.1241512298583984,0.0,1.5152015686035156,10.796134633012116,0.6599781974365624,1,4.122247557155788,4.122247557155788,2.7205935121589495,994.03125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleWoundHealing,1w_1t,1,1,2.546600341796875,0.0,3.990922212600708,12.103351315949112,0.250568652238487,1,4.702593941241503,4.702593941241503,1.1783226258817583,1310.16015625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleYeastColonies,1w_1t,1,1,3.508812189102173,0.0,3.352222442626953,12.572134739719331,0.2983095594385302,1,12.096297012176365,12.096297012176365,3.6084410325399405,1155.59375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+ExampleYeastPatches,1w_1t,1,1,3.55591082572937,0.0,3.5286455154418945,12.408702468965203,0.2833948594790399,1,5.990533472970128,5.990533472970128,1.6976863917768543,1161.3125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+cp_tutorial_3d_monolayer,1w_1t,1,1,4.238352060317993,0.0,11.372286319732666,21.959275498986244,0.0879330656901283,1,14.058660756796598,14.058660756796598,1.2362211398426242,1490.5078125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+cp_tutorial_advanced_segmentation_final,1w_1t,1,1,6.258520603179932,0.0,10.826166152954102,23.582449715118855,0.09236880220308952,1,36.253466514870524,36.253466514870524,3.3486892776884045,1161.5546875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+cp_tutorial_quality_control,1w_1t,1,1,2.458336353302002,0.0,0.6694271564483643,8.554739723913372,1.4938145104621763,1,4.787729676347226,4.787729676347226,7.151980062697866,872.07421875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+cp_tutorial_beginner_segmentation_final,1w_1t,1,1,4.205931663513184,0.0,7.010474681854248,17.030014875344932,0.14264369324211057,1,16.074153264053166,16.074153264053166,2.2928765873242702,1074.6484375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+cp_tutorial_pixel_based_classification,1w_1t,1,1,2.822808265686035,0.0,1.9511425495147705,10.231026147026569,0.5125202155263796,1,4.208680049050599,4.208680049050599,2.157033605820987,1083.15625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+cp_tutorial_translocation_final,1w_1t,1,1,3.1146538257598877,0.0,1.0031156539916992,9.29117836104706,0.9968940231574484,1,3.004684013314545,3.004684013314545,2.995351534350005,944.1484375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+cp4_supplement_combine_objects,1w_1t,1,1,2.2870330810546875,0.0,0.1528942584991455,7.976566542871296,6.540467966660689,1,1.8334413380362093,1.8334413380362093,11.991564340177339,839.62890625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b
+cp_tutorial_translocation_start,1w_1t,1,1,2.454098701477051,0.0,0.2975425720214844,8.005126714706421,3.360863600815395,1,2.1760300607420504,2.1760300607420504,7.31334022542807,880.6484375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,2b5c4e662dc67ed43cb7f22fe155ad3ada3f695cf7ab39ba4196cf20b2fa1c7b