test(pyamber): extend the OutputManager tests to cover its partitioning, output-port and emit semantics

[eugenegujing]

What changes were proposed in this PR?

This PR extends amber/src/test/python/core/architecture/packaging/test_output_manager.py (10 → 39 tests) to give the Python worker's OutputManager direct semantic assertions. Nearly every line of output_manager.py already executes under test_main_loop.py's end-to-end tests, but the behaviors below had no direct assertion anywhere in amber/src/test/python — the end-to-end suite reaches add_partitioning only through a fixture that sends OneToOnePartitioning and asserts an EmptyReturn, emit_state appears in the test tree solely as a monkeypatch target, and test_partitioners.py constructs partitioners directly rather than through the dispatch table.

• save_tuple_to_storage_if_needed: mirrors the five shapes its twin save_state_to_storage_if_needed already had — no-writer no-op, unknown-port no-op (asserting the other port's queue stays untouched), fan-out to every port when port_id is omitted, the single-selected-port path, and that the exact tuple object is enqueued unconverted; plus close_port_storage_writers stopping the tuple threads.
• add_output_port: the port_id.id None→0 and internal None→False normalizations, the "each port can only be added once" guard, and that a storage writer is only set up when a storage URI is given — all against a real OutputManager rather than the MagicMock the handler tests use.
• add_partitioning: all five Partitioning→Partitioner dispatch-table entries driven through OutputManager (parametrized), registration of only the channels sent from this worker, is_control None→False canonicalization, and OneToOnePartitioner's receiver selection by worker id.
• Emit chain: tuple_to_batch batching below/at batch size and fan-out to every partitioner (frame contents asserted via to_pydict), tuple_to_frame serialization against the port schema, emit_ecm flushing the pending batch before passing the ECM through by identity, and emit_state flush ordering, StateFrame wrapping and loop-column defaults.
• Query methods: is_missing_output_ports, get_port_ids ordering, and get_output_channel_ids listing add_partitioning's registrations.

One pre-existing test is strengthened: test_recreates_result_and_state_tables_and_reopens_writer now asserts the (uri, schema) pairing of create_document calls instead of the URI set alone, so swapping the result/state schemas can no longer pass. No production code is changed.

Any related issues, documentation, discussions?

Closes #7813

How was this PR tested?

All tests in this PR are new or strengthened unit tests; run them from amber/src/main/python with python -m pytest ../../test/python/core/architecture/packaging/ -q — 62 passed (39 in the extended file, including the 10 pre-existing ones unchanged in behavior). The suite is hermetic (no Arrow Flight, no Iceberg; storage writers are stubbed at the writer-triple boundary) and was verified stable across repeated runs. Each key assertion was mutation-checked: thirteen distinct hand-applied production mutations (dispatch-entry swaps, removal of each None normalization, removal of the once-only guard and channel filter, skipping the pending-batch flush, dropping the loop bookkeeping, gutting the fan-out, and swapping the result/state schemas) each turned exactly the intended test red, and every mutation was reverted byte-exact.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Fable 5)

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @aglinxinyuan
  You can notify them by mentioning @aglinxinyuan in a comment.

[github-actions]

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 3 worse · ⚪ 10 noise (<±5%) · 0 without baseline

Compared against main 2edbdf9 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	527	0.322	16,767/30,078/30,078 us	🔴 +8.2% / 🔴 +100.6%
🟢	bs=100 sw=10 sl=64	1,140	0.696	84,198/130,845/130,845 us	🟢 -9.8% / 🔴 +23.4%
⚪	bs=1000 sw=10 sl=64	1,352	0.825	738,552/782,940/782,940 us	⚪ within ±5% / 🟢 +29.6%

Baseline details

Latest main 2edbdf9 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	527 tuples/sec	568 tuples/sec	780.31 tuples/sec	-7.2%	-32.5%
bs=10 sw=10 sl=64	MB/s	0.322 MB/s	0.347 MB/s	0.476 MB/s	-7.2%	-32.4%
bs=10 sw=10 sl=64	p50	16,767 us	15,500 us	12,590 us	+8.2%	+33.2%
bs=10 sw=10 sl=64	p95	30,078 us	29,000 us	14,995 us	+3.7%	+100.6%
bs=10 sw=10 sl=64	p99	30,078 us	29,000 us	19,141 us	+3.7%	+57.1%
bs=100 sw=10 sl=64	throughput	1,140 tuples/sec	1,136 tuples/sec	1,006 tuples/sec	+0.4%	+13.3%
bs=100 sw=10 sl=64	MB/s	0.696 MB/s	0.693 MB/s	0.614 MB/s	+0.4%	+13.4%
bs=100 sw=10 sl=64	p50	84,198 us	83,643 us	99,387 us	+0.7%	-15.3%
bs=100 sw=10 sl=64	p95	130,845 us	145,122 us	106,056 us	-9.8%	+23.4%
bs=100 sw=10 sl=64	p99	130,845 us	145,122 us	113,689 us	-9.8%	+15.1%
bs=1000 sw=10 sl=64	throughput	1,352 tuples/sec	1,341 tuples/sec	1,043 tuples/sec	+0.8%	+29.6%
bs=1000 sw=10 sl=64	MB/s	0.825 MB/s	0.818 MB/s	0.637 MB/s	+0.9%	+29.6%
bs=1000 sw=10 sl=64	p50	738,552 us	739,063 us	965,215 us	-0.1%	-23.5%
bs=1000 sw=10 sl=64	p95	782,940 us	809,232 us	1,009,610 us	-3.2%	-22.5%
bs=1000 sw=10 sl=64	p99	782,940 us	809,232 us	1,039,108 us	-3.2%	-24.7%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,379.29,200,128000,527,0.322,16766.78,30078.40,30078.40
1,100,10,64,20,1754.00,2000,1280000,1140,0.696,84198.12,130845.29,130845.29
2,1000,10,64,20,14792.63,20000,12800000,1352,0.825,738552.31,782940.05,782940.05

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.38%. Comparing base (2edbdf9) to head (9a78606).

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #7815      +/-   ##
============================================
- Coverage     91.50%   91.38%   -0.13%     
+ Complexity     4501     4453      -48     
============================================
  Files          1177     1171       -6     
  Lines         47489    47032     -457     
  Branches       5324     5269      -55     
============================================
- Hits          43457    42979     -478     
- Misses         2366     2408      +42     
+ Partials       1666     1645      -21     

Flag	Coverage Δ		*Carryforward flag
access-control-service	81.00% <ø> (ø)		Carriedforward from 9c5aecc
agent-service	98.62% <ø> (ø)		Carriedforward from 9c5aecc
amber	88.11% <ø> (+0.07%)	⬆️	Carriedforward from 9c5aecc
computing-unit-managing-service	73.67% <ø> (ø)		Carriedforward from 9c5aecc
config-service	86.73% <ø> (ø)		Carriedforward from 9c5aecc
file-service	65.41% <ø> (-10.34%)	⬇️	Carriedforward from 9c5aecc
frontend	92.88% <ø> (-0.36%)	⬇️	Carriedforward from 9c5aecc
notebook-migration-service	78.86% <ø> (-0.28%)	⬇️	Carriedforward from 9c5aecc
pyamber	97.67% <ø> (+0.10%)	⬆️
workflow-compiling-service	77.19% <ø> (ø)		Carriedforward from 9c5aecc

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.