Support Worker-variant callbacks

[chrsmith]

⚠️ This is part of a stacked PR set, to be merged into feature/worker-callbacks. This will not go directly into main, until the overall feature is code complete.

---

What changed?

THIS IS IT! The actual PR that implements Worker callbacks!

This PR provides the implementation of the Worker-variant callback in the CHASM Callback component. A Worker-variant completion callback attached to a Workflow, Workflow Update, standalone Activity, or standalone Nexus Operation will result in a Nexus operation being invoked within the same namespace.

Additional changes/refactorings

The CallbackInfo.BlockedReason is now properly set. Previously it was left unimplemented in CHASM, and only callbacks attached to Workflows would report their status. We now wire through a destinationBlocked DestinationBlockedFn with the CHASM context for Callbacks, and inject it via CallbackDestinationBlockedProvider in service/history/fx.go.

Quirks/issues

We deliver all completion callbacks using the same request ID. This means however, that if multiple completion callbacks to the same (task queue, service, operation) were added in the same request (e.g. to StartNexusOperationExecution.) then all of the worker callback invocations would carry the same request ID. The Nexus handler could interpret subsequent calls as duplicates and ignore them.

This is something we can fix, by persisting a little more information in CallbackState but it didn't seem like a realistic scenario; attaching multiple worker callbacks, in the same request, to the same taskqueue/operation.

Why?

This is the new worker callbacks feature that will make developing "Nexus Connectors" (AKA fronting a Nexus service by a different API protocol) easier.

How did you test it?

• built
• run locally and tested manually
• covered by existing tests
• added new unit test(s)
• added new functional test(s)

Potential risks

While behind configuration flags, it's possible to attach a Worker-variant callback to Workflows, Workflow Updates, standalone Activities, and standalone Nexus Operations. In all cases they should behave the exact same. But there is opportunity for some subtle difference to cause problems.

NOTE: Worker-variant callbacks are NOT supported for HSM-backed workflows. Attaching a Worker callback to an HSM workflow will fail in glorious ways.

[github-actions]

Claude finished @chrsmith's task in 8m 48s —— View job

---

Reviewed 20 files, 3 findings.
• chrsmith/wc-support-worker-variant-callbacks

[chrsmith]

I looked into adding this as another commit to this PR, but it was an involved enough yak shave I'll save it for a new PR in the stack. (It requires some proto changes from the api package, as well as wiring through from all the execution types that register completion callbacks.)