Repository navigation
Progress, aborting, and streamed results for long-running async nodes #1608
Description
#4731 made async nodes run as independent background tasks that show a placeholder value until they finish. Long-running nodes still need:
- Progress reporting in the UI while a task runs, through a side channel so updates don't invalidate cached results
- Aborting and retrying a running task from the UI
- Intermediate results published by a task while it runs (like progressive image decoding or streamed output), rather than only a placeholder followed by the final value
Also:
- Sandbox the graph executor in a web worker so it can recover from panics
- Sandbox individual nodes that panic so they can be disabled and let the remainder still execute
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
0HyperCube commentedon Feb 15, 2024
We already have async nodes. However it is not trivial to communicate the state of the graph execution to the front end. How would you propose to do this?
Keavon commentedon Feb 18, 2024
At the moment, while we do technically have async execution, it still blocks our event loop which requires a full start-to-finish graph compilation and execution before showing the result and returning from the event loop. (At least that's how I think it works currently? It seems to behave that way but I haven't rigorously traced the code.)
I'll have to defer to @TrueDoctor for his thoughts on how this should be implemented in our current architecture.
TrueDoctor commentedon Apr 12, 2024
Nodes need to return some sort of meaningful data. What you are proposing is essentially polling a node for completion and returning some sort of default value until the polling is successful. This could just be implemented as a node. The more interesting issue is how we can express this in terms of the caching model. It could make sense to treat long-running tasks as external track their state there as a form of extracting the impure state.
[-]Independent, async node execution[/-][+]Independent, async execution for long-running nodes[/+]TrueDoctor commentedon Mar 4, 2025
https://discord.com/channels/731730685944922173/1059075292414619689/1276094924567089174
Idea:
Callback based async execution:
Currently, every node in Graphite is wrapped in an async wrapper which returns a future. This is pretty convenient to use when using async nodes but requires extra work for non async nodes and currently has quite a few subtle semantics you need to get right. This also invites problems where the node graph execution is "blocked" on the async function, while this is technically non-blocking it still shows up as render freeze for the user. Additionally, executing everything, even sync nodes in an async context leads to additional overhead fundamentally limits what performance we can achieve with the interpreted execution model.
Solution
Fine-grained callback based async execution in the node runtime. In this new system, nodes could decide if they want to return futures or regular "sync" values. We could still permit regular async based composition but make this be based on the info provided by the type system to decide if a branch of the node graph should be sync or async. Additionally, we would introduce an
AsyncTaskRegistrywhich would be used as an argument for some sort ofExecAsyncnode. When the graph is executed a future is passed to theExecAsyncnode, instead of passing on the future, the exec node add the task to the task registry and return some sort of default value. The task registry is then polled externally. Once the task spawned by the node succeeds, the hash value of the task registry is updated triggering a reexecuton of the async node. The node then checks if the task has completed and returns that result. This could potentially be extended to support progress reporting and aborting of tasks in the future.Drawbacks
While using a scope injected global task registry is definitely the easiest and least disruptive to implement, it could potentially be inefficient. The completion of one task would trigger the reexecution of every
ExecAsyncnode. We would need to do some testing to see if this is an actual issue though.One way to address this would be to actually provide each exec async node with its own task registry If the exec async nodes are inserted by the compiler, the task registries would need to be generated by the compiler as well. And to actually be able to update the input values, they would either needed to be be provided to the compiler as inputs or we would need to permit the interior mutability for input values which could make caching and incremental compilation harder in the future
[-]Independent, async execution for long-running nodes[/-][+]Progress, aborting, and streamed results for long-running async nodes[/+]