Repository navigation
Conversation
msgpackr decodes msgpack 64-bit integers as BigInt, and convertToValue had no branch for it, so the protobuf Value for that field had no kind. The Relaunch form then read the dataclass input as blank and resubmitted the empty value, which the backend rejects. Signed-off-by: 1fanwang <1fannnw@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Relaunching an execution whose dataclass input contains a large integer, above 4294967295 or below -2147483648, shows that field empty, and submitting the form sends the field back empty, so Flyte rejects the new execution. Relaunch now shows and sends the number.
Type
Are all requirements met?
Complete description
Python writes small integers in a compact form and larger ones in a wider form. The console decoded the wider form but dropped it before the form value was built, so the field arrived empty. The form showed nothing for it, and sending the form unchanged sent the empty field back, which Flyte refuses with the error in the linked issue.
The fix carries those integers through as ordinary numbers. The value holds numbers as doubles, so an integer beyond 2^53, about 9 quadrillion, rounds to the nearest number a double can hold. Anything smaller is exact.
Tracking Issue
fixes flyteorg/flyte#8105
Follow-up issue
N/A
Testing Done
End to end on a Flyte v1.16.4 sandbox: run a workflow whose dataclass input holds 10737418240, open that execution in the console served from this repository, press Relaunch and submit the form unchanged. Master's execution page does not load against flyteadmin since #942, and #948 fixes that, so the red run serves master 73d8292 plus that pull request, and the green run adds this change on top.
Sandbox and client tools on macOS arm64:
The workflow, run once with the large value. Relaunch starts from this execution:
Reproducer source: wf_8105.py
.venv/bin/pyflyte run --remote --project flytesnacks --domain development wf_8105.py wf --params '{"name": "vol", "volume_size": 10737418240}' ./flytectl get execution -p flytesnacks -d development axxqf52fctjt47dmbcdsThe console dev server, from the repository root with Node v22.17.0, behind a small proxy that serves the console and flyteadmin from one origin as a deployment does:
Reproducer source: proxy.js
The relaunch script drives Chromium through Relaunch, prints the form fields, submits, and saves the request. The check script decodes that request and reads the new execution's phase from flyteadmin:
Reproducer source: relaunch_8105.py
Reproducer source: check_exec.py
Red, on master plus #948. The params field is empty, the request carries volume_size with no value, and the new execution fails in flyteadmin with the error from the linked issue:
.venv/bin/python check_exec.py red aqjsdt2fgdwgjv5srf5c curl -s -w '\nHTTP %{http_code}\n' localhost:30080/api/v1/executions/flytesnacks/development/aqjsdt2fgdwgjv5srf5cGreen, with this change cherry-picked on the same base, after the dev server recompiles. The form shows the number, the request carries it, and the relaunched execution succeeds with the original value in its output:
The two browser console errors, the React key warning in NavigationItems and the refused connection, appear on both runs.
The probes below cover the integer boundaries. They run the real msgpack decoder and the real Relaunch form helpers on the bytes that released flytekit writes for a dataclass. The change also adds a regression test for CI.
First, the bytes flytekit writes for a dataclass with a string and an integer field:
Reproducer source: flytekit_bytes.py
Next, what msgpackr 1.11.2, the version in yarn.lock, returns for those bytes, run from the repository root. It returns a BigInt for the 64-bit forms, and the console's convertToValue in decodeMsgPackLiterals.ts had no case for a BigInt, which left the protobuf Value for that field with no kind.
Reproducer source: probe.mjs
Then the console itself. The probe decodes each literal with the console's decoder, reads it the way the Relaunch form does, and builds what the form submits when left unchanged. It is saved outside the repository and run from the repository root, first on the decoder from master and then on this change.
Reproducer source: relaunch_probe.ts
On the decoder from master:
git checkout upstream/master -- packages/oss-console/src/common/decodeMsgPackLiterals.ts TS_NODE_SKIP_PROJECT=true TS_NODE_TRANSPILE_ONLY=true TS_NODE_COMPILER_OPTIONS='{"module":"commonjs","esModuleInterop":true,"jsx":"react","target":"es2019","resolveJsonModule":true}' node -r ts-node/register relaunch_probe.tsWith this change:
git checkout HEAD -- packages/oss-console/src/common/decodeMsgPackLiterals.ts TS_NODE_SKIP_PROJECT=true TS_NODE_TRANSPILE_ONLY=true TS_NODE_COMPILER_OPTIONS='{"module":"commonjs","esModuleInterop":true,"jsx":"react","target":"es2019","resolveJsonModule":true}' node -r ts-node/register relaunch_probe.ts