Skip to content

fix(amber, v1.2): report a malformed PVE websocket handshake - #7866

Open
github-actions[bot] wants to merge 1 commit into
release/v1.2from
backport/7852-report-a-malformed-pve-websocket-handsha-v1.2
Open

fix(amber, v1.2): report a malformed PVE websocket handshake#7866
github-actions[bot] wants to merge 1 commit into
release/v1.2from
backport/7852-report-a-malformed-pve-websocket-handsha-v1.2

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Automated backport of #7852 to release/v1.2.

Source: c9a0ddf · automation run

Any related issues, documentation, discussions?

Backport of #7852.

How was this PR tested?

Release-branch CI runs on this branch once the conflicts are resolved and this PR is marked ready for review.

Was this PR authored or co-authored using generative AI tooling?

No.

@github-actions

Copy link
Copy Markdown
Contributor Author

The cherry-pick conflicted and was committed with conflict markers. Resolve the conflicts on this branch, then mark this PR ready for review.

Conflicting files:

  • amber/src/main/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveWebsocketResource.scala
  • amber/src/test/scala/org/apache/texera/web/resource/pythonvirtualenvironment/PveWebsocketResourceSpec.scala

@codecov-commenter

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
3887 1 3886 9
View the top 1 failed test(s) by shortest run time
org.apache.texera.amber.engine.e2e.DataProcessingSpec::Engine should execute headerlessCsv->keyword workflow with MATERIALIZED mode
Stack Traces | 120s run time
com.twitter.util.TimeoutException: 1.minutes
      at com.twitter.util.Promise.ready(Promise.scala:680)
      at com.twitter.util.Promise.result(Promise.scala:689)
      at com.twitter.util.Await$.$anonfun$result$1(Awaitable.scala:155)
      at com.twitter.concurrent.LocalScheduler$Activation.blocking(Scheduler.scala:189)
      at com.twitter.concurrent.LocalScheduler.blocking(Scheduler.scala:256)
      at com.twitter.concurrent.Scheduler$.blocking(Scheduler.scala:85)
      at com.twitter.util.Await$.result(Awaitable.scala:155)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.executeWorkflow(DataProcessingSpec.scala:152)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.$anonfun$new$13(DataProcessingSpec.scala:372)
      at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
      at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
      at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
      at org.scalatest.Transformer.apply(Transformer.scala:22)
      at org.scalatest.Transformer.apply(Transformer.scala:20)
      at org.scalatest.flatspec.AnyFlatSpecLike$$anon$5.apply(AnyFlatSpecLike.scala:1832)
      at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
      at org.scalatest.TestSuite.withFixture$(TestSuite.scala:195)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.super$withFixture(DataProcessingSpec.scala:71)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.$anonfun$withFixture$1(DataProcessingSpec.scala:71)
      at org.scalatest.Retries.withRetry(Retries.scala:345)
      at org.scalatest.Retries.withRetry$(Retries.scala:344)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.withRetry(DataProcessingSpec.scala:57)
      at org.scalatest.Retries.withRetry(Retries.scala:205)
      at org.scalatest.Retries.withRetry$(Retries.scala:205)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.withRetry(DataProcessingSpec.scala:57)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.withFixture(DataProcessingSpec.scala:71)
      at org.scalatest.flatspec.AnyFlatSpecLike.invokeWithFixture$1(AnyFlatSpecLike.scala:1830)
      at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$runTest$1(AnyFlatSpecLike.scala:1842)
      at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
      at org.scalatest.flatspec.AnyFlatSpecLike.runTest(AnyFlatSpecLike.scala:1842)
      at org.scalatest.flatspec.AnyFlatSpecLike.runTest$(AnyFlatSpecLike.scala:1824)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.org$scalatest$BeforeAndAfterEach$$super$runTest(DataProcessingSpec.scala:57)
      at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
      at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.runTest(DataProcessingSpec.scala:57)
      at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$runTests$1(AnyFlatSpecLike.scala:1900)
      at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
      at scala.collection.immutable.List.foreach(List.scala:323)
      at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
      at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:390)
      at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:427)
      at scala.collection.immutable.List.foreach(List.scala:323)
      at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
      at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
      at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
      at org.scalatest.flatspec.AnyFlatSpecLike.runTests(AnyFlatSpecLike.scala:1900)
      at org.scalatest.flatspec.AnyFlatSpecLike.runTests$(AnyFlatSpecLike.scala:1899)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.runTests(DataProcessingSpec.scala:57)
      at org.scalatest.Suite.run(Suite.scala:1114)
      at org.scalatest.Suite.run$(Suite.scala:1096)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.org$scalatest$flatspec$AnyFlatSpecLike$$super$run(DataProcessingSpec.scala:57)
      at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$run$1(AnyFlatSpecLike.scala:1945)
      at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
      at org.scalatest.flatspec.AnyFlatSpecLike.run(AnyFlatSpecLike.scala:1945)
      at org.scalatest.flatspec.AnyFlatSpecLike.run$(AnyFlatSpecLike.scala:1943)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.org$scalatest$BeforeAndAfterAll$$super$run(DataProcessingSpec.scala:57)
      at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
      at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
      at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
      at org.apache.texera.amber.engine.e2e.DataProcessingSpec.run(DataProcessingSpec.scala:57)
      at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
      at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
      at sbt.TestRunner.runTest$1(TestFramework.scala:153)
      at sbt.TestRunner.run(TestFramework.scala:168)
      at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.$anonfun$apply$1(TestFramework.scala:336)
      at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:296)
      at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:336)
      at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:336)
      at sbt.TestFunction.apply(TestFramework.scala:348)
      at sbt.Tests$.$anonfun$toTask$1(Tests.scala:436)
      at sbt.std.Transform$$anon$3.$anonfun$apply$2(Transform.scala:47)
      at sbt.std.Transform$$anon$4.work(Transform.scala:69)
      at sbt.Execute.$anonfun$submit$2(Execute.scala:283)
      at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
      at sbt.Execute.work(Execute.scala:292)
      at sbt.Execute.$anonfun$submit$1(Execute.scala:283)
      at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
      at sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
      at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
      at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
      at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
      at java.base/java.lang.Thread.run(Thread.java:840)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

### What changes were proposed in this PR?

`PveWebsocketResource.onOpen` read the handshake parameters in its
prologue, outside the `Future` that owns both the `catch` arm that turns
a failure into an `[ERR]` line and the pump that writes to the socket. A
handshake missing `cuid`, carrying an empty value list for it, or
carrying a non-numeric one threw out of `onOpen` itself, ahead of both —
so the endpoint's own error contract could not see it:

```
Before:  ?pveName=env&action=install  ->  NumberFormatException out of onOpen  ->  socket closes, nothing sent
After:   ?pveName=env&action=install  ->  [ERR] Missing required parameter: cuid  ->  __DONE__
```

The client side makes that silent close worse than a lost message:
`computing-unit-selection.component.ts` clears `isInstalling`/`isLocked`
only when `onmessage` sees `__DONE__`, and there is no `onclose`
handler, so the pip modal stayed locked on "installing" with an empty
log until the user navigated away.

The three reads move inside the `try`, so a malformed handshake now
travels the same path as any other failure — an `[ERR]` line followed by
the sentinel. A `requiredParam` helper rejects an absent key, an empty
value list, and a blank value alike; the last one is reachable from the
wire, since `?pveName=` arrives as `[""]` and used to resolve to a venv
directory named `" "` rather than failing.

| Handshake | Before | After |
| --- | --- | --- |
| no `cuid` | NPE out of `onOpen`, socket closes empty | `[ERR] Missing
required parameter: cuid` then `__DONE__` |
| `cuid` present, value list empty | IndexOutOfBoundsException, same |
same as above |
| `cuid=abc` | NumberFormatException, same | `[ERR] Invalid cuid: abc`
then `__DONE__` |
| `pveName=` (blank) | resolved a venv path with a whitespace name |
`[ERR] Missing required parameter: pveName` then `__DONE__` |

Well-formed handshakes are untouched: the parsed `cuid`/`pveName` reach
`PveManager` exactly as before.

### Any related issues, documentation, discussions?

Follows #7847, which added `PveWebsocketResourceSpec` and deliberately
left this path unpinned so that fixing it would not have to fight a test
that had cemented it.

### How was this PR tested?

Four cases added to `PveWebsocketResourceSpec` — `cuid` absent, `cuid`
with an empty value list, a non-numeric `cuid`, and a blank `pveName` —
each asserting the client receives both the `[ERR]` line and the
sentinel, and that the pump stops rather than parking in `queue.take()`.

```bash
sbt "WorkflowExecutionService/testOnly org.apache.texera.web.resource.pythonvirtualenvironment.PveWebsocketResourceSpec"
```

`Tests: succeeded 7, failed 0`. Written before the fix: all four failed
against the old source with the escaping exception itself
(`NullPointerException`, `IndexOutOfBoundsException`,
`NumberFormatException`) rather than an assertion mismatch, which is the
defect stated as a test. Hoisting the reads back out of the `Future`
fails all four again.

Mutation testing on the new code — 4 mutants, 4 killed: dropping the
`values == null` arm, dropping the `isEmpty` arm, hardcoding `cuid` into
the message instead of interpolating the parameter name, and swallowing
a bad `cuid` as `0`.

`PveResourceSpec`, the other suite in the package, is unaffected —
locally it reports 6 pre-existing failures that all come from the
Windows interpreter path (`Scripts\python.exe`), unrelated to the
endpoint.

```bash
sbt scalafmtCheckAll "scalafixAll --check"
```

Clean.

Backport notes -- two places where `release/v1.2` differs from the source commit's
branch, so the cherry-pick did not apply as-is:

- `PveManager.createNewPve`/`installUserPackages` still take an `isLocal` flag here, so
  `val isLocal = !KubernetesConfig.kubernetesComputingUnitEnabled` stays in `onOpen`'s
  prologue rather than moving into the `try` with the other three reads. It reads
  configuration, not the handshake, so none of the failure modes this commit fixes can
  originate from it, and the endpoint's error contract is unchanged by where it sits.
- The spec's `pythonBinFor` helper drops the source commit's Windows
  `Scripts/python.exe` arm. This branch's `PveManager.pythonBinPath` resolves
  `bin/python` unconditionally -- the platform split is a later change to `PveManager`
  that this backport does not carry -- so keeping the arm would assert an interpreter
  path the endpoint under test cannot produce.

### Was this PR authored or co-authored using generative AI tooling?

(backported from commit c9a0ddf)

Generated-by: Claude Code (Opus 5)
@aglinxinyuan
aglinxinyuan force-pushed the backport/7852-report-a-malformed-pve-websocket-handsha-v1.2 branch from f5bd718 to 11b6c3d Compare August 23, 2026 05:26
@aglinxinyuan
aglinxinyuan marked this pull request as ready for review August 23, 2026 05:26
Copilot AI lite review requested due to automatic review settings August 23, 2026 05:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown
Contributor Author

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @SarahAsad23
    You can notify them by mentioning @SarahAsad23 in a comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants