Skip to content

test(e2e): wait for the notebook kernel before clicking Run All - #581

Merged
tkislan merged 1 commit into
mainfrom
test/e2e-run-all-kernel-race
Oct 23, 2026
Merged

tkislan merged 1 commit into
mainfrom
test/e2e-run-all-kernel-race

Conversation

@tkislan

@tkislan tkislan commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

clickRunAll now waits until the notebook toolbar names a kernel before it clicks Run All. This fixes the intermittent E2E (execution) failure TimeoutError: notebook "Run All" button did not appear or could not be clicked. It has hit main (2026-10-17) and unrelated PRs: #580 (failed 3 retries, run), renovate/node-24.x, follow-redirects, nteract-transform-vega.

Root cause

On open, registerControllerForNotebook registers the Deepnote controller only after notebookInterpreters.resolve() returns, and that call waits on the Python extension. On a cold runner it can finish after the editor and its Run All button are already up.

VS Code 1.140 resolves Run All against the notebook's selected-or-only kernel (getKernels(notebook).selected). When there is none, it runs _notebook.selectKernel and waits on that picker. Nothing answers the picker, so the first attempt times out waiting for output. Mocha's retry then fails with the "Run All button did not appear" error. The CI failure screenshot shows that picker open over the toolbar.

I reproduced this locally with the same VS Code (1.140.0) and ChromeDriver (150.0.7871.124) as CI. The click landed at 14:03:36.766 and the controller registered at 14:03:36.883.

The toolbar's kernel label is computed from the same getKernels(notebook).selected. So the label naming a kernel (anything but "Select Kernel") is exactly the condition under which Run All won't open the picker.

Verification

All runs were local under Xvfb, using the same commands as e2e.yml:

  • Unchanged helper: the execution shard gave 6 passing and 1 failing, with the CI error.
  • Forced race: I injected a 15 s delay before interpreter resolution in the installed extension. With the old helper, helloWorld fails with the CI error. With the new helper, it passes.
  • With this change:
    • execution shard: 7 passing, in two runs.
    • Other clickRunAll callers: agent 8 passing, interpreterKernel 2 passing, snapshots 6 passing.

Note for local E2E runs

A freshly built E2E venv currently fails the chart and SQL tests for an unrelated reason. deepnote-toolkit on Python 3.12 resolves pyarrow 26, which needs NumPy 2 (BLU-6622). CI isn't affected because it restores a cached venv. Locally, run pip install "pyarrow<26" in the E2E venv.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CqyiyR2czV5oi4Y4bu76FS

Summary by CodeRabbit

  • Tests
    • Notebook end-to-end runs now wait for a usable kernel selection before triggering Run All. If no kernel becomes available before the wait times out, the test reports a clear error instead of proceeding while the notebook is still waiting at the kernel picker. This makes automated notebook runs more predictable.

On notebook open, the Deepnote controller registers only after the
interpreter resolves, which waits on the Python extension and can finish
after the editor and its Run All button are up. A Run All click in that
window finds no kernel, so VS Code opens its kernel picker and holds the
run on it. Nothing answers the picker, the first attempt times out waiting
for output, and Mocha's retry then fails with "Run All button did not
appear". This made the execution shard flaky on main and unrelated PRs.

clickRunAll now waits until the notebook toolbar's kernel label names a
kernel: the same selected-or-only kernel VS Code resolves Run All against.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CqyiyR2czV5oi4Y4bu76FS
@tkislan
tkislan requested a review from a team as a code owner October 23, 2026 15:00
@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 040ca25f-c588-4b9c-9919-8aa07a194619
📥 Commits

Reviewing files that changed from the base of the PR and between e696374 and 2e130aa.

📒 Files selected for processing: 1
  • test/e2e/helpers/notebook.ts

Included review availability: This review used your included allowance. 4 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.

📝 Walkthrough

clickRunAll now opens the notebook and waits until the toolbar shows a named kernel before clicking Run All. The new polling helper retries after read errors and reports a timeout if no named kernel appears.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: jamesbhobbs

Merge Risk: ⚪ Minimal · up to 2e130

The Run All test helper now waits for a named kernel. No actionable issue was established, so the change is mergeable after normal checks.

Pre-merge checks: Passed 5 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 5 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: waiting for the notebook kernel before clicking Run All.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Updates Docs Passed The pull request changes only test/e2e/helpers/notebook.ts. It updates E2E synchronization and does not implement a product feature. No OSS documentation or roadmap update is required for this test-…
Full details: Updates Docs

Explanation

The pull request changes only test/e2e/helpers/notebook.ts. It updates E2E synchronization and does not implement a product feature. No OSS documentation or roadmap update is required for this test-only change.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Oct 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 38%. Comparing base (e696374) to head (2e130aa).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@          Coverage Diff          @@
##            main    #581   +/-   ##
=====================================
  Coverage     38%     38%           
=====================================
  Files        822     822           
  Lines      41098   41098           
  Branches    9044    9044           
=====================================
  Hits       15622   15622           
  Misses     23404   23404           
  Partials    2072    2072           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@tkislan
tkislan merged commit 0e9e0a4 into main Oct 23, 2026
20 checks passed
@tkislan
tkislan deleted the test/e2e-run-all-kernel-race branch October 23, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants