Skip to content

Bundle the ggsql-jupyter kernel into the VS Code extension - #523

Open
samclark2015 wants to merge 8 commits into
mainfrom
sclark/bundle-kernel-in-vsix
Open

Bundle the ggsql-jupyter kernel into the VS Code extension#523
samclark2015 wants to merge 8 commits into
mainfrom
sclark/bundle-kernel-in-vsix

Conversation

@samclark2015

@samclark2015 samclark2015 commented Aug 18, 2026

Copy link
Copy Markdown

Installing the ggsql extension is now enough to run queries in Positron — no separate native installer. Groundwork for posit-dev/positron#14954.

Today the extension registers a runtime for a kernel that may not exist, and starting a session fails with KS-19: Kernel path not found: ggsql-jupyter.

What changes

Extension (ggsql-vscode/src/manager.ts, package.json)

  • Discovers a kernel at <extensionPath>/bundled/bin/ggsql-jupyter[.exe], which the per-platform VSIXes now carry.
  • New ggsql.kernelStrategybundled (default) | environment | path — modelled on air.executableStrategy. A ggsql.kernelPath set before this existed still means "use that path", via inspect() rather than get() so a set value is distinguishable from the default.
  • Fixes the phantom runtime. Discovery appended the bare binary name whether or not it was on PATH, and the accessibility check waved through any non-absolute path. With no kernel and no bundle, discovery now yields zero runtimes instead of one that fails at session start.
  • Runs the bundled kernel before offering it. A bundled binary can pass every filesystem check and still be rejected by the dynamic linker — on Linux that is the common case, not an edge one: the kernels are built on Ubuntu 24.04 and need GLIBC_2.39, while Positron supports back to Ubuntu 20.04 and RHEL 9. Discovery probes it with ggsql-jupyter --version (a flag the kernel gains in this PR; there was no way to ask before) and falls back to a kernel installed on the machine when it fails. selectKernelCandidates returns the host tier as a callback, so a bundled kernel that runs never pays for the PATH lookup; only the bundled kernel is probed — a kernel the user installed is taken at its word.
  • A probe success is cached against the extension version, keeping it to one spawn per update; a failure is not cached, so a host that gains the missing libraries starts working without waiting for an update. The Jupyter kernel spec is written only for a kernel that passed — it outlives the window, is what Quarto resolves, and has no fallback. A fallback that succeeds stays silent (the runtime's name in the picker already discloses where it came from); only the dead end — nothing runnable anywhere, whether the bundled kernel failed or the build carries none — raises one non-modal notice per extension version.
  • The bundled kernel gets a fixed runtimeId rather than one hashed from its path: that path contains the versioned extension directory, so hashing it would mint a new runtime on every update and lose runtime affinity and the session restore added in 56174ec. It also shows as plain ggsql, since it is the default.

Release (.github/workflows/release-packages.yml)

  • Each platform job uploads its ggsql-jupyter as an artifact, taken after signing so the extension ships the same binary the installer does.
  • build-vsix stages that artifact and packages six VSIXes: five platform targets plus a kernel-less universal build for everything else. publish-openvsx publishes them.
  • This has to live in release-packages.yml: Actions artifacts are run-scoped, and two workflows triggered by the same tag run in parallel, so a separate workflow could not reach the kernels. release-vscode.yaml is retired.
  • create-release's globs are now scoped per artifact directory. artifacts/**/*.exe would also have matched the raw ggsql-jupyter.exe, publishing one platform's bare kernel next to the installers.

Tests — 53 new (src/test/kernelDiscovery.test.ts now 50, plus 3 integration)

  • A Positron integration suite (src/test/integration/) that launches the bundled kernel via @posit-dev/positron-test-electron: asserts one registered ggsql runtime with the ggsql-bundled id under bundled/bin/, and that executeCode starts a session and returns a result. Everything else about bundling can pass while the binary fails to start; only this catches that. Verified locally against Positron daily 2026.09.0-100.
  • discoverAllRuntimes had no coverage, so "no kernel means no runtime" was only asserted at selectKernelCandidates, which returns candidates rather than runtimes. Testing it needed a seam: discovery writes a Jupyter kernel spec as a side effect, and with the default directory a test run would repoint the real kernelspec at a fixture, so GgsqlRuntimeManager takes an optional kernelSpecDir.
  • The probe needed a seam too: a fixture file cannot be a runnable kernel, so the manager takes an optional probe (and a globalState stand-in for the success cache), and the registration tests inject the verdict — handover to an installed kernel, the once-per-version warning, the cache surviving a new window. A separate kernel probe suite exercises the real probeKernel against actual binaries: exit zero, exit non-zero, not executable, missing.
  • Host discovery and the symlink dedupe are covered by redirecting HOME/PATH instead of depending on what the developer has installed; the strategy settings are pinned against the real configuration service, since a stubbed inspect() cannot prove the migration.
  • test-extension.yaml gains a three-OS matrix — discovery branches on the OS for the binary name, the PATH lookup, the executable-bit repair and the locations it searches — and a packaging check on every PR rather than only at release time, run after the tests so that .vscode-test/ is populated and an ignore rule letting a cache into the package actually fails.

Packaging (ggsql-vscode/.vscodeignore)

  • Removes the !**/*.d.ts re-inclusion. Both test caches are excluded, but that later rule pulled 269 stray .d.ts files back out of them and into the VSIX, because the last matching rule wins. Pre-existing on main (134 of them from .vscode-test/); nothing reads a .d.ts at runtime. The universal build drops from 149 files to 15.
  • Excludes .positron-test/, or vsce package walks the 2.9 GB Positron download and dies in its secret scanner on a directory symlink inside the app bundle.

Verified, not assumed

  • Open VSX takes the platform from the TargetPlatform attribute vsce package --target writes into extension.vsixmanifest, defaulting to universal when absent (ExtensionProcessor.getTargetPlatform()), and ovsx discards a target option for an already-packaged vsix. So the packaged file is published as-is, with no target passed to the publish action.
  • Packaged locally with a real kernel: 46.3 MiB binary, 16.21 MiB VSIX over 16 files (the plan projected ~15.6 MiB), and 106 KB for the kernel-less universal build.
  • extension/bundled/bin/ggsql-jupyter ships without adding a .vscodeignore rule, the zip preserves -rwxr-xr-x, and the universal build has neither the attribute nor the binary. build-vsix asserts all of this per target so a targeted-but-empty VSIX cannot be published.

Testing

CI covers the extension code on three platforms plus the kernel launch. Nothing on a PR exercises release-packages.yml, so it needs one manual dry run before merge — publishing is gated on refs/tags/v*, so a branch dispatch builds all five kernels and all six VSIXes and publishes nothing:

gh workflow run "Release Cargo and Installer Packages" --ref sclark/bundle-kernel-in-vsix

Not in this PR

  • win32-arm64 is not built (no runner produces that kernel). The universal VSIX is not a fallback: Positron's bootstrap appends ?targetPlatform=<target> and gets HTTP 403 for a target that was never published. The Positron-side entry needs either this target or a universal fallback in build/lib/extensions.ts.
  • Docs: ggsql-vscode/README.md, doc/get_started/tooling.qmd, and the stale dylibbundler note in INSTALLERS.md.
  • The VSIX version comes from ggsql-vscode/package.json, not the tag. Tagging v0.5.0 without bumping it would republish 0.4.1 and, with skipDuplicate, silently skip all six publishes.

🤖 Generated with Claude Code

samclark2015 and others added 5 commits August 18, 2026 10:07
The per-platform VSIXes will carry ggsql-jupyter at bundled/bin/, so
installing the extension is enough. Add a ggsql.kernelStrategy setting
(bundled | environment | path, modelled on air.executableStrategy) that
decides where manager.ts looks; a ggsql.kernelPath configured before the
strategy existed still means "use that path".

Also fix the phantom runtime: discovery appended the bare binary name
whether or not it was on PATH, and the accessibility check accepted any
non-absolute path, so a machine with no kernel got a registered runtime
that failed at session start with KS-19. With no kernel and no bundle,
discovery now yields nothing.

The bundled kernel gets a fixed runtimeId rather than one hashed from its
path, which contains the versioned extension directory: hashing it would
mint a new runtime on every update and lose runtime affinity and
restorable sessions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each platform job now uploads its ggsql-jupyter binary as an artifact,
taken after signing so the extension ships the same binary the installer
does. A build-vsix job stages that artifact into ggsql-vscode/bundled/bin
and packages one VSIX per target, plus a kernel-less universal build for
platforms without one; publish-openvsx then publishes the packaged files.

The VSIX build has to live in release-packages.yml rather than its own
workflow: Actions artifacts are scoped to a workflow run, and two
workflows triggered by the same tag run in parallel, so a separate
workflow could not reach the kernels. Building in one run also keeps the
kernel and the extension on the same commit.

Open VSX takes the platform from the TargetPlatform attribute that
vsce package --target writes into extension.vsixmanifest, so the packaged
file is published as-is with no target passed to the publish action.

create-release now names an artifact directory per glob instead of
matching an extension anywhere under artifacts/, which would have swept
the raw ggsql-jupyter.exe onto the release page alongside the installers.

win32-arm64 is not built: no runner produces that kernel yet.

release-vscode.yaml loses its tag trigger, which would otherwise race the
new publish, and is left as a manual path for the universal VSIX alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds the layer that was missing: a Positron integration suite that launches
the bundled kernel. The unit suites cover discovery precedence and the
release workflow proves the binary is inside the VSIX, but nothing until
now started the binary, which is the failure the bundling work is about.
It asserts one registered ggsql runtime with the ggsql-bundled id, and that
executeCode starts a session and returns a result.

Session creation needs positron.positron-supervisor, so the harness runs
with disableExtensions: false; the suite drives mocha itself because
extensionTestsPath must export run(). .vscode-test.mjs now globs one level
so the Positron suite does not also run under stock VS Code, where it
cannot pass.

discoverAllRuntimes was untested, so the "no kernel means no runtime"
requirement was only checked one level down at selectKernelCandidates,
which returns candidates rather than runtimes. Testing it needed a seam:
discovery writes a Jupyter kernel spec as a side effect, and with the
default directory a test run would repoint the real kernelspec at a
fixture. GgsqlRuntimeManager therefore takes an optional kernelSpecDir.

Also covered: host discovery and the symlink dedupe, by redirecting HOME
and PATH rather than depending on what the developer has installed; the
strategy settings read through the real configuration service, since a
stubbed inspect() cannot prove the migration; and resolveConfiguredPath.
The old fallback test passed vacuously on any machine without a kernel
installed, which was every machine including CI.

test-extension.yaml gains a three-OS matrix, because discovery branches on
the OS for the binary name, the PATH lookup, the executable-bit repair and
the locations it searches, and a packaging job asserting the universal VSIX
stays kernel-less on every PR rather than only at release time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Packaging from a working tree that had run the tests swept 269 stray .d.ts
files out of .vscode-test/ and .positron-test/ into the VSIX: both
directories are excluded, but the later `!**/*.d.ts` re-included every
.d.ts anywhere, and in .vscodeignore the last matching rule wins. Nothing
reads a .d.ts at runtime — esbuild bundles the one dependency — so the
negation only ever shipped junk, and it goes.

The Positron download cache also needed excluding outright. Without it
`vsce package` walked all 2.9 GB of it and died in the secret scanner on a
directory symlink inside the app bundle.

A clean checkout was unaffected, which is why CI never saw it, so the
packaging check moves into the job that has just run the tests and
therefore has a populated .vscode-test/. That also drops a job rather than
adding one, and it now asserts the absence of both caches.

Measured on darwin-arm64 with a real kernel: 46.3 MiB binary, 16.21 MiB
VSIX across 16 files, against the plan's ~15.6 MiB projection. The
kernel-less universal build is 106 KB.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
samclark2015 and others added 3 commits August 24, 2026 10:09
A bundled kernel is built for a platform, not for every system it can be
installed on. One linked against newer shared libraries than the host
provides is exec'd successfully and then rejected by the dynamic linker,
so it passes every filesystem check discovery makes and still cannot
serve a session. On the Linux builds this is the common case rather than
an edge one: the kernels are built on Ubuntu 24.04 and need GLIBC_2.39,
while Positron supports back to Ubuntu 20.04 and RHEL 9.

Run the bundled kernel before offering it, and put the host locations
behind it as a fallback tier. selectKernelCandidates() now returns a
KernelSelection carrying that tier as a callback, so the common case --
a bundled kernel that runs -- never pays for the PATH lookup.

Only the bundled kernel is probed; a kernel the user installed is taken
at its word. A success is cached against the extension version, keeping
it to one spawn per update; a failure is not, so a host that gains the
missing libraries starts working without waiting for an update. The
Jupyter kernel spec is written only for a kernel that passed, because it
outlives the window, is what Quarto resolves, and has no fallback.

A fallback that succeeds stays silent: the runtime's name in the picker
already discloses where it came from. Only the dead end interrupts --
nothing runnable anywhere, whether the bundled kernel failed or the build
carries none -- with one non-modal notice per extension version.

ggsql-jupyter gains --version, which the probe uses and which had no way
to be asked before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Uses a const array with `(typeof)[number]` to derive the type and a
type guard function, so the compiler enforces that KERNEL_STRATEGIES
and KernelStrategy stay in sync instead of relying on manual `as` casts.
On Windows a file that is not a valid executable fails the CreateProcess
call, which Node surfaces as a synchronous throw from execFile rather
than a callback error. probeKernel only handled the callback path, so
the promise rejected and discovery crashed instead of treating the
kernel as unrunnable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@samclark2015
samclark2015 marked this pull request as ready for review August 24, 2026 16:45
@samclark2015
samclark2015 requested a review from thomasp85 August 24, 2026 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant