From 20364490a478d10e0097fce223ad96bfa31459bd Mon Sep 17 00:00:00 2001 From: Sam Clark Date: Wed, 19 Aug 2026 14:04:08 -0500 Subject: [PATCH] Fall back to an installed kernel when the bundled one cannot run A bundled kernel is built for a platform, not for every system it can be installed on. One linked against newer shared libraries than the host provides is exec'd successfully and then rejected by the dynamic linker, so it passes every filesystem check discovery makes and still cannot serve a session. On the Linux builds this is the common case rather than an edge one: the kernels are built on Ubuntu 24.04 and need GLIBC_2.39, while Positron supports back to Ubuntu 20.04 and RHEL 9. Run the bundled kernel before offering it, and put the host locations behind it as a fallback tier. selectKernelCandidates() now returns a KernelSelection carrying that tier as a callback, so the common case -- a bundled kernel that runs -- never pays for the PATH lookup. Only the bundled kernel is probed; a kernel the user installed is taken at its word. A success is cached against the extension version, keeping it to one spawn per update; a failure is not, so a host that gains the missing libraries starts working without waiting for an update. The Jupyter kernel spec is written only for a kernel that passed, because it outlives the window, is what Quarto resolves, and has no fallback. A fallback that succeeds stays silent: the runtime's name in the picker already discloses where it came from. Only the dead end interrupts -- nothing runnable anywhere, whether the bundled kernel failed or the build carries none -- with one non-modal notice per extension version. ggsql-jupyter gains --version, which the probe uses and which had no way to be asked before. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 11 + ggsql-jupyter/src/main.rs | 3 + ggsql-vscode/CHANGELOG.md | 4 + ggsql-vscode/CLAUDE.md | 11 +- ggsql-vscode/src/extension.ts | 5 + ggsql-vscode/src/manager.ts | 262 ++++++++++++-- ggsql-vscode/src/test/kernelDiscovery.test.ts | 333 ++++++++++++++++-- 7 files changed, 563 insertions(+), 66 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 67c520ae4..9d762a765 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,17 @@ a kernel installed on the machine, and a fixed path in `ggsql.kernelPath`; configuring `ggsql.kernelPath` alone continues to mean that path is used. +- The extension now runs the bundled kernel before offering it, and falls back to + a kernel installed on the machine when it does not start. A bundled binary can + be built against newer system libraries than the host provides — every + filesystem check passes and the kernel still dies the moment it is launched — + so the extension runs `ggsql-jupyter --version` once per update and treats a + failure as "not usable here". When nothing on the machine can run, including + builds that carry no kernel at all, the extension says so once and points at + the install instructions instead of offering a runtime that cannot start. + +- `ggsql-jupyter` accepts `--version`. + ### Changed - Dodging now only takes effect where groups actually meet on a position. A layer whose grouping gives every group a position of its own — `colour` mapped diff --git a/ggsql-jupyter/src/main.rs b/ggsql-jupyter/src/main.rs index fccb8906b..b9e7f94d9 100644 --- a/ggsql-jupyter/src/main.rs +++ b/ggsql-jupyter/src/main.rs @@ -18,6 +18,9 @@ use std::process::Command; #[derive(Parser)] #[command(name = "ggsql-jupyter")] #[command(about = "Jupyter kernel for ggsql", long_about = None)] +// `--version` doubles as a liveness probe: the VS Code extension runs it to +// confirm a kernel binary actually loads on this machine before offering it. +#[command(version)] struct Args { /// Path to the Jupyter connection file #[arg(short = 'f', long = "connection-file")] diff --git a/ggsql-vscode/CHANGELOG.md b/ggsql-vscode/CHANGELOG.md index 0fe10a2b0..b44f5dd37 100644 --- a/ggsql-vscode/CHANGELOG.md +++ b/ggsql-vscode/CHANGELOG.md @@ -9,6 +9,10 @@ uses one installed on the machine. - Fixed: no ggsql runtime is offered when no kernel can be found, rather than one that fails at session start with `KS-19: Kernel path not found`. +- The bundled kernel is run before it is offered, and a kernel installed on the + machine is used instead when it does not start. If nothing on the machine can + run, the extension points at the install instructions once rather than offering + a runtime that fails at session start. ## 0.3.2 diff --git a/ggsql-vscode/CLAUDE.md b/ggsql-vscode/CLAUDE.md index dab22fdb4..54fd8c638 100644 --- a/ggsql-vscode/CLAUDE.md +++ b/ggsql-vscode/CLAUDE.md @@ -116,22 +116,25 @@ The extension ships the kernel: the per-platform VSIXes carry `ggsql-jupyter` at | Strategy | Candidates, in priority order | | --- | --- | -| `bundled` (default) | The bundled kernel alone. A build that carries none falls through to the host locations, so the platform-neutral VSIX behaves as it always did. | +| `bundled` (default) | The bundled kernel, with host locations behind it as a fallback tier reached only when it cannot run. A build that carries no kernel goes straight to the host locations. | | `environment` | Host locations, then the bundled kernel as the fallback. | | `path` | `ggsql.kernelPath` alone — neither the bundled kernel nor a host install stands in for it. An empty path is treated as `bundled`. | Host locations are, in order: Jupyter kernelspec directories (user then system), the native package install locations per platform, then `PATH`. -`selectKernelCandidates()` is the whole precedence rule with no filesystem in it, which is what `src/test/kernelDiscovery.test.ts` exercises; `discoverKernelPaths()` supplies it with what is actually on disk. +`selectKernelCandidates()` is the whole precedence rule with no filesystem in it, which is what `src/test/kernelDiscovery.test.ts` exercises; `discoverKernelPaths()` supplies it with what is actually on disk. It returns a `KernelSelection`: the `candidates` to offer, plus a `fallback()` callback for the tier behind them. The fallback is a callback rather than a list because reaching the host locations shells out to `which`/`where`, and the common case — a bundled kernel that runs — must not pay for it. -Four things here are load bearing: +Five things here are load bearing: +- **A kernel is run before it is offered.** Filesystem checks cannot tell whether a binary starts. The bundled kernel is built for the platform but not for every system it can be installed on: one linked against newer shared libraries than the host provides is exec'd successfully and then killed by the dynamic linker, which no `stat` or `access` call can see. `probeKernel()` runs `ggsql-jupyter --version` and requires exit 0; only the bundled kernel is probed, since a kernel the user installed is their own business. A success is cached in `globalState` against the extension version, so it costs one spawn per update rather than one per window; a failure is not cached, because it is cheap to repeat and a host that gains the missing libraries should start working without waiting for an update. - **Every candidate is an absolute path.** A candidate that is only a binary name satisfies each existence check further down and so registers a runtime that fails at session start with `KS-19: Kernel path not found`. `findOnPath()` returns `undefined` rather than the bare name, and `isKernelAccessible()` rejects any non-absolute path, so no kernel anywhere means **zero** runtimes rather than an unusable one. The single exception is a `ggsql.kernelPath` that resolves to nothing: it is passed through so discovery can report it as inaccessible in the log instead of ignoring the setting silently. - **The bundled kernel's `runtimeId` is fixed, not derived from its path.** Every other source hashes `kernelPath` to get one id per installed kernel, but the bundled path contains the versioned extension directory, so hashing it would mint a new runtime on every extension update and lose the workspace's runtime affinity and its restorable sessions. - **The bundled runtime is named plain `ggsql`.** The `ggsql ()` suffix is only worth showing for a kernel the user went out of their way to select. - **`ggsql.kernelPath` implies `path`.** Users configured that setting before a strategy existed, so a non-empty path with no explicitly set `kernelStrategy` still resolves to `path`. `resolveKernelStrategy()` reads the value through `inspect()` for that reason: `get()` cannot tell a set value from the default. -Discovery also writes the user-level Jupyter kernelspec for the bundled and system kernels, so Quarto and Jupyter can find ggsql without a session ever being started, and so the spec stops pointing into an extension directory an update has removed. +Discovery also writes the user-level Jupyter kernelspec for the bundled and system kernels, so Quarto and Jupyter can find ggsql without a session ever being started, and so the spec stops pointing into an extension directory an update has removed. Only a kernel that has passed the probe is written there: the spec outlives the window and is what Quarto resolves, and it has no fallback of its own. + +A fallback that succeeds is deliberately silent — the runtime's name in the picker already says where it came from, and the log records the handover. The one case that interrupts the user is the dead end: nothing runnable anywhere, whether because the bundled kernel failed its probe or because the build carries none (`win32-arm64` and the platform-neutral VSIX). `reportNoUsableKernel()` then shows a non-modal warning once per extension version, offering the install docs and the log. It is skipped under the `path` strategy, where the user named a binary and the log already reports it. ## Settings diff --git a/ggsql-vscode/src/extension.ts b/ggsql-vscode/src/extension.ts index 435f26a0a..b9d1c6be3 100644 --- a/ggsql-vscode/src/extension.ts +++ b/ggsql-vscode/src/extension.ts @@ -23,6 +23,11 @@ export function log(message: string): void { outputChannel.appendLine(`[${new Date().toISOString()}] ${message}`); } +/** Reveal the ggsql output channel, for notifications that offer it. */ +export function showLog(): void { + outputChannel.show(); +} + /** * Activates the extension. * diff --git a/ggsql-vscode/src/manager.ts b/ggsql-vscode/src/manager.ts index b21cd7df0..360abf09e 100644 --- a/ggsql-vscode/src/manager.ts +++ b/ggsql-vscode/src/manager.ts @@ -12,7 +12,7 @@ import * as cp from 'child_process'; import * as crypto from 'crypto'; import type * as positron from '@posit-dev/positron'; import type { JupyterKernelSpec, PositronSupervisorApi } from './types'; -import { log } from './extension'; +import { log, showLog } from './extension'; /** Where a kernel candidate was discovered */ type KernelSource = 'Bundled' | 'Setting' | 'Jupyter' | 'System' | 'Path'; @@ -210,6 +210,25 @@ export function resolveKernelStrategy(config: vscode.WorkspaceConfiguration): Ke return 'bundled'; } +/** + * The kernels a window should consider, in priority order, plus the tier to + * fall back to when none of them can run. + */ +export interface KernelSelection { + /** The strategy these candidates were chosen under. */ + strategy: KernelStrategy; + /** Candidates to offer, best first. */ + candidates: KernelCandidate[]; + /** + * Host kernels to consult when nothing in `candidates` turns out to be + * runnable. A callback, not a list, so that the common case — a bundled + * kernel that runs — never pays for the PATH lookup. + */ + fallback: () => KernelCandidate[]; +} + +const NO_FALLBACK = (): KernelCandidate[] => []; + /** * Apply a strategy to the places a kernel can come from. * @@ -221,25 +240,42 @@ export function selectKernelCandidates( bundledPath: string | undefined, configuredPath: string | undefined, hostKernels: () => KernelCandidate[], -): KernelCandidate[] { +): KernelSelection { const bundled: KernelCandidate[] = bundledPath ? [{ kernelPath: bundledPath, source: 'Bundled' }] : []; + let effective = strategy; if (strategy === 'path') { if (configuredPath) { - return [{ kernelPath: configuredPath, source: 'Setting' }]; + // The user named a binary. Nothing may quietly stand in for it, so + // there is no fallback tier here. + return { + strategy, + candidates: [{ kernelPath: configuredPath, source: 'Setting' }], + fallback: NO_FALLBACK, + }; } // Nothing to point at. Treat it as the default rather than registering // no runtime at all. log('ggsql.kernelStrategy is "path" but ggsql.kernelPath is empty; using the bundled kernel'); + effective = 'bundled'; } else if (strategy === 'environment') { - return [...hostKernels(), ...bundled]; + // Host kernels are already ahead of the bundled one, so a failure among + // them falls through to it within the same tier. + return { strategy, candidates: [...hostKernels(), ...bundled], fallback: NO_FALLBACK }; } + if (bundled.length > 0) { + // The bundled kernel is built for this platform but not for every + // system it can be installed on: it can be too new for the host's + // shared libraries, which only shows up when it is run. Host kernels + // stand behind it for exactly that case. + return { strategy: effective, candidates: bundled, fallback: hostKernels }; + } // A build that carries no kernel still looks for a host install, or it // would offer nothing at all. - return bundled.length > 0 ? bundled : hostKernels(); + return { strategy: effective, candidates: hostKernels(), fallback: NO_FALLBACK }; } /** @@ -270,19 +306,25 @@ function dedupeCandidates(candidates: KernelCandidate[]): KernelCandidate[] { * Discover the ggsql-jupyter kernels this window should offer, in priority * order. */ -export function discoverKernelPaths(context: vscode.ExtensionContext): KernelCandidate[] { +export function discoverKernelPaths(context: vscode.ExtensionContext): KernelSelection { const config = vscode.workspace.getConfiguration('ggsql'); const strategy = resolveKernelStrategy(config); log(`Kernel strategy: ${strategy}`); const configuredPath = config.get('kernelPath', '').trim(); - return dedupeCandidates(selectKernelCandidates( + const selection = selectKernelCandidates( strategy, bundledKernelPath(context), configuredPath === '' ? undefined : resolveConfiguredPath(configuredPath), discoverHostKernels, - )); + ); + + return { + strategy: selection.strategy, + candidates: dedupeCandidates(selection.candidates), + fallback: () => dedupeCandidates(selection.fallback()), + }; } /** @@ -308,6 +350,134 @@ export async function isKernelAccessible(kernelPath: string): Promise { } } +/** How long the probe waits for the kernel to report its version. */ +const KERNEL_PROBE_TIMEOUT_MS = 15000; + +/** Where the last successful probe is remembered, to keep it to one per update. */ +const PROBE_CACHE_KEY = 'ggsql.bundledKernelProbe'; + +/** Where the dead-end notice records the version it has already reported. */ +const NO_KERNEL_NOTICE_KEY = 'ggsql.noUsableKernelNotice'; + +/** Install instructions offered when no kernel on this machine can run. */ +const INSTALL_DOCS_URL = 'https://ggsql.org/get_started/installation.html'; + +interface ProbeCacheEntry { + extensionVersion: string; + kernelPath: string; +} + +/** Runs a kernel binary and reports whether it started. */ +export type KernelProbe = (kernelPath: string) => Promise; + +/** + * Run the kernel and see whether it starts. + * + * An accessibility check cannot answer this. A binary built against newer + * shared libraries than the host provides passes every filesystem test and + * still fails: the kernel is exec'd successfully and then the dynamic linker + * rejects it, so the process exits non-zero before it can serve a session. + * `--version` is the cheapest thing that exercises that whole path. + */ +export function probeKernel(kernelPath: string): Promise { + return new Promise(resolve => { + cp.execFile( + kernelPath, + ['--version'], + { timeout: KERNEL_PROBE_TIMEOUT_MS, windowsHide: true }, + err => { + if (err) { + log(`Kernel probe failed for ${kernelPath}: ${err.message}`); + } + resolve(!err); + }, + ); + }); +} + +/** + * Probe the bundled kernel, remembering a success across windows. + * + * Only a success is cached, and only for the extension version that produced + * it: a failure is cheap to repeat (the linker gives up immediately) and + * re-running it means a host that gains the libraries the kernel needs starts + * working without waiting for an extension update. + */ +async function probeBundledKernel( + context: vscode.ExtensionContext, + kernelPath: string, + probe: KernelProbe, +): Promise { + const extensionVersion = context.extension.packageJSON.version as string; + const cached = context.globalState.get(PROBE_CACHE_KEY); + if (cached?.extensionVersion === extensionVersion && cached.kernelPath === kernelPath) { + return true; + } + + const ok = await probe(kernelPath); + if (ok) { + await context.globalState.update(PROBE_CACHE_KEY, { extensionVersion, kernelPath }); + } + return ok; +} + +/** + * Decide whether a candidate can actually serve a session. + * + * The bundled kernel is additionally run, because it is the one the extension + * chose rather than the user, and it is the one that can be wrong about the + * system it landed on. A kernel the user installed is taken at its word. + */ +async function canRunKernel( + context: vscode.ExtensionContext, + candidate: KernelCandidate, + probe: KernelProbe, +): Promise { + if (!await isKernelAccessible(candidate.kernelPath)) { + return false; + } + if (candidate.source !== 'Bundled') { + return true; + } + return probeBundledKernel(context, candidate.kernelPath, probe); +} + +/** + * Tell the user that nothing on this machine can run ggsql queries. + * + * Only for the dead end: a fallback that succeeds is reported by the runtime's + * name in the picker and by the log, and needs no interruption. Shown once per + * extension version, and never awaited, so discovery does not sit waiting for + * the notification to be dismissed. + */ +function reportNoUsableKernel( + context: vscode.ExtensionContext, + bundledRejected: boolean, +): void { + const version = context.extension.packageJSON.version as string; + if (context.globalState.get(NO_KERNEL_NOTICE_KEY) === version) { + return; + } + void context.globalState.update(NO_KERNEL_NOTICE_KEY, version); + + const reason = bundledRejected + ? 'The ggsql kernel bundled with this extension cannot run on this system.' + : 'This build of the ggsql extension does not include a kernel.'; + log(`${reason} No kernel installed on this machine could be used instead.`); + + const install = 'Install ggsql'; + const showOutput = 'Show Log'; + void vscode.window + .showWarningMessage(`${reason} Install ggsql to run queries.`, install, showOutput) + .then(choice => { + if (choice === install) { + void vscode.env.openExternal(vscode.Uri.parse(INSTALL_DOCS_URL)); + } else if (choice === showOutput) { + showLog(); + } + }); +} + /** * Stable runtime identifier for a candidate. * @@ -512,6 +682,15 @@ export interface RuntimeManagerOptions { * kernelspec — the one Quarto and Jupyter resolve — at a test fixture. */ kernelSpecDir?: string; + + /** + * Liveness probe for the bundled kernel. Defaults to running it. + * + * Tests override it because a stand-in kernel cannot be a real executable + * on every platform: a shell script named ggsql-jupyter.exe is not + * something Windows can spawn. + */ + probe?: KernelProbe; } /** @@ -533,10 +712,12 @@ export class GgsqlRuntimeManager implements positron.LanguageRuntimeManager { private _context: vscode.ExtensionContext; private _kernelSpecDir: string; + private _probe: KernelProbe; constructor(context: vscode.ExtensionContext, options: RuntimeManagerOptions = {}) { this._context = context; this._kernelSpecDir = options.kernelSpecDir ?? getUserJupyterKernelDir(); + this._probe = options.probe ?? probeKernel; } /** @@ -547,33 +728,62 @@ export class GgsqlRuntimeManager implements positron.LanguageRuntimeManager { discoverAllRuntimes(): AsyncGenerator { const context = this._context; const kernelSpecDir = this._kernelSpecDir; + const probe = this._probe; const generator = async function* discoverGgsqlRuntimes() { log('Discovering ggsql runtimes...'); - const candidates = discoverKernelPaths(context); - log(`Found ${candidates.length} kernel candidate(s)`); - - for (const candidate of candidates) { - const accessible = await isKernelAccessible(candidate.kernelPath); - if (accessible) { - // Write the kernel spec to the user kernelspec dir - // immediately so that Quarto/Jupyter can discover ggsql - // even if no session is ever started. The bundled kernel - // additionally needs this on every extension update, or the - // spec keeps pointing into the removed extension directory. - if (candidate.source === 'System' || candidate.source === 'Bundled') { - writeKernelJson(kernelSpecDir, candidate.kernelPath); + const selection = discoverKernelPaths(context); + log(`Found ${selection.candidates.length} kernel candidate(s)`); + + let registered = 0; + let bundledRejected = false; + + async function* offer(candidates: KernelCandidate[]) { + for (const candidate of candidates) { + if (await canRunKernel(context, candidate, probe)) { + // Write the kernel spec to the user kernelspec dir + // immediately so that Quarto/Jupyter can discover ggsql + // even if no session is ever started. The bundled kernel + // additionally needs this on every extension update, or + // the spec keeps pointing into the removed extension + // directory. Only a kernel that has proven it runs is + // advertised this way: the spec outlives the window, and + // Quarto has no discovery of its own to fall back on. + if (candidate.source === 'System' || candidate.source === 'Bundled') { + writeKernelJson(kernelSpecDir, candidate.kernelPath); + } + + const metadata = generateMetadata(context, candidate); + log(`Yielding runtime: ${metadata.runtimeName} (${metadata.runtimeId}) at ${candidate.kernelPath}`); + registered++; + yield metadata; + } else { + if (candidate.source === 'Bundled') { + bundledRejected = true; + } + log(`Skipping unusable kernel (${candidate.source}): ${candidate.kernelPath}`); } + } + } + + yield* offer(selection.candidates); - const metadata = generateMetadata(context, candidate); - log(`Yielding runtime: ${metadata.runtimeName} (${metadata.runtimeId}) at ${candidate.kernelPath}`); - yield metadata; - } else { - log(`Skipping inaccessible kernel (${candidate.source}): ${candidate.kernelPath}`); + if (registered === 0) { + const fallback = selection.fallback(); + if (fallback.length > 0) { + log(`No usable kernel among the primary candidates; trying ${fallback.length} installed kernel(s)`); + yield* offer(fallback); } } + if (registered === 0 && selection.strategy !== 'path') { + // Under the path strategy the user named a binary that did not + // work out, which the log already reports; there is nothing to + // install that would change it. + reportNoUsableKernel(context, bundledRejected); + } + log('Runtime discovery complete'); }; diff --git a/ggsql-vscode/src/test/kernelDiscovery.test.ts b/ggsql-vscode/src/test/kernelDiscovery.test.ts index 337d9a79d..ffa149e49 100644 --- a/ggsql-vscode/src/test/kernelDiscovery.test.ts +++ b/ggsql-vscode/src/test/kernelDiscovery.test.ts @@ -9,10 +9,12 @@ import { discoverKernelPaths, generateMetadata, isKernelAccessible, + probeKernel, resolveConfiguredPath, resolveKernelStrategy, selectKernelCandidates, type KernelCandidate, + type KernelProbe, } from '../manager'; const EXTENSION_ID = 'ggsql.ggsql'; @@ -198,48 +200,69 @@ suite('kernel strategy', () => { suite('kernel candidate selection', () => { const bundled = '/ext/ggsql.ggsql-0.5.0-darwin-arm64/bundled/bin/ggsql-jupyter'; - test('bundled uses only the bundled kernel', () => { - const candidates = selectKernelCandidates('bundled', bundled, undefined, hostKernels); - assert.deepStrictEqual(candidates, [{ kernelPath: bundled, source: 'Bundled' }]); + test('bundled offers only the bundled kernel, with host kernels behind it', () => { + const selection = selectKernelCandidates('bundled', bundled, undefined, hostKernels); + assert.deepStrictEqual(selection.candidates, [{ kernelPath: bundled, source: 'Bundled' }]); + // Reachable only when the bundled kernel turns out not to run. + assert.deepStrictEqual(selection.fallback(), HOST); + }); + + test('the host lookup is not performed while choosing the bundled kernel', () => { + // The PATH lookup shells out to which/where. The default strategy is the + // common case and must not pay for it. + let calls = 0; + const counted = () => { + calls++; + return HOST; + }; + selectKernelCandidates('bundled', bundled, undefined, counted); + assert.strictEqual(calls, 0); }); test('bundled falls back to host kernels when the build carries none', () => { - // The platform-neutral VSIX ships no kernel and must keep working the - // way it does today. - const candidates = selectKernelCandidates('bundled', undefined, undefined, hostKernels); - assert.deepStrictEqual(candidates, HOST); + // The platform-neutral VSIX, and the win32-arm64 build, ship no kernel + // and must keep working through a host install. + const selection = selectKernelCandidates('bundled', undefined, undefined, hostKernels); + assert.deepStrictEqual(selection.candidates, HOST); + assert.deepStrictEqual(selection.fallback(), []); }); test('no bundle and no host kernel yields no candidates', () => { - // Regression test for the phantom runtime: discovery used to append the - // bare binary name unconditionally, so Positron registered a ggsql - // runtime that failed at session start with KS-19. - const candidates = selectKernelCandidates('bundled', undefined, undefined, noHostKernels); - assert.deepStrictEqual(candidates, []); + // Regression test for the phantom runtime: a ggsql runtime registered + // against a kernel that is not there fails at session start with KS-19. + const selection = selectKernelCandidates('bundled', undefined, undefined, noHostKernels); + assert.deepStrictEqual(selection.candidates, []); + assert.deepStrictEqual(selection.fallback(), []); }); test('environment puts host kernels ahead of the bundled one', () => { - const candidates = selectKernelCandidates('environment', bundled, undefined, hostKernels); - assert.deepStrictEqual(candidates, [...HOST, { kernelPath: bundled, source: 'Bundled' }]); + const selection = selectKernelCandidates('environment', bundled, undefined, hostKernels); + // One tier: the bundled kernel already stands behind the host ones. + assert.deepStrictEqual(selection.candidates, [...HOST, { kernelPath: bundled, source: 'Bundled' }]); + assert.deepStrictEqual(selection.fallback(), []); }); test('environment falls back to the bundled kernel', () => { - const candidates = selectKernelCandidates('environment', bundled, undefined, noHostKernels); - assert.deepStrictEqual(candidates, [{ kernelPath: bundled, source: 'Bundled' }]); + const selection = selectKernelCandidates('environment', bundled, undefined, noHostKernels); + assert.deepStrictEqual(selection.candidates, [{ kernelPath: bundled, source: 'Bundled' }]); }); test('path uses the configured kernel alone', () => { // Neither the bundled kernel nor a host install may quietly stand in for - // the one the user named. - const candidates = selectKernelCandidates('path', bundled, '/opt/ggsql/ggsql-jupyter', hostKernels); - assert.deepStrictEqual(candidates, [ + // the one the user named, so there is no fallback tier either. + const selection = selectKernelCandidates('path', bundled, '/opt/ggsql/ggsql-jupyter', hostKernels); + assert.deepStrictEqual(selection.candidates, [ { kernelPath: '/opt/ggsql/ggsql-jupyter', source: 'Setting' }, ]); + assert.deepStrictEqual(selection.fallback(), []); }); test('path with no configured kernel behaves as bundled', () => { - const candidates = selectKernelCandidates('path', bundled, undefined, hostKernels); - assert.deepStrictEqual(candidates, [{ kernelPath: bundled, source: 'Bundled' }]); + const selection = selectKernelCandidates('path', bundled, undefined, hostKernels); + assert.deepStrictEqual(selection.candidates, [{ kernelPath: bundled, source: 'Bundled' }]); + // Including the dead-end notice, which an empty setting must not suppress. + assert.strictEqual(selection.strategy, 'bundled'); + assert.deepStrictEqual(selection.fallback(), HOST); }); }); @@ -280,7 +303,7 @@ suite('kernel strategy from real settings', () => { // A bundled kernel is present and must lose to the setting. const { extensionPath } = extensionDirWithBundle(); assert.deepStrictEqual( - discoverKernelPaths(contextFor(extensionPath)), + discoverKernelPaths(contextFor(extensionPath)).candidates, [{ kernelPath: configured, source: 'Setting' }], ); }); @@ -288,7 +311,7 @@ suite('kernel strategy from real settings', () => { test('the environment strategy puts the bundled kernel last', async () => { const { extensionPath, kernelPath } = extensionDirWithBundle(); await set('kernelStrategy', 'environment'); - const candidates = discoverKernelPaths(contextFor(extensionPath)); + const candidates = discoverKernelPaths(contextFor(extensionPath)).candidates; // Whether this machine has host kernels is unknown, but the bundled one // is the fallback either way, so it comes last. assert.strictEqual(candidates.at(-1)?.source, 'Bundled'); @@ -344,7 +367,7 @@ suite('kernel accessibility', () => { suite('bundled kernel discovery', () => { test('the bundled kernel is the only candidate under the default strategy', () => { const { extensionPath, kernelPath } = extensionDirWithBundle(); - const candidates = discoverKernelPaths(contextFor(extensionPath)); + const candidates = discoverKernelPaths(contextFor(extensionPath)).candidates; assert.deepStrictEqual(candidates, [{ kernelPath, source: 'Bundled' }]); }); @@ -355,7 +378,7 @@ suite('bundled kernel discovery', () => { this.skip(); } const { extensionPath, kernelPath } = extensionDirWithBundle(0o644); - const candidates = discoverKernelPaths(contextFor(extensionPath)); + const candidates = discoverKernelPaths(contextFor(extensionPath)).candidates; assert.deepStrictEqual(candidates, [{ kernelPath, source: 'Bundled' }]); assert.ok(fs.statSync(kernelPath).mode & 0o111, 'the executable bit was not restored'); }); @@ -378,7 +401,7 @@ suite('host kernel discovery', () => { this.skip(); } const kernel = writeStubKernel(path.join(home, '.local', 'share', 'jupyter', 'kernels', 'ggsql')); - const candidates = discoverKernelPaths(contextFor(tempDir())); + const candidates = discoverKernelPaths(contextFor(tempDir())).candidates; assert.deepStrictEqual(candidates, [{ kernelPath: kernel, source: 'Jupyter' }]); for (const candidate of candidates) { assert.ok(path.isAbsolute(candidate.kernelPath), `${candidate.kernelPath} is not absolute`); @@ -401,7 +424,7 @@ suite('host kernel discovery', () => { fs.mkdirSync(dir, { recursive: true }); fs.symlinkSync(real, path.join(dir, binaryName)); } - const candidates = discoverKernelPaths(contextFor(tempDir())); + const candidates = discoverKernelPaths(contextFor(tempDir())).candidates; assert.strictEqual( candidates.length, 1, @@ -412,11 +435,14 @@ suite('host kernel discovery', () => { test('a bundled kernel outranks an installed one', () => { const hostKernel = writeStubKernel(path.join(home, '.local', 'share', 'jupyter', 'kernels', 'ggsql')); const { extensionPath, kernelPath } = extensionDirWithBundle(); + const selection = discoverKernelPaths(contextFor(extensionPath)); + assert.deepStrictEqual(selection.candidates, [{ kernelPath, source: 'Bundled' }]); + // The installed one is not gone, just behind: it is what the fallback + // tier reaches for when the bundled kernel cannot run. assert.deepStrictEqual( - discoverKernelPaths(contextFor(extensionPath)), - [{ kernelPath, source: 'Bundled' }], + selection.fallback(), + [{ kernelPath: hostKernel, source: 'Jupyter' }], ); - assert.ok(fs.existsSync(hostKernel), 'the host kernel was never there to be outranked'); }); }); @@ -431,17 +457,71 @@ suite('runtime registration', () => { return collected; } - function managerFor(extensionPath: string, kernelSpecDir: string): GgsqlRuntimeManager { + /** An in-memory stand-in for context.globalState, which the probe cache uses. */ + function memoryState(): vscode.Memento { + const store = new Map(); + return { + keys: () => [...store.keys()], + get: (key: string, defaultValue?: unknown) => + store.has(key) ? store.get(key) : defaultValue, + update: async (key: string, value: unknown) => { + store.set(key, value); + }, + } as unknown as vscode.Memento; + } + + /** + * A manager over a stand-in extension directory. + * + * The probe defaults to passing: a stand-in kernel cannot be a real + * executable on every platform, so running one for real is left to the + * `kernel probe` suite and these tests inject the verdict instead. + */ + function managerFor( + extensionPath: string, + kernelSpecDir: string, + options: { probe?: KernelProbe; globalState?: vscode.Memento } = {}, + ): { manager: GgsqlRuntimeManager; globalState: vscode.Memento } { + const globalState = options.globalState ?? memoryState(); const context = { extensionPath, + globalState, extension: { packageJSON: { version: realExtension().packageJSON.version } }, } as unknown as vscode.ExtensionContext; - return new GgsqlRuntimeManager(context, { kernelSpecDir }); + const manager = new GgsqlRuntimeManager(context, { + kernelSpecDir, + probe: options.probe ?? (async () => true), + }); + return { manager, globalState }; } + /** The key reportNoUsableKernel stamps once it has warned for this version. */ + const NOTICE_KEY = 'ggsql.noUsableKernelNotice'; + + // The dead-end notice is fire-and-forget, so it is captured rather than + // awaited. Stubbing it also keeps the suite from raising real notifications + // in the test window. + let warnings: string[] = []; + let realShowWarningMessage: typeof vscode.window.showWarningMessage; + + setup(() => { + warnings = []; + realShowWarningMessage = vscode.window.showWarningMessage; + (vscode.window as unknown as Record).showWarningMessage = + (message: string) => { + warnings.push(message); + return Promise.resolve(undefined); + }; + }); + + teardown(() => { + (vscode.window as unknown as Record).showWarningMessage = + realShowWarningMessage; + }); + test('the bundled kernel is registered as a single runtime', async () => { const { extensionPath, kernelPath } = extensionDirWithBundle(); - const runtimes = await collect(managerFor(extensionPath, tempDir()).discoverAllRuntimes()); + const runtimes = await collect(managerFor(extensionPath, tempDir()).manager.discoverAllRuntimes()); assert.strictEqual(runtimes.length, 1); assert.strictEqual(runtimes[0].runtimeId, 'ggsql-bundled'); assert.strictEqual(runtimes[0].runtimePath, kernelPath); @@ -454,7 +534,7 @@ suite('runtime registration', () => { // pointing into a directory that no longer exists. const { extensionPath, kernelPath } = extensionDirWithBundle(); const kernelSpecDir = tempDir(); - await collect(managerFor(extensionPath, kernelSpecDir).discoverAllRuntimes()); + await collect(managerFor(extensionPath, kernelSpecDir).manager.discoverAllRuntimes()); const spec = JSON.parse(fs.readFileSync(path.join(kernelSpecDir, 'kernel.json'), 'utf8')); assert.strictEqual(spec.argv[0], kernelPath); assert.strictEqual(spec.language, 'ggsql'); @@ -474,11 +554,153 @@ suite('runtime registration', () => { fs.mkdirSync(path.join(extensionPath, 'bundled', 'bin', binaryName), { recursive: true }); const kernelSpecDir = tempDir(); - const runtimes = await collect(managerFor(extensionPath, kernelSpecDir).discoverAllRuntimes()); + const runtimes = await collect(managerFor(extensionPath, kernelSpecDir).manager.discoverAllRuntimes()); assert.deepStrictEqual(runtimes, []); assert.strictEqual(fs.existsSync(path.join(kernelSpecDir, 'kernel.json')), false); }); + test('a bundled kernel that cannot run hands over to an installed one', async function () { + // The bundled kernel is built for the platform, not for every system on + // it: one built against newer shared libraries than the host provides + // execs and then dies under the dynamic linker. Nothing on the + // filesystem shows that, so the host install has to be reachable. + if (systemInstallPresent()) { + this.skip(); + } + const home = tempDir(); + isolateHostEnv(home); + try { + const hostKernel = writeStubKernel( + path.join(home, '.local', 'share', 'jupyter', 'kernels', 'ggsql'), + ); + const { extensionPath, kernelPath } = extensionDirWithBundle(); + const { manager } = managerFor(extensionPath, tempDir(), { + probe: async candidate => candidate !== kernelPath, + }); + + const runtimes = await collect(manager.discoverAllRuntimes()); + assert.strictEqual(runtimes.length, 1); + assert.strictEqual(runtimes[0].runtimePath, hostKernel); + // Named for where it came from, which is how the handover is + // disclosed without interrupting the user. + assert.strictEqual(runtimes[0].runtimeName, 'ggsql (Jupyter)'); + // A fallback that works is not worth interrupting anyone over. + assert.deepStrictEqual(warnings, []); + } finally { + restoreHostEnv(); + } + }); + + test('a bundled kernel that cannot run is not advertised to Jupyter', async function () { + // The kernel spec outlives the window and is what Quarto resolves, so + // pointing it at a binary that does not run would break tools that + // never see this extension's fallback. + if (systemInstallPresent()) { + this.skip(); + } + isolateHostEnv(tempDir()); + try { + const { extensionPath } = extensionDirWithBundle(); + const kernelSpecDir = tempDir(); + const { manager } = managerFor(extensionPath, kernelSpecDir, { + probe: async () => false, + }); + + const runtimes = await collect(manager.discoverAllRuntimes()); + assert.deepStrictEqual(runtimes, []); + assert.strictEqual(fs.existsSync(path.join(kernelSpecDir, 'kernel.json')), false); + } finally { + restoreHostEnv(); + } + }); + + test('a bundled kernel that cannot run and no installed one warns once', async function () { + if (systemInstallPresent()) { + this.skip(); + } + isolateHostEnv(tempDir()); + try { + const { extensionPath } = extensionDirWithBundle(); + const globalState = memoryState(); + + const first = managerFor(extensionPath, tempDir(), { + probe: async () => false, + globalState, + }); + assert.deepStrictEqual(await collect(first.manager.discoverAllRuntimes()), []); + assert.strictEqual(warnings.length, 1); + assert.match(warnings[0], /cannot run on this system/); + assert.strictEqual(globalState.get(NOTICE_KEY), realExtension().packageJSON.version); + + // Discovery runs on every window open; the notice must not repeat. + const second = managerFor(extensionPath, tempDir(), { + probe: async () => false, + globalState, + }); + assert.deepStrictEqual(await collect(second.manager.discoverAllRuntimes()), []); + assert.strictEqual(warnings.length, 1, 'the dead-end notice was shown twice'); + } finally { + restoreHostEnv(); + } + }); + + test('a bundled kernel that runs is not re-probed on the next window', async () => { + const { extensionPath } = extensionDirWithBundle(); + const globalState = memoryState(); + let probes = 0; + const probe: KernelProbe = async () => { + probes++; + return true; + }; + + const first = managerFor(extensionPath, tempDir(), { probe, globalState }); + assert.strictEqual((await collect(first.manager.discoverAllRuntimes())).length, 1); + assert.strictEqual(probes, 1); + + const second = managerFor(extensionPath, tempDir(), { probe, globalState }); + assert.strictEqual((await collect(second.manager.discoverAllRuntimes())).length, 1); + assert.strictEqual(probes, 1, 'the bundled kernel was probed again'); + }); + + test('a bundled kernel that runs never reaches for an installed one', async () => { + // The fallback costs a PATH lookup that shells out. The default + // strategy is the common case and must not pay for it. + const home = tempDir(); + isolateHostEnv(home); + try { + writeStubKernel(path.join(home, '.local', 'share', 'jupyter', 'kernels', 'ggsql')); + const { extensionPath, kernelPath } = extensionDirWithBundle(); + const { manager } = managerFor(extensionPath, tempDir()); + + const runtimes = await collect(manager.discoverAllRuntimes()); + assert.strictEqual(runtimes.length, 1); + assert.strictEqual(runtimes[0].runtimePath, kernelPath); + assert.strictEqual(runtimes[0].runtimeName, 'ggsql'); + } finally { + restoreHostEnv(); + } + }); + + test('a build with no kernel and nothing installed warns too', async function () { + // The win32-arm64 and platform-neutral VSIXes carry no kernel at all. + // That dead end is the same one, and gets the same notice. + if (systemInstallPresent()) { + this.skip(); + } + isolateHostEnv(tempDir()); + try { + const globalState = memoryState(); + const { manager } = managerFor(tempDir(), tempDir(), { globalState }); + assert.deepStrictEqual(await collect(manager.discoverAllRuntimes()), []); + assert.strictEqual(warnings.length, 1); + // Worded for a build that never carried a kernel, not a broken one. + assert.match(warnings[0], /does not include a kernel/); + assert.strictEqual(globalState.get(NOTICE_KEY), realExtension().packageJSON.version); + } finally { + restoreHostEnv(); + } + }); + test('a machine with no kernel at all registers nothing', async function () { if (systemInstallPresent()) { this.skip(); @@ -487,7 +709,7 @@ suite('runtime registration', () => { // than what the precedence rule returns. isolateHostEnv(tempDir()); try { - const runtimes = await collect(managerFor(tempDir(), tempDir()).discoverAllRuntimes()); + const runtimes = await collect(managerFor(tempDir(), tempDir()).manager.discoverAllRuntimes()); assert.deepStrictEqual(runtimes, []); } finally { restoreHostEnv(); @@ -545,6 +767,45 @@ suite('runtime metadata', () => { }); }); +suite('kernel probe', () => { + // The probe is what separates a kernel that is present from one that runs. + // The failure it exists for is a binary built against newer shared + // libraries than the host provides: exec succeeds, the dynamic linker then + // rejects it, and the process exits non-zero. + + test('a binary that exits non-zero does not pass', async function () { + if (process.platform === 'win32') { + this.skip(); + } + const dir = tempDir(); + const kernelPath = path.join(dir, binaryName); + fs.writeFileSync(kernelPath, '#!/bin/sh\nexit 1\n'); + fs.chmodSync(kernelPath, 0o755); + assert.strictEqual(await probeKernel(kernelPath), false); + }); + + test('a binary that exits zero passes', async function () { + if (process.platform === 'win32') { + this.skip(); + } + assert.strictEqual(await probeKernel(writeStubKernel(tempDir())), true); + }); + + test('a file that is not executable at all does not pass', async () => { + // The nearest reachable stand-in for a binary the loader rejects: the + // spawn fails rather than the process exiting non-zero, and the probe + // has to treat both the same way. + const kernelPath = path.join(tempDir(), binaryName); + fs.writeFileSync(kernelPath, 'not a real executable\n'); + fs.chmodSync(kernelPath, 0o644); + assert.strictEqual(await probeKernel(kernelPath), false); + }); + + test('a missing binary does not pass', async () => { + assert.strictEqual(await probeKernel(path.join(tempDir(), binaryName)), false); + }); +}); + suiteTeardown(() => { for (const dir of tempDirs) { fs.rmSync(dir, { recursive: true, force: true });