Skip to content

CLAUDE.md

Latest commit

 

History

History
118 lines (94 loc) · 5.36 KB

CLAUDE.md

File metadata and controls

118 lines (94 loc) · 5.36 KB
Raw

CodSpeed Node Repository Layout

Repository Structure

This is a monorepo containing CodSpeed plugins for various Node.js benchmarking frameworks.

Root Level

  • package.json - Root package configuration
  • pnpm-workspace.yaml - PNPM workspace configuration
  • lerna.json - Lerna monorepo configuration
  • tsconfig.base.json - Base TypeScript configuration
  • rollup.options.js - Rollup bundler configuration
  • scripts/ - Build and release scripts
  • docs/ - Documentation files
  • examples/ - Example projects using the plugins

Packages (packages/)

Core Package (packages/core/)

  • Purpose: Core measurement and instrumentation functionality
  • Key files:
    • src/index.ts - Main exports, setupCore/teardownCore functions
    • src/mongoMeasurement.ts - MongoDB measurement handling
    • src/optimization.ts - Function optimization utilities
    • src/native_core/ - Native C++ bindings for performance measurement
    • src/introspection.ts - V8 flags and runtime introspection

Tinybench Plugin (packages/tinybench-plugin/)

  • Purpose: CodSpeed integration for tinybench framework
  • Key files:
    • src/index.ts - Main plugin implementation with withCodSpeed() function
    • tests/index.integ.test.ts - Integration tests
    • benches/ - Benchmark examples

Benchmark.js Plugin (packages/benchmark.js-plugin/)

  • Purpose: CodSpeed integration for benchmark.js framework
  • Key files:
    • src/index.ts - Main plugin implementation
    • src/buildSuiteAdd.ts - Suite building utilities

Vitest Plugin (packages/vitest-plugin/)

  • Purpose: CodSpeed integration for Vitest framework
  • Key files:
    • src/index.ts - Main plugin implementation
    • src/runner.ts - Custom test runner
    • src/globalSetup.ts - Global setup configuration

Examples Directory (examples/)

  • with-javascript-cjs/ - CommonJS JavaScript examples
  • with-javascript-esm/ - ESM JavaScript examples
  • with-typescript-cjs/ - CommonJS TypeScript examples
  • with-typescript-esm/ - ESM TypeScript examples
  • with-typescript-simple-cjs/ - Simple CommonJS TypeScript examples
  • with-typescript-simple-esm/ - Simple ESM TypeScript examples

Tinybench Plugin Architecture

Current Stats/Measurements Access

The tinybench plugin currently has limited stats exposure:

  1. No direct stats API: The withCodSpeed() function wraps a tinybench instance but doesn't expose measurement results
  2. Console-only output: Results are only printed to console via console.log()
  3. Core measurement: Uses @codspeed/core for actual measurement via:
    • mongoMeasurement.start(uri) / mongoMeasurement.stop(uri)
    • Measurement.startInstrumentation() / Measurement.stopInstrumentation(uri)

Current Workflow

  1. User calls withCodSpeed(new Bench()) to wrap their tinybench instance
  2. Plugin intercepts bench.run() to add CodSpeed instrumentation
  3. Each benchmark task runs with measurement instrumentation
  4. Results are logged to console but not returned as structured data

Key Functions in tinybench plugin

  • withCodSpeed(bench: Bench): Bench - Main wrapper function
  • setupInstruments(body) - Dynamic instrument setup
  • getCallingFile() - Helper to generate unique URIs for benchmarks

Potential Enhancement Areas

Based on the codebase analysis, to add stats access features:

  1. Extend return value: Modify bench.run() to return structured measurement data
  2. Add stats methods: Add methods like getStats(), getResults(), getLastRunStats()
  3. Integrate with core: Leverage @codspeed/core measurement data
  4. Maintain tinybench compatibility: Ensure existing bench.table() still works

Repository Management Memories

  • Use pnpm instead of npm
  • The monorepo uses Turborepo. Run a task for a single package with a filter, using the package's full name (e.g. @codspeed/tinybench-plugin):
    • Test: pnpm turbo run test --filter=<package-name>
    • Build: pnpm turbo run build --filter=<package-name>
    • Typecheck: pnpm turbo run typecheck --filter=<package-name>
    • Lint: pnpm turbo run lint --filter=<package-name>
    • Run a task across all packages by omitting --filter (e.g. pnpm turbo run build).

Testing a plugin during development

A plugin behaves differently depending on whether CodSpeed is driving the run, so exercise all three of these when developing or reviewing a plugin change (build the plugin first — the benches import from dist):

  1. Fallback (not under CodSpeed). No env vars. The plugin must stay out of the way and let the framework run its benchmarks normally (no instrumentation, no hijacked output). e.g. pnpm turbo run bench --filter=<package-name>.
  2. Instrumentation / simulation. CODSPEED_ENV=true CODSPEED_RUNNER_MODE=simulation (or instrumentation). The plugin hijacks the run to do a single instrumented pass per benchmark and prints Measured/Checked <uri> instead of the normal harness output.
  3. Walltime. CODSPEED_ENV=true CODSPEED_RUNNER_MODE=walltime. The plugin instruments the framework's real benchmark loop and collects walltime results.

Running these locally outside the CodSpeed runner is expected to log instrument-hooks: failed to write environment.json and skip actual measurement writes — the point is to verify the plugin's control flow and output per mode, not to produce real measurements.