Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/actions/bundle-codeql-packs/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Bundle CodeQL packs
description: |
Bundles a set of CodeQL query packs via `codeql pack bundle`, reusing the cache
directory populated by the `install-qlt` action. This wraps the CLI directly rather than
a `qlt pack` subcommand, since QLT's own pack-publication CLI surface wasn't verified
when this action was written; it can be swapped for a native `qlt pack bundle` call
once that's confirmed.
inputs:
codeql-bin:
description: Path to the `codeql` binary, e.g. the `codeql-bin` output of `install-qlt`.
required: true

cache-dir:
description: The cache directory to pass as `--common-caches`, e.g. the `cache-dir` output of `install-qlt`.
required: true

packs:
description: |
One `<output-file>.tgz <pack-src-path>` pair per line, e.g.:
common-cpp-coding-standards.tgz cpp/common/src
common-c-coding-standards.tgz c/common/src
required: true

runs:
using: composite
steps:
- name: Bundle packs
shell: bash
env:
CODEQL_BIN: ${{ inputs.codeql-bin }}
CACHE_DIR: ${{ inputs.cache-dir }}
PACKS: ${{ inputs.packs }}
run: |
while read -r output pack_path; do
[ -z "$output" ] && continue
echo "::debug::Bundling $pack_path -> $output"
"$CODEQL_BIN" pack bundle --common-caches="$CACHE_DIR" --output="$output" "$pack_path"
done <<< "$PACKS"
208 changes: 207 additions & 1 deletion .github/actions/install-qlt/action.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
name: Fetch and Install QLT
description: |
Fetches and installs QLT.
Fetches and installs QLT.

Optionally also installs CodeQL and the repo's query packs via `qlt codeql run install`,
restoring a cache directory covering both the downloaded bundle and compiled query
artifacts, when `language` is supplied. Existing callers that only pass `qlt-version`/
`add-to-path` are unaffected: the original "Install QLT" step and `qlt-home` output are
unchanged, and all the CodeQL-related steps below are skipped unless `language` is set.
inputs:
qlt-version:
description: |
Expand All @@ -14,11 +20,121 @@ inputs:
required: false
default: 'true'

language:
description: |
The language pack(s) to install and to scope the cache key to, e.g. `cpp` or `c`.
Space-separate multiple values (e.g. `cpp c`) to install packs for more than one
language in a single call. Leave unset to only install QLT itself (original behavior).
When `custom-bundle` is `false` this only scopes the cache key/precompile target (no
packs are built), so pass whatever language the queries being tested/precompiled are in.
required: false
default: ''

custom-bundle:
description: |
Whether to build and install a custom bundle containing this repo's own query packs
(`qlt codeql run install --custom-bundle --packs <language>`), or a plain CodeQL CLI +
standard library checkout (`qlt codeql run install`, using the `codeql-cli-version`/
`codeql-standard-library-version` set above) for testing/using the standard library
directly rather than this repo's packs. Ignored unless `language` is set.
required: false
default: 'true'

codeql-cli-version:
description: |
The version of the CodeQL CLI to use, e.g. `2.23.9`. Leave unset to use an already
committed `qlt.conf.json` as-is. Ignored unless `language` is set.

QLT has no option to point at a differently-named/located config file: it always reads
`qlt.conf.json` from `--base`. Setting this input runs `qlt codeql set version`, which
*overwrites* `qlt.conf.json` with only the CLI/standard-library/bundle versions (any
other committed settings, e.g. `ExportedCustomizationPacks`, are dropped from the file,
though `--packs`/`--cache-dir` passed explicitly to the install step below still apply
since those are passed on the command line, not read back from the file). Only needed
when a caller wants to override the committed `qlt.conf.json` per matrix entry, e.g. to
test multiple CLI/standard-library combinations from a single `qlt.conf.json`.
required: false
default: ''

codeql-standard-library-version:
description: |
The tag or commit to use from the CodeQL Standard Library, e.g. `codeql-cli/v2.23.9`.
Ignored unless `language` is set.
required: false
default: ''

codeql-bundle-version:
description: |
The CodeQL bundle version to use, e.g. `codeql-bundle-v2.23.9`. Ignored unless
`language` is set.
required: false
default: ''

base:
description: The base path to find the query repository. Ignored unless `language` is set.
required: false
default: ${{ github.workspace }}

cache-dir:
description: |
Directory used for the downloaded bundle and compiled query cache when `custom-bundle`
is `true` (passed to the `codeql-bundle` tool's own `--cache-dir`, and to `codeql`'s own
`--common-caches` in the precompile step below). This is separate from, and in addition
to, `~/.qlt` (see the `install-cache-paths` output) which is where QLT itself installs
CodeQL for every install mode. Ignored unless `language` is set (and, for caching
purposes, unless `custom-bundle` is also `true`).
required: false
default: ${{ github.workspace }}/.qlt-cache

compilation-cache-size:
description: |
If set, precompiles `language` right after install via `codeql query compile
--common-caches=<cache-dir> --compilation-cache-size=<this value>`, so callers don't
each need to hand-roll that step. QLT itself has no equivalent setting (its own
`--cache-dir` is a fingerprinted download cache, not a size-bounded compile cache), so
this action runs the CLI directly. Ignored unless `language` is set.
required: false
default: ''

precompile:
description: |
Pass `--precompile` to the `codeql query compile` step described by
`compilation-cache-size`. Ignored unless both `language` and `compilation-cache-size`
are set.
required: false
default: 'false'

outputs:
qlt-home:
description: 'The directory containing the QLT installation'
value: ${{ steps.install-qlt.outputs.qlt-home }}

cache-dir:
description: |
The single directory passed as `--cache-dir`/`--common-caches` to `codeql-bundle`/
`codeql` (e.g. for a `bundle-codeql-packs` or further `codeql query compile` call).
Empty unless `language` is set.
value: ${{ inputs.language != '' && inputs.cache-dir || '' }}

install-cache-paths:
description: |
Newline-separated list of paths to restore/save with `actions/cache` to cover the whole
CodeQL install: always `~/.qlt` (where QLT installs CodeQL for every install mode), plus
`cache-dir` too when `custom-bundle` is `true`. Pass straight through to
`save-qlt-cache`'s `cache-dir` input (which forwards it, unmodified, to
`actions/cache/save`'s multiline-capable `path`) — do not use this for `--common-caches`/
`--cache-dir` CLI flags, which expect a single directory (see the `cache-dir` output).
Empty unless `language` is set.
value: ${{ steps.qlt-cache-paths.outputs.paths }}

cache-primary-key:
description: The primary cache key to pass to a cache-save step at the end of the job. Empty unless `language` is set.
value: ${{ steps.restore-qlt-cache.outputs.cache-primary-key }}

codeql-bin:
description: Path to the `codeql` binary installed by QLT. Empty unless `language` is set.
value: ${{ steps.locate-codeql-bin.outputs.codeql-bin }}

runs:
using: composite
steps:
Expand Down Expand Up @@ -80,3 +196,93 @@ runs:

popd
echo -e "\e[0;32m[QLT]\e[0m Done."

# Everything below is opt-in: skipped entirely unless `language` is supplied, so existing
# callers that only install QLT itself see no behavior change.
- name: Configure QLT for this CodeQL version
if: inputs.language != '' && inputs.codeql-cli-version != ''
shell: bash
run: |
qlt codeql set version \
--cli-version "${{ inputs.codeql-cli-version }}" \
--standard-library-version "${{ inputs.codeql-standard-library-version }}" \
--bundle-version "${{ inputs.codeql-bundle-version }}" \
--base "${{ inputs.base }}" \
--automation-type actions

- name: Determine QLT cache paths
if: inputs.language != ''
id: qlt-cache-paths
shell: bash
run: |
# QLT installs CodeQL under ~/.qlt regardless of install mode (packages or custom
# bundles); --cache-dir is a second, separate cache used only by custom-bundle builds.
{
echo "paths<<EOF"
echo "~/.qlt"
if [ "${{ inputs.custom-bundle }}" == "true" ] && [ -n "${{ inputs.cache-dir }}" ]; then
echo "${{ inputs.cache-dir }}"
fi
echo "EOF"
} >> "$GITHUB_OUTPUT"

- name: Restore QLT cache
if: inputs.language != ''
id: restore-qlt-cache
uses: actions/cache/restore@v6
with:
path: ${{ steps.qlt-cache-paths.outputs.paths }}
key: qlt-${{ runner.os }}-${{ inputs.custom-bundle }}-${{ inputs.language }}-${{ hashFiles('qlt.conf.json') }}-${{ github.run_id }}
restore-keys: qlt-${{ runner.os }}-${{ inputs.custom-bundle }}-${{ inputs.language }}-${{ hashFiles('qlt.conf.json') }}

- name: Install CodeQL + packs via QLT
if: inputs.language != ''
shell: bash
run: |
ARGS=(--cache-dir "${{ inputs.cache-dir }}" --base "${{ inputs.base }}" --automation-type actions)
if [ "${{ inputs.custom-bundle }}" == "true" ]; then
# Build this repo's own query packs into the bundle.
ARGS+=(--custom-bundle --packs ${{ inputs.language }})
fi
# Otherwise: plain CLI + standard library checkout, from qlt.conf.json's CodeQLCLI /
# CodeQLStandardLibrary (set by "Configure QLT for this CodeQL version" above).
qlt codeql run install "${ARGS[@]}"

- name: Locate installed codeql binary
if: inputs.language != ''
id: locate-codeql-bin
shell: bash
env:
CACHE_DIR: ${{ inputs.cache-dir }}
run: |
# With --automation-type actions, the install step above already exported
# QLT_CODEQL_PATH (and QLT_CODEQL_HOME) to GITHUB_ENV, so it's available here
# directly. Fall back to searching the cache dir / PATH for older QLT versions.
CODEQL_BIN="$QLT_CODEQL_PATH"
if [ -z "$CODEQL_BIN" ]; then
CODEQL_BIN="$(find "$CACHE_DIR" -type f -name codeql -perm -u+x 2>/dev/null | head -n1)"
fi
if [ -z "$CODEQL_BIN" ]; then
CODEQL_BIN="$(command -v codeql)"
fi
echo "::debug::Resolved codeql binary to $CODEQL_BIN"
echo "codeql-bin=$CODEQL_BIN" >> "$GITHUB_OUTPUT"

- name: Precompile queries
if: inputs.language != '' && inputs.compilation-cache-size != ''
shell: bash
env:
CODEQL_BIN: ${{ steps.locate-codeql-bin.outputs.codeql-bin }}
CACHE_DIR: ${{ inputs.cache-dir }}
CACHE_SIZE: ${{ inputs.compilation-cache-size }}
PRECOMPILE: ${{ inputs.precompile }}
run: |
# No QLT equivalent to --compilation-cache-size exists (its own --cache-dir is a
# fingerprinted download cache, not a size-bounded compile cache), so this shells
# out to the CLI directly rather than a qlt subcommand.
# --verbosity=progress++ surfaces per-query compilation-cache hit/miss messages.
ARGS=(query compile --common-caches="$CACHE_DIR" --compilation-cache-size="$CACHE_SIZE" --threads 0 --verbosity=progress++)
if [ "$PRECOMPILE" == "true" ]; then
ARGS+=(--precompile)
fi
"$CODEQL_BIN" "${ARGS[@]}" ${{ inputs.language }}
81 changes: 81 additions & 0 deletions .github/actions/run-qlt-unit-tests/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
name: Run and validate CodeQL unit tests via QLT
description: |
Runs the repository's unit tests for a given language via `qlt test run
execute-unit-tests`, then validates the results via `qlt test run
validate-unit-tests`, failing the step if any test failed.

Replaces a hand-rolled `codeql test run` slicing script plus a separate
results-validation job with QLT's own test-running/validation subcommands.
Unverified: whether QLT's runner supports the same degree of parallelism/RAM
tuning as the `--slice`/`--ram` flags used by the script it replaces.
inputs:
language:
description: The language to run tests for, e.g. `cpp` or `c`.
required: true

num-threads:
description: Number of threads to use for the test runner.
required: false
default: "8"

work-dir:
description: Where to place intermediate execution output files.
required: false
default: ${{ runner.temp }}

base:
description: The base path to find the query repository.
required: false
default: ${{ github.workspace }}

verbosity:
description: |
CodeQL `--verbosity` level for the test run, e.g. `progress++` to show per-test
progress. Set to `''` to leave verbosity at CodeQL's own default.
required: false
default: "progress++"

codeql-args:
description: Extra arguments to pass through to CodeQL.
required: false
default: ""

outputs:
results-directory:
description: The directory containing the raw test execution results.
value: ${{ inputs.work-dir }}

runs:
using: composite
steps:
- name: Execute unit tests
shell: bash
env:
CODEQL_ARGS: ${{ inputs.codeql-args }}
VERBOSITY: ${{ inputs.verbosity }}
run: |
if [ -n "$VERBOSITY" ]; then
CODEQL_ARGS="$CODEQL_ARGS --verbosity=$VERBOSITY"
fi
ARGS=(
--num-threads "${{ inputs.num-threads }}"
--language "${{ inputs.language }}"
--runner-os "${{ runner.os }}"
--work-dir "${{ inputs.work-dir }}"
--base "${{ inputs.base }}"
--automation-type actions
)
if [ -n "$CODEQL_ARGS" ]; then
ARGS+=(--codeql-args "$CODEQL_ARGS")
fi
qlt test run execute-unit-tests "${ARGS[@]}"

- name: Validate unit test results
shell: bash
run: |
# Deliberately omit --pretty-print: per QLT's own docs, that mode does not
# exit with a failure code on test failures, which we need for CI gating.
qlt test run validate-unit-tests \
--results-directory "${{ inputs.work-dir }}" \
--base "${{ inputs.base }}" \
--automation-type actions
22 changes: 22 additions & 0 deletions .github/actions/save-qlt-cache/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Save QLT cache
description: |
Persists the cache directory populated by the `install-qlt` action (downloaded bundle and
compiled query artifacts) so subsequent runs can reuse it. Call this at the end of the job,
typically with `if: always()` so the cache is saved even if the test run failed.
inputs:
cache-dir:
description: The cache directory output by the `install-qlt` action.
required: true

cache-primary-key:
description: The cache-primary-key output by the `install-qlt` action.
required: true

runs:
using: composite
steps:
- name: Save QLT cache
uses: actions/cache/save@v6
with:
path: ${{ inputs.cache-dir }}
key: ${{ inputs.cache-primary-key }}
Loading