diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..46ed9a8 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,51 @@ +name: CI + +on: + pull_request: + push: + branches: [master] + # The template resolves the SF Compute release index at build time, so + # the repo can break without a commit: a release that drops the LTS the + # default keys on, or an index that stops answering, fails validate. + # The daily run finds that before a reader does. + # + # GitHub disables scheduled workflows in a public repository after 60 + # days without repository activity, and this repo is meant to go quiet. + # Re-enable it from the Actions tab when that happens; firing + # workflow_dispatch from the release pipeline instead would be immune. + schedule: + - cron: "17 6 * * *" + workflow_dispatch: + +permissions: + contents: read + +jobs: + check: + runs-on: ubuntu-latest + env: + # `packer init` fetches the qemu plugin from GitHub releases; the + # unauthenticated rate limit is low enough to flake a shared runner. + PACKER_GITHUB_API_TOKEN: ${{ github.token }} + steps: + - uses: actions/checkout@v4 + + - uses: hashicorp/setup-packer@v3 + with: + version: "1.15.0" + + - name: Install plugins + run: packer init . + + - name: Check formatting + run: packer fmt -check -diff . + + # -evaluate-datasources makes validate fetch the release index and + # resolve the base image, which is the part worth checking: plain + # validate stops at an unresolved checksum. Firmware paths are not + # checked by validate, so the runner needs no OVMF. + - name: Validate, resolving the live release index + run: packer validate -evaluate-datasources . + + - name: Lint shell scripts + run: shellcheck ./*.sh diff --git a/README.md b/README.md index aa1a325..f929b40 100644 --- a/README.md +++ b/README.md @@ -61,31 +61,53 @@ packer build -only='qemu.sfc_base' -var 'accelerator=kvm' . ``` Expect roughly 15 minutes on Linux with KVM, and a few hours on macOS under -emulation. Packer downloads a 3.3 GB base image first. The result is a sparse +emulation. Packer downloads the base image first (~3.5 GB). The result is a sparse 20 GiB raw file — it takes far less room on disk, but `sf images upload` transfers the full 20 GiB. Need more space for your software? Add `-var 'disk_size=40G'` (the platform caps images at 75 GiB). -`-only` matters: without it Packer builds **both** starting points at once. If -you mistype the name, Packer runs nothing and still exits 0, so check that it -actually printed a build. +`-only` matters: without it Packer builds **both** starting points at once. +Mistype the name and Packer lists the valid ones and exits without building. + +Want to check the template before building? `validate` doesn't fetch anything +unless you ask it to, so it can't yet know which base image to download — pass +`-evaluate-datasources`: + +```bash +packer validate -evaluate-datasources . +``` + +Without the flag it fails with `invalid checksum: encoding/hex: invalid byte: +U+003C '<'`, which doesn't sound like the real problem. `packer build` needs no +such flag. Re-running? Packer refuses to overwrite `output/`. Pass `-force`, or `rm -rf output/` first. -The image lands at `output/sfc_base/ubuntu-24.04-cuda-13.2-custom.raw`. -Upload it: +The image lands at `output/sfc_base/sfc-base-custom.raw`. Upload it: ```bash -sf images upload --name my-image --file output/sfc_base/ubuntu-24.04-cuda-13.2-custom.raw +sf images upload --name my-image --file output/sfc_base/sfc-base-custom.raw ``` +Before you push 20 GiB, confirm it is what the platform expects: + +```bash +qemu-img info output/sfc_base/sfc-base-custom.raw +``` + +You want `file format: raw` and a virtual size at or under 75 GiB. The real +test is booting an instance from it — do that before you rely on the image. + To start from stock Ubuntu instead: ```bash -packer build -only='qemu.ubuntu' -var 'accelerator=kvm' . +packer build -only='qemu.ubuntu' . ``` +That one lands at `output/ubuntu/ubuntu-noble-custom.raw` and uploads the same +way. + ## Making it yours Edit **`customize.sh`**. That is the whole customization surface — it @@ -93,37 +115,71 @@ runs as root inside the build VM on both starting points. Everything else in this repo is boilerplate that makes the resulting image boot correctly on SF Compute. -If you only ever build on our base images, delete `install-nvidia.sh`, its -`provisioner` block in `build.pkr.hcl`, and the `source "qemu" "ubuntu"` -block — otherwise the repo still offers a path that no longer installs a -driver. +If you only ever build on our base images, you can delete the stock-Ubuntu +path — otherwise the repo still offers a route that no longer installs a +driver. Remove all four, or the template will not load: + +- `install-nvidia.sh` +- its `provisioner` block in `build.pkr.hcl` +- the `source "qemu" "ubuntu"` block +- the `"source.qemu.ubuntu"` line from the `build` block's `sources` list + +The `nvidia_driver_branch` variable becomes unused too, and can go. +`ubuntu_codename` stays: the base-image path uses it to pick the LTS. + +Going the other way, keeping only stock Ubuntu, also means removing the +`data "http"` block, the `sfc_*` locals and the `manifest` post-processor. +Packer evaluates data sources for every build, so until they are gone the +stock-Ubuntu path still needs the release index to be reachable. ## Choosing a base image -`build.pkr.hcl` pins a specific release by default, so repeated builds -reproduce the same bytes. To see what is current: +By default the build resolves the current release at build time and, within +it, the Ubuntu 24.04 image with the newest CUDA — there is no version to keep +up to date in this repo. Look for the `Trying ...` line in Packer's output: it +names the release tag and the image it picked. + +To see what is available in the SF Compute release index (`tiny-llama.sfcc.xyz`, +operated by SF Compute): ```bash curl -fsSL https://tiny-llama.sfcc.xyz/latest.json | jq -r '.tag, (.images[].name)' ``` -`latest.json` is the published index of the newest release: the tag, and -every image in it with its URL, SHA256 and size. Point the build at one -with `-var`: +Each release carries one image per CUDA minor for each of the two newest Ubuntu +LTS releases. To build on a different LTS, change `ubuntu_codename`; you still +get its newest CUDA: + +```bash +packer build -only='qemu.sfc_base' -var 'ubuntu_codename=resolute' . +``` + +To build on a specific CUDA, name the image with `base_image_name`: ```bash -packer build -only='qemu.sfc_base' \ - -var 'base_image_tag=' \ - -var 'base_image_name=ubuntu-22.04-cuda-13.1' . +packer build -only='qemu.sfc_base' -var 'base_image_name=' . ``` -Then pin that tag in `build.pkr.hcl` so your builds stay reproducible. +The output filename is the same whichever image was picked; +`output/sfc_base/manifest.json` records which one it was. -Integrity is checked against the `SHA256SUMS` published alongside each -release; Packer fetches it and picks the line matching the image it is -downloading. Releases published before `SHA256SUMS` existed don't have -one, and Packer will fail with a 404 — for those, pass -`-var 'base_image_checksum=none'`. +**Pin the release if you need a repeatable starting point.** Tracking the latest +means a rebuild next month begins from a different base image — different driver, +CUDA and kernel. Pinning fixes that. It does not make the build bit-for-bit +reproducible: `customize.sh` still installs whatever the Ubuntu archive has that +day. + +```bash +packer build -only='qemu.sfc_base' -var 'base_image_tag=' . +``` + +You don't supply a checksum. The release index lists each image's sha256 beside +its URL and the build enforces it, so a corrupted or truncated download fails +instead of producing a broken image. + +Each build writes `output/sfc_base/manifest.json` recording the release and +image it resolved. That is where to find the values to pass to `base_image_tag` +and `base_image_name` if you want to rebuild on the same base. ## What the boilerplate is doing @@ -139,6 +195,24 @@ cloud-init datasource when an instance launches. The root password in `cloud-init/user-data` exists only so Packer can log in during the build, and `finalize.sh` removes it. +## When a build fails + +| What you see | What to do | +|---|---| +| `E: Unable to locate package packer` | Packer is not in the Ubuntu archive. See [Requirements](#requirements). | +| `packer init` reports an unknown command | Your Packer predates 1.7. Ubuntu 22.04 ships 1.6.6; install a current one per Requirements. | +| `Output directory 'output/...' already exists` | `packer build -force ...`, or `rm -rf output/`. | +| `Datasource.Execute failed: ... Response code: 404` | The tag you pinned has no release index. Use the tag from `latest.json` (the current release) or from an earlier build's `output/sfc_base/manifest.json`, or drop `-var base_image_tag`. If you did not pin a tag, the index itself is missing: check with `curl -fsSL https://tiny-llama.sfcc.xyz/latest.json`. | +| `Datasource.Execute failed: ... dial tcp ... no such host`, or `connection refused` | The build looks up the base image over the network before it starts, for either starting point. Check you can reach `https://tiny-llama.sfcc.xyz/latest.json`; behind a proxy, set `HTTPS_PROXY`. | +| `Call to function "jsondecode" failed: invalid character '<'` | Something returned HTML instead of the release index — usually a captive portal or a proxy login page. Get past it in a browser, then retry. | +| `invalid checksum: encoding/hex: invalid byte` from `packer validate` | Plain `validate` does not run data sources, so the digest is still unresolved. Use `packer validate -evaluate-datasources .`. | +| `Call to function "max" failed: must pass at least one number` | No image in that release matches: `base_image_name` is not in it, or `ubuntu_codename` is an LTS it does not carry. List the names with the `jq` command under [Choosing a base image](#choosing-a-base-image). | +| `Could not access KVM kernel module` | No `/dev/kvm`, or you are not in the `kvm` group. Drop `-var 'accelerator=kvm'` to build under emulation. | +| QEMU exits immediately on macOS | Drop `-var 'accelerator=kvm'`; macOS has no KVM. | +| `Could not open '/usr/share/OVMF/...'` | Firmware auto-detection missed yours. Pass `-var 'efi_firmware_code=...' -var 'efi_firmware_vars=...'`; both must be the same variant, so a 4M code file needs a 4M vars file. | +| `Timeout waiting for SSH` | Usually a slow emulated boot. Build on Linux with KVM, or raise `ssh_timeout` in `build.pkr.hcl`. | +| A step you added to `customize.sh` failed | Packer deletes `output/` on failure. Re-run with `-on-error=abort` to keep the VM and disk so you can look. | + ## Image requirements If you build your own template from scratch rather than copying this @@ -159,21 +233,6 @@ The SF Compute base images ship all of it. See the [Images documentation](https://docs.sfcompute.com/preview/images) for the full picture. -## When a build fails - -| What you see | What to do | -|---|---| -| `E: Unable to locate package packer` | Packer is not in the Ubuntu archive. See [Requirements](#requirements). | -| `packer init` reports an unknown command | Your Packer predates 1.7. Ubuntu 22.04 ships 1.6.6; install a current one per Requirements. | -| `Output directory 'output/...' already exists` | `packer build -force ...`, or `rm -rf output/`. | -| A 404 fetching `SHA256SUMS` | The tag you pinned predates checksums. Add `-var 'base_image_checksum=none'`. | -| `Could not access KVM kernel module` | No `/dev/kvm`, or you are not in the `kvm` group. Drop `-var 'accelerator=kvm'` to build under emulation. | -| QEMU exits immediately on macOS | Drop `-var 'accelerator=kvm'`; macOS has no KVM. | -| `Could not open '/usr/share/OVMF/...'` | Firmware auto-detection missed yours. Pass `-var 'efi_firmware_code=...' -var 'efi_firmware_vars=...'`; both must be the same variant, so a 4M code file needs a 4M vars file. | -| `Timeout waiting for SSH` | Usually a slow emulated boot. Build on Linux with KVM, or raise `ssh_timeout` in `build.pkr.hcl`. | -| A step you added to `customize.sh` failed | Packer deletes `output/` on failure. Re-run with `-on-error=abort` to keep the VM and disk so you can look. | -| The build succeeded but `packer` printed no build | You mistyped `-only`. Packer exits 0 having run nothing. | - ## Files | File | Purpose | diff --git a/build.pkr.hcl b/build.pkr.hcl index e29246e..6db869c 100644 --- a/build.pkr.hcl +++ b/build.pkr.hcl @@ -32,31 +32,49 @@ packer { variable "base_image_tag" { type = string - default = "v20260924.213848" + default = "" description = <<-EOT - Which SF Compute base image release to build on (qemu.sfc_base only). - - This is a pin, on purpose: the same tag gives you the same bytes - every time. To move to the current release: - - curl -fsSL https://tiny-llama.sfcc.xyz/latest.json | jq -r .tag - - That endpoint also lists every image in the release with its URL, - SHA256 and size -- see the README. + Pin the build to one SF Compute base image release (qemu.sfc_base only). + + Empty, the default, resolves whatever is current at build time. The + build prints the release it picked, so check the log if you need to + know which one you got. + + Set this to a tag for a repeatable starting point. It does not make + the build bit-for-bit reproducible -- customize.sh still installs + whatever the Ubuntu archive has that day -- but it fixes the base + image, so a rebuild next month starts from the same driver, CUDA and + kernel. Tags look like v20260930.063326. latest.json carries the + current one and each build's output/sfc_base/manifest.json records + the one it used; there is no list of older tags, and releases + published before the index existed cannot be pinned. EOT } variable "base_image_name" { - type = string - default = "ubuntu-24.04-cuda-13.2" - description = "Which image within the release (qemu.sfc_base only). Names look like ubuntu--cuda-; `curl -fsSL https://tiny-llama.sfcc.xyz/latest.json | jq -r '.images[].name'` lists them." + type = string + default = "" + + description = <<-EOT + Which image within the release (qemu.sfc_base only). + + Empty, the default, takes the image with the newest CUDA for + ubuntu_codename. A release carries one image per CUDA minor per + Ubuntu LTS and the set moves as NVIDIA and Canonical ship, so a + name written here goes stale; a rule does not. + + Set this to build on a specific CUDA instead. Names look like + ubuntu--cuda-, and + `curl -fsSL https://tiny-llama.sfcc.xyz/latest.json | jq -r '.images[].name'` + lists the ones in the current release. + EOT } variable "ubuntu_codename" { type = string default = "noble" - description = "Ubuntu cloud image to build on (qemu.ubuntu only). noble = 24.04 LTS, jammy = 22.04 LTS." + description = "Which Ubuntu to build on, for either starting point. noble = 24.04 LTS, resolute = 26.04 LTS. For qemu.ubuntu this names the cloud image; for qemu.sfc_base it picks the LTS when base_image_name is empty." } variable "nvidia_driver_branch" { @@ -69,7 +87,7 @@ variable "nvidia_driver_branch" { Must exist in the Ubuntu archive for your codename: nvidia-headless--server in -updates/restricted and nvidia-fabricmanager- in -updates/multiverse. 590 - is present for both jammy and noble. + is present for both noble and resolute. Ubuntu may resolve the branch you name forward: as of this writing the 590 packages are transitional and depend on their 595 @@ -82,24 +100,6 @@ variable "nvidia_driver_branch" { EOT } -variable "base_image_checksum" { - type = string - default = "" - - description = <<-EOT - Override the integrity check on the base image (qemu.sfc_base only). - - Empty (the default) derives it from the tag, as - "file://SHA256SUMS" -- Packer fetches that file and picks - the line matching the image it is downloading, exactly as it does - for Ubuntu's cloud images. - - Releases published before SHA256SUMS existed have no such file and - Packer fails with a 404. For those, and only those, pass - -var 'base_image_checksum=none' to skip the check. - EOT -} - variable "efi_firmware_code" { type = string default = "" @@ -142,15 +142,40 @@ variable "disk_size" { EOT } +# The release index lists each image's sha256 beside its URL, so the digest +# below comes from the same document as the download and the two cannot +# disagree. +data "http" "sfc_release" { + url = var.base_image_tag == "" ? "https://tiny-llama.sfcc.xyz/latest.json" : "https://tiny-llama.sfcc.xyz/${var.base_image_tag}/manifest.json" +} + locals { - # Public, unauthenticated CDN serving the SF Compute base images. - sfc_base_url = "https://tiny-llama.sfcc.xyz/${var.base_image_tag}" + sfc_release = jsondecode(data.http.sfc_release.body) + + # The images this build may pick from: the one named, or every image in + # the release for ubuntu_codename. Empty when the name or codename is not + # in the release, which fails the build here rather than downloading + # something else. + # The variant filter keeps a future non-base build published into the + # same release (a "-minimal", say) from tying with the base image on + # CUDA and winning on list order. + sfc_candidates = var.base_image_name != "" ? [ + for i in local.sfc_release.images : i if i.name == var.base_image_name + ] : [ + for i in local.sfc_release.images : i + if i.ubuntu_codename == var.ubuntu_codename && try(i.variant, "base") == "base" + ] - ubuntu_base_url = "https://cloud-images.ubuntu.com/${var.ubuntu_codename}/current" + # Newest CUDA wins. Ranked numerically rather than by string, so a + # hypothetical 13.10 outranks 13.9 instead of sorting between 13.1 and + # 13.2. A version with no minor ranks as .0. + sfc_cuda_rank = [ + for i in local.sfc_candidates : + parseint(split(".", i.cuda_version)[0], 10) * 1000 + try(parseint(split(".", i.cuda_version)[1], 10), 0) + ] + sfc_image = local.sfc_candidates[index(local.sfc_cuda_rank, max(local.sfc_cuda_rank...))] - # Variable defaults can't reference locals, so the "empty means derive - # it" fallback lives here rather than on the variable. - base_image_checksum = var.base_image_checksum != "" ? var.base_image_checksum : "file:${local.sfc_base_url}/SHA256SUMS" + ubuntu_base_url = "https://cloud-images.ubuntu.com/${var.ubuntu_codename}/current" # Probe for OVMF, newest layout first. Packer's own default is /usr/share/OVMF/ # OVMF_VARS.fd, which does not exist on Ubuntu 24.04 -- the build gets @@ -201,13 +226,13 @@ source "qemu" "sfc_base" { # The qcow2 rather than the .raw published alongside it: same image, # ~3G instead of ~12G over the wire. format = "raw" below converts on # the way out, so what lands in output/ is still raw. - iso_url = "${local.sfc_base_url}/${var.base_image_name}.qcow2" - # Released tags publish a SHA256SUMS covering every artifact; Packer - # picks the line matching the iso_url basename. Releases built before - # SHA256SUMS existed have no such file -- see base_image_checksum. - iso_checksum = local.base_image_checksum + iso_url = local.sfc_image.qcow2.url + iso_checksum = "sha256:${local.sfc_image.qcow2.sha256}" - vm_name = "${var.base_image_name}-custom.raw" + # A fixed name, since the base image is resolved at build time: the + # upload command in the README stays copy-pasteable whatever the + # release picked. manifest.json beside it records which image that was. + vm_name = "sfc-base-custom.raw" output_directory = "output/sfc_base" disk_image = true @@ -304,4 +329,19 @@ build { script = "finalize.sh" environment_vars = ["DEBIAN_FRONTEND=noninteractive"] } + + # Otherwise the resolved tag and image survive only in terminal output. + # These are what you pass to base_image_tag and base_image_name to + # rebuild on the same base. + post-processor "manifest" { + only = ["qemu.sfc_base"] + output = "output/sfc_base/manifest.json" + + custom_data = { + base_image_tag = local.sfc_release.tag + base_image_name = local.sfc_image.name + base_image_url = local.sfc_image.qcow2.url + base_image_sha256 = local.sfc_image.qcow2.sha256 + } + } } diff --git a/cloud-init/user-data b/cloud-init/user-data index 1f2ee15..32c90e6 100644 --- a/cloud-init/user-data +++ b/cloud-init/user-data @@ -15,6 +15,6 @@ chpasswd: password: packer type: text runcmd: - # The unit is ssh.service on both jammy and noble (sshd is an alias). + # The unit is ssh.service on both noble and resolute (sshd is an alias). - printf 'PasswordAuthentication yes\nPermitRootLogin yes\n' > /etc/ssh/sshd_config.d/10-packer.conf - systemctl restart ssh