Skip to content

fix: 2.2.2 round on scalo 2.14.22 - #134

Merged
catinspace-au merged 7 commits into
mainfrom
fix/round-222
Oct 29, 2026
Merged

catinspace-au merged 7 commits into
mainfrom
fix/round-222

Conversation

@catinspace-au

@catinspace-au catinspace-au commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

The 2.2.2 round for dfe-transform-vector, on scalo 2.14.22.

  • Rebuild on scalo 2.14.22. The app builds no Kafka client, so scalo's transport-kafka feature becomes transport-kafka-config and librdkafka leaves the binary (cargo tree -e normal -i rdkafka-sys prints nothing). The dev-dependency keeps transport-kafka for the broker tests.
  • The Kafka Secret group does NOT mount security.protocol. The DFE Kafka user Secret (dfe-infra helm/charts/kafka, dfe-kafka-user) carries only username, password and sasl.mechanism, so a required mount of that key would stop the pod. dfe-infra already sets DFE_TRANSFORM_KAFKA_SECURITY_PROTOCOL as plain env from kafka.securityProtocol (argocd/values/apps/dfe-transform-vector/profile-single.yaml, profile-scale.yaml). The rest of the earlier commit (shared validate argv, the .secrets reload window) stays.
  • emit_dockerfile returns the generator's Result, the contract literal ends in DeploymentContract::default(), and the Dockerfile is regenerated: no curl, netcat or ping, a WORKDIR, writable paths owned by appuser, a bash /dev/tcp health check.
  • The image-filesystem test helper reads every RUN mkdir -p line, quoted or not, since scalo now emits one for the writable paths.
  • build.rust.optimize.pgo.workload_setup_cmd builds pgo-driver before the PGO workload clock starts.
  • libz-sys 1.1.30 -> 1.1.31 for RUSTSEC-2026-0418 (its build script printed the build environment into its output file under target/).
  • build.rust.glibc_max: "2.41", and Build and lint on Namespace runners.

Local gate: clippy (shipped code strict, tests and benches relaxed, all features) clean, and the full suite passes locally with Docker (358 tests).

build.rust.glibc_max is 2.41, the glibc of the trixie-slim runtime image, so hyperi-ci packaging fails a binary linked against anything newer.
…col through the chart, and close the .secrets reload window

Three follow-ups from moving config validation to --no-environment.

The older validate tests built their own argv. They call validate_args now, so
they cannot drift from what the app runs.

The Kafka Secret group carried the SASL mechanism but not the security
protocol, which the app already reads from DFE_TRANSFORM_KAFKA_SECURITY_PROTOCOL
and which turns TLS on for both endpoints when it names SSL. The group now
mounts it from the Secret's security.protocol key beside the mechanism.

The reload race on .secrets is real at the file level. assemble removed the
whole config directory and rewrote each credential in place, so a process
reading the credential files during a re-assembly found them missing or empty:
7,173 missing and 62 empty reads in 300 re-assemblies with a reader polling one
file. assemble now leaves the .secrets directory in place, each credential is
written to a sibling and moved over the old file, and a credential the new
config does not need is removed afterwards.

Closes #96
Release Builds run on Namespace 32x64 for both arches and Quality on 8x16. Test stays on ARC because it pulls images through Harbor.
The app builds no Kafka client, so scalo's transport-kafka feature becomes transport-kafka-config and librdkafka leaves the binary, while the dev-dependency keeps transport-kafka for the broker tests. The Kafka Secret group no longer mounts security.protocol: the DFE Kafka user Secret carries no such key, so the required mount would stop the pod, and the deployment already sets DFE_TRANSFORM_KAFKA_SECURITY_PROTOCOL as plain env. emit_dockerfile returns the generator's Result, the contract literal ends in DeploymentContract::default(), the Dockerfile is regenerated, and the image-filesystem test helper reads every mkdir line, quoted or not.
On an uncached arm64 runner the pgo-driver compile inside scripts/pgo-workload.sh can outlast the whole workload timeout, as it did for dfe-receiver. hyperi-ci's pgo.workload_setup_cmd builds it first, under its own limit, so the timeout covers only the load.
libz-sys 1.1.30 built with cc 1.6 or newer printed the full build environment into its build-script output file under target/. 1.1.31 fixes it, and cargo audit and cargo deny fail the Quality job on the older one.
@catinspace-au
catinspace-au merged commit 205eb6f into main Oct 29, 2026
17 checks passed
@catinspace-au
catinspace-au deleted the fix/round-222 branch October 29, 2026 15:09
catinspace-au added a commit that referenced this pull request Oct 29, 2026
The production base image carries no curl since scalo 2.14.22, so #134's release run failed its Container job on the Vector download ("curl: not found") and published nothing. The download layer now installs curl, downloads and checks the archive, and purges curl in the same RUN. This release carries #134.

Release: true
@github-actions

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant