build: baseline byte-identity vs portable builds — adopt Tycho-native baselineReplace?

[joaodinissf]

Why this issue

While verifying #1475 (headless Docker build env), the full in-container verify fails at compare-version-with-baselines:

Baseline and reactor have the same fully qualified version, but different content

for bundles untouched since the last release (xtext.scope, xtext.export). Root cause, established by unpacking both jars: the payloads are byte-identical — every class, the manifest, everything. The archives differ only in zip directory-entry order, because the baseline was packed on a CI (ext4) filesystem and the local build packed from a bind mount whose readdir returns entries in a different order. The compare-version-with-baselines mojo defaults to the bytes comparator (raw archive comparison), so archive-ordering noise fails the gate. The same applies to any non-CI environment; CI passes only because it is the environment that produced the baseline.

So today's gate conflates two different guarantees:

1. The one we care about: same version ⇒ same published content (protects p2 consumers and release promotion).
2. An accidental one: builds only "work" in the canonical CI environment.

What was measured

• Payloads byte-identical, ordering-only divergence (verified by extraction + entry-order diff).
• Setting project.build.outputTimestamp (Maven reproducible-packing mode, honored by Tycho 5) makes builds byte-identical across filesystems — verified A/B (bind mount vs native fs), with a negative control (without the property, the same A/B diverges). Cost: repacks every artifact once → a one-time repo-wide baseline "wave", and the timestamp must effectively be frozen forever (bumping it per release would re-trigger the wave every release), making jar entry dates meaningless.
• The content-aware comparator family (zip/class/manifest) that ships with Tycho passes the failing bundles and still detects real content changes. Its ignore-list (verified from the 5.0.4 bytecode): zip order/timestamps, META-INF/maven/**, and manifest churn headers (Bnd-LastModified, Built-By, Build-Jdk*, Created-By, Tool, …).

How the ecosystem handles this

• Eclipse/Tycho mainstream does not try to make rebuilds byte-identical. The standard tycho-p2-plugin baseline support compares with the content-aware comparators and — with baselineReplace=all, the default — replaces rebuilt-but-unchanged artifacts with the baseline's original bytes at packaging time. Published byte-identity holds by construction: an unchanged bundle's released jar is carried forward; rebuild nondeterminism never reaches the repo. ("P2 assumes that the same artifact type, id and version represent the same artifact.")
• Maven mainstream commits project.build.outputTimestamp and auto-bumps it per release (maven-release-plugin) — fine there because that ecosystem never byte-compares release N against N−1; it would break our cross-release gate.

Proposal to discuss: adopt Tycho-native baseline replace

1. Configure baseline in the packaging step (tycho-p2-plugin / p2-metadata-default in ddk-parent): baselines=${baseline.repo.url}, baselineMode=failCommon, baselineReplace=all.
2. Retire (or demote) the compare-version-with-baselines execution — its protection becomes redundant: change detection happens content-aware in step 1, and same-version-different-bytes can no longer be published at all.
3. Verify the release pipeline composes — confirm the baseline-replaced artifacts are what ddk-repository publishes, and dry-run once to see the substitution logged.

What the semantics become

• Guarantee (strengthened): a released version is literally the same file forever — same version ⇒ same bytes at publication, by construction rather than by check.
• Trigger for version bumps (relaxed, deliberately): moves from "any byte drift" to "semantic content drift" (comparator ignore-list above). Manifest churn headers / archive repacking no longer force bumps.
• Release repos may contain jars older than the release build — by design; unchanged bundles ship their original release-time bytes.
• Local/container builds stop false-failing with no flags or skips: ci: add reusable headless Docker build environment #1475's documented workflow becomes green as-is.
• No transition wave: replace absorbs packing differences from day one (unlike the outputTimestamp route).

Alternatives considered (and why not preferred)

• Frozen project.build.outputTimestamp: proven to work (see measurements), simplest diff; but jar timestamps become a meaningless constant, one repo-wide baseline wave at adoption, and it protects rebuild identity, which nothing consumes — promotion needs published identity.
• Property-gated local skip of the gate (the mojo has no CLI property for skip; a pom-level ${...} hook is required either way): CI-neutral and simple, but local builds lose the check entirely, and the underlying conflation remains.
• Switching the gate's comparator to zip unconditionally: weakens the byte gate on the promotion path — rejected.

Open questions

• failCommon vs warn for baselineMode in CI?
• Keep the old gate as belt-and-braces (zip comparator) or delete it?
• Any consumers relying on jar entry timestamps or on "shipped == freshly built" assumptions?

---

Investigation and drafting by Claude (session-assisted), on behalf of @joaodinissf. Evidence: in-container A/B builds + comparator bytecode inspection; details reproducible on request.