Repository navigation
ci: run ownCloud core's acceptance tests against the image - #738
Merged
Merged
Conversation
Each release line builds its amd64 image once and runs core's tests/acceptance/run.sh from the release's -qa tarball against it, with MariaDB and the testing app: the @smoketest API subset plus the CLI suite on every push and PR, and the full API suite weekly, on manual runs and on PRs labelled full-acceptance. Known failures are listed per release in .github/acceptance/expected-failures-*.md. The build job, which publishes, now waits for these tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com>
Capture the server logs before matching them (grep -q under pipefail can fail the pipeline), stop when MariaDB, the server or the testing app does not come up, re-run CI only for the full-acceptance label, and cancel a superseded PR run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com>
A workflow-level concurrency group let an unrelated label cancel a PR's real run and left skipped checks that branch protection treats as passing, and on master it could cancel pending publish runs. Full runs now come only from the weekly schedule and manual runs. The server readiness loop also stops as soon as the container exits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com>
The smoke lists hold the CLI failures this PR's first run found on the current images. The full lists add API scenarios that failed in a local full run and again on their own against the published images. @local_storage scenarios are skipped: uploads into a freshly created local mount get 423 Locked, and which ones fail varies from run to run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com>
Behat in 11.0.1's test setup prints failed scenarios as 'file.feature:67 (on line 77)'. run.sh's check that a listed scenario really failed greps for the path at the end of the line, so every expected failure counted as an unexpected pass. Patch that grep after extracting the -qa tarball, and stop if the line is not found. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com>
oc-tmueller
marked this pull request as ready for review
October 23, 2026 10:47
Member
Author
|
@phil-davis @jvillafanez please have a look - thank you |
jvillafanez
approved these changes
Oct 26, 2026
|
Should we include redis as part of the setup since it's recommended? It could affect some parts of the code. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runs ownCloud core's acceptance tests against the image built from this repository, for both release lines. The build job, which publishes, waits for them.
How
.github/workflows/acceptance.yml(reusable, called once per release line frommain.yml):tests/acceptance/run.sh --remotefrom the release's-qatarball (sha256-checked) against the image, with MariaDB 10.11 and thetestingapp (.github/acceptance/run-part.sh).@smokeTestAPI subset (2 parts) and the CLI suite (2 parts). The weekly schedule and manual runs (workflow_dispatch, new) run the full API suite (10 parts) and the CLI suite..github/acceptance/expected-failures-<version>-<smoke|full>.md, in core'srun.shformat. Any other failure turns the job red, and so does a listed scenario that passes.Draft: the expected-failures lists are still empty. This PR's first run shows which scenarios fail on the current images, for environment reasons such as no federated second server or no external storage. Those entries are added next, until the smoke runs are green. The full lists come from a full run.
Trade-offs, open for review
buildneeds the wholeacceptancematrix, so a red run on one line also holds back the other line's publish. Keeping them independent needsbuildsplit into one job per line.buildpushes. Testing that image would need the shareddocker-build.ymlto expose it.acceptanceandbuildmatrices.composer.jsonat the release tag.symfony/yamlis pinned below 5.4.52, with Composer's advisory block disabled for it, because 10.16'sbehat.ymlexceeds that version's alias cap. This is the same as BayernBox CI.-qatarball ships its scripts0644, so the workflow restores the exec bits.run.shcallslint-expected-failures.shdirectly.Verified locally: the harness runs a CLI slice against the published
owncloud/server:11.0.1(7/7 passed, expected-failures check exit 0).actionlintand editorconfig are clean.🤖 Generated with Claude Code