Repository navigation
docs(#1094): unify SLO documentation - #1184
Conversation
Merge the content of the error-budgets research (SLOs-SLIs.md and users-expectations.md in packit/research) into the official Service Level Objectives page so there is a single authoritative document. The page now contains: - job-outcome definitions (success / failure / error) - the three SLOs with the existing official thresholds (SLO2 keeps "5 minutes after the Copr build has finished", not the research's 12 hours) - the SLIs per objective and how they are measured (Prometheus Pushgateway, histograms with buckets aligned to SLO thresholds, Grafana), checked against packit-service monitoring code - a rationale section explaining why each objective and indicator was chosen, based on the collected user expectations - guidance for other teams defining their own SLOs - a note that the page supersedes the research, which stays as historical background SLO headings keep their text, so existing anchors do not change. Note: the full site build (make import && yarn build) could not run in the sandbox because git clones fail on TLS verification. The page was compiled with @mdx-js/mdx 1.6.22 (the MDX version used by Docusaurus 2.4.1) instead. pre-commit could not fetch hook repos; check-merge-conflict and detect-private-key were run directly and passed; prettier excludes docs/. Closes #1094 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
🤖 Finished Review · ✅ Success · Started 8:18 AM UTC · Completed 8:29 AM UTC Commit: Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $3.90 |
|
Build succeeded. ✔️ pre-commit SUCCESS in 1m 12s |
TomasTomecek
left a comment
There was a problem hiding this comment.
LGTM, I actually like the laid out casual tone
| The objectives below talk about jobs (builds and tests) ending with a status. | ||
| To make them unambiguous, we distinguish these outcomes: | ||
|
|
||
| - **success**: the job finished and everything is awesome |
|
<!-- fullsend:risk-assessment --> DetailsDocs-only PR from a bot: 2 files, 161 lines, no protected, security-sensitive, CI or dependency paths, and it changes a file with no activity since 2024-01. The composite is 0.50x1.25 + 0.30x1.57 + 0.20x1.6 = 1.42, which rounds to 1. Previous run<!-- fullsend:risk-assessment --> DetailsThis is a docs-only PR from a bot, with no protected, security-sensitive, CI or dependency files, but the score rounds up to 2 (0.50x1.25 + 0.30x1.57 + 0.20x2.33 = 1.56) for three reasons: the 161-line change counts as a medium blast radius (change-size sub-score 3), the edited SLO page has not changed since January 2024, and the linked issue is over 180 days old with one maintainer suggestion (SLI screenshots) not done. Previous run (2)<!-- fullsend:risk-assessment --> DetailsThis PR changes one documentation file (156 lines, which puts blast radius at medium) and touches no protected, security-sensitive, CI or dependency paths; the author is a bot, the file has had no recent churn or fix history, and the size fits the single-task issue it closes, giving 0.50x1.25 + 0.30x1.57 + 0.20x1.6 = 1.42, rounded to 1. Previous run (3)<!-- fullsend:risk-assessment --> DetailsThis bot-authored PR changes only docs, edits a single file, touches no protected or security-sensitive paths and has no dependency or CI changes, so most signals score 1; the weighted total of 1.53 rounds up to 2 because the 145-line diff counts as a medium blast radius, the target doc was last changed over 180 days ago, and linked issue #1094 has been open for more than 180 days. Previous run (4)<!-- fullsend:risk-assessment --> DetailsThis bot PR changes one docs file and touches no protected, security, CI or dependency paths; it scores 1.53, rounded up to 2 (moderate), because of the medium blast radius of a 130-line change, a file untouched for more than 2.5 years, a linked issue open for more than 180 days, and retirement of the research page deferred to packit/research. Previous run (5)<!-- fullsend:risk-assessment --> DetailsDocs-only edit to one file by a known bot (130 lines), with no protected, security, CI or dependency paths, a file with no churn since January 2024, and a linked issue whose scope matches the PR (composite 0.5x1.25 + 0.3x1.57 + 0.2x1.8 = 1.46, rounded to 1). |
ReviewFindingsLow
Info
Next steps:
Previous run<!-- fullsend:review-findings-v2:eyJ2ZXJzaW9uIjoyLCJhY3Rpb24iOiJyZXF1ZXN0LWNoYW5nZXMiLCJmaW5kaW5ncyI6W3sic2V2ZXJpdHkiOiJsb3ciLCJjYXRlZ29yeSI6ImVkZ2UtY2FzZSIsImZpbGUiOiJkb2NzL3NlcnZpY2UtbGV2ZWwtb2JqZWN0aXZlcy1kZXRhaWxzLm1kIiwiaWQiOiJmX2RiMzc1NmViYjBjOTg4NmQiLCJsaW5lIjozLCJhY3Rpb25hYmxlIjp0cnVlfSx7InNldmVyaXR5IjoibG93IiwiY2F0ZWdvcnkiOiJtaXNsZWFkaW5nLWxhYmVsIiwiZmlsZSI6ImRvY3Mvc2VydmljZS1sZXZlbC1vYmplY3RpdmVzLm1kIiwiaWQiOiJmX2Q4NDkyYjg2M2VkMWFlNzEiLCJhY3Rpb25hYmxlIjpmYWxzZX0seyJzZXZlcml0eSI6ImxvdyIsImNhdGVnb3J5IjoidW5kZXItZW5naW5lZXJpbmciLCJmaWxlIjoiZG9jcy9zZXJ2aWNlLWxldmVsLW9iamVjdGl2ZXMtZGV0YWlscy5tZCIsImlkIjoiZl8xNWZkNDBhYWU3ODlhMDE5IiwibGluZSI6MTUyLCJhY3Rpb25hYmxlIjpmYWxzZX0seyJzZXZlcml0eSI6ImluZm8iLCJjYXRlZ29yeSI6ImRlc2lnbi1kaXJlY3Rpb24iLCJmaWxlIjoiZG9jcy9zZXJ2aWNlLWxldmVsLW9iamVjdGl2ZXMtZGV0YWlscy5tZCIsImlkIjoiZl8xMGVmYzFjYjE5ZDViYzBmIiwiYWN0aW9uYWJsZSI6ZmFsc2V9XX0= --> ReviewFindingsMedium / Low / Info
Next steps:
Previous run (2)<!-- fullsend:review-findings-v2:eyJ2ZXJzaW9uIjoyLCJhY3Rpb24iOiJhcHByb3ZlIiwiZmluZGluZ3MiOlt7InNldmVyaXR5IjoibG93IiwiY2F0ZWdvcnkiOiJkZXNpZ24tc21lbGwiLCJmaWxlIjoiZG9jcy9zZXJ2aWNlLWxldmVsLW9iamVjdGl2ZXMubWQiLCJpZCI6ImZfZWJhN2U1YzJkNjliOTM5NyIsImxpbmUiOjEzMywiYWN0aW9uYWJsZSI6ZmFsc2V9LHsic2V2ZXJpdHkiOiJsb3ciLCJjYXRlZ29yeSI6ImNvbXBsZXhpdHktcmF0aW8iLCJmaWxlIjoiZG9jcy9zZXJ2aWNlLWxldmVsLW9iamVjdGl2ZXMubWQiLCJpZCI6ImZfODc0NmIxNzNhODNjMDg0NiIsImxpbmUiOjk1LCJhY3Rpb25hYmxlIjpmYWxzZX1dfQ== --> ReviewFindingsMedium / Low / Info
Previous run (3)<!-- fullsend:review-findings-v2:eyJ2ZXJzaW9uIjoyLCJhY3Rpb24iOiJyZXF1ZXN0LWNoYW5nZXMiLCJmaW5kaW5ncyI6W3sic2V2ZXJpdHkiOiJsb3ciLCJjYXRlZ29yeSI6ImVkZ2UtY2FzZSIsImZpbGUiOiJkb2NzL3NlcnZpY2UtbGV2ZWwtb2JqZWN0aXZlcy5tZCIsImlkIjoiZl8xMzVkMzk4NWY1ODI1ZTdmIiwibGluZSI6MTAyLCJhY3Rpb25hYmxlIjp0cnVlfSx7InNldmVyaXR5IjoibG93IiwiY2F0ZWdvcnkiOiJsb2dpYy1lcnJvciIsImZpbGUiOiJkb2NzL3NlcnZpY2UtbGV2ZWwtb2JqZWN0aXZlcy5tZCIsImlkIjoiZl80MGRhNjFjNzRiOWY5OTJiIiwibGluZSI6MTI0LCJhY3Rpb25hYmxlIjpmYWxzZX1dfQ== --> ReviewFindingsMedium / Low / Info
Next steps:
Previous run (4)<!-- fullsend:review-findings-v2:eyJ2ZXJzaW9uIjoyLCJhY3Rpb24iOiJyZXF1ZXN0LWNoYW5nZXMiLCJmaW5kaW5ncyI6W3sic2V2ZXJpdHkiOiJsb3ciLCJjYXRlZ29yeSI6ImxvZ2ljLWVycm9yIiwiZmlsZSI6ImRvY3Mvc2VydmljZS1sZXZlbC1vYmplY3RpdmVzLm1kIiwiaWQiOiJmXzVmNTM5ZjY1MDQwMTQ0ZjIiLCJsaW5lIjo4NiwiYWN0aW9uYWJsZSI6dHJ1ZX0seyJzZXZlcml0eSI6ImxvdyIsImNhdGVnb3J5IjoibG9naWMtZXJyb3IiLCJmaWxlIjoiZG9jcy9zZXJ2aWNlLWxldmVsLW9iamVjdGl2ZXMubWQiLCJpZCI6ImZfN2ZlZjgzNjUzMjhlMjA1NCIsImxpbmUiOjcwLCJhY3Rpb25hYmxlIjp0cnVlfV19 --> ReviewFindingsLow
Info
Labels: Docs-only SLO documentation change not tied to a specific integration, matching the area label on linked issue #1094. Next steps:
Previous run (5)<!-- fullsend:review-findings-v2:eyJ2ZXJzaW9uIjoyLCJhY3Rpb24iOiJyZXF1ZXN0LWNoYW5nZXMiLCJmaW5kaW5ncyI6W3sic2V2ZXJpdHkiOiJsb3ciLCJjYXRlZ29yeSI6ImxvZ2ljLWVycm9yIiwiZmlsZSI6ImRvY3Mvc2VydmljZS1sZXZlbC1vYmplY3RpdmVzLm1kIiwiaWQiOiJmX2NiMjY5ODUzYmE2NjEyN2YiLCJsaW5lIjo3MCwiYWN0aW9uYWJsZSI6dHJ1ZX0seyJzZXZlcml0eSI6ImxvdyIsImNhdGVnb3J5IjoiZG9jLXN0eWxlIiwiZmlsZSI6ImRvY3Mvc2VydmljZS1sZXZlbC1vYmplY3RpdmVzLm1kIiwiaWQiOiJmX2UwNjY3ZGUyZjBjYTkxNTciLCJsaW5lIjoxMDEsImFjdGlvbmFibGUiOnRydWV9XX0= --> ReviewFindingsLow
On the reviewer note about SLO1's threshold: git history shows that commit 76caf5b (2023-03-22, "Update SLO1 definition") deliberately changed SLO1 from "a status update within 30 seconds" to "the first status update within 15 seconds". That came after the late-2022 change to 30s for all statuses. So the published 15-second value is the intended current threshold, and this PR is right to keep it. Labels: The PR only rewrites a documentation page (SLO docs). Next steps:
|
|
🤖 Finished Fix · ✅ Success · Started 8:30 AM UTC · Completed 8:34 AM UTC Commit: Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $0.64 |
- State the real 25-second limit of the SLO1 "no status" counter in the indicators table and note it is looser than the 15-second SLO1 limit. - Capitalize the first word after the colon in the rationale subheadings to match the Objectives headings. Note: pre-commit could not fetch hook repositories (TLS verification failure in the sandbox). check-merge-conflict and detect-private-key (pre-commit-hooks v5.0.0) were run directly and passed; the prettier hook excludes docs/. Addresses #1184 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🔧 Fix agent — iteration 1 (bot-triggered)Fixed both low-severity findings in docs/service-level-objectives.md. The SLO1 'no status' counter now states its real 25-second limit, and the rationale subheadings are capitalized. The repo has no test or lint suite, and a full site build needs network clones that fail in the sandbox. Instead, the page was compiled with @mdx-js/mdx 1.6.22 (the version Docusaurus 2.4.1 uses) without errors. pre-commit could not fetch its hook repos (TLS failure), so check-merge-conflict and detect-private-key were run directly and passed. The prettier hook excludes docs/. Project CI on the previous head was all passing. Fixed (3):
Tests: passed CI inspections (4):
Conflict update: skipped; strategy Decision points
Next steps:
|
|
🤖 Review · ❌ Terminated · Started 8:34 AM UTC · Ended 8:45 AM UTC Commit: |
|
Build succeeded. ✔️ pre-commit SUCCESS in 1m 11s |
|
🤖 Finished Review · ✅ Success · Started 8:34 AM UTC · Completed 8:45 AM UTC Commit: Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $4.10 |
|
🤖 Finished Fix · ✅ Success · Started 8:46 AM UTC · Completed 8:50 AM UTC Commit: Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $0.75 |
Address the second round of review feedback on PR #1184: - Describe the SLO1 counter accurately: it counts PRs or commits whose first status took longer than 25 seconds, and neither SLO1 indicator sees events that never get a status. - Say that the histograms give the share of jobs that finished within the time threshold, whatever the outcome, so they approximate SLO2 and SLO3. Explain how error outcomes are handled for test runs (counted) and for builds (missing from the histogram). Verification: pre-commit could not fetch its hook repositories (TLS verification failure in the sandbox). The prettier hook excludes docs/. check-merge-conflict and detect-private-key were checked with equivalent pattern searches and passed. The page compiles with @mdx-js/mdx 1.6.22. Addresses #1184 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🔧 Fix agent — iteration 2 (bot-triggered)Addressed both low-severity logic-error findings in docs/service-level-objectives.md. The SLO1 counter is now described as counting first statuses that took longer than 25s, and the page notes that dropped events are not captured. The How we measure section now says the histograms measure only time, whatever the outcome, so they approximate SLO2/SLO3, and it explains how error outcomes are handled for test runs and builds. pre-commit could not fetch its hook repos (sandbox TLS failure). The applicable hooks were checked with equivalent pattern searches, and the page compiles with @mdx-js/mdx 1.6.22. Fixed (3):
Tests: passed CI inspections (4):
Conflict update: skipped; strategy Decision points
Next steps:
|
|
🤖 Review · ❌ Terminated · Started 8:51 AM UTC · Ended 9:02 AM UTC Commit: |
|
Build succeeded. ✔️ pre-commit SUCCESS in 1m 11s |
|
🤖 Finished Review · ✅ Success · Started 8:51 AM UTC · Completed 9:01 AM UTC Commit: Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $4.03 |
|
/fs-fix I would keep the service-level-objectives.md page, as simple as it is, for non developer users. You could instead add a new more detailed page (add a link to it from the service-level-objectives.md page and we can remove the link to the research) for those users, probably developers, that want to know more how are our SLOs built. In that case it could also be useful to add pictures for our SLIs. The graphana dashboards for SLIs are called Packit and Fedora CI. |
|
🤖 Finished Fix · ✅ Success · Started 12:51 PM UTC · Completed 12:55 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $0.83 |
Follow the /fs-fix instruction on PR #1184: - Restore docs/service-level-objectives.md to the simple page for non-developer users. It is now the same as on main, except the link to the error-budgets research is replaced by a link to the new details page. - Add docs/service-level-objectives-details.md ("How our SLOs are built") for readers who want to know more. It covers job outcomes, the SLIs, how we measure them, why we picked them, guidance for other teams, and background. It names the Grafana dashboards that show the SLIs: Packit and Fedora CI. SLI screenshots were not added. The sandbox cannot download the example image from the PR comment, and Grafana sits behind a firewall. Screenshots from the Packit and Fedora CI dashboards need to be added by someone with access. Verification: pre-commit could not fetch its hook repositories (TLS verification failure). check-merge-conflict and detect-private-key from pre-commit-hooks v5.0.0 were run directly and passed. The prettier hook excludes docs/. Both pages compile with @mdx-js/mdx 1.6.22. Addresses #1184 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🔧 Fix agent — iteration 4 (human-triggered)Following the /fs-fix instruction, service-level-objectives.md is back to the simple page from main. It now links to a new 'How our SLOs are built' page instead of the research. The detailed SLO/SLI content moved to that page, which also names the Packit and Fedora CI Grafana dashboards. SLI screenshots could not be added from the sandbox and need to be added by someone with Grafana access. Fixed (4):
Disagreed (1):
Tests: passed CI inspections (4):
Conflict update: skipped; strategy Decision points
Next steps:
|
|
🤖 Review · ❌ Terminated · Started 12:56 PM UTC · Ended 1:07 PM UTC Commit: |
|
Build succeeded. ✔️ pre-commit SUCCESS in 1m 10s |
Superseded by updated review
|
🤖 Finished Review · ✅ Success · Started 12:56 PM UTC · Completed 1:07 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $4.26 |
|
🤖 Finished Fix · ✅ Success · Started 1:08 PM UTC · Completed 1:11 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $0.69 |
Address the fourth round of review feedback on PR #1184: - Set sidebar_position to 12.5 on the details page. Before, it shared position 12 with service-level-objectives.md (and reproduce-locally.md), so the autogenerated sidebar could put the details page above the overview. Docusaurus 2.4.1 validates sidebar_position as a plain number and sorts by it, so fractional values work. Not changed in this commit (they need a human or another repository): the PR description still describes the first single-page version, the packit/research follow-up has no tracking issue, and the Grafana screenshots are still missing. Verification: pre-commit could not fetch its hook repositories (TLS verification failure). check-merge-conflict and detect-private-key from pre-commit-hooks v5.0.0 were run directly and passed. The prettier hook excludes docs/. The repository has no test suite, and the full site build needs network access for `make import`. Addresses #1184 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🔧 Fix agent — iteration 5 (bot-triggered)Fixed the one finding that needed a code change: the SLO details page now uses sidebar_position 12.5, so it appears after the overview page it builds on instead of tying with it at 12. The other findings need a human: updating the PR description (two-page design, missing Grafana screenshots), opening a packit/research follow-up issue or changing 'Closes #1094' to 'Refs #1094', and adding the screenshots. All project CI checks pass. No merge conflict was reported. Fixed (2):
Disagreed (4):
Tests: passed CI inspections (4):
Conflict update: skipped; strategy Decision points
Next steps:
|
|
🤖 Review · ❌ Terminated · Started 1:12 PM UTC · Ended 1:36 PM UTC Commit: |
|
Build succeeded. ✔️ pre-commit SUCCESS in 1m 09s |
| [what percentage of the measured jobs](https://prometheus.io/docs/practices/histograms/) | ||
| finished within the threshold, instead of relying on averages. | ||
|
|
||
| The histograms record only the time, whatever the outcome of the job. This means |
There was a problem hiding this comment.
[low] logic-error
The lead sentence "The histograms record only the time, whatever the outcome of the job" contradicts the bullets that follow it. Lines 65-67 say the build-reporting histogram is recorded only when Packit reports success or failure. Lines 71-73 say a failed build is not measured at all when its SRPM build already failed or a newer build exists for the same target. Readers may conclude that every finished build is in the SLO2 histogram, whatever its outcome.
Suggested fix: Reword line 58 so it does not say recording ignores the outcome. For example: "The histograms record only durations, not job outcomes, and some outcomes are not recorded at all. This means the current indicators approximate SLO2 and SLO3 rather than compute them exactly:"
| them, or drop objectives nobody talks about, as you learn more. | ||
|
|
||
| ## Background | ||
|
|
There was a problem hiding this comment.
[low] under-engineering
The PR closes #1094, and the page says it supersedes the error-budgets research. But the research is still imported from packit/research (make import, docusaurus.config.js) and stays published with no historical notice and no link to the new page, so the site still shows two SLO/SLI descriptions. The packit/research follow-up has no tracking issue. The research file lives in another repository and could not be checked here.
Suggested fix: Open a tracking issue or PR in packit/research to add a historical notice to research/monitoring/error-budgets/SLOs-SLIs.md that links to the official SLO pages, or change "Closes #1094" to a non-closing reference until that is done.
|
🤖 Finished Review · ✅ Success · Started 1:12 PM UTC · Completed 1:36 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $3.86 |
|
🤖 Finished Fix · ❌ Failure (running pre-script: exit status 1: Fix iteration 6 exceeds bot cap of 5. Escalating to human. The review→fix loop has run 6 times without converging. A human can still direct the agent with /fs-fix (u…) · Started 1:37 PM UTC · Completed 1:37 PM UTC Commit: Effort: high |
TomasTomecek
left a comment
There was a problem hiding this comment.
LGTM and agreed this is a much better structure
|
Build succeeded (gate pipeline). ✔️ pre-commit SUCCESS in 1m 08s |

Summary
Unifies Packit's SLO documentation into the official page
docs/service-level-objectives.md. The content of the error-budgets research (SLOs-SLIs.mdandusers-expectations.mdin packit/research) is merged in, so the website has one authoritative page instead of an official page plus a research page.The page now has:
packit_service/worker/monitoring.pyandjobs.pyin packit-service.Notes for reviewers
first_initial_status_timehistogram has a 15s bucket. However, theno_status_after_25_scounter uses 25s, and the week 47 2022 weekly post said SLO1 was raised from 15s to 30s (packit-service#1776). I did not change the published thresholds because the issue didn't ask for that.packit/researchthroughmake import, so it can't be retired here.research/monitoring/error-budgets/SLOs-SLIs.mdin packit/research should be changed to point to this page or marked as historical.Testing
make import && yarn build) could not run in the sandbox because git clones fail TLS verification. Instead, I compiled the page with@mdx-js/mdx@1.6.22(the MDX version Docusaurus 2.4.1 uses). The table, headings and admonition compiled without errors. All new links are external, and the image paths are unchanged. CI's test-deploy job runs the full build.check-merge-conflictanddetect-private-key(pre-commit-hooks v5.0.0) directly, and both passed. The prettier hook excludesdocs/.🤖 Generated with Claude Code
Closes #1094
Post-script verification
agent/1094-unify-slo-docs)c0e3687835a9f7786e74d7610b56d2c684879ec9..HEAD)