Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Contributing

Thank you for your interest in the observability stack. Bug reports, feature requests and
pull requests are welcome. Report vulnerabilities as [SECURITY.md](SECURITY.md) describes,
not in a public issue.

## Report a bug or ask for a feature

Open an [issue](https://github.com/CMLPlatform/monitoring/issues). For a bug, say what you
did, what you expected and what happened, with the version you used. A question the README
and docs don't answer is a gap in the docs: open an issue for it.

## Scope and response

One person maintains this stack, alongside other work. Expect a reply to an issue or pull
request within about two weeks.

Before a large pull request, such as a new service or a change to how projects connect,
open an issue and agree on the change there first. A small fix needs no issue. A pull
request outside what the stack sets out to do is closed with a link to this section.

## Set up and check

You need Docker with the Compose plugin and [`just`](https://github.com/casey/just).
`just hooks` installs the git hooks (with [prek](https://prek.j178.dev), which sets up the
Python environment for the citation hooks itself). `just check` runs the static checks and
validates every config, `just smoke` starts a throwaway copy of the stack and sends it
telemetry, and `just fmt` formats the YAML. CI runs `just lint`, `just validate` and
`just smoke`, plus `just infra-validate` when `infra/` changes. The
[Checks](../README.md#checks) table in the README lists every recipe.

## Write commits and pull requests

- **Commits** follow [Conventional Commits 1.0.0](https://www.conventionalcommits.org/en/v1.0.0/):
`type(scope): summary`, in the imperative, lower case, with no period. Add a body only
when the change needs one, with a bullet per logical chunk. The commit-msg hook refuses a
first line that breaks the rule.
- **The pull request title** becomes the squash commit subject, so it follows the same rule.
- **The changelog** ([CHANGELOG.md](../CHANGELOG.md)) follows
[Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/). Write each entry for
the people who run the stack or connect a project to it, not as a commit subject. Mark
breaking changes **Breaking**, and put upgrade steps first under the version.
- **The release notes** are that version's changelog section.

## Who maintains this

Simon van Lierde maintains the stack: reviews pull requests, merges them and makes
releases. The repository belongs to [CMLPlatform](https://github.com/CMLPlatform) on
GitHub. Only maintainers can merge to `main`. A new maintainer is added by an existing one,
by name in this section. If no maintainer can continue, the repository is archived with a
note in the README, and anyone may fork it under its license.
48 changes: 48 additions & 0 deletions .github/SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Security

<!-- The advisories link below works only once Settings > Code security >
Private vulnerability reporting is on. -->

## Report a vulnerability

Report it privately, through
[a GitHub security advisory](https://github.com/CMLPlatform/monitoring/security/advisories/new).
Don't open a public issue or pull request for it: anyone can read those before there is a fix.

Please include:

- What the problem is and where it is: a file, a service or an image.
- Steps to reproduce it, or a proof of concept.
- The impact you expect, and the version you saw it in.

I acknowledge a report within 14 days, then tell you whether I can reproduce it and when I
expect a fix.

## Fixes and disclosure

A fix ships in a new release. When it is out, I publish the GitHub security advisory for
it, with a CVE where one applies, and credit you unless you'd rather not be named. The
CHANGELOG entry for that release links the advisory.

## Scope

This repository is a self-hosted stack, not a published package: Compose files, the
configuration of the services they run, and the scripts and templates that projects copy to
send it telemetry. It stores the logs, traces and metrics those projects send.

In scope:

- The Compose files, the service configuration in `config/` and the dashboards.
- How the stack exposes itself: the OTLP endpoint and its token check, Grafana sign-in, the
Cloudflare Tunnel overlay and Access rule, and the guards that keep an exposed stack from
starting with default settings.
- `bootstrap.sh`, `scripts/`, the spoke templates in `templates/` and the OpenTofu config in
`infra/`.
- The CI and release workflows in `.github/workflows`.
- The image versions this repository pins, when the problem affects this stack.

Out of scope:

- A problem in Grafana, Loki, Tempo, Prometheus, the OpenTelemetry Collector or another
upstream image that doesn't depend on how this repository configures it. Report it upstream.
- A host someone runs this stack on: its firewall, Docker setup or `.env` secrets.
40 changes: 40 additions & 0 deletions .github/workflows/scorecard.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: scorecard

# OpenSSF Scorecard: rates the repository's supply-chain practices, publishes the score for the
# README badge, and uploads the findings to code scanning. With publish_results, the job may only
# use approved actions and no workflow-level env or write permissions.
on:
push:
branches: [main]
schedule:
- cron: "41 4 * * 2" # Tuesdays, 04:41 UTC
# Branch protection is one of the checks.
branch_protection_rule:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
analysis:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
security-events: write # to upload the SARIF file
id-token: write # to publish the results
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: ossf/scorecard-action@fa7117db83c52d787ae5a89f350999701505d78e # v2.5.0
with:
results_file: results.sarif
results_format: sarif
publish_results: true
- uses: github/codeql-action/upload-sarif@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
sarif_file: results.sarif
17 changes: 16 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Installed by `just hooks` (prek).
# pre-commit gitleaks on the staged diff
# pre-commit gitleaks on the staged diff; CITATION.cff against its schema
# and citesync
# commit-msg Conventional Commits shape on the first line
# pre-push `just check`, plus `just infra-validate` when infra/ changed
# `prek run` runs the pre-commit stage on demand; `--hook-stage pre-push` the rest.
Expand Down Expand Up @@ -36,3 +37,17 @@ repos:
pass_filenames: false
files: ^infra/
stages: [pre-push]

# GitHub's "Cite this repository" silently drops a CITATION.cff that fails its schema.
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0494aa7fa1314e98913a41227d8f0a515b2c1049 # frozen: 0.38.2
hooks:
- id: check-citation-file-format

# CITATION.cff is the source: any codemeta.json or .zenodo.json must agree with it, and
# `prek run citesync-write --hook-stage manual --all-files` rewrites them to match.
- repo: https://github.com/simonvanlierde/citesync
rev: a3c5a9794449e78a33583e456af41bb24ebd240e # frozen: v0.1.0
hooks:
- id: citesync
- id: citesync-write
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,15 @@ Notable changes to this stack. Format follows

## [Unreleased]

### Added

- `CITATION.cff`, so GitHub offers a citation and a Zenodo archive of a release
has the metadata it needs.
- `SECURITY.md` says how to report a vulnerability and what is in scope;
`CONTRIBUTING.md` covers issues, pull requests and the checks.
- An OpenSSF Scorecard run, weekly and on each push to `main`, with its score
as a README badge.

### Changed

- `bootstrap.sh` creates only a `watchdog` healthchecks.io check per
Expand Down
13 changes: 13 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# GitHub reads this for its "Cite this repository" button, and Zenodo when it
# archives a release. Edit this file only: there is no .zenodo.json, since Zenodo
# reads only that file when one exists.
cff-version: 1.2.0
message: If you use this software, please cite it as below.
title: "Observability stack"
abstract: "Central OpenTelemetry observability stack for CML research platforms: Grafana, Loki, Tempo, Prometheus"
authors:
- given-names: "Simon"
family-names: "van Lierde"
orcid: "https://orcid.org/0009-0006-6953-909X"
repository-code: "https://github.com/CMLPlatform/monitoring"
license: MIT
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Observability stack

[![CI](https://github.com/CMLPlatform/monitoring/actions/workflows/ci.yml/badge.svg)](https://github.com/CMLPlatform/monitoring/actions/workflows/ci.yml)
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/CMLPlatform/monitoring/badge)](https://scorecard.dev/viewer/?uri=github.com/CMLPlatform/monitoring)
[![OpenTelemetry](https://img.shields.io/badge/OpenTelemetry-OTLP-425cc7?logo=opentelemetry&logoColor=white)](config/otel-collector.yaml)
[![Grafana LGTM](https://img.shields.io/badge/Grafana-Loki%20%7C%20Tempo%20%7C%20Prometheus-f46800?logo=grafana&logoColor=white)](compose.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
Expand Down Expand Up @@ -108,9 +109,10 @@ top of [infra/main.tf](infra/main.tf).
| `just check` | `lint` plus `validate` |
| `just smoke` | Boots an isolated copy of the stack and asserts it works end to end |
| `just restore-check` | Rehearses backup and restore on the smoke stack |
| `just hooks` | Installs the git hooks: gitleaks at commit, a Conventional Commits check on the message, `check` at push, `infra-validate` at push when `infra/` changed |
| `just hooks` | Installs the git hooks: gitleaks, a CITATION.cff schema check and citesync at commit, a Conventional Commits check on the message, `check` at push, `infra-validate` at push when `infra/` changed |

Every check runs in a pinned container. Nothing is installed on the host.
Every `just` check runs in a pinned container. Nothing is installed on the host.
The citation hooks are the exception: prek builds their Python environments itself.

`just smoke` covers what has no offline validator. It boots the stack in
production shape with JWT auth on, then asserts that every dashboard and
Expand Down Expand Up @@ -201,5 +203,8 @@ file.
| [docs/RUNBOOK.md](docs/RUNBOOK.md) | Operations: rotating secrets, disk pressure, backup and restore |
| [docs/adr/](docs/adr/) | Why the stack looks like this |
| [CHANGELOG.md](CHANGELOG.md) | Release history |
| [CONTRIBUTING.md](.github/CONTRIBUTING.md) | Issues, pull requests, checks, commit style |
| [SECURITY.md](.github/SECURITY.md) | Reporting a vulnerability, and what is in scope |

Licensed under the [MIT License](LICENSE).
Licensed under the [MIT License](LICENSE). To cite the stack, use GitHub's
"Cite this repository" button, which reads [CITATION.cff](CITATION.cff).
Loading