Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .ai/base.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ Verify the real API before using or documenting it. Key packages:
- `pkg/mutation/selectors/` — available container selectors
- `pkg/feature/feature.go` — `NewVersionGate`, `Mutation[T]`
- `pkg/recording/` — resource event recording
- `pkg/metrics/` — Prometheus implementation of `component.MetricsRecorder`: condition metrics plus the per-resource
apply counters
- `pkg/testing/` — testing utilities (`golden/` for snapshot tests, `integration/` for integration helpers)

When changing a public API, also check `examples/` for real usage patterns and to identify what else needs updating.
Expand Down
2 changes: 2 additions & 0 deletions .github/copilot-instructions.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,15 +90,15 @@ package defaults to `certificate` (the lowercased kind), and the version default
segment, which matches the API-version pattern). The four generated files:

- `builder.go` registers the scaffolded default handlers, exposes the fluent configuration API (`WithMutation`,
`WithGuard`, `WithDataGuard`, `WithOptionalData`, the `WithCustom*` status setters), and `Build()` returns the
`Resource`.
`WithGuard`, `WithDataGuard`, `WithOptionalData`, `WithMetricsIdentifier`, the `WithCustom*` status setters), and
`Build()` returns the `Resource`.
- `builder_test.go` tests `Build()` validation, that a registered mutation applies through the mutator, declared data
extraction with `ExtractInto`, and `WithDataGuard`/`WithOptionalData` gating.
- `mutator.go` defines `Mutator`, which records metadata and object edits and applies them in a single pass when
`Apply()` runs.
- `resource.go` defines `Resource`, which delegates every lifecycle method to the generic base: `Identity`, `Object`,
`Mutate`, the variant's status and suspension methods, `GuardStatus`, `ExtractData`, `ProducedData`, `ConsumedData`,
`RecordObservation`, `Preview`, `RegisteredMutations`, and `FiringSet`.
`Mutate`, `MetricsIdentifier`, the variant's status and suspension methods, `GuardStatus`, `ExtractData`,
`ProducedData`, `ConsumedData`, `RecordObservation`, `Preview`, `RegisteredMutations`, and `FiringSet`.

Follow the printed next steps: run `go mod tidy` so `github.com/cert-manager/cert-manager/pkg/apis/certmanager/v1`
resolves in your module, then `go test ./certificate/...` to verify the generated package builds and passes before you
Expand Down
95 changes: 90 additions & 5 deletions docs/component.md
Original file line number Diff line number Diff line change
Expand Up @@ -646,7 +646,7 @@ recCtx := component.ReconcileContext{
Client: r.Client, // sigs.k8s.io/controller-runtime/pkg/client
Scheme: r.Scheme, // *runtime.Scheme
EventRecorder: r.EventRecorder, // events.EventRecorder, from manager.GetEventRecorder(name)
Metrics: r.Metrics, // component.MetricsRecorder (condition metrics), optional
Metrics: r.Metrics, // component.MetricsRecorder, optional; see Metrics
APIReader: r.APIReader, // client.Reader, from manager.GetAPIReader(); direct reads on a status conflict
Owner: owner, // the CRD that owns this component
}
Expand All @@ -655,9 +655,9 @@ err = comp.Reconcile(ctx, recCtx)
```

Dependencies are passed explicitly so components stay testable and decoupled from global state. The `Metrics` field is
optional; when set, the framework records Prometheus metrics for every condition reported during a reconcile, using the
recorder from [go-crd-condition-metrics](https://github.com/sourcehawk/go-crd-condition-metrics). Leave it `nil` to opt
out.
optional; when set, the framework records Prometheus metrics for every condition reported during a reconcile and for
every resource it applies. Use the recorder from `pkg/metrics`, or implement `component.MetricsRecorder` to record
elsewhere. Leave it `nil` to opt out. See [Metrics](#metrics).

`EventRecorder` takes a `k8s.io/client-go/tools/events.EventRecorder`. The manager accessor that returns one,
`GetEventRecorder(name)`, was added in controller-runtime v0.23; on v0.22.x, build the recorder from client-go instead,
Expand Down Expand Up @@ -763,7 +763,7 @@ follows the server, and the next reconcile stages it again.
source.

After a successful update, `FlushStatus` records metrics for every condition on the owner. If `Metrics` is `nil`,
recording is skipped.
recording is skipped. See [Metrics](#metrics).

This split is what lets a controller with several components stage several conditions during one reconcile and persist
them in a single write. Persisting after each component would race the components' writes and produce 409 conflicts. See
Expand Down Expand Up @@ -821,6 +821,91 @@ applies to a status-only controller too, as in the validation-only example above
app.Status.ObservedGeneration = app.Generation
```

## Metrics

The framework records two families of Prometheus metrics through the single `ReconcileContext.Metrics` recorder:
**condition metrics**, one gauge per owner and condition type, and **resource metrics**, counters describing what the
framework did to each managed resource.

`pkg/metrics` implements both. Build the collectors once per process and register them with controller-runtime's
registry, then give each controller its own recorder:

```go
var (
conditions = ocm.NewOperatorConditionsGauge("myoperator")
collectors = metrics.NewCollectors()
)

func init() {
ctrlmetrics.Registry.MustRegister(conditions, collectors)
}

recCtx := component.ReconcileContext{
// ...
Metrics: metrics.NewRecorder("webapp-controller", conditions, collectors),
}
```

The controller name becomes the `controller` label on every series the recorder emits. Passing `nil` for either
collector disables that family; passing `nil` for `Metrics` itself disables both.

### Resource metrics

| Series | Type | Labels |
| --------------------------------- | ------- | ------------------------------------------------------------------------ |
| `ocf_resource_apply_total` | counter | `controller`, `owner_kind`, `component`, `resource`, `kind`, `operation` |
| `ocf_resource_apply_errors_total` | counter | `controller`, `owner_kind`, `component`, `resource`, `kind` |

`operation` is `created`, `updated` or `none`, the same classification the framework reports through
[`ConvergingOperation`](#status-model) and the apply event.

Both counters cover managed resources on the reconcile path and the suspension path. Read-only resources, deletions and
orphans are not applies and record nothing.

Exactly one of the two counters moves per apply attempt, never both. The error counter covers every failure of the
attempt: mutating the desired object, the patch itself, and the classification that follows it. The one exception is a
failure so early that the framework has not worked out what it is applying, where no kind exists to label a series with;
those surface as an error condition on the owner instead.

The reading that matters most is the `updated` rate. In steady state, a converged resource applies as `none` on every
pass and `updated` stays flat:

```promql
rate(ocf_resource_apply_total{operation="updated"}[5m])
```

A resource whose `updated` rate never settles to zero is being rewritten on every reconcile even though nothing changed.
Events report the same thing, but client-go's spam filter truncates them within seconds under exactly those conditions,
which is why the counter exists.

### The resource identifier

The `resource` label comes from `WithMetricsIdentifier` on the resource's builder:

```go
secret.NewBuilder(tlsSecret).
WithMetricsIdentifier("tls").
Build()
```

It is a Prometheus label value, not a name of anything in Kubernetes. When unset, the framework labels the resource with
its lowercased kind, so metrics work without any configuration. Set an identifier to tell two resources of the same kind
apart within one component, or when the object's name carries a generated suffix. `Build` rejects a blank identifier;
omit the call to accept the default.

**The identifier must be low-cardinality and stable across reconciles**: a constant, or a value drawn from a small fixed
set. Deriving it from a per-owner value such as the owning custom resource's name creates one time series per owner, and
the framework never removes a series once created.

Kept to that rule, the series set is bounded by the operator's static topology. No owner name or namespace appears in
these labels, so the same handful of series covers three owners or three thousand, and deleting a resource or an owner
leaves nothing behind that needs reaping. This is why there is no resource-metric counterpart to `RemoveConditionsFor`:
a resource that goes away simply stops incrementing, and deleting a counter series mid-flight would read downstream as a
counter reset and corrupt `rate()` and `increase()`.

Per-owner detail lives in the condition metrics instead, which are keyed by owner name and namespace and are cleaned up
with `RemoveConditionsFor` when the owner is deleted.

## Declared Data

Resources inside one component pass observed values to each other through **data cells**. A cell is created in the
Expand Down
23 changes: 19 additions & 4 deletions docs/custom-resource.md
Original file line number Diff line number Diff line change
Expand Up @@ -715,6 +715,14 @@ func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder {
return b
}

// WithMetricsIdentifier sets the resource's identifier for resource-level metrics,
// used as the value of the `resource` label. It must be low-cardinality and stable
// across reconciles; when unset the framework labels the resource by kind.
func (b *Builder) WithMetricsIdentifier(identifier string) *Builder {
b.base.WithMetricsIdentifier(identifier)
return b
}

// WithCustomConvergeStatus overrides the default convergence status handler.
func (b *Builder) WithCustomConvergeStatus(
handler func(concepts.ConvergingOperation, *examplev1.MessageQueue) (concepts.AliveStatusWithReason, error),
Expand Down Expand Up @@ -890,6 +898,7 @@ func (r *Resource) FiringSet() ([]string, error) {
var _ concepts.MutationInspector = (*Resource)(nil)
var _ concepts.DataProducer = (*Resource)(nil)
var _ concepts.DataConsumer = (*Resource)(nil)
var _ concepts.MetricsIdentifiable = (*Resource)(nil)
```

!!! warning "Do not omit `Preview`"
Expand All @@ -907,6 +916,11 @@ always if your builder exposes `ExtractInto`, `WithDataGuard`, or `WithOptionalD
[build-time topology validation](component.md#build-time-validation) silently passes, `DataTopology()` omits the
resource, and its cells are never cleared at the start of a reconcile.

Forward `MetricsIdentifier` whenever your builder exposes `WithMetricsIdentifier`. It satisfies
`concepts.MetricsIdentifiable`, which is how the framework reads the identifier at apply time. Without it the builder
accepts an identifier and the framework silently labels the resource by kind instead. See
[Metrics](component.md#metrics).

Forward `RecordObservation` whenever the resource may be registered read-only and declares an extraction. The framework
feeds the fetched cluster object back to the resource before extraction runs; without it, the extraction would see the
inert base passed to the builder rather than live cluster state.
Expand Down Expand Up @@ -1162,10 +1176,11 @@ implications.
### Static resources

Static resources have the simplest implementation. They do not participate in convergence, grace, or suspension
reporting. The builder uses `generic.NewStaticBuilder`, which supports `WithMutation`, `WithGuard`, `WithDataGuard`, and
`WithOptionalData`, plus a package-level `ExtractInto`. The resource wrapper needs only `Identity`, `Object`, `Mutate`,
`GuardStatus`, `ExtractData`, `ProducedData`, `ConsumedData`, `RecordObservation`, `Preview`, `RegisteredMutations`, and
`FiringSet`. `pkg/primitives/configmap` is a complete reference.
reporting. The builder uses `generic.NewStaticBuilder`, which supports `WithMutation`, `WithGuard`, `WithDataGuard`,
`WithOptionalData`, and `WithMetricsIdentifier`, plus a package-level `ExtractInto`. The resource wrapper needs only
`Identity`, `Object`, `Mutate`, `MetricsIdentifier`, `GuardStatus`, `ExtractData`, `ProducedData`, `ConsumedData`,
`RecordObservation`, `Preview`, `RegisteredMutations`, and `FiringSet`. `pkg/primitives/configmap` is a complete
reference.

### Task resources

Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ self-induced update conflicts.
| `Client` | `client.Client` | The controller-runtime client. |
| `Scheme` | `*runtime.Scheme` | The operator scheme. |
| `EventRecorder` | `events.EventRecorder` | For Kubernetes events. `GetEventRecorder(name)` on the manager, v0.23 and later. |
| `Metrics` | `component.MetricsRecorder` | Optional. Pass `nil` to skip status-condition metrics. |
| `Metrics` | `component.MetricsRecorder` | Optional. `metrics.NewRecorder(...)` records condition and resource metrics; pass `nil` to skip both. See [Metrics](component.md#metrics). |
| `APIReader` | `client.Reader` | Optional, recommended. `GetAPIReader()` on the manager; `FlushStatus` reads through it on a 409 so the retry sees the live owner, not the informer cache. |
| `Owner` | `component.OperatorCRD` | The owner object you fetched. Your CRD satisfies this via `GetStatusConditions`. |

Expand Down
20 changes: 20 additions & 0 deletions docs/primitives.md
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,26 @@ The interface deliberately omits operations that are not common to all three kin
replica field), and the StatefulSet-only VolumeClaimTemplate methods. Reach for the concrete mutator type when you need
those.

## Metrics Identifier

Every primitive builder exposes `WithMetricsIdentifier`, which sets the value of the `resource` label on the framework's
per-resource apply counters:

```go
res, err := secret.NewBuilder(tlsSecret).
WithMetricsIdentifier("tls").
Build()
```

When unset, the framework labels the resource with its lowercased kind (`secret`, `deployment`), so the counters work
without configuration. Set an identifier to tell two resources of the same kind apart within one component, or when the
object's name carries a generated suffix.

The identifier is a Prometheus label value, not a Kubernetes name, so it must be low-cardinality and stable across
reconciles. Never derive it from a per-owner value such as the owning custom resource's name. `Build` rejects a blank
identifier; omit the call to accept the default. See [Metrics](component.md#metrics) for the series, their labels, and
the cardinality contract.

## Built-in Primitives

| Primitive | Category | Documentation |
Expand Down
10 changes: 5 additions & 5 deletions e2e/component/suite_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
"github.com/sourcehawk/operator-component-framework/e2e/framework"

ocm "github.com/sourcehawk/go-crd-condition-metrics/pkg/crd-condition-metrics"
"github.com/sourcehawk/operator-component-framework/pkg/metrics"

appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
Expand Down Expand Up @@ -55,15 +56,14 @@ var _ = BeforeSuite(func() {

By("creating E2E reconciler")
recorder := events.NewFakeRecorder(1000)
metrics := &ocm.ConditionMetricRecorder{
Controller: "e2e-component",
OperatorConditionsGauge: ocm.NewOperatorConditionsGauge("e2e_component"),
}
metricsRecorder := metrics.NewRecorder(
"e2e-component", ocm.NewOperatorConditionsGauge("e2e_component"), metrics.NewCollectors(),
)
clusterReconciler = framework.NewClusterE2EReconciler(
mgr.GetClient(),
mgr.GetScheme(),
recorder,
metrics,
metricsRecorder,
mgr.GetAPIReader(),
)

Expand Down
12 changes: 6 additions & 6 deletions e2e/primitives/suite_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
"github.com/sourcehawk/operator-component-framework/e2e/framework"

ocm "github.com/sourcehawk/go-crd-condition-metrics/pkg/crd-condition-metrics"
"github.com/sourcehawk/operator-component-framework/pkg/metrics"

appsv1 "k8s.io/api/apps/v1"
autoscalingv2 "k8s.io/api/autoscaling/v2"
Expand Down Expand Up @@ -81,22 +82,21 @@ var _ = BeforeSuite(func() {

By("creating E2E reconcilers")
recorder := events.NewFakeRecorder(1000)
metrics := &ocm.ConditionMetricRecorder{
Controller: "e2e-primitives",
OperatorConditionsGauge: ocm.NewOperatorConditionsGauge("e2e_primitives"),
}
metricsRecorder := metrics.NewRecorder(
"e2e-primitives", ocm.NewOperatorConditionsGauge("e2e_primitives"), metrics.NewCollectors(),
)
reconciler = framework.NewE2EReconciler(
mgr.GetClient(),
mgr.GetScheme(),
recorder,
metrics,
metricsRecorder,
mgr.GetAPIReader(),
)
clusterReconciler = framework.NewClusterE2EReconciler(
mgr.GetClient(),
mgr.GetScheme(),
recorder,
metrics,
metricsRecorder,
mgr.GetAPIReader(),
)

Expand Down
12 changes: 5 additions & 7 deletions examples/component-prerequisites/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
"github.com/sourcehawk/operator-component-framework/examples/component-prerequisites/app"
"github.com/sourcehawk/operator-component-framework/examples/component-prerequisites/resources"
sharedapp "github.com/sourcehawk/operator-component-framework/examples/shared/app"
"github.com/sourcehawk/operator-component-framework/pkg/metrics"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/meta"
Expand Down Expand Up @@ -47,13 +48,10 @@ func main() {

gauge := ocm.NewOperatorConditionsGauge("example")
controller := &app.Controller{
Client: fakeClient,
Scheme: scheme,
EventRecorder: events.NewFakeRecorder(100),
Metrics: &ocm.ConditionMetricRecorder{
Controller: "example",
OperatorConditionsGauge: gauge,
},
Client: fakeClient,
Scheme: scheme,
EventRecorder: events.NewFakeRecorder(100),
Metrics: metrics.NewRecorder("example", gauge, metrics.NewCollectors()),
NewConfigMapResource: resources.NewConfigMapResource,
NewDeploymentResource: resources.NewDeploymentResource,
}
Expand Down
Loading
Loading