@@ -4,10 +4,11 @@ SQLMesh can add structured metadata to warehouse statements so a query-history
44tool can connect a statement to the plan or run, model, and evaluation that issued
55it. The metadata is a versioned JSON object in a ` QUERY_CONTEXT ` SQL comment.
66
7- Enable ` query_execution_context ` on a connection and set a provider on its
8- engine adapter. The provider returns a ` QueryExecutionContext ` . The flag defaults
9- to ` false ` . For YAML configuration, set the option under the gateway's
10- ` connection ` :
7+ Set ` query_execution_context ` to ` true ` on a connection to enable bounded query
8+ context comments on supported adapters. The flag defaults to ` false ` . With this
9+ flag enabled, ` Context.apply() ` and built-in scheduler runs add model context to
10+ SQL on supported adapters without a custom provider. For YAML configuration,
11+ set the option under the gateway's ` connection ` :
1112
1213``` yaml
1314gateways :
@@ -17,20 +18,59 @@ gateways:
1718 query_execution_context : true
1819` ` `
1920
20- The flag enables context comments but does not infer model context. Without a
21- provider, SQLMesh adds parent metadata only when a plan or run correlation ID is
22- available. SQLMesh calls the provider after rendering each statement and before
23- executing it. The call runs in the current worker's context, so concurrent model
24- evaluations can return different IDs for a shared adapter.
21+ ## Automatic model context
2522
26- To build a provider from native observer events, see the
27- [query execution context example](../examples/query_execution_context.md).
23+ For each model batch, SQLMesh creates a context with a new operation ID, the
24+ model's canonical name, an opaque snapshot ID, and the batch interval and index
25+ when available. Intervals use UTC and an exclusive end.
2826
29- ## Configure a provider
27+ The context stays active for SQL inside the batch, including nested audits.
28+ Automatic context uses the broad phases ` materialize`, `audit`, and `ddl`. The
29+ ` role` field does not identify the engine operation for each SQL statement.
3030
31- Set the provider on warehouse adapters before loading a Python ` Context`. When
32- ` Context` creates its snapshot evaluator, it clones those adapters and preserves
33- the provider and flag.
31+ Outside a model batch, SQLMesh can add parent metadata when a plan or run
32+ correlation ID is available. Use a custom provider when you need caller-defined
33+ context for other statements.
34+
35+ # # Read the context from an observer
36+
37+ SQLMesh establishes the model context before it calls an observer's `start`
38+ callback. It restores the context after the `finish` callback. Both callbacks can
39+ read the active immutable value with `get_query_execution_context()` :
40+
41+ ` ` ` python
42+ from sqlmesh.core.query_execution import get_query_execution_context
43+
44+
45+ def observe(event, facts, error):
46+ context = get_query_execution_context()
47+ if context is not None and context.evaluation is not None:
48+ print(context.evaluation.evaluation_id)
49+ ` ` `
50+
51+ For console observer registration and callback behavior, see
52+ [Execution observations](../reference/execution_observations.md).
53+
54+ SQLMesh forwards each observer event to the configured console observer once. If
55+ the console has no observer, model context still works. Nested observer scopes
56+ restore the outer observer and context on exit. Use `observer_scope(None)` to
57+ disable the built-in observer for its block.
58+
59+ An observer callback can read the active context with
60+ ` get_query_execution_context()` . SQL issued inside the callback does not use that
61+ automatic context. A custom adapter provider remains authoritative. Without one,
62+ an active plan or run correlation ID can still produce parent-only metadata.
63+
64+ # # Configure a custom provider
65+
66+ Use a custom provider when you need caller-defined operation IDs or context
67+ outside a model batch. Set it on warehouse adapters before loading a Python
68+ ` Context` . When `Context` creates its snapshot evaluator, it clones those
69+ adapters and preserves the provider and flag.
70+
71+ SQLMesh calls the provider after rendering each statement and before executing
72+ it. The provider runs in the current worker's context, so concurrent workers can
73+ return different values for a shared adapter.
3474
3575` ` ` python
3676from sqlmesh.core.context import Context
@@ -53,23 +93,32 @@ with observer_scope(observer):
5393
5494` QueryContextObserver` returns the immutable context active in its worker, or
5595` None` when no context applies. It assigns an evaluation ID when SQLMesh reports
56- a model batch and restores the previous context when that event finishes. It also
96+ a model batch and restores the previous context when that event finishes. It
5797forwards the same event, facts, and error to the observer returned by the console.
5898
99+ A custom provider replaces the built-in provider for its adapter. If it returns
100+ ` None` , SQLMesh does not use automatic model context for that statement. SQLMesh
101+ can still add parent-only metadata when the adapter has a correlation ID.
102+
59103Adapters created with `with_settings` inherit the flag and provider. Pass
60104` query_execution_context_provider=None` to clear the inherited provider. Setting
61105the flag to `false` disables provider calls and comment output.
62106
63107When `state_connection` is omitted, the built-in scheduler uses the warehouse
64- connection for its state adapter and disables query-context comments on that
65- adapter. To add context to state SQL, configure `state_connection` explicitly
66- and set a provider on its adapter.
108+ connection for its state adapter and disables query context on that adapter. An
109+ explicitly configured `state_connection` uses its own `query_execution_context`
110+ setting. The automatic observer activates only if a supported warehouse adapter
111+ has `query_execution_context` enabled and no custom provider. An enabled state
112+ adapter without a custom provider can use that context during a model batch.
113+ Setting the flag only on `state_connection` does not activate the observer. Use a
114+ custom provider for state SQL outside a model batch.
67115
68116# # Read the payload
69117
70118The version-one payload uses the SQLMesh model's canonical name and opaque
71- snapshot identifier. Operation intervals use UTC half-open boundaries. Fields
72- that are unavailable are omitted.
119+ snapshot identifier. Operation intervals use UTC half-open boundaries, and the
120+ payload includes `batch_index` when available. Fields that are unavailable are
121+ omitted.
73122
74123` ` ` json
75124{
@@ -93,9 +142,9 @@ that are unavailable are omitted.
93142}
94143` ` `
95144
96- The provider assigns an operation ID to the statements for one model evaluation,
97- including nested audits . A later evaluation gets a new ID . The adapter preserves
98- the provider's ID. Consumers can compare it with the execution observer's record .
145+ Each model batch gets one operation ID, which nested audits reuse. The built-in
146+ provider creates this ID . A custom provider supplies its own . The adapter preserves
147+ the provider's ID when it adds the context comment .
99148
100149The adapter's correlation ID sets `parent` and replaces any parent supplied by
101150the provider. If the adapter has no correlation ID, SQLMesh clears the provider's
0 commit comments