Skip to content

Commit 7e024b5

Browse files
kvlongekvlonge
committed
docs(dagster): describe the failure description, metadata and log levels
1 parent f718243 commit 7e024b5

2 files changed

Lines changed: 80 additions & 0 deletions

File tree

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/integrations/dagster-reference.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,46 @@ A running or completed SQLBuild CLI subprocess.
119119
5. If the DAG artifact is available, results are matched to the correct Dagster asset keys
120120
6. Raises `Failure` if the process exited non-zero and `raise_on_error` is set
121121

122+
### Logs
123+
124+
The command is logged at INFO. When Dagster selected a subset of assets, the selection is logged
125+
as a count at INFO, with the selector file path when SQLBuild read the selection from one; the
126+
full list of selectors is logged at DEBUG. Each check that did not pass is logged at ERROR, or at
127+
WARNING for a warn-severity check, with its failing row count when the check reports one, so it
128+
shows in the default run log view.
129+
130+
### Failure output
131+
132+
The `Failure` that `get_error()` returns, and that `wait()` and `stream()` raise, starts its
133+
description with what failed: up to 10 failed checks and assets, each with its name, model and
134+
failing row count (or status), followed by "and N more" when there are more. The command and its
135+
exit code follow, then the details of each failed asset.
136+
137+
```
138+
Failed: 1 check
139+
audit not_null on orders.order_id: 2 failing rows
140+
141+
SQLBuild CLI command failed with exit code 1: sqb audit --json-output ...
142+
```
143+
144+
Checks that only warned are not listed; they do not fail the command. The failure metadata holds:
145+
146+
| Key | Description |
147+
|-----|-------------|
148+
| `command` | The command that ran |
149+
| `project_dir` | The project directory |
150+
| `selection_count` | How many selectors Dagster passed to SQLBuild |
151+
| `selector_file` | The selector file the selection was passed in, or empty |
152+
| `execution_status` | The status of the execution JSON |
153+
| `incomplete_assets` | Assets that did not complete, as `kind:name (status)` |
154+
| `failed_checks` | Up to 100 failed checks, each with `kind`, `name`, `asset`, `column`, `status` and `failing_rows` |
155+
| `failed_check_count` | How many checks failed |
156+
| `microbatch_limits` | Microbatch limit evidence of incomplete assets, when there is any |
157+
158+
The metadata no longer lists the selection itself: `selection` was replaced by `selection_count`
159+
and `selector_file`. Without an execution JSON, the metadata holds the non-empty `stdout_tail` and
160+
`stderr_tail` instead of the execution keys.
161+
122162
## sqlbuild_assets
123163

124164
Decorator that creates a Dagster multi-asset definition from a SQLBuild DAG artifact.

‎website/src/content/docs/docs/integrations/dagster-reference.mdx‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,46 @@ A running or completed SQLBuild CLI subprocess.
107107
5. If the DAG artifact is available, results are matched to the correct Dagster asset keys
108108
6. Raises `Failure` if the process exited non-zero and `raise_on_error` is set
109109

110+
### Logs
111+
112+
The command is logged at INFO. When Dagster selected a subset of assets, the selection is logged
113+
as a count at INFO, with the selector file path when SQLBuild read the selection from one; the
114+
full list of selectors is logged at DEBUG. Each check that did not pass is logged at ERROR, or at
115+
WARNING for a warn-severity check, with its failing row count when the check reports one, so it
116+
shows in the default run log view.
117+
118+
### Failure output
119+
120+
The `Failure` that `get_error()` returns, and that `wait()` and `stream()` raise, starts its
121+
description with what failed: up to 10 failed checks and assets, each with its name, model and
122+
failing row count (or status), followed by "and N more" when there are more. The command and its
123+
exit code follow, then the details of each failed asset.
124+
125+
```
126+
Failed: 1 check
127+
audit not_null on orders.order_id: 2 failing rows
128+
129+
SQLBuild CLI command failed with exit code 1: sqb audit --json-output ...
130+
```
131+
132+
Checks that only warned are not listed; they do not fail the command. The failure metadata holds:
133+
134+
| Key | Description |
135+
|-----|-------------|
136+
| `command` | The command that ran |
137+
| `project_dir` | The project directory |
138+
| `selection_count` | How many selectors Dagster passed to SQLBuild |
139+
| `selector_file` | The selector file the selection was passed in, or empty |
140+
| `execution_status` | The status of the execution JSON |
141+
| `incomplete_assets` | Assets that did not complete, as `kind:name (status)` |
142+
| `failed_checks` | Up to 100 failed checks, each with `kind`, `name`, `asset`, `column`, `status` and `failing_rows` |
143+
| `failed_check_count` | How many checks failed |
144+
| `microbatch_limits` | Microbatch limit evidence of incomplete assets, when there is any |
145+
146+
The metadata no longer lists the selection itself: `selection` was replaced by `selection_count`
147+
and `selector_file`. Without an execution JSON, the metadata holds the non-empty `stdout_tail` and
148+
`stderr_tail` instead of the execution keys.
149+
110150
## sqlbuild_assets
111151

112152
Decorator that creates a Dagster multi-asset definition from a SQLBuild DAG artifact.

0 commit comments

Comments
 (0)