diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/scope.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/scope.md index 929a65c0d9..7a17d9e551 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/scope.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/scope.md @@ -26,7 +26,9 @@ Online: https://sqlbuild.com/docs/cli/scope/ The command is read-only and offline. It does not connect to a warehouse, require warehouse credentials, or inspect relations. It reads the compiler-owned declaration-scope index and reuses a deterministic cache when the relevant source and configuration fingerprint has not changed. Building a cold cache uses the normal compiler expansion path, which loads the configured adapter and may execute authored Python macros; a warm cache hit reconstructs scope facts without importing macro modules. -Text output is deterministic, bounded, and requires no interactive pager. +Text output is deterministic, bounded, and requires no interactive pager. In a color terminal it +uses the same color conventions as other commands; redirected output and `NO_COLOR` produce plain +text. ## Usage @@ -90,6 +92,7 @@ These identities are for inspection. SQL authored inside the owning model contin | `--page-size N` | Return at most `N` declarations per paged section (default: `100`). | | `--after CURSOR` | Continue after a declaration identity returned as the previous section's next cursor. | | `--paths MODE` | Render paths as `relative`, `compact`, or `none`. | +| `--verbose`, `-v` | Show each declaration's scope details and column, and keep empty optional sections. | | `--json` | Write the canonical versioned JSON report instead of the text tree. | Filters combine in a fixed order: definition path, kind, glob, then usage. `--dependency-depth` @@ -107,11 +110,17 @@ The default text report keeps distinct facts in distinct sections: - **Scope chain** starts with the resource's owner folder, then shows parent folders and the project declaration roles. Its compact labels identify the access rule checked at each path. -Ordinary text rows show each declaration's qualified identity, definition location, visibility, -reason for appearing, declaration-role root, and safe type or signature metadata where useful. -`--explain` additionally shows its owning path, narrowest required placement, current consumers, -dependencies, and the consumers affected by a placement mismatch. JSON includes the complete -structured declaration report. +By default, each text row shows a usage marker, the declaration's qualified identity, and its +`file:line` definition. A one-line legend explains the markers. Empty optional sections +(**Relationship grants**, **Nearby unavailable**, **Private retained**, **Relationship retained**, +and **Diagnostics**) are omitted; the core sections are always shown, even when empty. + +`--verbose` restores the full rows: kind, visibility, reason for appearing, declaration-role root, +safe type or signature metadata where useful, and the definition column. It also shows empty +optional sections. `--explain` always shows the explained declaration in full, plus its owning +path, narrowest required placement, current consumers, dependencies, and the consumers affected by +a placement mismatch. JSON includes the complete structured declaration report regardless of +`--verbose`. ### See the whole scope at once @@ -160,7 +169,7 @@ Scope Path: models/commerce/orders.sql Used (1) - └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:5:1 + └─ ● macro:formatted_order_total models/commerce/macros/orders.py:5 Scope chain ├─ exact-owner-private models/commerce (1) @@ -168,20 +177,22 @@ Scope chain └─ project global (2) Available (1 of 3, 2 collapsed) - └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:5:1 + └─ ● macro:formatted_order_total models/commerce/macros/orders.py:5 … 2 globals collapsed; run sqb scope model:orders --globals all -Relationship grants (0 of 0) - (none) +● used by this resource ○ not used by this resource --verbose for scope details -Nearby unavailable (0 of 0) - (none) - -Diagnostics (0) - (none) Completeness: complete ``` +With `--verbose`, the same `Used` section shows why each declaration is visible: + +```console +$ sqb scope model:orders --verbose +Used (1) + └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:5:1 +``` + This answers several questions without opening declaration files: what the model actually uses, where that declaration came from, which owner and parent folders SQLBuild checks, how many project-wide declarations are available, and whether the result is complete. The scope-chain count @@ -202,9 +213,9 @@ Scope Path: models/commerce/orders.sql Used (3) - ├─ ● macro:add_tax [macro; project; dependency; params 1; role macros] macros/currency.py:1:1 - ├─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:5:1 - └─ ● macro:round_money [macro; project; dependency; params 1; role macros] macros/currency.py:5:1 + ├─ ● macro:add_tax macros/currency.py:1 + ├─ ● macro:formatted_order_total models/commerce/macros/orders.py:5 + └─ ● macro:round_money macros/currency.py:5 ``` The remaining report sections follow the `Used` section as usual. The dependency graph comes from diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/declaration-scopes/explorer.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/declaration-scopes/explorer.md index b30beb77ac..f7e084f556 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/declaration-scopes/explorer.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/declaration-scopes/explorer.md @@ -75,8 +75,8 @@ Scope Path: models/commerce/orders.sql Used (2) - ├─ ● constant:minimum_order_value [constant; descendant-public; inherited_ancestor; type integer; role models/commerce/constants] models/commerce/constants/limits.sql:1:1 - └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:4:1 + ├─ ● constant:minimum_order_value models/commerce/constants/limits.sql:1 + └─ ● macro:formatted_order_total models/commerce/macros/orders.py:4 Scope chain ├─ exact-owner-private models/commerce (3) @@ -84,13 +84,13 @@ Scope chain └─ project global (2) Available (3 of 5, 2 collapsed) - ├─ ● constant:minimum_order_value [constant; descendant-public; inherited_ancestor; type integer; role models/commerce/constants] models/commerce/constants/limits.sql:1:1 - ├─ ○ enum:order_status [enum; descendant-public; inherited_ancestor; members 4; type VARCHAR; role models/commerce/enums] models/commerce/enums/status.sql:1:1 - └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:4:1 + ├─ ● constant:minimum_order_value models/commerce/constants/limits.sql:1 + ├─ ○ enum:order_status models/commerce/enums/status.sql:1 + └─ ● macro:formatted_order_total models/commerce/macros/orders.py:4 … 2 globals collapsed; run sqb scope model:orders --globals all -Diagnostics (0) - (none) +● used by this resource ○ not used by this resource --verbose for scope details + Completeness: complete ``` @@ -98,7 +98,9 @@ Completeness: complete dependency when `--dependency-depth` is nonzero. `○` marks a declaration present in the section but not in `Used`. -The compact labels translate to ordinary folder rules: +Add `--verbose` to see each declaration's access details, for example +`[enum; descendant-public; inherited_ancestor; members 4; type VARCHAR; role models/commerce/enums]`. +Those labels translate to ordinary folder rules: | Output label | Meaning in this report | |--------------|------------------------| @@ -122,10 +124,10 @@ declarations. The expanded `Used` section is: ```console $ sqb scope model:orders --used-only --dependency-depth 1 Used (4) - ├─ ● constant:minimum_order_value [constant; descendant-public; inherited_ancestor; type integer; role models/commerce/constants] models/commerce/constants/limits.sql:1:1 - ├─ ● macro:add_tax [macro; project; dependency; params 1; role macros] macros/currency.py:1:1 - ├─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:4:1 - └─ ● macro:round_money [macro; project; dependency; params 1; role macros] macros/currency.py:5:1 + ├─ ● constant:minimum_order_value models/commerce/constants/limits.sql:1 + ├─ ● macro:add_tax macros/currency.py:1 + ├─ ● macro:formatted_order_total models/commerce/macros/orders.py:4 + └─ ● macro:round_money macros/currency.py:5 ``` SQLBuild derives these edges from actual Python calls, including nested calls and calls reached @@ -166,6 +168,8 @@ Explanation Required path: models/commerce Promotion impact: model:orders +● used by this resource ○ not used by this resource --verbose for scope details + Diagnostics (1) ERROR S008 models/commerce/macros/orders.py: Declaration 'macro:formatted_order_total' is currently descendant-public at 'models/commerce' (models/commerce/macros/orders.py); required exact-owner-private at 'models/commerce'. Consumers: model:orders. Move it to 'models/commerce/_sqlbuild/_macros/' Completeness: complete @@ -181,8 +185,8 @@ cannot use it. The relevant section is: ```console $ sqb scope model:orders --include-nearby Nearby unavailable (2 of 2) - ├─ ○ enum:finance_status [enum; descendant-public; sibling_scope; members 3; type VARCHAR; role models/finance/enums] models/finance/enums/status.sql:1:1 - └─ ○ macro:calculate_margin [macro; descendant-public; sibling_scope; params 2; role models/finance/macros] models/finance/macros/margin.py:4:1 + ├─ ○ enum:finance_status models/finance/enums/status.sql:1 + └─ ○ macro:calculate_margin models/finance/macros/margin.py:4 ``` The declarations are known, but they belong to a different folder branch. Here, the inspected @@ -218,13 +222,13 @@ report sections are: ```console $ sqb scope test:orders__completed_only --used-only Used (1) - └─ ● enum:order_status [enum; descendant-public; expected_model through model:orders; members 4; type VARCHAR; role models/commerce/enums] models/commerce/enums/status.sql:1:1 + └─ ● enum:order_status models/commerce/enums/status.sql:1 Relationship grants (1 of 1) - └─ ● enum:order_status [enum; descendant-public; expected_model through model:orders; members 4; type VARCHAR; role models/commerce/enums] models/commerce/enums/status.sql:1:1 + └─ ● enum:order_status models/commerce/enums/status.sql:1 ``` -The compact reason `expected_model through model:orders` means **available through expected output +With `--verbose`, these rows show the reason `expected_model through model:orders`, which means **available through expected output for model `orders`**. With multiple expected models, SQLBuild combines their eligible file-based declarations into one deterministic set. Tests include macros in that union; scenarios include only enums and constants. Neither receives declarations defined inside a model's `MODEL()` header. @@ -244,19 +248,15 @@ Move preview Destination: models/finance/orders.sql Ownership root: models Retained (2) - ├─ ○ macro:add_tax [macro; project; global; params 1; role macros] macros/currency.py:1:1 - └─ ○ macro:round_money [macro; project; global; params 1; role macros] macros/currency.py:5:1 + ├─ ○ macro:add_tax macros/currency.py:1 + └─ ○ macro:round_money macros/currency.py:5 Gained (2) - ├─ ○ enum:finance_status [enum; descendant-public; inherited_ancestor; members 3; type VARCHAR; role models/finance/enums] models/finance/enums/status.sql:1:1 - └─ ○ macro:calculate_margin [macro; descendant-public; inherited_ancestor; params 2; role models/finance/macros] models/finance/macros/margin.py:4:1 + ├─ ○ enum:finance_status models/finance/enums/status.sql:1 + └─ ○ macro:calculate_margin models/finance/macros/margin.py:4 Lost (3) - ├─ ● constant:minimum_order_value [constant; descendant-public; inherited_ancestor; type integer; role models/commerce/constants] models/commerce/constants/limits.sql:1:1 - ├─ ○ enum:order_status [enum; descendant-public; inherited_ancestor; members 4; type VARCHAR; role models/commerce/enums] models/commerce/enums/status.sql:1:1 - └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:4:1 - Private retained (0) - (none) - Relationship retained (0) - (none) + ├─ ● constant:minimum_order_value models/commerce/constants/limits.sql:1 + ├─ ○ enum:order_status models/commerce/enums/status.sql:1 + └─ ● macro:formatted_order_total models/commerce/macros/orders.py:4 Invalidated usages (2) - constant:minimum_order_value - macro:formatted_order_total @@ -287,16 +287,12 @@ Scope chain └─ project global (2) Available (3 of 5, 2 collapsed) - ├─ ○ constant:minimum_order_value [constant; descendant-public; inherited_ancestor; type integer; role models/commerce/constants] models/commerce/constants/limits.sql:1:1 - ├─ ○ enum:order_status [enum; descendant-public; inherited_ancestor; members 4; type VARCHAR; role models/commerce/enums] models/commerce/enums/status.sql:1:1 - └─ ○ macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:4:1 + ├─ ○ constant:minimum_order_value models/commerce/constants/limits.sql:1 + ├─ ○ enum:order_status models/commerce/enums/status.sql:1 + └─ ○ macro:formatted_order_total models/commerce/macros/orders.py:4 … 2 globals collapsed; run sqb scope --at models/commerce/returns/new_return.sql --globals all -Relationship grants (0 of 0) - (none) - -Nearby unavailable (0 of 0) - (none) +● used by this resource ○ not used by this resource --verbose for scope details Diagnostics (1) ERROR S013 models/commerce/returns/new_return.sql: Runtime usage and relationship facts are unavailable for a prospective path @@ -327,8 +323,6 @@ Scope folders sqb scope model:orders --browse global/macros sqb scope model:orders --list global/macros -Diagnostics (0) - (none) Completeness: complete ``` diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/exploring-a-project.md b/src/sqlbuild/.agents/skills/sqlbuild/references/exploring-a-project.md index b1c4d8a4d2..9b492a696d 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/exploring-a-project.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/exploring-a-project.md @@ -44,7 +44,9 @@ Common questions: Report sections: **Used**, **Scope chain** (owner folder, parents, project globals), **Available** (globals collapsed; `--globals all` expands them), **Relationship grants** (tests and scenarios), -**Nearby unavailable**, **Diagnostics**. Use `--json` for machine-readable output. +**Nearby unavailable**, **Diagnostics**. Rows show each declaration's identity and `file:line`, and +empty optional sections are hidden; add `--verbose` for each declaration's access details and every +section. Use `--json` for machine-readable output. Placement rules the scope report reflects: diff --git a/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py b/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py index aa1d7a50ea..9f68afdacd 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py @@ -89,6 +89,8 @@ def dispatch_cli_command(*, args: CliNamespace, handlers: CliEntrypointHandlers) paths=args.scope_paths, json_output=args.json, no_cache=args.no_cache, + verbose=args.verbose, + no_color=args.no_color, ) ) diff --git a/src/sqlbuild/cli/commands/_helpers/entry/parser.py b/src/sqlbuild/cli/commands/_helpers/entry/parser.py index b0789d403a..adfbfcf97f 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/parser.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/parser.py @@ -528,6 +528,7 @@ def _add_inspection_parsers( ) scope_parser.add_argument("--json", action="store_true", default=False) scope_parser.add_argument("--no-cache", action="store_true", default=False) + scope_parser.add_argument("--verbose", "-v", action="store_true", default=False) debug_parser: argparse.ArgumentParser = subparsers.add_parser(CliCommand.DEBUG) debug_parser.add_argument("--json", action="store_true", default=False) diff --git a/src/sqlbuild/cli/commands/_helpers/janitor_output/outputs.py b/src/sqlbuild/cli/commands/_helpers/janitor_output/outputs.py index af1b6480bf..5126d0d108 100644 --- a/src/sqlbuild/cli/commands/_helpers/janitor_output/outputs.py +++ b/src/sqlbuild/cli/commands/_helpers/janitor_output/outputs.py @@ -17,6 +17,7 @@ ) from sqlbuild.executor.janitor.models import JanitorExecutionResult, JanitorPlan from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.main.count_noun import format_count_noun def write_janitor_disabled(*, invocation: JanitorInvocation) -> None: @@ -121,7 +122,11 @@ def write_janitor_completion( def _deleted_message(*, result: JanitorExecutionResult) -> str: - archived_prefix: str = f"Archived {len(result.archived)} relations. " if result.archived else "" + archived_prefix: str = ( + f"Archived {format_count_noun(count=len(result.archived), singular='relation')}. " + if result.archived + else "" + ) return archived_prefix + _deletion_summary(result=result) @@ -138,19 +143,21 @@ def _deletion_summary(*, result: JanitorExecutionResult) -> str: + len(result.deleted_archives) + len(result.deleted_query_diff_artifacts) ) + objects: str = format_count_noun(count=deleted_object_count, singular="object") pruned_state_count: int = len(result.pruned_direct_state) + len(result.pruned_virtual_state) non_checkpoint_state_count: int = deleted_state_count - len(result.deleted_checkpoints) if non_checkpoint_state_count or pruned_state_count: pruned_state_label: str = ( - "state tables" if result.pruned_virtual_state else "direct state tables" + "state table" if result.pruned_virtual_state else "direct state table" ) return ( - f"Deleted {deleted_object_count} objects, deleted {deleted_state_count} " - f"state items, and pruned {pruned_state_count} {pruned_state_label}." + f"Deleted {objects}, deleted " + f"{format_count_noun(count=deleted_state_count, singular='state item')}, and pruned " + f"{format_count_noun(count=pruned_state_count, singular=pruned_state_label)}." ) if result.deleted_checkpoints: - return ( - f"Deleted {deleted_object_count} objects and " - f"{len(result.deleted_checkpoints)} checkpoints." + checkpoints: str = format_count_noun( + count=len(result.deleted_checkpoints), singular="checkpoint" ) - return f"Deleted {deleted_object_count} objects." + return f"Deleted {objects} and {checkpoints}." + return f"Deleted {objects}." diff --git a/src/sqlbuild/cli/commands/_helpers/scope/command.py b/src/sqlbuild/cli/commands/_helpers/scope/command.py index 9ba033d642..57223c228d 100644 --- a/src/sqlbuild/cli/commands/_helpers/scope/command.py +++ b/src/sqlbuild/cli/commands/_helpers/scope/command.py @@ -28,6 +28,7 @@ ScopeReportFilters, ) from sqlbuild.compiler.scopes.types import DeclarationKind, DiagnosticSeverity +from sqlbuild.presentation.main.supports_color import supports_color def run_scope_command( @@ -107,7 +108,11 @@ def run_scope_command( output: str = ( serialize_scope_report(report=result) if request.json_output - else render_scope_result(result=result, request=request) + else render_scope_result( + result=result, + request=request, + use_color=output_stream is None and not request.no_color and supports_color(), + ) ) stream.write(output) return 1 if _failed(result=result) else 0 diff --git a/src/sqlbuild/cli/commands/_helpers/scope/output.py b/src/sqlbuild/cli/commands/_helpers/scope/output.py index 3a6268067f..b55529ffa6 100644 --- a/src/sqlbuild/cli/commands/_helpers/scope/output.py +++ b/src/sqlbuild/cli/commands/_helpers/scope/output.py @@ -2,6 +2,8 @@ from __future__ import annotations +from collections.abc import Callable +from dataclasses import dataclass from pathlib import PurePosixPath from shlex import quote @@ -25,133 +27,198 @@ ScopeSection, SourceLocation, ) +from sqlbuild.compiler.scopes.types import DiagnosticSeverity +from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.main.count_noun import format_count_noun + +_USED_MARKER: str = "●" +_UNUSED_MARKER: str = "○" +_LEGEND_MARKERS: str = ( + f"{_USED_MARKER} used by this resource {_UNUSED_MARKER} not used by this resource" +) +_LEGEND_VERBOSE_HINT: str = "--verbose for scope details" + + +@dataclass(frozen=True) +class _ScopeText: + """Styling and detail options shared by one scope text rendering.""" + + request: ScopeCommandRequest + style: CliStyle + + @property + def verbose(self) -> bool: + return self.request.verbose def render_scope_result( - *, result: ScopeReport | ScopeBrowseResult | ScopeListResult, request: ScopeCommandRequest + *, + result: ScopeReport | ScopeBrowseResult | ScopeListResult, + request: ScopeCommandRequest, + use_color: bool = False, ) -> str: """Render one compiler-owned scope result without deriving scope facts.""" + text: _ScopeText = _ScopeText(request=request, style=CliStyle(use_color=use_color)) if isinstance(result, ScopeBrowseResult): - return _render_browse(result=result, request=request) + return _render_browse(result=result, text=text) if isinstance(result, ScopeListResult): - return _render_list(result=result, request=request) - return _render_report(report=result, request=request) + return _render_list(result=result, text=text) + return _render_report(report=result, text=text) -def _render_report(*, report: ScopeReport, request: ScopeCommandRequest) -> str: +def _render_report(*, report: ScopeReport, text: _ScopeText) -> str: + request: ScopeCommandRequest = text.request + style: CliStyle = text.style resource: ScopeResourceReport = report.resource - lines: list[str] = ["Scope", f" Target: {resource.target}"] + lines: list[str] = [ + style.section("Scope"), + _field(label="Target", value=resource.target, text=text), + ] if resource.identity is not None: - lines.append(f" Resource: {resource.identity}") + lines.append(_field(label="Resource", value=resource.identity, text=text)) if resource.path is not None: - lines.append(f" Path: {_path(value=resource.path, mode=request.paths)}") + lines.append( + _field(label="Path", value=_path(value=resource.path, mode=request.paths), text=text) + ) labels: list[str] = [] if resource.prospective: labels.append("prospective") if resource.directory: labels.append("directory") if resource.duplicate_count: - labels.append(f"{resource.duplicate_count} matches") - if labels: - lines.append(f" Status: {', '.join(labels)}") - lines.extend(["", f"Used ({len(report.used)})"]) - lines.extend( - _declaration_lines( - declarations=report.used, - request=request, - used=frozenset(item.identity for item in report.used), + labels.append( + format_count_noun(count=resource.duplicate_count, singular="match", plural="matches") ) - ) - lines.extend(["", "Scope chain"]) + if labels: + lines.append(_field(label="Status", value=", ".join(labels), text=text)) + used: frozenset[str] = frozenset(item.identity for item in report.used) + lines.extend(["", _header(title="Used", count=f"({len(report.used)})", text=text)]) + lines.extend(_declaration_lines(declarations=report.used, text=text, used=used)) + lines.extend(["", style.section("Scope chain")]) if report.scope_chain: for index, entry in enumerate(report.scope_chain): - connector: str = "└─" if index == len(report.scope_chain) - 1 else "├─" + chain_path: str = _path(value=entry.path, mode=request.paths) lines.append( - f" {connector} {_scope_label(entry.kind)} " - f"{_path(value=entry.path, mode=request.paths)} " - f"({entry.declaration_count})" + f" {_connector(last=index == len(report.scope_chain) - 1, text=text)} " + f"{_scope_label(entry.kind)} " + f"{style.muted(f'{chain_path} ({entry.declaration_count})')}" ) else: - lines.append(" (none)") - used: frozenset[str] = frozenset(item.identity for item in report.used) - for title, declarations, section in ( - ("Available", report.available, _section(report=report, name="available")), + lines.append(f" {style.muted('(none)')}") + for title, declarations, section, optional in ( + ("Available", report.available, _section(report=report, name="available"), False), ( "Relationship grants", report.relationship_scope, _section(report=report, name="relationship_scope"), + True, ), ( "Nearby unavailable", report.nearby_unavailable, _section(report=report, name="nearby_unavailable"), + True, ), ): - lines.extend(["", _section_title(title=title, section=section, count=len(declarations))]) - lines.extend(_declaration_lines(declarations=declarations, request=request, used=used)) - if section is not None and section.collapsed_count: + collapsed_count: int = section.collapsed_count if section is not None else 0 + if optional and not text.verbose and not declarations and not collapsed_count: + continue + lines.extend( + [ + "", + _header( + title=title, + count=_section_count(section=section, count=len(declarations)), + text=text, + ), + ] + ) + lines.extend(_declaration_lines(declarations=declarations, text=text, used=used)) + if collapsed_count: + global_count: str = format_count_noun(count=collapsed_count, singular="global") lines.append( - f" … {section.collapsed_count} globals collapsed; run " - f"{_follow_up(request=request, globals_all=True)}" + " " + + style.muted( + f"… {global_count} collapsed; run " + f"{_follow_up(request=request, globals_all=True)}" + ) ) if report.explanation is not None: - lines.extend(["", "Explanation"]) + lines.extend(["", style.section("Explanation")]) if report.explanation.declaration is None: - lines.append(" (declaration not resolved)") + lines.append(f" {style.muted('(declaration not resolved)')}") else: lines.extend( - _explanation_lines( - declaration=report.explanation.declaration, - request=request, - used=used, - ) + _explanation_lines(declaration=report.explanation.declaration, text=text, used=used) ) if report.move_preview is not None: - lines.extend(["", *_move_lines(move=report.move_preview, request=request, used=used)]) - lines.extend(["", *_diagnostic_lines(report.diagnostics), _completeness_line(report.complete)]) + lines.extend(["", *_move_lines(move=report.move_preview, text=text, used=used)]) + if not text.verbose: + lines.extend(["", style.muted(f"{_LEGEND_MARKERS} {_LEGEND_VERBOSE_HINT}")]) + lines.extend( + [ + "", + *_diagnostic_lines(diagnostics=report.diagnostics, text=text), + _completeness_line(complete=report.complete, text=text), + ] + ) return "\n".join(lines) + "\n" -def _render_browse(*, result: ScopeBrowseResult, request: ScopeCommandRequest) -> str: +def _render_browse(*, result: ScopeBrowseResult, text: _ScopeText) -> str: + request: ScopeCommandRequest = text.request + style: CliStyle = text.style lines: list[str] = [ - "Scope folders", - f" Path: {_path(value=result.folder, mode=request.paths)}", + style.section("Scope folders"), + _field(label="Path", value=_path(value=result.folder, mode=request.paths), text=text), "", ] if not result.folders: - lines.append(" (none)") + lines.append(f" {style.muted('(none)')}") for index, folder in enumerate(result.folders): - connector: str = "└─" if index == len(result.folders) - 1 else "├─" kinds: str = ( ", ".join(f"{kind} {count}" for kind, count in folder.kind_counts) or "no declarations" ) + counts: str = ( + f"{format_count_noun(count=folder.descendant_count, singular='declaration')}, " + f"{folder.used_count} used, " + f"{format_count_noun(count=folder.child_count, singular='child', plural='children')}; " + f"{kinds}" + ) lines.append( - f" {connector} {folder.name}/ {folder.descendant_count} declarations, " - f"{folder.used_count} used, {folder.child_count} children; {kinds}" + f" {_connector(last=index == len(result.folders) - 1, text=text)} " + f"{folder.name}/ {style.muted(counts)}" ) - lines.append(f" {_follow_up(request=request, browse=folder.path)}") - lines.append(f" {_follow_up(request=request, list_path=folder.path)}") - lines.extend(["", *_diagnostic_lines(result.diagnostics), _completeness_line(result.complete)]) + lines.append(f" {style.muted(_follow_up(request=request, browse=folder.path))}") + lines.append(f" {style.muted(_follow_up(request=request, list_path=folder.path))}") + lines.extend( + [ + "", + *_diagnostic_lines(diagnostics=result.diagnostics, text=text), + _completeness_line(complete=result.complete, text=text), + ] + ) return "\n".join(lines) + "\n" -def _render_list(*, result: ScopeListResult, request: ScopeCommandRequest) -> str: +def _render_list(*, result: ScopeListResult, text: _ScopeText) -> str: + request: ScopeCommandRequest = text.request + style: CliStyle = text.style section: ScopeSection = result.section lines: list[str] = [ - "Scope declarations", - f" Path: {_path(value=result.folder, mode=request.paths)}", - f" Showing: {section.returned} of {section.total}", + style.section("Scope declarations"), + _field(label="Path", value=_path(value=result.folder, mode=request.paths), text=text), + _field(label="Showing", value=f"{section.returned} of {section.total}", text=text), "", ] - lines.extend( - _declaration_lines(declarations=result.declarations, request=request, used=frozenset()) - ) + lines.extend(_declaration_lines(declarations=result.declarations, text=text, used=frozenset())) if section.next_cursor is not None: lines.extend( [ "", - "Continue: " + style.muted("Continue:") + + " " + _follow_up( request=request, list_path=result.folder, @@ -159,42 +226,71 @@ def _render_list(*, result: ScopeListResult, request: ScopeCommandRequest) -> st ), ] ) - lines.extend(["", *_diagnostic_lines(result.diagnostics), _completeness_line(section.complete)]) + if not text.verbose and result.declarations: + lines.extend(["", style.muted(_LEGEND_VERBOSE_HINT)]) + lines.extend( + [ + "", + *_diagnostic_lines(diagnostics=result.diagnostics, text=text), + _completeness_line(complete=section.complete, text=text), + ] + ) return "\n".join(lines) + "\n" def _declaration_lines( *, declarations: tuple[DeclarationReport, ...], - request: ScopeCommandRequest, + text: _ScopeText, used: frozenset[str], + used_marker_style: Callable[[str], str] | None = None, + name_style: Callable[[str], str] | None = None, + detailed: bool | None = None, ) -> list[str]: + style: CliStyle = text.style if not declarations: - return [" (none)"] + return [f" {style.muted('(none)')}"] + show_details: bool = text.verbose if detailed is None else detailed + resolved_used_marker_style: Callable[[str], str] = used_marker_style or style.success + resolved_name_style: Callable[[str], str] = name_style or style.object_name lines: list[str] = [] for index, declaration in enumerate(declarations): - connector: str = "└─" if index == len(declarations) - 1 else "├─" - marker: str = "●" if declaration.identity in used else "○" - details: list[str] = [declaration.kind, _scope_label(declaration.scope)] - if declaration.visibility is not None: - provenance: str = declaration.visibility.reason - if declaration.visibility.through is not None: - provenance += f" through {declaration.visibility.through}" - details.append(provenance) - if declaration.inaccessible_reason is not None: - details.append(declaration.inaccessible_reason) - details.extend(_metadata_parts(declaration)) - line = f" {connector} {marker} {declaration.identity} [{'; '.join(details)}]" - if request.paths != SCOPE_PATH_NONE: + marker: str = ( + resolved_used_marker_style(_USED_MARKER) + if declaration.identity in used + else style.muted(_UNUSED_MARKER) + ) + line: str = ( + f" {_connector(last=index == len(declarations) - 1, text=text)} {marker} " + f"{resolved_name_style(declaration.identity)}" + ) + if show_details: + line += f" {style.muted(f'[{"; ".join(_declaration_details(declaration))}]')}" + if text.request.paths != SCOPE_PATH_NONE: location: SourceLocation = declaration.definition - line += ( - f" {_path(value=location.path, mode=request.paths)}:" - f"{location.line}:{location.column}" + position: str = ( + f"{location.line}:{location.column}" if show_details else f"{location.line}" + ) + line += " " + style.muted( + f"{_path(value=location.path, mode=text.request.paths)}:{position}" ) lines.append(line) return lines +def _declaration_details(declaration: DeclarationReport) -> list[str]: + details: list[str] = [declaration.kind, _scope_label(declaration.scope)] + if declaration.visibility is not None: + provenance: str = declaration.visibility.reason + if declaration.visibility.through is not None: + provenance += f" through {declaration.visibility.through}" + details.append(provenance) + if declaration.inaccessible_reason is not None: + details.append(declaration.inaccessible_reason) + details.extend(_metadata_parts(declaration)) + return details + + def _scope_label(scope: str) -> str: return { "global": "project", @@ -233,9 +329,12 @@ def _metadata_parts(declaration: DeclarationReport) -> list[str]: def _explanation_lines( - *, declaration: DeclarationReport, request: ScopeCommandRequest, used: frozenset[str] + *, declaration: DeclarationReport, text: _ScopeText, used: frozenset[str] ) -> list[str]: - lines: list[str] = _declaration_lines(declarations=(declaration,), request=request, used=used) + style: CliStyle = text.style + lines: list[str] = _declaration_lines( + declarations=(declaration,), text=text, used=used, detailed=True + ) required_scope: str | None = ( _scope_label(declaration.required_scope) if declaration.required_scope is not None else None ) @@ -250,52 +349,115 @@ def _explanation_lines( ("Promotion impact", declaration.promotion_impact), ) for label, value in facts: + rendered: str if isinstance(value, tuple): - rendered: str = ", ".join(value) if value else "(none)" + rendered = ", ".join(value) if value else style.muted("(none)") else: - rendered = str(value) if value is not None else "(none)" - lines.append(f" {label}: {rendered}") + rendered = str(value) if value is not None else style.muted("(none)") + lines.append(f" {style.muted(f'{label}:')} {rendered}") return lines -def _move_lines( - *, move: MovePreview, request: ScopeCommandRequest, used: frozenset[str] -) -> list[str]: +def _move_lines(*, move: MovePreview, text: _ScopeText, used: frozenset[str]) -> list[str]: + request: ScopeCommandRequest = text.request + style: CliStyle = text.style lines: list[str] = [ - "Move preview", - f" Resource: {move.resource}", - f" Destination: {_path(value=move.destination, mode=request.paths)}", - f" Ownership root: {_path(value=move.new_ownership_root, mode=request.paths)}", + style.section("Move preview"), + _field(label="Resource", value=move.resource, text=text), + _field( + label="Destination", value=_path(value=move.destination, mode=request.paths), text=text + ), + _field( + label="Ownership root", + value=_path(value=move.new_ownership_root, mode=request.paths), + text=text, + ), ] - for title, values in ( - ("Retained", move.retained), - ("Gained", move.gained), - ("Lost", move.lost), - ("Private retained", move.private_retained), - ("Relationship retained", move.relationship_retained), - ): - lines.append(f" {title} ({len(values)})") + lost_warning: bool = bool(move.lost) + sections: tuple[ + tuple[ + str, + tuple[DeclarationReport, ...], + bool, + Callable[[str], str] | None, + Callable[[str], str] | None, + Callable[[str], str] | None, + ], + ..., + ] = ( + ("Retained", move.retained, False, None, None, None), + ("Gained", move.gained, False, None, None, style.success), + ( + "Lost", + move.lost, + False, + style.warning_strong if lost_warning else None, + style.error, + None, + ), + ("Private retained", move.private_retained, True, None, None, None), + ("Relationship retained", move.relationship_retained, True, None, None, None), + ) + for title, values, optional, title_style, used_marker_style, name_style in sections: + if optional and not text.verbose and not values: + continue + header: str = _header( + title=title, + count=f"({len(values)})", + text=text, + title_style=title_style, + count_style=style.warning if title_style is not None else None, + ) + lines.append(f" {header}") lines.extend( " " + line - for line in _declaration_lines(declarations=values, request=request, used=used) + for line in _declaration_lines( + declarations=values, + text=text, + used=used, + used_marker_style=used_marker_style, + name_style=name_style, + ) ) - lines.append(f" Invalidated usages ({len(move.invalidated_usages)})") - lines.extend(f" - {identity}" for identity in move.invalidated_usages) - if not move.invalidated_usages: - lines.append(" (none)") + invalidated: tuple[str, ...] = move.invalidated_usages + invalidated_header: str = _header( + title="Invalidated usages", + count=f"({len(invalidated)})", + text=text, + title_style=style.error_strong if invalidated else None, + count_style=style.error if invalidated else None, + ) + lines.append(f" {invalidated_header}") + lines.extend(f" {style.error(f'- {identity}')}" for identity in invalidated) + if not invalidated: + lines.append(f" {style.muted('(none)')}") return lines -def _diagnostic_lines(diagnostics: tuple[ScopeDiagnostic, ...]) -> list[str]: - lines: list[str] = [f"Diagnostics ({len(diagnostics)})"] +def _diagnostic_lines(*, diagnostics: tuple[ScopeDiagnostic, ...], text: _ScopeText) -> list[str]: + style: CliStyle = text.style + if not diagnostics and not text.verbose: + return [] + lines: list[str] = [ + _header( + title="Diagnostics", + count=f"({len(diagnostics)})", + text=text, + title_style=style.error_strong if diagnostics else None, + count_style=style.error if diagnostics else None, + ) + ] if not diagnostics: - lines.append(" (none)") + lines.append(f" {style.muted('(none)')}") for diagnostic in diagnostics: - location: str = f" {diagnostic.path}" if diagnostic.path is not None else "" - lines.append( - f" {diagnostic.severity.value.upper()} {diagnostic.code.value}{location}: " - f"{diagnostic.message}" + severity: str = diagnostic.severity.value.upper() + severity_text: str = ( + style.error_strong(severity) + if diagnostic.severity is DiagnosticSeverity.ERROR + else style.warning_strong(severity) ) + location: str = f" {style.muted(diagnostic.path)}" if diagnostic.path is not None else "" + lines.append(f" {severity_text} {diagnostic.code.value}{location}: {diagnostic.message}") return lines @@ -303,15 +465,37 @@ def _section(*, report: ScopeReport, name: str) -> ScopeSection | None: return next((section for section in report.sections if section.name == name), None) -def _section_title(*, title: str, section: ScopeSection | None, count: int) -> str: +def _section_count(*, section: ScopeSection | None, count: int) -> str: if section is None: - return f"{title} ({count})" + return f"({count})" suffix: str = f", {section.collapsed_count} collapsed" if section.collapsed_count else "" - return f"{title} ({section.returned} of {section.total}{suffix})" + return f"({section.returned} of {section.total}{suffix})" + + +def _header( + *, + title: str, + count: str, + text: _ScopeText, + title_style: Callable[[str], str] | None = None, + count_style: Callable[[str], str] | None = None, +) -> str: + style: CliStyle = text.style + return f"{(title_style or style.section)(title)} {(count_style or style.muted)(count)}" + + +def _field(*, label: str, value: str, text: _ScopeText) -> str: + return f" {text.style.muted(f'{label}:')} {value}" + + +def _connector(*, last: bool, text: _ScopeText) -> str: + return text.style.muted("└─" if last else "├─") -def _completeness_line(complete: bool) -> str: - return f"Completeness: {'complete' if complete else 'partial'}" +def _completeness_line(*, complete: bool, text: _ScopeText) -> str: + style: CliStyle = text.style + state: str = style.success("complete") if complete else style.warning("partial") + return f"{style.muted('Completeness:')} {state}" def _path(*, value: str | None, mode: str) -> str: @@ -370,6 +554,8 @@ def _follow_up( args.extend(("--globals", request.globals)) if request.paths != SCOPE_PATH_RELATIVE: args.extend(("--paths", request.paths)) + if request.verbose: + args.append("--verbose") if request.no_cache: args.append("--no-cache") return " ".join(quote(item) for item in args) diff --git a/src/sqlbuild/cli/commands/classes/build_progress_callbacks.py b/src/sqlbuild/cli/commands/classes/build_progress_callbacks.py index 1a7fe8c318..24ce769d05 100644 --- a/src/sqlbuild/cli/commands/classes/build_progress_callbacks.py +++ b/src/sqlbuild/cli/commands/classes/build_progress_callbacks.py @@ -50,12 +50,14 @@ from sqlbuild.executor.testing.models import SqlTestExecutionResult, StepResult from sqlbuild.executor.testing.types import SqlTestOutcome from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.classes.transient_line_coordinator import TransientLineCoordinator from sqlbuild.presentation.main.coded_error_text import format_coded_error from sqlbuild.presentation.main.completion_line import format_completion_line from sqlbuild.presentation.main.inline_error_lines import format_inline_error_lines from sqlbuild.presentation.main.status_cell import format_status_cell from sqlbuild.presentation.main.summary_footer import format_summary_footer from sqlbuild.presentation.main.terminal_columns import terminal_columns +from sqlbuild.presentation.main.transient_line_coordinator import shared_transient_line_coordinator from sqlbuild.presentation.main.tree_connector import tree_connector from sqlbuild.presentation.types import CompletionState from sqlbuild.runtime.contracts.types import ExecutionResourceKind @@ -177,12 +179,13 @@ def __init__( self._debug: bool = debug self._is_tty: bool = hasattr(sys.stdout, "isatty") and sys.stdout.isatty() and not debug self._stream = sys.stderr if debug else sys.stdout + self._lines: TransientLineCoordinator = shared_transient_line_coordinator() self._start_time: float = time.monotonic() self._current_node_name: str = "" self._current_node_type: ExecutionResourceKind = ExecutionResourceKind.TABLE self._current_sub_message: str = "" self._spinner_frame_index: int = 0 - self._write_lock: threading.Lock = threading.Lock() + self._spinner_line_active: bool = False self._runtime_diagnostic_lock: threading.Lock = threading.Lock() self._spinner_stop_event: threading.Event | None = None self._spinner_thread: threading.Thread | None = None @@ -286,7 +289,10 @@ def on_node_start(self, *, name: str, resource_kind: ExecutionResourceKind) -> N self._current_sub_message = "" if self._is_tty: self._hide_cursor() - self._write_spinner_line() + with self._lines.lock: + self._spinner_line_active = True + self._lines.claim(stream=self._stream, owner=self) + self._write_spinner_line() self._start_spinner_loop() def on_sub_progress(self, message: str) -> None: @@ -320,10 +326,7 @@ def _write_runtime_diagnostic(self, message: str) -> None: _RUNTIME_DIAGNOSTICS_LOGGER.debug(message) if self._debug: return - with self._write_lock: - prefix: str = "\r\033[K" if self._is_tty else "" - self._stream.write(f"{prefix}{message}\n") - self._stream.flush() + self._lines.write_persistent(stream=self._stream, text=f"{message}\n") def _write_runtime_diagnostic_summaries(self) -> None: with self._runtime_diagnostic_lock: @@ -344,7 +347,24 @@ def _write_runtime_diagnostic_summaries(self) -> None: f"{_query_diagnostic_message(last_query)}" ) + def clear_transient_line(self) -> None: + """Erase the live spinner row so a persistent line can take its place.""" + + with self._lines.lock: + self._stream.write("\r\033[K") + self._stream.flush() + + def redraw_transient_line(self) -> None: + """Draw the live spinner row again below persistent output.""" + + self._write_spinner_line() + def _write_spinner_line(self) -> None: + with self._lines.lock: + if self._spinner_line_active: + self._draw_spinner_line() + + def _draw_spinner_line(self) -> None: ctr: str = f"{self._counter + 1}/{self._total}".rjust(len(str(self._total)) * 2 + 1) display_type: str = materialization_type_display(self._current_node_type) status: str = self._style.status(status=_ACTIVE_SPINNER_FRAMES[self._spinner_frame_index]) @@ -357,9 +377,8 @@ def _write_spinner_line(self) -> None: max_plain_width: int = max(terminal_columns() - 2, _MIN_NAME_WIDTH) if len(plain_part) > max_plain_width: plain_part = f"{plain_part[: max_plain_width - 4]}... " - with self._write_lock: - self._stream.write(f"\r\033[K{plain_part}{status}") - self._stream.flush() + self._stream.write(f"\r\033[K{plain_part}{status}") + self._stream.flush() def _start_spinner_loop(self) -> None: self._stop_spinner_loop() @@ -388,7 +407,7 @@ def _spin_until_stopped(self, stop_event: threading.Event) -> None: def _hide_cursor(self) -> None: if self._cursor_hidden: return - with self._write_lock: + with self._lines.lock: self._stream.write("\033[?25l") self._stream.flush() self._cursor_hidden = True @@ -396,7 +415,7 @@ def _hide_cursor(self) -> None: def _show_cursor(self) -> None: if not self._cursor_hidden: return - with self._write_lock: + with self._lines.lock: self._stream.write("\033[?25h") self._stream.flush() self._cursor_hidden = False @@ -430,7 +449,9 @@ def _write_node_result(self, node_result: object) -> None: self._stop_spinner_loop() if self._is_tty: - with self._write_lock: + with self._lines.lock: + self._spinner_line_active = False + self._lines.release(owner=self) self._stream.write("\r\033[K") self._stream.flush() self._show_cursor() @@ -660,7 +681,9 @@ def _write_failed_test_result(self, *, test_result: SqlTestExecutionResult) -> N self._stop_spinner_loop() if self._is_tty: - with self._write_lock: + with self._lines.lock: + self._spinner_line_active = False + self._lines.release(owner=self) self._stream.write("\r\033[K") self._stream.flush() self._show_cursor() diff --git a/src/sqlbuild/cli/commands/classes/load_progress_reporter.py b/src/sqlbuild/cli/commands/classes/load_progress_reporter.py index 0510ae75de..d65d1ca9ca 100644 --- a/src/sqlbuild/cli/commands/classes/load_progress_reporter.py +++ b/src/sqlbuild/cli/commands/classes/load_progress_reporter.py @@ -14,8 +14,10 @@ from sqlbuild.executor.load.models import LoadExecutionResult from sqlbuild.executor.scheduling.types import ExecutionStatus from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.classes.transient_line_coordinator import TransientLineCoordinator from sqlbuild.presentation.main.completion_line import format_completion_line from sqlbuild.presentation.main.summary_footer import format_summary_footer +from sqlbuild.presentation.main.transient_line_coordinator import shared_transient_line_coordinator from sqlbuild.presentation.types import CompletionState from sqlbuild.spec.contracts.models import SourceEntry @@ -46,6 +48,7 @@ def __init__( total_count: int, ) -> None: self._stream: TextIO = stream + self._lines: TransientLineCoordinator = shared_transient_line_coordinator() self._style: CliStyle = CliStyle(use_color=use_color) self._source_order: dict[str, int] = source_order self._total_count: int = total_count @@ -55,7 +58,8 @@ def __init__( self._spinner_frame_index: int = 0 self._spinner_stop_event: threading.Event | None = None self._spinner_thread: threading.Thread | None = None - self._write_lock: threading.Lock = threading.Lock() + self._write_lock: threading.RLock = self._lines.lock + self._spinner_line_active: bool = False self._cursor_hidden: bool = False self._projector: NativeProgressProjector | None = current_native_progress_projector() if self._projector is not None: @@ -69,7 +73,10 @@ def on_start(self, source: SourceEntry) -> None: self._current_source = source self._current_sub_message = "" self._hide_cursor() - self._write_spinner_line() + with self._write_lock: + self._spinner_line_active = True + self._lines.claim(stream=self._stream, owner=self) + self._write_spinner_line() self._start_spinner_loop() def on_progress(self, *, source: SourceEntry, message: str) -> None: @@ -92,6 +99,8 @@ def on_complete(self, result: LoadExecutionResult) -> None: self._stop_spinner_loop() if self._is_tty: with self._write_lock: + self._spinner_line_active = False + self._lines.release(owner=self) self._stream.write("\r\033[K") self._stream.flush() self._show_cursor() @@ -123,7 +132,24 @@ def on_complete(self, result: LoadExecutionResult) -> None: self._stream.write(self._style.error(f" {result.error_message}\n")) self._stream.flush() + def clear_transient_line(self) -> None: + """Erase the live spinner row so a persistent line can take its place.""" + + with self._write_lock: + self._stream.write("\r\033[K") + self._stream.flush() + + def redraw_transient_line(self) -> None: + """Draw the live spinner row again below persistent output.""" + + self._write_spinner_line() + def _write_spinner_line(self) -> None: + with self._write_lock: + if self._spinner_line_active: + self._draw_spinner_line() + + def _draw_spinner_line(self) -> None: source: SourceEntry | None = self._current_source if source is None: return @@ -139,9 +165,8 @@ def _write_spinner_line(self) -> None: line: str = ( f" {ordinal}/{self._total_count} {resource_kind:<10}{name_display:<48} {status}" ) - with self._write_lock: - self._stream.write(f"\r\033[K{line}") - self._stream.flush() + self._stream.write(f"\r\033[K{line}") + self._stream.flush() def _start_spinner_loop(self) -> None: self._stop_spinner_loop() diff --git a/src/sqlbuild/cli/commands/models.py b/src/sqlbuild/cli/commands/models.py index d7e0504971..626d53e259 100644 --- a/src/sqlbuild/cli/commands/models.py +++ b/src/sqlbuild/cli/commands/models.py @@ -1230,6 +1230,8 @@ class ScopeCommandRequest: paths: str = "relative" json_output: bool = False no_cache: bool = False + verbose: bool = False + no_color: bool = False @dataclass(frozen=True) diff --git a/src/sqlbuild/cli/output/_helpers/plan_text.py b/src/sqlbuild/cli/output/_helpers/plan_text.py index 60f240c2ea..510990d0e9 100644 --- a/src/sqlbuild/cli/output/_helpers/plan_text.py +++ b/src/sqlbuild/cli/output/_helpers/plan_text.py @@ -50,6 +50,7 @@ from sqlbuild.presentation.classes.cli_style import CliStyle from sqlbuild.presentation.main.aligned_name_value import format_aligned_name_value from sqlbuild.presentation.main.append_overflow_line import append_overflow_line +from sqlbuild.presentation.main.count_header import count_header_style from sqlbuild.presentation.main.resolve_name_column_width import resolve_name_column_width from sqlbuild.presentation.main.surface_header import format_surface_header from sqlbuild.presentation.main.tree_connector import tree_connector @@ -112,7 +113,9 @@ def format_plan( resolved_display_options: DisplayOptions = display_options or DisplayOptions() style: CliStyle = CliStyle(use_color=True) - resolved_section_header_style: Callable[[str], str] = section_header_style or style.plan_section + resolved_section_header_style: Callable[[str], str] = count_header_style( + style=style, title_style=section_header_style or style.plan_section + ) active_model_entries: tuple[ModelPlanEntry, ...] = tuple( entry for entry in plan.model_entries if entry.action != PlanAction.SKIP diff --git a/src/sqlbuild/cli/progress/classes/audit_progress_reporter.py b/src/sqlbuild/cli/progress/classes/audit_progress_reporter.py index f23afafc05..ba59b10e59 100644 --- a/src/sqlbuild/cli/progress/classes/audit_progress_reporter.py +++ b/src/sqlbuild/cli/progress/classes/audit_progress_reporter.py @@ -15,6 +15,8 @@ from sqlbuild.executor.auditing.main.resource_id import audit_resource_id from sqlbuild.executor.auditing.models import AuditExecutionResult from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.classes.transient_line_coordinator import TransientLineCoordinator +from sqlbuild.presentation.main.transient_line_coordinator import shared_transient_line_coordinator _LABEL_WIDTH: int = 10 _NAME_WIDTH: int = 50 @@ -34,6 +36,7 @@ def __init__( self._entries: tuple[AuditPlanEntry, ...] = entries self._worker_limit: int = worker_limit self._stream: TextIO = stream + self._lines: TransientLineCoordinator = shared_transient_line_coordinator() self._style: CliStyle = CliStyle(use_color=use_color) self._is_tty: bool = hasattr(stream, "isatty") and stream.isatty() self._projector: NativeProgressProjector | None = current_native_progress_projector() @@ -45,7 +48,8 @@ def __init__( self._running: int = 0 self._completed: int = 0 self._current_group: str | None = None - self._lock: threading.RLock = threading.RLock() + self._lock: threading.RLock = self._lines.lock + self._closed: bool = False self._cursor_hidden: bool = False self._on_result: Callable[[AuditExecutionResult], None] | None = None @@ -92,6 +96,8 @@ def close(self) -> None: """Restore the terminal after the final aggregate update.""" with self._lock: + self._closed = True + self._lines.release(owner=self) if self._is_tty: self._clear_aggregate() if self._cursor_hidden: @@ -140,9 +146,23 @@ def _write_result(self, result: AuditExecutionResult) -> None: if self._on_result is not None: self._on_result(result) + def clear_transient_line(self) -> None: + """Erase the aggregate row so a persistent line can take its place.""" + + with self._lock: + self._clear_aggregate() + self._stream.flush() + + def redraw_transient_line(self) -> None: + """Draw the aggregate row again below persistent output.""" + + with self._lock: + self._render_aggregate() + def _render_aggregate(self) -> None: - if not self._is_tty: + if not self._is_tty or self._closed: return + self._lines.claim(stream=self._stream, owner=self) if not self._cursor_hidden: self._stream.write("\033[?25l") self._cursor_hidden = True diff --git a/src/sqlbuild/cli/progress/classes/connection_progress_reporter.py b/src/sqlbuild/cli/progress/classes/connection_progress_reporter.py index 18768f021a..57253b2757 100644 --- a/src/sqlbuild/cli/progress/classes/connection_progress_reporter.py +++ b/src/sqlbuild/cli/progress/classes/connection_progress_reporter.py @@ -5,7 +5,9 @@ from typing import TextIO from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.classes.transient_line_coordinator import TransientLineCoordinator from sqlbuild.presentation.main.phase_line import format_phase_line +from sqlbuild.presentation.main.transient_line_coordinator import shared_transient_line_coordinator class ConnectionProgressReporter: @@ -22,14 +24,14 @@ def __init__( ) -> None: self._adapter_name: str = adapter_name self._stream: TextIO = stream + self._lines: TransientLineCoordinator = shared_transient_line_coordinator() self._blank_line_before_start: bool = blank_line_before_start self._blank_line_after_complete: bool = blank_line_after_complete self._style: CliStyle = CliStyle(use_color=use_color) def on_connection_start(self, connection_count: int) -> None: if self._blank_line_before_start: - self._stream.write("\n") - self._stream.flush() + self._write("") self._write(self._style.muted(self._start_message(connection_count))) def on_connection_complete(self, *, connection_count: int, elapsed_seconds: float) -> None: @@ -60,5 +62,4 @@ def _start_message(self, connection_count: int) -> str: return f"Connecting to {self._adapter_name} ({connection_count} connections)..." def _write(self, message: str) -> None: - self._stream.write(f"{message}\n") - self._stream.flush() + self._lines.write_persistent(stream=self._stream, text=f"{message}\n") diff --git a/src/sqlbuild/cli/progress/classes/native_progress_projector.py b/src/sqlbuild/cli/progress/classes/native_progress_projector.py index 7a11db8c23..24978fe052 100644 --- a/src/sqlbuild/cli/progress/classes/native_progress_projector.py +++ b/src/sqlbuild/cli/progress/classes/native_progress_projector.py @@ -9,6 +9,8 @@ from typing import TextIO, cast from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.classes.transient_line_coordinator import TransientLineCoordinator +from sqlbuild.presentation.main.transient_line_coordinator import shared_transient_line_coordinator from sqlbuild.runtime.observability.constants import ( RESOURCE_TERMINALS, RETRY_SCHEDULED_EVENT, @@ -88,9 +90,10 @@ class NativeProgressProjector: def __init__(self, *, stream: TextIO, use_color: bool) -> None: self._stream: TextIO = stream + self._lines: TransientLineCoordinator = shared_transient_line_coordinator() self._style: CliStyle = CliStyle(use_color=use_color) self._is_tty: bool = hasattr(stream, "isatty") and stream.isatty() - self._lock: threading.RLock = threading.RLock() + self._lock: threading.RLock = self._lines.lock self._seen_event_ids: set[str] = set() self._resource_ordinals: dict[str, int] = {} self._resource_total: int = 0 @@ -466,8 +469,7 @@ def _statement_context(self, event: LifecycleEvent) -> str: return f"model={resource_name} phase={phase} kind={statement_kind}" def _write(self, line: str) -> None: - self._stream.write(f"{line}\n") - self._stream.flush() + self._lines.write_persistent(stream=self._stream, text=f"{line}\n") _CURRENT_PROJECTOR: ContextVar[NativeProgressProjector | None] = ContextVar( diff --git a/src/sqlbuild/cli/progress/classes/nested_command_progress_callbacks.py b/src/sqlbuild/cli/progress/classes/nested_command_progress_callbacks.py index fd4b4d9365..26d7fa4ab2 100644 --- a/src/sqlbuild/cli/progress/classes/nested_command_progress_callbacks.py +++ b/src/sqlbuild/cli/progress/classes/nested_command_progress_callbacks.py @@ -11,8 +11,10 @@ ) from sqlbuild.cli.progress.models import NestedProgressChildRow from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.classes.transient_line_coordinator import TransientLineCoordinator from sqlbuild.presentation.main.inline_error_lines import format_inline_error_lines from sqlbuild.presentation.main.terminal_columns import terminal_columns +from sqlbuild.presentation.main.transient_line_coordinator import shared_transient_line_coordinator from sqlbuild.presentation.main.tree_connector import tree_connector _LABEL_WIDTH: int = 10 @@ -47,6 +49,7 @@ def __init__( self._total: int = total self._label: str = label self._stream: TextIO = stream + self._lines: TransientLineCoordinator = shared_transient_line_coordinator() self._use_color: bool = use_color self._style: CliStyle = CliStyle(use_color=use_color) self._name_width: int = max(_NAME_WIDTH, name_width) @@ -56,7 +59,8 @@ def __init__( self._active_name: str = "" self._active_group_is_new: bool = False self._spinner_frame_index: int = 0 - self._write_lock: threading.Lock = threading.Lock() + self._write_lock: threading.RLock = self._lines.lock + self._spinner_line_active: bool = False self._spinner_stop_event: threading.Event | None = None self._spinner_thread: threading.Thread | None = None self._cursor_hidden: bool = False @@ -83,7 +87,10 @@ def on_item_start( self._stream.flush() if self._is_tty: self._hide_cursor() - self._write_spinner_line() + with self._write_lock: + self._spinner_line_active = True + self._lines.claim(stream=self._stream, owner=self) + self._write_spinner_line() self._start_spinner_loop() def on_item_complete( @@ -111,6 +118,8 @@ def on_item_complete( self._stop_spinner_loop() if self._is_tty: with self._write_lock: + self._spinner_line_active = False + self._lines.release(owner=self) self._stream.write("\r\033[K") self._stream.flush() self._show_cursor() @@ -162,11 +171,29 @@ def _write_error( else: self._stream.write(f"{continuation_pad}{line}\n") + def clear_transient_line(self) -> None: + """Erase the live spinner row so a persistent line can take its place.""" + + with self._write_lock: + self._stream.write("\r\033[K") + self._stream.flush() + + def redraw_transient_line(self) -> None: + """Draw the live spinner row again below persistent output.""" + + self._write_spinner_line() + def _write_spinner_line(self) -> None: - spinner: str = self._style.status(status=_ACTIVE_SPINNER_FRAMES[self._spinner_frame_index]) - self._spinner_frame_index = (self._spinner_frame_index + 1) % len(_ACTIVE_SPINNER_FRAMES) - name: str = self._active_name with self._write_lock: + if not self._spinner_line_active: + return + spinner: str = self._style.status( + status=_ACTIVE_SPINNER_FRAMES[self._spinner_frame_index] + ) + self._spinner_frame_index = (self._spinner_frame_index + 1) % len( + _ACTIVE_SPINNER_FRAMES + ) + name: str = self._active_name self._stream.write( f"\r\033[K {self._label:<{_LABEL_WIDTH}}{name:<{self._name_width}} {spinner}" ) diff --git a/src/sqlbuild/cli/progress/classes/planning_progress_reporter.py b/src/sqlbuild/cli/progress/classes/planning_progress_reporter.py index 249959bbb2..7dbe0d24d6 100644 --- a/src/sqlbuild/cli/progress/classes/planning_progress_reporter.py +++ b/src/sqlbuild/cli/progress/classes/planning_progress_reporter.py @@ -76,6 +76,7 @@ def _is_planning_completion_message(message: str) -> bool: "Built ", "Checked ", "Compiled ", + "Evaluated ", "Applied ", "Finalized ", "Generated ", diff --git a/src/sqlbuild/compiler/planner/_helpers/migrations/display.py b/src/sqlbuild/compiler/planner/_helpers/migrations/display.py new file mode 100644 index 0000000000..7183e87708 --- /dev/null +++ b/src/sqlbuild/compiler/planner/_helpers/migrations/display.py @@ -0,0 +1,131 @@ +"""Plan text for model migrations and identity handovers.""" + +from __future__ import annotations + +from collections.abc import Callable, Sequence + +from sqlbuild.compiler.migrations.types import ( + MigrationCompatibility, + MigrationDecision, + MigrationDiscovery, +) +from sqlbuild.compiler.planner.models import ModelMigrationPlanEntry, PlanOutput +from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.main.append_overflow_line import append_overflow_line +from sqlbuild.presentation.main.count_header import count_header_style +from sqlbuild.presentation.main.visible_entries import visible_entries +from sqlbuild.presentation.models import DisplayOptions + + +def format_model_migrations( + *, lines: list[str], plan: PlanOutput, display_options: DisplayOptions +) -> list[str]: + """Format declared and discovered model migrations, then identity handovers.""" + + result: list[str] = list(lines) + style: CliStyle = CliStyle(use_color=True) + header_style: Callable[[str], str] = count_header_style( + style=style, title_style=style.plan_section + ) + title: str + renamed: bool + for title, renamed in (("Migrations", False), ("Renamed", True)): + entries: tuple[ModelMigrationPlanEntry, ...] = tuple( + entry + for entry in plan.migration_entries + if (entry.decision == MigrationDecision.RENAMED) is renamed + ) + if not entries: + continue + result.append(header_style(f"{title} ({len(entries)})")) + visible: Sequence[ModelMigrationPlanEntry] = visible_entries( + entries=entries, options=display_options + ) + entry: ModelMigrationPlanEntry + for entry in visible: + result.extend(_migration_lines(entry=entry, style=style)) + result = append_overflow_line( + lines=result, + total_count=len(entries), + visible_count=len(visible), + indent=" ", + options=display_options, + ) + return result + + +def _migration_lines(*, entry: ModelMigrationPlanEntry, style: CliStyle) -> list[str]: + origin: str = entry.origin.qualified_name or entry.origin.name + destination: str = entry.destination.qualified_name or entry.destination.name + relation_move: str = f"{style.muted(f'{origin} ->')} {destination}" + name: str = style.object_name(entry.model_name) + if entry.decision == MigrationDecision.RENAMED: + return [f" {name} {relation_move} {style.muted('(identity handed over)')}"] + decision: str = _decision_text(decision=entry.decision, style=style) + rows: list[str] = [f" {name} {decision} {relation_move}"] + if entry.decision.checks_compatibility: + rows.append( + _property_row( + label="compatibility", + value=_compatibility_text(compatibility=entry.compatibility, style=style), + style=style, + ) + ) + if entry.transfer is not None: + details: tuple[str | None, ...] = ( + entry.transfer.label + + ( + "" + if entry.transfer_fallback is None + else f" ({entry.transfer_fallback.label} if refused)" + ), + entry.storage_transition, + None if entry.promotion is None else f"promote by {entry.promotion.label}", + ) + rows.append( + _property_row( + label="transfer", + value=", ".join(detail for detail in details if detail), + style=style, + ) + ) + if entry.discovery != MigrationDiscovery.MANUAL: + rows.append(_property_row(label="discovery", value=entry.discovery.value, style=style)) + if entry.completed_at is not None: + rows.append( + _property_row( + label="completed", + value=( + f"{entry.completed_at.isoformat()} on target '{entry.target_name or 'default'}'" + ), + style=style, + ) + ) + blocking: bool = ( + entry.decision.blocks_build or entry.compatibility == MigrationCompatibility.INCOMPATIBLE + ) + finding_style: Callable[[str], str] = style.error if blocking else style.warning + rows.extend(f" {finding_style(f'! {finding}')}" for finding in entry.compatibility_findings) + return rows + + +def _property_row(*, label: str, value: str, style: CliStyle) -> str: + return f" {style.muted(label)} {value}" + + +def _decision_text(*, decision: MigrationDecision, style: CliStyle) -> str: + if decision.blocks_build: + return style.error_strong(decision.label) + if decision in (MigrationDecision.SUPERSEDED_REPLACE, MigrationDecision.FORCED_REPLACE): + return style.warning_strong(decision.label) + if decision == MigrationDecision.DONE: + return style.muted(decision.label) + return style.accent_strong(decision.label) + + +def _compatibility_text(*, compatibility: MigrationCompatibility, style: CliStyle) -> str: + if compatibility == MigrationCompatibility.COMPATIBLE: + return style.success(compatibility.value) + if compatibility == MigrationCompatibility.INCOMPATIBLE: + return style.error(compatibility.value) + return compatibility.value diff --git a/src/sqlbuild/compiler/planner/main/pre_build/pre_build_work_display.py b/src/sqlbuild/compiler/planner/main/pre_build/pre_build_work_display.py index bb4ec60431..8797e0d342 100644 --- a/src/sqlbuild/compiler/planner/main/pre_build/pre_build_work_display.py +++ b/src/sqlbuild/compiler/planner/main/pre_build/pre_build_work_display.py @@ -2,10 +2,8 @@ from __future__ import annotations -from collections.abc import Sequence - -from sqlbuild.compiler.migrations.types import MigrationDecision, MigrationDiscovery -from sqlbuild.compiler.planner.models import ModelMigrationPlanEntry, PlanOutput +from sqlbuild.compiler.planner._helpers.migrations.display import format_model_migrations +from sqlbuild.compiler.planner.models import PlanOutput from sqlbuild.presentation.main.append_overflow_line import append_overflow_line from sqlbuild.presentation.main.visible_entries import visible_entries from sqlbuild.presentation.models import DisplayOptions @@ -54,65 +52,4 @@ def format_pre_build_work( ) if entry.irreversible_warning is not None: lines.append(f" WARNING: {entry.irreversible_warning}") - return _format_model_migrations(lines=lines, plan=plan, display_options=display_options) - - -def _format_model_migrations( - *, lines: list[str], plan: PlanOutput, display_options: DisplayOptions -) -> list[str]: - title: str - renamed: bool - for title, renamed in (("Migrations", False), ("Renamed", True)): - entries: tuple[ModelMigrationPlanEntry, ...] = tuple( - entry - for entry in plan.migration_entries - if (entry.decision == MigrationDecision.RENAMED) is renamed - ) - if not entries: - continue - lines.append(f"{title} ({len(entries)})") - visible: Sequence[ModelMigrationPlanEntry] = visible_entries( - entries=entries, options=display_options - ) - entry: ModelMigrationPlanEntry - for entry in visible: - lines.extend(_migration_lines(entry)) - lines = append_overflow_line( - lines=lines, - total_count=len(entries), - visible_count=len(visible), - indent=" ", - options=display_options, - ) - return lines - - -def _migration_lines(entry: ModelMigrationPlanEntry) -> list[str]: - origin: str = entry.origin.qualified_name or entry.origin.name - destination: str = entry.destination.qualified_name or entry.destination.name - if entry.decision == MigrationDecision.RENAMED: - return [f" {entry.model_name} {origin} -> {destination} (identity handed over)"] - rows: list[str] = [f" {entry.model_name} {entry.decision.label} {origin} -> {destination}"] - if entry.decision.checks_compatibility: - rows.append(f" compatibility {entry.compatibility.value}") - if entry.transfer is not None: - details: tuple[str | None, ...] = ( - entry.transfer.label - + ( - "" - if entry.transfer_fallback is None - else f" ({entry.transfer_fallback.label} if refused)" - ), - entry.storage_transition, - None if entry.promotion is None else f"promote by {entry.promotion.label}", - ) - rows.append(f" transfer {', '.join(detail for detail in details if detail)}") - if entry.discovery != MigrationDiscovery.MANUAL: - rows.append(f" discovery {entry.discovery.value}") - if entry.completed_at is not None: - rows.append( - f" completed {entry.completed_at.isoformat()} on target " - f"'{entry.target_name or 'default'}'" - ) - rows.extend(f" ! {finding}" for finding in entry.compatibility_findings) - return rows + return format_model_migrations(lines=lines, plan=plan, display_options=display_options) diff --git a/src/sqlbuild/presentation/_helpers/structure.py b/src/sqlbuild/presentation/_helpers/structure.py index b110c14007..766ce030c1 100644 --- a/src/sqlbuild/presentation/_helpers/structure.py +++ b/src/sqlbuild/presentation/_helpers/structure.py @@ -2,6 +2,8 @@ from __future__ import annotations +from collections.abc import Callable + from sqlbuild.presentation.classes.cli_style import CliStyle from sqlbuild.presentation.constants import ( PHASE_FAIL_GLYPH, @@ -27,6 +29,29 @@ def format_surface_header(*, style: CliStyle, title: str, context: str | None = return rendered +def count_header_style( + *, style: CliStyle, title_style: Callable[[str], str] | None = None +) -> Callable[[str], str]: + """Return a header style that renders ``Title (count)`` as a bold title and dim count.""" + + resolved_title_style: Callable[[str], str] = title_style or style.section + + def render(text: str) -> str: + title, separator, count = text.rpartition(" (") + if not separator or not text.endswith(")"): + return resolved_title_style(text) + return f"{resolved_title_style(title)} {style.muted(f'({count}')}" + + return render + + +def format_count_noun(*, count: int, singular: str, plural: str | None = None) -> str: + """Render a count with the singular noun for one and the plural noun otherwise.""" + + noun: str = singular if count == 1 else (plural or f"{singular}s") + return f"{count} {noun}" + + def format_phase_line(*, style: CliStyle, ok: bool, label: str, summary: str | None = None) -> str: """Render a concise phase-completion line: state glyph, label, dim summary.""" diff --git a/src/sqlbuild/presentation/_helpers/transient_line.py b/src/sqlbuild/presentation/_helpers/transient_line.py new file mode 100644 index 0000000000..7a74e98f60 --- /dev/null +++ b/src/sqlbuild/presentation/_helpers/transient_line.py @@ -0,0 +1,13 @@ +"""Process-wide transient terminal line coordination.""" + +from __future__ import annotations + +from sqlbuild.presentation.classes.transient_line_coordinator import TransientLineCoordinator + +_SHARED_COORDINATOR: TransientLineCoordinator = TransientLineCoordinator() + + +def shared_transient_line_coordinator() -> TransientLineCoordinator: + """Return the coordinator shared by every terminal writer in this process.""" + + return _SHARED_COORDINATOR diff --git a/src/sqlbuild/presentation/classes/cli_style.py b/src/sqlbuild/presentation/classes/cli_style.py index 1ebc904204..4981b60221 100644 --- a/src/sqlbuild/presentation/classes/cli_style.py +++ b/src/sqlbuild/presentation/classes/cli_style.py @@ -24,6 +24,7 @@ label=TextStyle(_DIM), value=TextStyle(_BLUE), accent=TextStyle(_BLUE), + accent_strong=TextStyle(_BLUE + _BOLD), plan_section=TextStyle(_BOLD), object_name=TextStyle(""), command=TextStyle(_DIM), @@ -66,6 +67,9 @@ def value(self, text: str) -> str: def accent(self, text: str) -> str: return self.theme.accent.apply(text=text, use_color=self.use_color) + def accent_strong(self, text: str) -> str: + return self.theme.accent_strong.apply(text=text, use_color=self.use_color) + def plan_section(self, text: str) -> str: return self.theme.plan_section.apply(text=text, use_color=self.use_color) diff --git a/src/sqlbuild/presentation/classes/transient_line_coordinator.py b/src/sqlbuild/presentation/classes/transient_line_coordinator.py new file mode 100644 index 0000000000..f3eddb2db6 --- /dev/null +++ b/src/sqlbuild/presentation/classes/transient_line_coordinator.py @@ -0,0 +1,55 @@ +"""Coordinator for one live transient terminal line and persistent writes.""" + +from __future__ import annotations + +import threading +from typing import TextIO + +from sqlbuild.presentation.types import TransientLineOwner + + +class TransientLineCoordinator: + """Keep persistent terminal lines from landing on a live spinner or status line.""" + + def __init__(self) -> None: + self.lock: threading.RLock = threading.RLock() + self._active: tuple[TextIO, TransientLineOwner] | None = None + + def claim(self, *, stream: TextIO, owner: TransientLineOwner) -> None: + """Record the owner of the transient line currently drawn on a terminal stream.""" + + with self.lock: + self._active = (stream, owner) + + def release(self, *, owner: TransientLineOwner) -> None: + """Forget a transient owner once its line is no longer drawn.""" + + with self.lock: + if self._active is not None and self._active[1] is owner: + self._active = None + + def write_persistent(self, *, stream: TextIO, text: str) -> None: + """Write lasting text, clearing and then redrawing any live transient line.""" + + with self.lock: + owner: TransientLineOwner | None = self._interrupted_owner(stream=stream) + if owner is not None: + owner.clear_transient_line() + stream.write(text) + stream.flush() + if owner is not None: + owner.redraw_transient_line() + + def _interrupted_owner(self, *, stream: TextIO) -> TransientLineOwner | None: + if self._active is None: + return None + owner_stream: TextIO + owner: TransientLineOwner + owner_stream, owner = self._active + if owner_stream is stream or (_is_tty(owner_stream) and _is_tty(stream)): + return owner + return None + + +def _is_tty(stream: TextIO) -> bool: + return hasattr(stream, "isatty") and stream.isatty() diff --git a/src/sqlbuild/presentation/classes/transient_status_reporter.py b/src/sqlbuild/presentation/classes/transient_status_reporter.py index b09d89222c..f89b1595fd 100644 --- a/src/sqlbuild/presentation/classes/transient_status_reporter.py +++ b/src/sqlbuild/presentation/classes/transient_status_reporter.py @@ -11,7 +11,9 @@ from sqlbuild.errors.contracts.exceptions import SharedInputError from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.classes.transient_line_coordinator import TransientLineCoordinator from sqlbuild.presentation.main._progress_spinners_disabled import progress_spinners_disabled +from sqlbuild.presentation.main.transient_line_coordinator import shared_transient_line_coordinator class TransientStatusReporter: @@ -25,6 +27,7 @@ def __init__( enabled: bool = True, ) -> None: self._stream: TextIO = stream + self._lines: TransientLineCoordinator = shared_transient_line_coordinator() self._style: CliStyle = CliStyle(use_color=use_color) self._enabled: bool = ( enabled @@ -38,55 +41,66 @@ def __init__( self._status_context: AbstractContextManager[Status] | None = None self._status: Status | None = None self._active: bool = False + self._message: str | None = None atexit.register(self.close) def start(self, message: str) -> None: if self._enabled: - self.close() - if self._console is None: - raise SharedInputError( - "status reporter was enabled without an initialized console", - code="G001", - ) - self._status_context = self._console.status(status=message, spinner="dots") - self._status = self._status_context.__enter__() - self._active = True + with self._lines.lock: + self.close() + self._message = message + self._enter_status() + self._lines.claim(stream=self._stream, owner=self) + self._active = True return self._active = True self._write_message(message=message, dim_output=True) def update(self, message: str) -> None: - if self._enabled and self._status is not None: - self._status.update(message) - return + if self._enabled: + with self._lines.lock: + if self._message is not None: + self._message = message + if self._status is not None: + self._status.update(message) + return self._write_message(message=message, dim_output=True) def complete(self, *, message: str, blank_line_after: bool = False) -> None: self.close() self._write_message(message=message, dim_output=True) if blank_line_after: - self._stream.write("\n") - self._stream.flush() + self.write_blank_line() def complete_styled(self, *, message: str, blank_line_after: bool = False) -> None: """Complete with a pre-styled message written verbatim.""" self.close() - self._stream.write(f"{message}\n") - if blank_line_after: - self._stream.write("\n") - self._stream.flush() + self._lines.write_persistent( + stream=self._stream, text=f"{message}\n" + ("\n" if blank_line_after else "") + ) def error(self, message: str) -> None: self.close() self._write_message(message=message, dim_output=False) def close(self) -> None: - if self._status_context is not None: - self._status_context.__exit__(None, None, None) - self._status_context = None - self._status = None - self._active = False + with self._lines.lock: + self._lines.release(owner=self) + self._exit_status() + self._message = None + self._active = False + + def clear_transient_line(self) -> None: + """Erase the live status so a persistent line can be written in its place.""" + + self._exit_status() + + def redraw_transient_line(self) -> None: + """Restart the live status below persistent output written while it was active.""" + + if self._message is not None and self._status_context is None: + self._enter_status() def report_preflight_progress(self, message: str) -> None: """Render one preflight progress update.""" @@ -100,10 +114,23 @@ def report_preflight_progress(self, message: str) -> None: self.update(message) def write_blank_line(self) -> None: - self._stream.write("\n") - self._stream.flush() + self._lines.write_persistent(stream=self._stream, text="\n") + + def _enter_status(self) -> None: + if self._console is None or self._message is None: + raise SharedInputError( + "status reporter was enabled without an initialized console", + code="G001", + ) + self._status_context = self._console.status(status=self._message, spinner="dots") + self._status = self._status_context.__enter__() + + def _exit_status(self) -> None: + if self._status_context is not None: + self._status_context.__exit__(None, None, None) + self._status_context = None + self._status = None def _write_message(self, *, message: str, dim_output: bool) -> None: formatted_message: str = self._style.muted(message) if dim_output else message - self._stream.write(f"{formatted_message}\n") - self._stream.flush() + self._lines.write_persistent(stream=self._stream, text=f"{formatted_message}\n") diff --git a/src/sqlbuild/presentation/main/count_header.py b/src/sqlbuild/presentation/main/count_header.py new file mode 100644 index 0000000000..65078f7993 --- /dev/null +++ b/src/sqlbuild/presentation/main/count_header.py @@ -0,0 +1,16 @@ +"""Public counted section header style entry.""" + +from __future__ import annotations + +from collections.abc import Callable + +from sqlbuild.presentation._helpers.structure import count_header_style as _count_header_style +from sqlbuild.presentation.classes.cli_style import CliStyle + + +def count_header_style( + *, style: CliStyle, title_style: Callable[[str], str] | None = None +) -> Callable[[str], str]: + """Return a header style that renders ``Title (count)`` as a bold title and dim count.""" + + return _count_header_style(style=style, title_style=title_style) diff --git a/src/sqlbuild/presentation/main/count_noun.py b/src/sqlbuild/presentation/main/count_noun.py new file mode 100644 index 0000000000..2fa3a2f787 --- /dev/null +++ b/src/sqlbuild/presentation/main/count_noun.py @@ -0,0 +1,11 @@ +"""Public counted noun rendering entry.""" + +from __future__ import annotations + +from sqlbuild.presentation._helpers.structure import format_count_noun as _format_count_noun + + +def format_count_noun(*, count: int, singular: str, plural: str | None = None) -> str: + """Render a count with the singular noun for one and the plural noun otherwise.""" + + return _format_count_noun(count=count, singular=singular, plural=plural) diff --git a/src/sqlbuild/presentation/main/transient_line_coordinator.py b/src/sqlbuild/presentation/main/transient_line_coordinator.py new file mode 100644 index 0000000000..8a97433a08 --- /dev/null +++ b/src/sqlbuild/presentation/main/transient_line_coordinator.py @@ -0,0 +1,14 @@ +"""Public shared transient line coordinator entry.""" + +from __future__ import annotations + +from sqlbuild.presentation._helpers.transient_line import ( + shared_transient_line_coordinator as _shared_transient_line_coordinator, +) +from sqlbuild.presentation.classes.transient_line_coordinator import TransientLineCoordinator + + +def shared_transient_line_coordinator() -> TransientLineCoordinator: + """Return the coordinator shared by every terminal writer in this process.""" + + return _shared_transient_line_coordinator() diff --git a/src/sqlbuild/presentation/models.py b/src/sqlbuild/presentation/models.py index e50b77e91e..18264c6368 100644 --- a/src/sqlbuild/presentation/models.py +++ b/src/sqlbuild/presentation/models.py @@ -39,6 +39,7 @@ class CliTheme: label: TextStyle value: TextStyle accent: TextStyle + accent_strong: TextStyle plan_section: TextStyle object_name: TextStyle command: TextStyle diff --git a/src/sqlbuild/presentation/types.py b/src/sqlbuild/presentation/types.py index 04b1df05d6..f0e598c6cc 100644 --- a/src/sqlbuild/presentation/types.py +++ b/src/sqlbuild/presentation/types.py @@ -1,6 +1,7 @@ """Type-layer declarations for presentation.""" from enum import StrEnum +from typing import Protocol class CompletionState(StrEnum): @@ -9,3 +10,15 @@ class CompletionState(StrEnum): OK = "ok" WARN = "warn" FAIL = "fail" + + +class TransientLineOwner(Protocol): + """A renderer that owns the transient line currently drawn in place on a terminal.""" + + def clear_transient_line(self) -> None: + """Erase the drawn transient line so a persistent line can take its place.""" + ... + + def redraw_transient_line(self) -> None: + """Draw the transient line again below the persistent output.""" + ... diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/dbt/helpers.py b/tests/e2e/src/sqlbuild/cli/commands/main/dbt/helpers.py index 3d17efdb68..b3f7d14d57 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/dbt/helpers.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/dbt/helpers.py @@ -2,14 +2,10 @@ import json import os -import pty import subprocess -import threading from collections.abc import Callable -from functools import partial from pathlib import Path from shutil import copytree -from typing import cast import pytest @@ -85,66 +81,6 @@ def write_dbt_init_orders_model(*, workspace: Path, amount_cents: int) -> None: ) -def _capture_pty_output( - *, master_fd: int, output_parts: list[bytes], reader_done: threading.Event -) -> None: - try: - for chunk in iter(partial(os.read, master_fd, 4096), b""): - output_parts.append(chunk) - except OSError: - return - finally: - reader_done.set() - - -def run_sqb_with_pty( - *, command: tuple[str, ...], project_dir: Path, input_text: str, timeout_seconds: float = 60.0 -) -> subprocess.CompletedProcess[str]: - """Run sqb through a real PTY and return captured terminal output.""" - - master_fd: int - slave_fd: int - master_fd, slave_fd = pty.openpty() - process_env: dict[str, str] = dict(os.environ) - process_env["TERM"] = "xterm-256color" - process: subprocess.Popen[bytes] = subprocess.Popen( - ["uv", "run", "sqb", "--project-dir", str(project_dir), *command], - cwd=REPO_ROOT, - stdin=slave_fd, - stdout=slave_fd, - stderr=slave_fd, - env=process_env, - close_fds=True, - ) - os.close(slave_fd) - output_parts: list[bytes] = [] - reader_done: threading.Event = threading.Event() - reader: threading.Thread = threading.Thread( - target=_capture_pty_output, - kwargs={"master_fd": master_fd, "output_parts": output_parts, "reader_done": reader_done}, - daemon=True, - ) - reader.start() - try: - os.write(master_fd, input_text.encode()) - process.wait(timeout=timeout_seconds) - except subprocess.TimeoutExpired: - process.kill() - process.wait() - raise subprocess.TimeoutExpired(command, timeout_seconds) from None - finally: - reader_done.wait(timeout=1.0) - os.close(master_fd) - reader.join(timeout=1.0) - output: str = b"".join(output_parts).decode(errors="replace") - return subprocess.CompletedProcess( - args=("sqb", *command), - returncode=cast(int, process.returncode), - stdout=output, - stderr="", - ) - - def prepare_dbt_interop_project(*, tmp_path: Path) -> Path: """Copy the reusable dbt interop fixture and return its SQLBuild project root.""" diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/dbt/test_init.py b/tests/e2e/src/sqlbuild/cli/commands/main/dbt/test_init.py index cfce0e4744..9f48c6bb72 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/dbt/test_init.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/dbt/test_init.py @@ -13,10 +13,13 @@ ) from tests.e2e.src.sqlbuild.cli.commands.main.dbt.helpers import ( prepare_dbt_init_duckdb_workspace, - run_sqb_with_pty, skip_unless_dbt_is_runnable, ) -from tests.e2e.src.sqlbuild.cli.commands.shared.helpers import query_duckdb, run_sqb +from tests.e2e.src.sqlbuild.cli.commands.shared.helpers import ( + query_duckdb, + run_sqb, + run_sqb_with_pty, +) pytestmark: pytest.MarkDecorator = pytest.mark.dbt diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/janitor/test_janitor.py b/tests/e2e/src/sqlbuild/cli/commands/main/janitor/test_janitor.py index d59b115508..21eef5918e 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/janitor/test_janitor.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/janitor/test_janitor.py @@ -94,7 +94,7 @@ def test_given_default_config_when_running_janitor_then_it_reports_disabled( "main.partition_state relation matches exclude pattern 'partition_*'", "main._sqlbuild_fingerprints relation matches exclude pattern", "main._sqlbuild_microbatches relation matches exclude pattern", - "Archived 1 relations.", + "Archived 1 relation.", ), expected_existing_tables=( "orders", @@ -622,7 +622,7 @@ def test_given_virtual_checkpoint_refs_when_janitor_then_preserves_physical_vers "expired VDEs pruned", "Eligible expired VDEs", "pr expired virtual environment", - "Deleted 0 objects, deleted 1 state items, and pruned 0 direct state tables.", + "Deleted 0 objects, deleted 1 state item, and pruned 0 direct state tables.", ), expected_virtual_environment_names_after=("dev",), ) @@ -998,7 +998,7 @@ def test_given_virtual_checkpoints_over_limit_when_running_janitor_then_it_prune "detached VDEs pruned", "Eligible detached VDEs", "dev detached virtual environment", - "Deleted 1 objects, deleted 1 state items, and pruned 0 direct state tables.", + "Deleted 1 object, deleted 1 state item, and pruned 0 direct state tables.", ), expected_virtual_environment_count_after=0, expected_ref_count_after=0, @@ -1110,7 +1110,7 @@ def test_given_detached_vde_when_running_janitor_then_it_cleans_refs_and_physica "detached VDEs pruned", "Eligible detached VDEs", "dev detached virtual environment", - "Deleted 0 objects, deleted 1 state items, and pruned 0 direct state tables.", + "Deleted 0 objects, deleted 1 state item, and pruned 0 direct state tables.", ), expected_virtual_environment_count_after=0, ) diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/janitor/test_janitor_archive.py b/tests/e2e/src/sqlbuild/cli/commands/main/janitor/test_janitor_archive.py index 193fd3d19a..dbd110d627 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/janitor/test_janitor_archive.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/janitor/test_janitor_archive.py @@ -59,7 +59,7 @@ "main.customers -> main._sqb_archive__", "age 20d, delete after", "main.products relation is newer than 14 days", - "Archived 1 relations.", + "Archived 1 relation.", ), expected_second_run_fragments=( "relations to archive 0", @@ -405,7 +405,7 @@ def test_given_small_identifier_limit_when_archiving_long_name_then_fits_name_an expected_second_stdout_fragments=( "archives to delete 1", "Archives to delete", - "Deleted 1 objects", + "Deleted 1 object", ), ) ], @@ -486,7 +486,7 @@ def test_given_audit_write_interrupted_after_rename_when_janitor_reruns_then_arc "main.old_orders -> main._sqb_archive__", "main.LegacyCustomers relation name is not a plain lowercase identifier and may " "require quoting; janitor does not act on it", - "Archived 1 relations.", + "Archived 1 relation.", ), ) ], diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/plan/_test_types.py b/tests/e2e/src/sqlbuild/cli/commands/main/plan/_test_types.py index 60d4fa93bd..26a69b025e 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/plan/_test_types.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/plan/_test_types.py @@ -61,3 +61,12 @@ class VirtualSourceFreshnessPlanE2ETestCase: include_freshness: bool = True source_freshness_type: str = "timestamp" warehouse_column_type: str = "TIMESTAMP" + + +@dataclass(frozen=True) +class TerminalPlanProgressE2ETestCase: + description: str + runs: int + columns: int + expected_screen_fragments: tuple[str, ...] + expected_raw_fragments: tuple[str, ...] diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/plan/test_terminal_progress.py b/tests/e2e/src/sqlbuild/cli/commands/main/plan/test_terminal_progress.py new file mode 100644 index 0000000000..d89032a16c --- /dev/null +++ b/tests/e2e/src/sqlbuild/cli/commands/main/plan/test_terminal_progress.py @@ -0,0 +1,85 @@ +"""E2E coverage for plan progress and migration output in a real terminal.""" + +from __future__ import annotations + +import subprocess +from pathlib import Path + +import pytest + +from tests.e2e.src.sqlbuild.cli.commands.main.model_migrations.helpers import ( + build, + load_raw_orders, + write_orders_project, +) +from tests.e2e.src.sqlbuild.cli.commands.main.plan._test_types import ( + TerminalPlanProgressE2ETestCase, +) +from tests.e2e.src.sqlbuild.cli.commands.shared.helpers import ( + SPINNER_GLYPHS, + assert_fragments_in_order, + render_terminal_screen, + run_sqb_with_pty, +) + + +@pytest.mark.parametrize( + "test_case", + [ + TerminalPlanProgressE2ETestCase( + description="renamed model plan leaves only persistent lines on screen", + runs=3, + columns=100, + expected_screen_fragments=( + "Compiled project.", + "Evaluated rules.", + "Connecting to duckdb...", + "✓ Warehouse connected duckdb", + "Generated plan.", + "Plan ready 1 selected", + "Migrations (1)", + "└── stg_customer_orders migrate main.stg_orders -> main.stg_customer_orders", + "├── compatibility compatible", + "Models (1)", + ), + expected_raw_fragments=( + "\x1b[1mMigrations\x1b[0m \x1b[2m(1)\x1b[0m", + "\x1b[34m\x1b[1mmigrate\x1b[0m", + "\x1b[2mmain.stg_orders ->\x1b[0m main.stg_customer_orders", + "\x1b[2mcompatibility\x1b[0m \x1b[32mcompatible\x1b[0m", + "\x1b[1mModels\x1b[0m \x1b[2m(1)\x1b[0m", + ), + ) + ], + ids=lambda case: case.description, +) +def test_given_renamed_model_when_planning_in_terminal_then_no_transient_line_survives( + tmp_path: Path, test_case: TerminalPlanProgressE2ETestCase +) -> None: + project_dir: Path = write_orders_project(tmp_path=tmp_path, name="stg_orders", migrate_from="") + load_raw_orders(project_dir=project_dir, last_day=3) + initial: subprocess.CompletedProcess[str] = build(project_dir=project_dir) + _ = write_orders_project( + tmp_path=tmp_path, name="stg_customer_orders", migrate_from="stg_orders" + ) + + results: list[subprocess.CompletedProcess[str]] = [ + run_sqb_with_pty(command=("plan",), project_dir=project_dir, columns=test_case.columns) + for _ in range(test_case.runs) + ] + + assert initial.returncode == 0, initial.stdout + initial.stderr + for result in results: + screen: tuple[str, ...] = render_terminal_screen( + output=result.stdout, columns=test_case.columns + ) + rendered: str = "\n".join(screen) + assert result.returncode == 0, rendered + assert not any(glyph in rendered for glyph in SPINNER_GLYPHS), rendered + assert any(line.startswith("✓ Warehouse connected") for line in screen), rendered + assert_fragments_in_order(rendered, test_case.expected_screen_fragments) + assert all(fragment in result.stdout for fragment in test_case.expected_raw_fragments) + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/test/_test_types.py b/tests/e2e/src/sqlbuild/cli/commands/main/test/_test_types.py index b78f987df6..89b60c7b31 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/test/_test_types.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/test/_test_types.py @@ -186,3 +186,11 @@ class SetOperationExpectedE2ETestCase: test_files: dict[str, str] expected_exit_code: int expected_output_fragments: tuple[str, ...] + + +@dataclass(frozen=True) +class TerminalTestProgressE2ETestCase: + description: str + columns: int + expected_screen_fragments: tuple[str, ...] + expected_summary: str diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/test/test_terminal_progress.py b/tests/e2e/src/sqlbuild/cli/commands/main/test/test_terminal_progress.py new file mode 100644 index 0000000000..bf2f874642 --- /dev/null +++ b/tests/e2e/src/sqlbuild/cli/commands/main/test/test_terminal_progress.py @@ -0,0 +1,66 @@ +"""E2E coverage for sqb test progress rows in a real terminal.""" + +from __future__ import annotations + +import subprocess +from pathlib import Path + +import pytest + +from tests.e2e.src.sqlbuild.cli.commands.main.test._test_types import ( + TerminalTestProgressE2ETestCase, +) +from tests.e2e.src.sqlbuild.cli.commands.shared.helpers import ( + SPINNER_GLYPHS, + assert_fragments_in_order, + prepare_waffle_shop, + render_terminal_screen, + run_sqb, + run_sqb_with_pty, +) + + +@pytest.mark.parametrize( + "test_case", + [ + TerminalTestProgressE2ETestCase( + description="test rows replace their spinner and statement narration stays separate", + columns=100, + expected_screen_fragments=( + "✓ Warehouse connected duckdb", + "test test_stg_orders", + "PASS= FAIL= TOTAL=", + ), + expected_summary="PASS=5 FAIL=0 TOTAL=5", + ) + ], + ids=lambda case: case.description, +) +def test_given_built_project_when_testing_in_terminal_then_no_spinner_row_survives( + tmp_path: Path, test_case: TerminalTestProgressE2ETestCase +) -> None: + project_dir: Path = prepare_waffle_shop(tmp_path) + built: subprocess.CompletedProcess[str] = run_sqb( + command=("--no-color", "build"), project_dir=project_dir + ) + + result: subprocess.CompletedProcess[str] = run_sqb_with_pty( + command=("test",), project_dir=project_dir, columns=test_case.columns, timeout_seconds=180.0 + ) + screen: tuple[str, ...] = render_terminal_screen( + output=result.stdout, columns=test_case.columns + ) + rendered: str = "\n".join(screen) + + assert built.returncode == 0, built.stdout + built.stderr + assert result.returncode == 0, rendered + assert not any(glyph in rendered for glyph in SPINNER_GLYPHS), rendered + assert not any(line.lstrip().startswith("test ") and "statement" in line for line in screen), ( + rendered + ) + assert_fragments_in_order(rendered, test_case.expected_screen_fragments) + assert test_case.expected_summary in rendered + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/e2e/src/sqlbuild/cli/commands/shared/helpers.py b/tests/e2e/src/sqlbuild/cli/commands/shared/helpers.py index c3281e77bc..9481008692 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/shared/helpers.py +++ b/tests/e2e/src/sqlbuild/cli/commands/shared/helpers.py @@ -2,16 +2,23 @@ from __future__ import annotations +import fcntl import os +import pty import re +import struct import subprocess +import termios +import threading from collections.abc import Callable, Mapping +from functools import partial from pathlib import Path from shutil import copytree -from typing import Any +from typing import Any, cast REPO_ROOT: Path = Path(__file__).resolve().parents[7] WAFFLE_SHOP_DIR: Path = REPO_ROOT / "tests" / "e2e" / "fixtures" / "waffle_shop" +SPINNER_GLYPHS: tuple[str, ...] = ("⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏") SOURCE_LOADER_STRATEGIES_DIR: Path = ( REPO_ROOT / "tests" / "e2e" / "fixtures" / "source_loader_strategies" ) @@ -213,6 +220,145 @@ def run_sqb( ) +def run_sqb_with_pty( + *, + command: tuple[str, ...], + project_dir: Path, + input_text: str = "", + timeout_seconds: float = 60.0, + columns: int = 120, +) -> subprocess.CompletedProcess[str]: + """Run sqb through a real PTY of the given width and return the raw terminal output.""" + + master_fd: int + slave_fd: int + master_fd, slave_fd = pty.openpty() + fcntl.ioctl(slave_fd, termios.TIOCSWINSZ, struct.pack("HHHH", 40, columns, 0, 0)) + process_env: dict[str, str] = dict(os.environ) + for variable in _PTY_UNSET_ENVIRONMENT_VARIABLES: + process_env.pop(variable, None) + process_env["TERM"] = "xterm-256color" + process: subprocess.Popen[bytes] = subprocess.Popen( + ["uv", "run", "sqb", "--project-dir", str(project_dir), *command], + cwd=REPO_ROOT, + stdin=slave_fd, + stdout=slave_fd, + stderr=slave_fd, + env=process_env, + close_fds=True, + ) + os.close(slave_fd) + output_parts: list[bytes] = [] + reader_done: threading.Event = threading.Event() + reader: threading.Thread = threading.Thread( + target=_capture_pty_output, + kwargs={"master_fd": master_fd, "output_parts": output_parts, "reader_done": reader_done}, + daemon=True, + ) + reader.start() + try: + os.write(master_fd, input_text.encode()) + process.wait(timeout=timeout_seconds) + except subprocess.TimeoutExpired: + process.kill() + process.wait() + raise subprocess.TimeoutExpired(command, timeout_seconds) from None + finally: + reader_done.wait(timeout=1.0) + os.close(master_fd) + reader.join(timeout=1.0) + output: str = b"".join(output_parts).decode(errors="replace") + return subprocess.CompletedProcess( + args=("sqb", *command), + returncode=cast(int, process.returncode), + stdout=output, + stderr="", + ) + + +_PTY_UNSET_ENVIRONMENT_VARIABLES: frozenset[str] = frozenset( + {"COLUMNS", "LINES", "NO_COLOR", "SQLBUILD_NO_PROGRESS"} +) + + +def _capture_pty_output( + *, master_fd: int, output_parts: list[bytes], reader_done: threading.Event +) -> None: + try: + for chunk in iter(partial(os.read, master_fd, 4096), b""): + output_parts.append(chunk) + except OSError: + return + finally: + reader_done.set() + + +_TERMINAL_TOKEN_PATTERN: re.Pattern[str] = re.compile( + r"(?P\x1b\[(?P[?0-9;]*)(?P[A-Za-z]))" + r"|(?P\r)|(?P\n)|(?P[^\x1b\r\n])" +) + + +class _TerminalScreen: + """Minimal auto-wrapping terminal that understands the progress control sequences.""" + + def __init__(self, *, columns: int) -> None: + self.columns: int = columns + self.rows: list[list[str]] = [[]] + self.row: int = 0 + self.column: int = 0 + + def carriage_return(self, match: re.Match[str]) -> None: + del match + self.column = 0 + + def line_feed(self, match: re.Match[str]) -> None: + del match + self._move_to_row(self.row + 1) + self.column = 0 + + def character(self, match: re.Match[str]) -> None: + wrap: bool = self.column >= self.columns + self._move_to_row(self.row + int(wrap)) + self.column *= int(not wrap) + line: list[str] = self.rows[self.row] + line.extend(" " for _ in range(self.column + 1 - len(line))) + line[self.column] = match.group(0) + self.column += 1 + + def control(self, match: re.Match[str]) -> None: + handlers: dict[str, Callable[[str], None]] = { + "A": self._cursor_up, + "K": self._erase_line, + } + handlers.get(match.group("command"), self._ignore)(match.group("parameters")) + + def lines(self) -> tuple[str, ...]: + return tuple("".join(line).rstrip() for line in self.rows) + + def _move_to_row(self, row: int) -> None: + self.row = row + self.rows.extend([] for _ in range(row + 1 - len(self.rows))) + + def _cursor_up(self, parameters: str) -> None: + self.row = max(self.row - int(parameters or "1"), 0) + + def _erase_line(self, parameters: str) -> None: + del self.rows[self.row][self.column * int(parameters != "2") :] + + def _ignore(self, parameters: str) -> None: + del parameters + + +def render_terminal_screen(*, output: str, columns: int) -> tuple[str, ...]: + """Replay raw terminal output on a minimal emulator and return the visible final lines.""" + + screen: _TerminalScreen = _TerminalScreen(columns=columns) + for match in _TERMINAL_TOKEN_PATTERN.finditer(output): + getattr(screen, str(match.lastgroup))(match) + return screen.lines() + + def query_duckdb(*, db_path: Path, sql: str) -> list[tuple[Any, ...]]: """Open a DuckDB file and execute a query, returning all rows.""" diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/__init__.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/_test_types.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/_test_types.py new file mode 100644 index 0000000000..cc9423a686 --- /dev/null +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/_test_types.py @@ -0,0 +1,12 @@ +from dataclasses import dataclass + + +@dataclass(frozen=True) +class JanitorCompletionTestCase: + description: str + archived: int + deleted: int + deleted_state_backups: int + deleted_checkpoints: int + pruned_direct_state: int + expected_output: str diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/helpers.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/helpers.py new file mode 100644 index 0000000000..7a57be2679 --- /dev/null +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/helpers.py @@ -0,0 +1,11 @@ +"""Janitor output test builders.""" + +from __future__ import annotations + +from typing import Any + + +def placeholder_candidates(count: int) -> tuple[Any, ...]: + """Return count opaque candidates; completion wording only depends on the counts.""" + + return tuple(object() for _ in range(count)) diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/test_outputs.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/test_outputs.py new file mode 100644 index 0000000000..c364340aa6 --- /dev/null +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/janitor_output/test_outputs.py @@ -0,0 +1,86 @@ +"""Janitor completion summary wording.""" + +from __future__ import annotations + +from types import SimpleNamespace +from typing import cast + +import pytest + +from sqlbuild.cli.commands._helpers.janitor_output.outputs import write_janitor_completion +from sqlbuild.cli.commands.models import JanitorInvocation +from sqlbuild.executor.janitor.models import JanitorExecutionResult +from tests.unit.src.sqlbuild.cli.commands._helpers.janitor_output._test_types import ( + JanitorCompletionTestCase, +) +from tests.unit.src.sqlbuild.cli.commands._helpers.janitor_output.helpers import ( + placeholder_candidates, +) + + +@pytest.mark.parametrize( + "test_case", + ( + JanitorCompletionTestCase( + description="single archive and object use singular nouns", + archived=1, + deleted=1, + deleted_state_backups=0, + deleted_checkpoints=0, + pruned_direct_state=0, + expected_output="Archived 1 relation. Deleted 1 object.\n", + ), + JanitorCompletionTestCase( + description="single state item and pruned table use singular nouns", + archived=0, + deleted=0, + deleted_state_backups=1, + deleted_checkpoints=0, + pruned_direct_state=1, + expected_output=( + "Deleted 0 objects, deleted 1 state item, and pruned 1 direct state table.\n" + ), + ), + JanitorCompletionTestCase( + description="single checkpoint uses a singular noun", + archived=2, + deleted=2, + deleted_state_backups=0, + deleted_checkpoints=1, + pruned_direct_state=0, + expected_output="Archived 2 relations. Deleted 2 objects and 1 checkpoint.\n", + ), + JanitorCompletionTestCase( + description="plural counts keep plural nouns", + archived=0, + deleted=3, + deleted_state_backups=2, + deleted_checkpoints=0, + pruned_direct_state=2, + expected_output=( + "Deleted 3 objects, deleted 2 state items, and pruned 2 direct state tables.\n" + ), + ), + ), + ids=lambda case: case.description, +) +def test_given_janitor_counts_when_writing_completion_then_nouns_match_counts( + test_case: JanitorCompletionTestCase, capsys: pytest.CaptureFixture[str] +) -> None: + result: JanitorExecutionResult = JanitorExecutionResult( + archived=placeholder_candidates(test_case.archived), + deleted=placeholder_candidates(test_case.deleted), + deleted_state_backups=placeholder_candidates(test_case.deleted_state_backups), + deleted_checkpoints=placeholder_candidates(test_case.deleted_checkpoints), + pruned_direct_state=placeholder_candidates(test_case.pruned_direct_state), + ) + + write_janitor_completion( + invocation=cast(JanitorInvocation, SimpleNamespace(use_color=False)), result=result + ) + + assert capsys.readouterr().out == test_case.expected_output + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/_test_types.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/_test_types.py index c3beaee1a1..5e45a8382d 100644 --- a/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/_test_types.py +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/_test_types.py @@ -7,3 +7,19 @@ class ScopeOutputCase: description: str expected_exit_code: int + + +@dataclass(frozen=True) +class ScopeDetailLevelCase: + description: str + verbose: bool + expected_fragments: tuple[str, ...] + unexpected_fragments: tuple[str, ...] + + +@dataclass(frozen=True) +class ScopeColourCase: + description: str + use_color: bool + expected_fragments: tuple[str, ...] + unexpected_fragments: tuple[str, ...] diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/helpers.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/helpers.py new file mode 100644 index 0000000000..68d465ef22 --- /dev/null +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/helpers.py @@ -0,0 +1,23 @@ +"""Scope output test builders.""" + +from __future__ import annotations + +from dataclasses import replace + +from sqlbuild.compiler.scopes.main.preview_scope_move import preview_scope_move +from sqlbuild.compiler.scopes.main.query_scope_report import query_scope_report +from sqlbuild.compiler.scopes.models import ScopeLookup, ScopeReport, ScopeReportFilters +from tests.unit.src.sqlbuild.compiler.scopes.helpers import report_scope_lookup + + +def orders_move_report() -> ScopeReport: + """Return the orders report with a move preview into models/marts.""" + + lookup: ScopeLookup = report_scope_lookup() + report: ScopeReport = query_scope_report( + lookup=lookup, target="model:orders", at=None, directory=False, filters=ScopeReportFilters() + ) + move, diagnostics = preview_scope_move( + lookup=lookup, resource="model:orders", destination="models/marts/orders.sql" + ) + return replace(report, move_preview=move, diagnostics=(*report.diagnostics, *diagnostics)) diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/test_output.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/test_output.py index ac053b3485..414b4dfa5b 100644 --- a/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/test_output.py +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/test_output.py @@ -2,15 +2,138 @@ from __future__ import annotations +from dataclasses import replace from io import StringIO import pytest from sqlbuild.cli.commands._helpers.scope.command import run_scope_command +from sqlbuild.cli.commands._helpers.scope.output import render_scope_result from sqlbuild.cli.commands.models import ScopeCommandRequest -from tests.unit.src.sqlbuild.cli.commands._helpers.scope._test_types import ScopeOutputCase +from tests.unit.src.sqlbuild.cli.commands._helpers.scope._test_types import ( + ScopeColourCase, + ScopeDetailLevelCase, + ScopeOutputCase, +) +from tests.unit.src.sqlbuild.cli.commands._helpers.scope.helpers import orders_move_report from tests.unit.src.sqlbuild.compiler.scopes.helpers import report_scope_lookup +_BOLD: str = "\033[1m" +_DIM: str = "\033[2m" +_GREEN: str = "\033[32m" +_YELLOW: str = "\033[33m" +_RED: str = "\033[38;5;167m" +_RESET: str = "\033[0m" +_MOVE_REQUEST: ScopeCommandRequest = ScopeCommandRequest( + target="model:orders", as_path="models/marts/orders.sql" +) + + +@pytest.mark.parametrize( + "test_case", + ( + ScopeDetailLevelCase( + description="default is compact with a legend and no empty optional sections", + verbose=False, + expected_fragments=( + " Status: 1 match\n", + " ├─ ● enum:mart_status models/marts/_enums/mart_status.sql:1\n", + " … 1 global collapsed; run sqb scope model:orders " + "--as-path models/marts/orders.sql --globals all\n", + " Lost (1)\n └─ ● enum:order_status models/staging/enums/order_status.sql:1\n", + " Invalidated usages (1)\n - enum:order_status\n", + "● used by this resource ○ not used by this resource --verbose for scope details\n", + "Completeness: complete\n", + ), + unexpected_fragments=( + "[", + ":1:1", + "Private retained", + "Nearby unavailable", + "Diagnostics", + "\033[", + ), + ), + ScopeDetailLevelCase( + description="verbose restores bracketed details, columns, and empty sections", + verbose=True, + expected_fragments=( + " Status: 1 match\n", + " ├─ ● enum:mart_status [enum; exact-owner-private; expected_model through " + "model:expected_orders; role models/marts/_enums] " + "models/marts/_enums/mart_status.sql:1:1\n", + "Nearby unavailable (1 of 1)\n (none)\n", + " Private retained (0)\n (none)\n", + "Diagnostics (0)\n (none)\nCompleteness: complete\n", + ), + unexpected_fragments=("used by this resource", "\033["), + ), + ), + ids=lambda case: case.description, +) +def test_given_move_preview_when_rendering_then_detail_level_follows_verbose( + test_case: ScopeDetailLevelCase, +) -> None: + stream: StringIO = StringIO() + + exit_code: int = run_scope_command( + request=replace(_MOVE_REQUEST, verbose=test_case.verbose), + load_scope_index=lambda **_kwargs: report_scope_lookup().index, + output_stream=stream, + ) + + assert exit_code == 0 + assert all(fragment in stream.getvalue() for fragment in test_case.expected_fragments), ( + stream.getvalue() + ) + assert not any(fragment in stream.getvalue() for fragment in test_case.unexpected_fragments) + + +@pytest.mark.parametrize( + "test_case", + ( + ScopeColourCase( + description="colour marks headers, used markers, losses, gains, and completeness", + use_color=True, + expected_fragments=( + f"{_BOLD}Scope{_RESET}\n {_DIM}Target:{_RESET} model:orders\n", + f"{_BOLD}Used{_RESET} {_DIM}(3){_RESET}\n", + f" {_DIM}├─{_RESET} {_GREEN}●{_RESET} enum:mart_status " + f"{_DIM}models/marts/_enums/mart_status.sql:1{_RESET}\n", + f"{_BOLD}Available{_RESET} {_DIM}(2 of 3, 1 collapsed){_RESET}\n", + f" {_DIM}├─{_RESET} {_DIM}○{_RESET} constant:warehouse_password", + f" {_DIM}└─{_RESET} {_GREEN}●{_RESET} {_GREEN}enum:mart_status{_RESET}", + f" {_YELLOW}{_BOLD}Lost{_RESET} {_YELLOW}(1){_RESET}\n" + f" {_DIM}└─{_RESET} {_RED}●{_RESET} enum:order_status", + f" {_RED}{_BOLD}Invalidated usages{_RESET} {_RED}(1){_RESET}\n" + f" {_RED}- enum:order_status{_RESET}\n", + f"{_DIM}Completeness:{_RESET} {_GREEN}complete{_RESET}\n", + ), + unexpected_fragments=(), + ), + ScopeColourCase( + description="colour off renders the same report without ANSI sequences", + use_color=False, + expected_fragments=( + "Used (3)\n", + " Lost (1)\n └─ ● enum:order_status", + "Completeness: complete\n", + ), + unexpected_fragments=("\033[",), + ), + ), + ids=lambda case: case.description, +) +def test_given_colour_setting_when_rendering_move_preview_then_applies_colour_roles( + test_case: ScopeColourCase, +) -> None: + rendered: str = render_scope_result( + result=orders_move_report(), request=_MOVE_REQUEST, use_color=test_case.use_color + ) + + assert all(fragment in rendered for fragment in test_case.expected_fragments), rendered + assert not any(fragment in rendered for fragment in test_case.unexpected_fragments) + @pytest.mark.parametrize( "test_case", (ScopeOutputCase("browse", 0),), ids=lambda case: case.description @@ -28,7 +151,8 @@ def test_given_browse_result_when_rendering_then_counts_and_follow_ups_are_exact assert exit_code == test_case.expected_exit_code assert "Scope folders" in stream.getvalue() - assert "declarations" in stream.getvalue() + assert "constants/ 1 declaration, 0 used, 0 children; constant 1" in stream.getvalue() + assert "macros/ 1 declaration, 1 used, 0 children; macro 1" in stream.getvalue() assert "sqb scope model:orders --browse" in stream.getvalue() assert "sqb scope model:orders --list" in stream.getvalue() diff --git a/tests/unit/src/sqlbuild/cli/commands/main/entry/test_scope.py b/tests/unit/src/sqlbuild/cli/commands/main/entry/test_scope.py index bf480371c2..8d6ccfcc18 100644 --- a/tests/unit/src/sqlbuild/cli/commands/main/entry/test_scope.py +++ b/tests/unit/src/sqlbuild/cli/commands/main/entry/test_scope.py @@ -35,6 +35,7 @@ def record(request: ScopeCommandRequest) -> int: exit_code: int = _main_with_dependencies( argv=( + "--no-color", "scope", "model:orders", "--as-path", @@ -65,6 +66,7 @@ def record(request: ScopeCommandRequest) -> int: "compact", "--json", "--no-cache", + "--verbose", ), handlers=build_handlers(run_scope=record), ) @@ -88,6 +90,8 @@ def record(request: ScopeCommandRequest) -> int: paths="compact", json_output=True, no_cache=True, + verbose=True, + no_color=True, ) ] assert ScopeCommandRequest.__dataclass_params__.frozen is True diff --git a/tests/unit/src/sqlbuild/cli/commands/main/scope/test_scope.py b/tests/unit/src/sqlbuild/cli/commands/main/scope/test_scope.py index 9e94ad4741..8e705df7b0 100644 --- a/tests/unit/src/sqlbuild/cli/commands/main/scope/test_scope.py +++ b/tests/unit/src/sqlbuild/cli/commands/main/scope/test_scope.py @@ -8,6 +8,7 @@ import pytest from sqlbuild.cli.commands._helpers.runtime import adapter_context, connection +from sqlbuild.cli.commands._helpers.scope import command as scope_command from sqlbuild.cli.commands._helpers.scope.command import run_scope_command from sqlbuild.cli.commands.models import ScopeCommandRequest from sqlbuild.compiler.scopes.models import ScopeCompleteness, ScopeIndex @@ -65,6 +66,54 @@ def test_given_scope_index_when_rendering_json_twice_then_bytes_are_deterministi assert "/home/" not in outputs[0] +@pytest.mark.parametrize( + "test_case", (ScopeCommandCase("verbose json", 0),), ids=lambda case: case.description +) +def test_given_verbose_flag_when_rendering_json_then_bytes_match_default_json( + test_case: ScopeCommandCase, +) -> None: + index: ScopeIndex = report_scope_lookup(extra_globals=3).index + outputs: list[str] = [] + for verbose in (False, True): + stream: StringIO = StringIO() + exit_code: int = run_scope_command( + request=ScopeCommandRequest( + target="model:orders", + as_path="models/marts/orders.sql", + json_output=True, + verbose=verbose, + ), + load_scope_index=lambda **_kwargs: index, + output_stream=stream, + ) + assert exit_code == test_case.expected_exit_code + outputs.append(stream.getvalue()) + assert outputs[0] == outputs[1] + assert "\x1b[" not in outputs[0] + + +@pytest.mark.parametrize( + "test_case", (ScopeCommandCase("no color flag", 0),), ids=lambda case: case.description +) +def test_given_no_color_flag_when_rendering_text_to_color_terminal_then_output_is_plain( + monkeypatch: pytest.MonkeyPatch, + capsys: pytest.CaptureFixture[str], + test_case: ScopeCommandCase, +) -> None: + monkeypatch.setattr(scope_command, "supports_color", lambda: True) + index: ScopeIndex = report_scope_lookup().index + outputs: list[str] = [] + for no_color in (False, True): + exit_code: int = run_scope_command( + request=ScopeCommandRequest(target="model:orders", no_color=no_color), + load_scope_index=lambda **_kwargs: index, + ) + assert exit_code == test_case.expected_exit_code + outputs.append(capsys.readouterr().out) + assert "\x1b[" in outputs[0] + assert "\x1b[" not in outputs[1] + + @pytest.mark.parametrize( "test_case", (ScopeCommandCase("partial", 1),), ids=lambda case: case.description ) diff --git a/tests/unit/src/sqlbuild/cli/commands/shared/_helpers/test_planning_progress.py b/tests/unit/src/sqlbuild/cli/commands/shared/_helpers/test_planning_progress.py index 1d514490b0..4b7457ac7e 100644 --- a/tests/unit/src/sqlbuild/cli/commands/shared/_helpers/test_planning_progress.py +++ b/tests/unit/src/sqlbuild/cli/commands/shared/_helpers/test_planning_progress.py @@ -72,6 +72,11 @@ def test_given_planning_messages_when_reporting_then_writes_expected_output( message="Finalized dbt run.", expected_is_completion=True, ), + PlanningCompletionMessageTestCase( + description="treats evaluated rules as completion", + message="Evaluated rules. (built-in 0.00s, custom 0.00s)", + expected_is_completion=True, + ), ], ids=lambda case: case.description, ) diff --git a/tests/unit/src/sqlbuild/cli/progress/classes/_test_types.py b/tests/unit/src/sqlbuild/cli/progress/classes/_test_types.py index 7b8e70ca21..9d7acc928a 100644 --- a/tests/unit/src/sqlbuild/cli/progress/classes/_test_types.py +++ b/tests/unit/src/sqlbuild/cli/progress/classes/_test_types.py @@ -56,3 +56,10 @@ class StatementMonitorRaceCase: description: str query_id: str expected_submission_count: int + + +@dataclass(frozen=True) +class LockOrderCase: + description: str + timeout_seconds: float + expected_alive_threads: int diff --git a/tests/unit/src/sqlbuild/cli/progress/classes/test_native_progress_projector.py b/tests/unit/src/sqlbuild/cli/progress/classes/test_native_progress_projector.py index a1db1233a7..009ef2b432 100644 --- a/tests/unit/src/sqlbuild/cli/progress/classes/test_native_progress_projector.py +++ b/tests/unit/src/sqlbuild/cli/progress/classes/test_native_progress_projector.py @@ -1,5 +1,6 @@ from __future__ import annotations +import time from dataclasses import replace from io import StringIO from threading import Event, Thread @@ -17,9 +18,11 @@ invocation_scope, run_scope, ) +from sqlbuild.presentation.main.transient_line_coordinator import shared_transient_line_coordinator from sqlbuild.runtime.observability.classes.operation_lifecycle import publish_retry_scheduled from tests.unit.src.sqlbuild.cli.progress.classes._test_types import ( CursorCleanupCase, + LockOrderCase, NativeProjectionCase, RetryProjectionCase, StartFlushCase, @@ -734,3 +737,53 @@ def test_given_many_runtime_schema_events_when_projected_then_output_does_not_gr projector.consume(second_terminal) assert stream.getvalue() == test_case.expected_output + + +@pytest.mark.parametrize( + "test_case", + ( + LockOrderCase( + description="reporter holding the line lock claims a terminal", + timeout_seconds=5.0, + expected_alive_threads=0, + ), + ), + ids=lambda case: case.description, +) +def test_given_line_lock_held_when_projector_writes_concurrently_then_no_deadlock( + test_case: LockOrderCase, +) -> None: + projector: NativeProgressProjector = NativeProgressProjector(stream=StringIO(), use_color=False) + line_lock_held: Event = Event() + + def project_resource() -> None: + dispatcher: EventDispatcher = EventDispatcher() + dispatcher.subscribe_lifecycle(subscriber=projector.consume, accepts_opaque=False) + line_lock_held.wait() + with ( + invocation_scope("inv-lock-order"), + run_scope("run-lock-order"), + dispatcher_scope(dispatcher), + ResourceAttemptLifecycle( + resource_id="task:writer", resource_kind="task", resource_name="writer" + ) as lifecycle, + ): + lifecycle.skipped(skip_code="scheduler") + + def claim_terminal_under_line_lock() -> None: + with shared_transient_line_coordinator().lock: + line_lock_held.set() + time.sleep(0.2) + projector.consume_resource_terminal(resource_name="writer") + + threads: tuple[Thread, ...] = ( + Thread(target=project_resource, daemon=True), + Thread(target=claim_terminal_under_line_lock, daemon=True), + ) + for thread in threads: + thread.start() + for thread in threads: + thread.join(timeout=test_case.timeout_seconds) + + alive: int = sum(thread.is_alive() for thread in threads) + assert alive == test_case.expected_alive_threads diff --git a/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/__init__.py b/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/_test_types.py b/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/_test_types.py new file mode 100644 index 0000000000..58de063742 --- /dev/null +++ b/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/_test_types.py @@ -0,0 +1,19 @@ +from dataclasses import dataclass + +from sqlbuild.compiler.migrations.types import MigrationCompatibility, MigrationDecision + + +@dataclass(frozen=True) +class MigrationDecisionStyleTestCase: + description: str + decision: MigrationDecision + compatibility: MigrationCompatibility + findings: tuple[str, ...] + expected_fragments: tuple[str, ...] + + +@dataclass(frozen=True) +class MigrationPlainTextTestCase: + description: str + decision: MigrationDecision + expected_text: str diff --git a/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/helpers.py b/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/helpers.py new file mode 100644 index 0000000000..50b408132d --- /dev/null +++ b/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/helpers.py @@ -0,0 +1,55 @@ +"""Migration plan display test builders.""" + +from __future__ import annotations + +from sqlbuild.adapter.contract.types import MigrationTransfer +from sqlbuild.compiler.compile.models import CompiledRelationLocation +from sqlbuild.compiler.migrations.types import ( + MigrationCompatibility, + MigrationDecision, + MigrationDiscovery, + MigrationPromotion, +) +from sqlbuild.compiler.planner.main.pre_build.pre_build_work_display import format_pre_build_work +from sqlbuild.compiler.planner.models import ModelMigrationPlanEntry, PlanOutput +from sqlbuild.presentation.models import DisplayOptions + + +def migration_entry( + *, + decision: MigrationDecision, + compatibility: MigrationCompatibility = MigrationCompatibility.COMPATIBLE, + findings: tuple[str, ...] = (), +) -> ModelMigrationPlanEntry: + """Return one daily_revenue migration moving prod.revenue with the given decision.""" + + return ModelMigrationPlanEntry( + model_name="daily_revenue", + discovery=MigrationDiscovery.AUTOMATIC, + decision=decision, + compatibility=compatibility, + origin_model="revenue", + origin=CompiledRelationLocation( + database=None, schema="prod", name="revenue", qualified_name="prod.revenue" + ), + destination=CompiledRelationLocation( + database=None, + schema="prod", + name="daily_revenue", + qualified_name="prod.daily_revenue", + ), + target_name=None, + transfer=MigrationTransfer.COPY, + promotion=MigrationPromotion.TRANSACTIONAL_RENAME, + compatibility_findings=findings, + ) + + +def styled_migration_text(entry: ModelMigrationPlanEntry) -> str: + """Return the colour-styled pre-build text for one migration.""" + + return "\n".join( + format_pre_build_work( + lines=[], plan=PlanOutput(migration_entries=(entry,)), display_options=DisplayOptions() + ) + ) diff --git a/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/test_pre_build_work_display.py b/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/test_pre_build_work_display.py new file mode 100644 index 0000000000..d1a7356104 --- /dev/null +++ b/tests/unit/src/sqlbuild/compiler/planner/main/pre_build/test_pre_build_work_display.py @@ -0,0 +1,150 @@ +"""Migration plan text colour roles and plain rendering.""" + +from __future__ import annotations + +import pytest + +from sqlbuild.cli.output.main.plan import format_plan +from sqlbuild.compiler.migrations.types import MigrationCompatibility, MigrationDecision +from sqlbuild.compiler.planner.models import PlanOutput +from tests.unit.src.sqlbuild.compiler.planner.main.pre_build._test_types import ( + MigrationDecisionStyleTestCase, + MigrationPlainTextTestCase, +) +from tests.unit.src.sqlbuild.compiler.planner.main.pre_build.helpers import ( + migration_entry, + styled_migration_text, +) + +_BOLD_BLUE: str = "\033[34m\033[1m" +_BOLD_YELLOW: str = "\033[33m\033[1m" +_BOLD_RED: str = "\033[38;5;167m\033[1m" +_DIM: str = "\033[2m" +_GREEN: str = "\033[32m" +_RED: str = "\033[38;5;167m" +_RESET: str = "\033[0m" + + +@pytest.mark.parametrize( + "test_case", + ( + MigrationDecisionStyleTestCase( + description="migrate is bold blue with a dim origin arrow and green compatibility", + decision=MigrationDecision.MIGRATE, + compatibility=MigrationCompatibility.COMPATIBLE, + findings=(), + expected_fragments=( + f"\033[1mMigrations{_RESET} {_DIM}(1){_RESET}", + f" daily_revenue {_BOLD_BLUE}migrate{_RESET} " + f"{_DIM}prod.revenue ->{_RESET} prod.daily_revenue", + f" {_DIM}compatibility{_RESET} {_GREEN}compatible{_RESET}", + f" {_DIM}transfer{_RESET} physical copy, promote by transactional rename", + f" {_DIM}discovery{_RESET} automatic", + ), + ), + MigrationDecisionStyleTestCase( + description="redo is bold blue", + decision=MigrationDecision.REDO, + compatibility=MigrationCompatibility.COMPATIBLE, + findings=(), + expected_fragments=(f"{_BOLD_BLUE}redo{_RESET}",), + ), + MigrationDecisionStyleTestCase( + description="done is dim", + decision=MigrationDecision.DONE, + compatibility=MigrationCompatibility.NOT_CHECKED, + findings=(), + expected_fragments=(f"{_DIM}done{_RESET}",), + ), + MigrationDecisionStyleTestCase( + description="superseded replace is bold yellow", + decision=MigrationDecision.SUPERSEDED_REPLACE, + compatibility=MigrationCompatibility.COMPATIBLE, + findings=(), + expected_fragments=(f"{_BOLD_YELLOW}superseded replace{_RESET}",), + ), + MigrationDecisionStyleTestCase( + description="forced replace is bold yellow", + decision=MigrationDecision.FORCED_REPLACE, + compatibility=MigrationCompatibility.COMPATIBLE, + findings=(), + expected_fragments=(f"{_BOLD_YELLOW}forced replace{_RESET}",), + ), + MigrationDecisionStyleTestCase( + description="conflict is bold red with red incompatibility and blocking reason", + decision=MigrationDecision.CONFLICT, + compatibility=MigrationCompatibility.INCOMPATIBLE, + findings=("column amount_cents changed type",), + expected_fragments=( + f"{_BOLD_RED}conflict{_RESET}", + f" {_DIM}compatibility{_RESET} {_RED}incompatible{_RESET}", + f" {_RED}! column amount_cents changed type{_RESET}", + ), + ), + MigrationDecisionStyleTestCase( + description="origin missing is bold red", + decision=MigrationDecision.ORIGIN_MISSING, + compatibility=MigrationCompatibility.NOT_CHECKED, + findings=(), + expected_fragments=(f"{_BOLD_RED}origin missing{_RESET}",), + ), + MigrationDecisionStyleTestCase( + description="renamed hands over identity with dim origin and note", + decision=MigrationDecision.RENAMED, + compatibility=MigrationCompatibility.NOT_CHECKED, + findings=(), + expected_fragments=( + f"\033[1mRenamed{_RESET} {_DIM}(1){_RESET}", + f" daily_revenue {_DIM}prod.revenue ->{_RESET} prod.daily_revenue " + f"{_DIM}(identity handed over){_RESET}", + ), + ), + ), + ids=lambda case: case.description, +) +def test_given_migration_decision_when_formatting_then_applies_colour_roles( + test_case: MigrationDecisionStyleTestCase, +) -> None: + rendered: str = styled_migration_text( + migration_entry( + decision=test_case.decision, + compatibility=test_case.compatibility, + findings=test_case.findings, + ) + ) + + assert all(fragment in rendered for fragment in test_case.expected_fragments), rendered + + +@pytest.mark.parametrize( + "test_case", + ( + MigrationPlainTextTestCase( + description="colour off keeps the migration tree wording unchanged", + decision=MigrationDecision.MIGRATE, + expected_text=( + "Migrations (1)\n" + "└── daily_revenue migrate prod.revenue -> prod.daily_revenue\n" + " ├── compatibility compatible\n" + " ├── transfer physical copy, promote by transactional rename\n" + " └── discovery automatic" + ), + ), + ), + ids=lambda case: case.description, +) +def test_given_colour_off_when_formatting_plan_then_migration_text_is_plain( + test_case: MigrationPlainTextTestCase, +) -> None: + rendered: str = format_plan( + plan=PlanOutput(migration_entries=(migration_entry(decision=test_case.decision),)), + use_color=False, + include_header=False, + ) + + assert rendered.strip("\n") == test_case.expected_text + assert "\033[" not in rendered + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/presentation/classes/_test_types.py b/tests/unit/src/sqlbuild/presentation/classes/_test_types.py index 5c507b74b4..bedaf3ccc2 100644 --- a/tests/unit/src/sqlbuild/presentation/classes/_test_types.py +++ b/tests/unit/src/sqlbuild/presentation/classes/_test_types.py @@ -38,3 +38,10 @@ class ProgressSpinnersDisabledTestCase: description: str env_value: str | None expected_disabled: bool + + +@dataclass(frozen=True) +class TransientLineCoordinatorTestCase: + description: str + persistent_stream_is_tty: bool + expected_events: tuple[str, ...] diff --git a/tests/unit/src/sqlbuild/presentation/classes/helpers.py b/tests/unit/src/sqlbuild/presentation/classes/helpers.py index eba5de6efc..8551ac1c9e 100644 --- a/tests/unit/src/sqlbuild/presentation/classes/helpers.py +++ b/tests/unit/src/sqlbuild/presentation/classes/helpers.py @@ -1,5 +1,6 @@ from __future__ import annotations +import io from collections.abc import Callable from types import MappingProxyType from typing import cast @@ -23,3 +24,32 @@ def apply_no_progress_env(*, monkeypatch: pytest.MonkeyPatch, env_value: str | N _NO_PROGRESS_ENV_ACTIONS: MappingProxyType[ bool, Callable[[pytest.MonkeyPatch, str | None], None] ] = MappingProxyType({True: _delete_no_progress_env, False: _set_no_progress_env}) + + +class RecordingStream(io.StringIO): + """In-memory stream that records writes into a shared event log.""" + + def __init__(self, *, events: list[str], tty: bool) -> None: + super().__init__() + self._events: list[str] = events + self._tty: bool = tty + + def isatty(self) -> bool: + return self._tty + + def write(self, text: str) -> int: + self._events.append(f"write:{text}") + return super().write(text) + + +class RecordingSpinnerOwner: + """Transient line owner that records clear and redraw requests.""" + + def __init__(self, *, events: list[str]) -> None: + self._events: list[str] = events + + def clear_transient_line(self) -> None: + self._events.append("clear") + + def redraw_transient_line(self) -> None: + self._events.append("redraw") diff --git a/tests/unit/src/sqlbuild/presentation/classes/test_transient_line_coordinator.py b/tests/unit/src/sqlbuild/presentation/classes/test_transient_line_coordinator.py new file mode 100644 index 0000000000..30522b4ea5 --- /dev/null +++ b/tests/unit/src/sqlbuild/presentation/classes/test_transient_line_coordinator.py @@ -0,0 +1,76 @@ +from __future__ import annotations + +import pytest + +from sqlbuild.presentation.classes.transient_line_coordinator import TransientLineCoordinator +from tests.unit.src.sqlbuild.presentation.classes._test_types import ( + TransientLineCoordinatorTestCase, +) +from tests.unit.src.sqlbuild.presentation.classes.helpers import ( + RecordingSpinnerOwner, + RecordingStream, +) + + +@pytest.mark.parametrize( + "test_case", + ( + TransientLineCoordinatorTestCase( + description="persistent line on the live terminal clears then redraws the spinner", + persistent_stream_is_tty=True, + expected_events=("clear", "write:Warehouse connected\n", "redraw"), + ), + TransientLineCoordinatorTestCase( + description="redirected output does not disturb the terminal spinner", + persistent_stream_is_tty=False, + expected_events=("write:Warehouse connected\n",), + ), + ), + ids=lambda case: case.description, +) +def test_given_live_spinner_when_writing_persistent_line_then_line_never_joins_spinner( + test_case: TransientLineCoordinatorTestCase, +) -> None: + events: list[str] = [] + coordinator: TransientLineCoordinator = TransientLineCoordinator() + owner: RecordingSpinnerOwner = RecordingSpinnerOwner(events=events) + coordinator.claim(stream=RecordingStream(events=[], tty=True), owner=owner) + + coordinator.write_persistent( + stream=RecordingStream(events=events, tty=test_case.persistent_stream_is_tty), + text="Warehouse connected\n", + ) + + assert tuple(events) == test_case.expected_events + + +@pytest.mark.parametrize( + "test_case", + ( + TransientLineCoordinatorTestCase( + description="released spinner is neither cleared nor redrawn", + persistent_stream_is_tty=True, + expected_events=("write:Warehouse connected\n",), + ), + ), + ids=lambda case: case.description, +) +def test_given_released_spinner_when_writing_persistent_line_then_writes_directly( + test_case: TransientLineCoordinatorTestCase, +) -> None: + events: list[str] = [] + coordinator: TransientLineCoordinator = TransientLineCoordinator() + owner: RecordingSpinnerOwner = RecordingSpinnerOwner(events=events) + coordinator.claim(stream=RecordingStream(events=[], tty=True), owner=owner) + coordinator.release(owner=owner) + + coordinator.write_persistent( + stream=RecordingStream(events=events, tty=test_case.persistent_stream_is_tty), + text="Warehouse connected\n", + ) + + assert tuple(events) == test_case.expected_events + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/presentation/main/_test_types.py b/tests/unit/src/sqlbuild/presentation/main/_test_types.py index c8607939de..635b17ba07 100644 --- a/tests/unit/src/sqlbuild/presentation/main/_test_types.py +++ b/tests/unit/src/sqlbuild/presentation/main/_test_types.py @@ -26,3 +26,20 @@ class InlineErrorLinesTestCase: description: str content_width: int expected_lines: list[str] + + +@dataclass(frozen=True) +class CountHeaderTestCase: + description: str + text: str + use_color: bool + expected_rendered: str + + +@dataclass(frozen=True) +class CountNounTestCase: + description: str + count: int + singular: str + plural: str | None + expected_text: str diff --git a/tests/unit/src/sqlbuild/presentation/main/test_count_header.py b/tests/unit/src/sqlbuild/presentation/main/test_count_header.py new file mode 100644 index 0000000000..4b73b9ace4 --- /dev/null +++ b/tests/unit/src/sqlbuild/presentation/main/test_count_header.py @@ -0,0 +1,51 @@ +from __future__ import annotations + +import pytest + +from sqlbuild.presentation.classes.cli_style import CliStyle +from sqlbuild.presentation.main.count_header import count_header_style +from tests.unit.src.sqlbuild.presentation.main._test_types import CountHeaderTestCase + + +@pytest.mark.parametrize( + "test_case", + ( + CountHeaderTestCase( + description="counted header renders a bold title and a dim count", + text="Models (3)", + use_color=True, + expected_rendered="\033[1mModels\033[0m \033[2m(3)\033[0m", + ), + CountHeaderTestCase( + description="page summary in parentheses is dimmed as the count", + text="Available (2 of 5, 3 collapsed)", + use_color=True, + expected_rendered="\033[1mAvailable\033[0m \033[2m(2 of 5, 3 collapsed)\033[0m", + ), + CountHeaderTestCase( + description="header without a count is only bold", + text="Providers", + use_color=True, + expected_rendered="\033[1mProviders\033[0m", + ), + CountHeaderTestCase( + description="color off keeps the header text unchanged", + text="Migrations (1)", + use_color=False, + expected_rendered="Migrations (1)", + ), + ), + ids=lambda case: case.description, +) +def test_given_header_text_when_styling_then_title_is_bold_and_count_is_dim( + test_case: CountHeaderTestCase, +) -> None: + style: CliStyle = CliStyle(use_color=test_case.use_color) + + rendered: str = count_header_style(style=style, title_style=style.plan_section)(test_case.text) + + assert rendered == test_case.expected_rendered + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/presentation/main/test_count_noun.py b/tests/unit/src/sqlbuild/presentation/main/test_count_noun.py new file mode 100644 index 0000000000..31a49911d3 --- /dev/null +++ b/tests/unit/src/sqlbuild/presentation/main/test_count_noun.py @@ -0,0 +1,32 @@ +from __future__ import annotations + +import pytest + +from sqlbuild.presentation.main.count_noun import format_count_noun +from tests.unit.src.sqlbuild.presentation.main._test_types import CountNounTestCase + + +@pytest.mark.parametrize( + "test_case", + ( + CountNounTestCase("one uses the singular", 1, "relation", None, "1 relation"), + CountNounTestCase("zero uses the plural", 0, "object", None, "0 objects"), + CountNounTestCase("many uses the plural", 3, "state item", None, "3 state items"), + CountNounTestCase("irregular plural for many", 2, "match", "matches", "2 matches"), + CountNounTestCase("irregular noun for one", 1, "child", "children", "1 child"), + ), + ids=lambda case: case.description, +) +def test_given_count_when_formatting_noun_then_uses_singular_only_for_one( + test_case: CountNounTestCase, +) -> None: + assert ( + format_count_noun( + count=test_case.count, singular=test_case.singular, plural=test_case.plural + ) + == test_case.expected_text + ) + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/website/src/content/docs/docs/cli/scope.mdx b/website/src/content/docs/docs/cli/scope.mdx index 3cd67cba3d..82ef6de260 100644 --- a/website/src/content/docs/docs/cli/scope.mdx +++ b/website/src/content/docs/docs/cli/scope.mdx @@ -7,7 +7,9 @@ description: "Inspect declaration visibility, usage, placement, and move impact The command is read-only and offline. It does not connect to a warehouse, require warehouse credentials, or inspect relations. It reads the compiler-owned declaration-scope index and reuses a deterministic cache when the relevant source and configuration fingerprint has not changed. Building a cold cache uses the normal compiler expansion path, which loads the configured adapter and may execute authored Python macros; a warm cache hit reconstructs scope facts without importing macro modules. -Text output is deterministic, bounded, and requires no interactive pager. +Text output is deterministic, bounded, and requires no interactive pager. In a color terminal it +uses the same color conventions as other commands; redirected output and `NO_COLOR` produce plain +text. ## Usage @@ -71,6 +73,7 @@ These identities are for inspection. SQL authored inside the owning model contin | `--page-size N` | Return at most `N` declarations per paged section (default: `100`). | | `--after CURSOR` | Continue after a declaration identity returned as the previous section's next cursor. | | `--paths MODE` | Render paths as `relative`, `compact`, or `none`. | +| `--verbose`, `-v` | Show each declaration's scope details and column, and keep empty optional sections. | | `--json` | Write the canonical versioned JSON report instead of the text tree. | Filters combine in a fixed order: definition path, kind, glob, then usage. `--dependency-depth` @@ -88,11 +91,17 @@ The default text report keeps distinct facts in distinct sections: - **Scope chain** starts with the resource's owner folder, then shows parent folders and the project declaration roles. Its compact labels identify the access rule checked at each path. -Ordinary text rows show each declaration's qualified identity, definition location, visibility, -reason for appearing, declaration-role root, and safe type or signature metadata where useful. -`--explain` additionally shows its owning path, narrowest required placement, current consumers, -dependencies, and the consumers affected by a placement mismatch. JSON includes the complete -structured declaration report. +By default, each text row shows a usage marker, the declaration's qualified identity, and its +`file:line` definition. A one-line legend explains the markers. Empty optional sections +(**Relationship grants**, **Nearby unavailable**, **Private retained**, **Relationship retained**, +and **Diagnostics**) are omitted; the core sections are always shown, even when empty. + +`--verbose` restores the full rows: kind, visibility, reason for appearing, declaration-role root, +safe type or signature metadata where useful, and the definition column. It also shows empty +optional sections. `--explain` always shows the explained declaration in full, plus its owning +path, narrowest required placement, current consumers, dependencies, and the consumers affected by +a placement mismatch. JSON includes the complete structured declaration report regardless of +`--verbose`. ### See the whole scope at once @@ -142,7 +151,7 @@ Scope Path: models/commerce/orders.sql Used (1) - └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:5:1 + └─ ● macro:formatted_order_total models/commerce/macros/orders.py:5 Scope chain ├─ exact-owner-private models/commerce (1) @@ -150,20 +159,22 @@ Scope chain └─ project global (2) Available (1 of 3, 2 collapsed) - └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:5:1 + └─ ● macro:formatted_order_total models/commerce/macros/orders.py:5 … 2 globals collapsed; run sqb scope model:orders --globals all -Relationship grants (0 of 0) - (none) +● used by this resource ○ not used by this resource --verbose for scope details -Nearby unavailable (0 of 0) - (none) - -Diagnostics (0) - (none) Completeness: complete ``` +With `--verbose`, the same `Used` section shows why each declaration is visible: + +```console +$ sqb scope model:orders --verbose +Used (1) + └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:5:1 +``` + This answers several questions without opening declaration files: what the model actually uses, where that declaration came from, which owner and parent folders SQLBuild checks, how many project-wide declarations are available, and whether the result is complete. The scope-chain count @@ -184,9 +195,9 @@ Scope Path: models/commerce/orders.sql Used (3) - ├─ ● macro:add_tax [macro; project; dependency; params 1; role macros] macros/currency.py:1:1 - ├─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:5:1 - └─ ● macro:round_money [macro; project; dependency; params 1; role macros] macros/currency.py:5:1 + ├─ ● macro:add_tax macros/currency.py:1 + ├─ ● macro:formatted_order_total models/commerce/macros/orders.py:5 + └─ ● macro:round_money macros/currency.py:5 ``` The remaining report sections follow the `Used` section as usual. The dependency graph comes from diff --git a/website/src/content/docs/docs/concepts/declaration-scopes/explorer.mdx b/website/src/content/docs/docs/concepts/declaration-scopes/explorer.mdx index 102f93afc1..d8c89944d6 100644 --- a/website/src/content/docs/docs/concepts/declaration-scopes/explorer.mdx +++ b/website/src/content/docs/docs/concepts/declaration-scopes/explorer.mdx @@ -64,8 +64,8 @@ Scope Path: models/commerce/orders.sql Used (2) - ├─ ● constant:minimum_order_value [constant; descendant-public; inherited_ancestor; type integer; role models/commerce/constants] models/commerce/constants/limits.sql:1:1 - └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:4:1 + ├─ ● constant:minimum_order_value models/commerce/constants/limits.sql:1 + └─ ● macro:formatted_order_total models/commerce/macros/orders.py:4 Scope chain ├─ exact-owner-private models/commerce (3) @@ -73,13 +73,13 @@ Scope chain └─ project global (2) Available (3 of 5, 2 collapsed) - ├─ ● constant:minimum_order_value [constant; descendant-public; inherited_ancestor; type integer; role models/commerce/constants] models/commerce/constants/limits.sql:1:1 - ├─ ○ enum:order_status [enum; descendant-public; inherited_ancestor; members 4; type VARCHAR; role models/commerce/enums] models/commerce/enums/status.sql:1:1 - └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:4:1 + ├─ ● constant:minimum_order_value models/commerce/constants/limits.sql:1 + ├─ ○ enum:order_status models/commerce/enums/status.sql:1 + └─ ● macro:formatted_order_total models/commerce/macros/orders.py:4 … 2 globals collapsed; run sqb scope model:orders --globals all -Diagnostics (0) - (none) +● used by this resource ○ not used by this resource --verbose for scope details + Completeness: complete ``` @@ -87,7 +87,9 @@ Completeness: complete dependency when `--dependency-depth` is nonzero. `○` marks a declaration present in the section but not in `Used`. -The compact labels translate to ordinary folder rules: +Add `--verbose` to see each declaration's access details, for example +`[enum; descendant-public; inherited_ancestor; members 4; type VARCHAR; role models/commerce/enums]`. +Those labels translate to ordinary folder rules: | Output label | Meaning in this report | |--------------|------------------------| @@ -111,10 +113,10 @@ declarations. The expanded `Used` section is: ```console $ sqb scope model:orders --used-only --dependency-depth 1 Used (4) - ├─ ● constant:minimum_order_value [constant; descendant-public; inherited_ancestor; type integer; role models/commerce/constants] models/commerce/constants/limits.sql:1:1 - ├─ ● macro:add_tax [macro; project; dependency; params 1; role macros] macros/currency.py:1:1 - ├─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:4:1 - └─ ● macro:round_money [macro; project; dependency; params 1; role macros] macros/currency.py:5:1 + ├─ ● constant:minimum_order_value models/commerce/constants/limits.sql:1 + ├─ ● macro:add_tax macros/currency.py:1 + ├─ ● macro:formatted_order_total models/commerce/macros/orders.py:4 + └─ ● macro:round_money macros/currency.py:5 ``` SQLBuild derives these edges from actual Python calls, including nested calls and calls reached @@ -155,6 +157,8 @@ Explanation Required path: models/commerce Promotion impact: model:orders +● used by this resource ○ not used by this resource --verbose for scope details + Diagnostics (1) ERROR S008 models/commerce/macros/orders.py: Declaration 'macro:formatted_order_total' is currently descendant-public at 'models/commerce' (models/commerce/macros/orders.py); required exact-owner-private at 'models/commerce'. Consumers: model:orders. Move it to 'models/commerce/_sqlbuild/_macros/' Completeness: complete @@ -170,8 +174,8 @@ cannot use it. The relevant section is: ```console $ sqb scope model:orders --include-nearby Nearby unavailable (2 of 2) - ├─ ○ enum:finance_status [enum; descendant-public; sibling_scope; members 3; type VARCHAR; role models/finance/enums] models/finance/enums/status.sql:1:1 - └─ ○ macro:calculate_margin [macro; descendant-public; sibling_scope; params 2; role models/finance/macros] models/finance/macros/margin.py:4:1 + ├─ ○ enum:finance_status models/finance/enums/status.sql:1 + └─ ○ macro:calculate_margin models/finance/macros/margin.py:4 ``` The declarations are known, but they belong to a different folder branch. Here, the inspected @@ -207,13 +211,13 @@ report sections are: ```console $ sqb scope test:orders__completed_only --used-only Used (1) - └─ ● enum:order_status [enum; descendant-public; expected_model through model:orders; members 4; type VARCHAR; role models/commerce/enums] models/commerce/enums/status.sql:1:1 + └─ ● enum:order_status models/commerce/enums/status.sql:1 Relationship grants (1 of 1) - └─ ● enum:order_status [enum; descendant-public; expected_model through model:orders; members 4; type VARCHAR; role models/commerce/enums] models/commerce/enums/status.sql:1:1 + └─ ● enum:order_status models/commerce/enums/status.sql:1 ``` -The compact reason `expected_model through model:orders` means **available through expected output +With `--verbose`, these rows show the reason `expected_model through model:orders`, which means **available through expected output for model `orders`**. With multiple expected models, SQLBuild combines their eligible file-based declarations into one deterministic set. Tests include macros in that union; scenarios include only enums and constants. Neither receives declarations defined inside a model's `MODEL()` header. @@ -233,19 +237,15 @@ Move preview Destination: models/finance/orders.sql Ownership root: models Retained (2) - ├─ ○ macro:add_tax [macro; project; global; params 1; role macros] macros/currency.py:1:1 - └─ ○ macro:round_money [macro; project; global; params 1; role macros] macros/currency.py:5:1 + ├─ ○ macro:add_tax macros/currency.py:1 + └─ ○ macro:round_money macros/currency.py:5 Gained (2) - ├─ ○ enum:finance_status [enum; descendant-public; inherited_ancestor; members 3; type VARCHAR; role models/finance/enums] models/finance/enums/status.sql:1:1 - └─ ○ macro:calculate_margin [macro; descendant-public; inherited_ancestor; params 2; role models/finance/macros] models/finance/macros/margin.py:4:1 + ├─ ○ enum:finance_status models/finance/enums/status.sql:1 + └─ ○ macro:calculate_margin models/finance/macros/margin.py:4 Lost (3) - ├─ ● constant:minimum_order_value [constant; descendant-public; inherited_ancestor; type integer; role models/commerce/constants] models/commerce/constants/limits.sql:1:1 - ├─ ○ enum:order_status [enum; descendant-public; inherited_ancestor; members 4; type VARCHAR; role models/commerce/enums] models/commerce/enums/status.sql:1:1 - └─ ● macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:4:1 - Private retained (0) - (none) - Relationship retained (0) - (none) + ├─ ● constant:minimum_order_value models/commerce/constants/limits.sql:1 + ├─ ○ enum:order_status models/commerce/enums/status.sql:1 + └─ ● macro:formatted_order_total models/commerce/macros/orders.py:4 Invalidated usages (2) - constant:minimum_order_value - macro:formatted_order_total @@ -276,16 +276,12 @@ Scope chain └─ project global (2) Available (3 of 5, 2 collapsed) - ├─ ○ constant:minimum_order_value [constant; descendant-public; inherited_ancestor; type integer; role models/commerce/constants] models/commerce/constants/limits.sql:1:1 - ├─ ○ enum:order_status [enum; descendant-public; inherited_ancestor; members 4; type VARCHAR; role models/commerce/enums] models/commerce/enums/status.sql:1:1 - └─ ○ macro:formatted_order_total [macro; descendant-public; inherited_ancestor; params 1; role models/commerce/macros] models/commerce/macros/orders.py:4:1 + ├─ ○ constant:minimum_order_value models/commerce/constants/limits.sql:1 + ├─ ○ enum:order_status models/commerce/enums/status.sql:1 + └─ ○ macro:formatted_order_total models/commerce/macros/orders.py:4 … 2 globals collapsed; run sqb scope --at models/commerce/returns/new_return.sql --globals all -Relationship grants (0 of 0) - (none) - -Nearby unavailable (0 of 0) - (none) +● used by this resource ○ not used by this resource --verbose for scope details Diagnostics (1) ERROR S013 models/commerce/returns/new_return.sql: Runtime usage and relationship facts are unavailable for a prospective path @@ -316,8 +312,6 @@ Scope folders sqb scope model:orders --browse global/macros sqb scope model:orders --list global/macros -Diagnostics (0) - (none) Completeness: complete ```