Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ The playground runs on local DuckDB, with no warehouse credentials.
update every reference, and the next build
[migrates](https://sqlbuild.com/docs/concepts/models/migrations/) the existing table instead of
rebuilding it. The old name keeps working through a compatibility view.
`sqb rename column:<model>.<column>` renames an incremental model's column in place.
`sqb rename <model>.<column>` renames an incremental model's column in place.
- **Replay on change.** When a model's SQL changes, choose how far back to reprocess, from only the
new data to the last 14 days to a full rebuild, with
[`replay_on_change`](https://sqlbuild.com/docs/concepts/incremental/#replay-on-change).
Expand Down
2 changes: 1 addition & 1 deletion src/sqlbuild/.agents/skills/sqlbuild/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ the trigger applies.
| Know what a model can see: which macros, enums and constants are visible, used, or unavailable | `sqb scope model:<name>` |
| Fix "unknown macro/enum/constant", or decide where a declaration should live | `sqb scope model:<name> --explain macro:<name>` |
| Preview whether moving a file breaks declaration visibility | `sqb scope model:<name> --as-path <new/path.sql>` |
| Rename a model, move a model file, or rename a column | `sqb rename model:<old> <new>`, `sqb mv model:<name> <path/or/folder/>`, `sqb rename column:<model>.<column> <new>` |
| Rename a model, move a model file, or rename a column | `sqb rename <old> <new>`, `sqb mv <name> <path/or/folder/>`, `sqb rename <model>.<column> <new>` (`model:`/`column:` prefixes optional) |
| Find what a model depends on, or what breaks if it changes | `sqb lineage <model> --direction both` |
| Trace where a column comes from, or who consumes it before renaming or dropping it | `sqb lineage <model>.<column> --direction downstream` |
| See which intermediate models a SQL test will really execute | `sqb test --select <model> --inspect` |
Expand Down
18 changes: 10 additions & 8 deletions src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/rename.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,19 +24,21 @@ Nothing is written until the edited project compiles. The command applies its ed
## Usage

```bash
sqb rename model:<name> <new_name> [flags]
sqb mv model:<name> <path/to/new_file.sql | folder/> [flags]
sqb rename column:<model>.<column> <new_name> [--cascade] [flags]
sqb rename <model> <new_name> [flags]
sqb mv <model> <path/to/new_file.sql | folder/> [flags]
sqb rename <model>.<column> <new_name> [--cascade] [flags]
```

```bash
sqb rename model:stg_orders stg_order_lines
sqb mv model:fact_orders models/reporting/ # keeps the name
sqb mv model:fact_orders models/reporting/order_facts.sql # moves and renames
sqb rename column:stg_orders.amount revenue
sqb rename column:stg_orders.amount revenue --cascade
sqb rename stg_orders stg_order_lines
sqb mv fact_orders models/reporting/ # keeps the name
sqb mv fact_orders models/reporting/order_facts.sql # moves and renames
sqb rename stg_orders.amount revenue
sqb rename stg_orders.amount revenue --cascade
```

Targets are written the way [`sqb lineage`](lineage.md) reads them: a bare name is a model, and `<model>.<column>` is one of its columns. The `model:` and `column:` prefixes are optional, so `sqb rename model:stg_orders stg_order_lines` and `sqb rename column:stg_orders.amount revenue` still work. A target that names a source, seed, or function is refused, because those are not renamed by these commands.

A model rename keeps the file in its folder and renames it to `<new_name>.sql`. `sqb mv` takes a new file path, which also sets the model name, or a folder ending in `/`, which keeps it.

## What changes
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Column migrations apply to incremental models, including microbatch models, and

## Declaring a rename

[`sqb rename column:<model>.<column>`](../../cli/rename.md) renames the column in the model and every query that reads it, and adds the `migrate_from` declaration below for you.
[`sqb rename <model>.<column>`](../../cli/rename.md) renames the column in the model and every query that reads it, and adds the `migrate_from` declaration below for you.

Add `migrate_from` to the renamed column in the model header, naming the old column:

Expand Down
8 changes: 6 additions & 2 deletions src/sqlbuild/cli/commands/_helpers/entry/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,9 @@ def _add_refactor_parsers(
CliCommand.RENAME, help="rename a model or column and update every reference"
)
rename_parser.add_argument(
"refactor_target", metavar="TARGET", help="model:<name> or column:<model>.<column>"
"refactor_target",
metavar="TARGET",
help="<model> or <model>.<column>; the model: and column: prefixes are optional",
)
rename_parser.add_argument("refactor_new_name", metavar="NEW_NAME")
rename_parser.add_argument(
Expand All @@ -166,7 +168,9 @@ def _add_refactor_parsers(
move_parser: argparse.ArgumentParser = subparsers.add_parser(
CliCommand.MV, help="move a model file, optionally renaming it, and update references"
)
move_parser.add_argument("refactor_target", metavar="TARGET", help="model:<name>")
move_parser.add_argument(
"refactor_target", metavar="TARGET", help="<model>; the model: prefix is optional"
)
move_parser.add_argument(
"refactor_destination", metavar="DESTINATION", help="new .sql path or folder/"
)
Expand Down
40 changes: 34 additions & 6 deletions src/sqlbuild/cli/commands/_helpers/lineage/selection.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

from sqlbuild.cli.commands.constants import (
BOTH_DIRECTIONS,
COLUMN_TARGET_KIND,
COLUMN_TARGET_SEPARATOR,
DOWNSTREAM_DIRECTION,
PATH_BETWEEN_MARKER,
Expand All @@ -25,6 +26,7 @@
LineageGraph,
LineageNode,
LineageSelectionAnchors,
LineageTarget,
ParsedLineagePathSelector,
ParsedLineageSelector,
RelationLineageIndex,
Expand Down Expand Up @@ -64,6 +66,7 @@ class _ColumnLineageCandidateSelection:
str(CompiledResourceType.SEED),
str(CompiledResourceType.UDF),
str(CompiledResourceType.TABLE_FN),
COLUMN_TARGET_KIND,
}
)

Expand All @@ -85,14 +88,39 @@ def parse_depth(raw_depth: str) -> int | None:
def normalize_lineage_target(*, graph: ProjectGraph | RelationLineageIndex, target: str) -> str:
"""Strip a printed kind prefix such as `model:` after checking it matches the resource."""

kind, separator, remainder = target.partition(_KIND_SEPARATOR)
if not separator or kind not in _TARGET_KINDS:
resolved: LineageTarget = resolve_lineage_target(all_keys=graph.all_keys, target=target)
if resolved.kind is None:
return target
name: str = remainder.split(COLUMN_TARGET_SEPARATOR, maxsplit=1)[0]
key: CompiledObjectKey | None = graph.all_keys.get(name)
if key is None or str(key.resource_type) != kind:
if not resolved.kind_matches:
raise _unknown_target_error(target)
return remainder
return resolved.name


def resolve_lineage_target(*, all_keys: dict[str, CompiledObjectKey], target: str) -> LineageTarget:
"""Resolve `[kind:]<name>` or `[kind:]<model>.<column>` against project resource names."""

kind, separator, remainder = target.partition(_KIND_SEPARATOR)
written_kind: str | None = kind if separator and kind in _TARGET_KINDS else None
name: str = remainder if written_kind is not None else target
resource_name, column_separator, column_name = name.partition(COLUMN_TARGET_SEPARATOR)
key: CompiledObjectKey | None = all_keys.get(resource_name)
column: str | None = column_name if column_separator else None
kind_matches: bool
if written_kind is None:
kind_matches = True
elif written_kind == COLUMN_TARGET_KIND:
kind_matches = (
key is not None and key.resource_type == CompiledResourceType.MODEL and bool(column)
)
else:
kind_matches = key is not None and str(key.resource_type) == written_kind
return LineageTarget(
kind=written_kind,
name=name,
key=key,
column_name=column,
kind_matches=kind_matches,
)


def select_target_lineage(
Expand Down
55 changes: 4 additions & 51 deletions src/sqlbuild/cli/commands/_helpers/refactor/command.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,10 @@
render_refactor_json,
render_refactor_text,
)
from sqlbuild.cli.commands._helpers.refactor.target import refactor_request_for_target
from sqlbuild.cli.commands.exceptions import CliUserError
from sqlbuild.cli.commands.models import RefactorCommandRequest, RefactorCompile
from sqlbuild.cli.commands.types import CliCommand
from sqlbuild.compiler.compile.models import CompilerDiagnostic
from sqlbuild.compiler.refactoring.constants import (
COLUMN_KIND_PREFIX,
COLUMN_TARGET_SEPARATOR,
MODEL_KIND_PREFIX,
)
from sqlbuild.compiler.refactoring.exceptions import RefactorInputError
from sqlbuild.compiler.refactoring.main.commit_refactor_plan import commit_refactor_plan
from sqlbuild.compiler.refactoring.main.plan_column_rename import plan_column_rename
Expand Down Expand Up @@ -69,7 +64,9 @@ def _run(
_write_errors(errors=before.errors)
return 1
status.complete(message=f"Compiled project. ({time.monotonic() - started:.2f}s)")
refactor_request: RefactorRequest = _refactor_request(request=request)
refactor_request: RefactorRequest = refactor_request_for_target(
request=request, all_keys=before.project.graph.all_keys
)
status.start("Planning edits...")
plan: RefactorPlan = (
plan_column_rename(project=before.project, request=refactor_request)
Expand Down Expand Up @@ -171,50 +168,6 @@ def _files(plan: RefactorPlan) -> str:
return format_count_noun(count=len(plan.changes), singular="file")


def _refactor_request(*, request: RefactorCommandRequest) -> RefactorRequest:
target: str = request.target.strip()
if request.command == CliCommand.MV:
if not target.startswith(MODEL_KIND_PREFIX):
raise CliUserError(
f"sqb mv moves models; got '{target}'",
code="C954",
help="write the target as model:<name>",
)
return RefactorRequest(
operation=RefactorOperation.MOVE_MODEL,
model_name=target.removeprefix(MODEL_KIND_PREFIX),
new_name="",
destination=request.destination or "",
)
new_name: str = (request.new_name or "").strip()
if target.startswith(MODEL_KIND_PREFIX):
if request.cascade:
raise CliUserError("--cascade applies to column renames only", code="C954")
return RefactorRequest(
operation=RefactorOperation.RENAME_MODEL,
model_name=target.removeprefix(MODEL_KIND_PREFIX),
new_name=new_name,
)
if target.startswith(COLUMN_KIND_PREFIX) and COLUMN_TARGET_SEPARATOR in target:
model_name: str
column_name: str
model_name, column_name = target.removeprefix(COLUMN_KIND_PREFIX).rsplit(
COLUMN_TARGET_SEPARATOR, 1
)
return RefactorRequest(
operation=RefactorOperation.RENAME_COLUMN,
model_name=model_name,
column_name=column_name,
new_name=new_name,
cascade=request.cascade,
)
raise CliUserError(
f"cannot rename '{target}'",
code="C954",
help="write the target as model:<name> or column:<model>.<column>",
)


def _relative(
*, errors: tuple[CompilerDiagnostic, ...], staging_dir: Path
) -> tuple[CompilerDiagnostic, ...]:
Expand Down
83 changes: 83 additions & 0 deletions src/sqlbuild/cli/commands/_helpers/refactor/target.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
"""Read the target of `sqb rename` and `sqb mv` the way `sqb lineage` reads targets."""

from __future__ import annotations

from sqlbuild.cli.commands._helpers.lineage.selection import resolve_lineage_target
from sqlbuild.cli.commands.constants import COLUMN_TARGET_KIND, COLUMN_TARGET_SEPARATOR
from sqlbuild.cli.commands.exceptions import CliUserError
from sqlbuild.cli.commands.models import LineageTarget, RefactorCommandRequest
from sqlbuild.cli.commands.types import CliCommand
from sqlbuild.compiler.compile.models import CompiledObjectKey
from sqlbuild.compiler.compile.types import CompiledResourceType
from sqlbuild.compiler.refactoring.models import RefactorRequest
from sqlbuild.compiler.refactoring.types import RefactorOperation

_RENAME_USAGE: str = (
"write the target as <model> or <model>.<column>; the model: and column: prefixes are optional"
)
_MOVE_USAGE: str = "write the target as <model>; the model: prefix is optional"
_RENAME_SCOPE: str = "sqb rename renames models and model columns"
_MOVE_SCOPE: str = "sqb mv moves models"


def refactor_request_for_target(
*, request: RefactorCommandRequest, all_keys: dict[str, CompiledObjectKey]
) -> RefactorRequest:
"""Read a `sqb rename` or `sqb mv` target the way `sqb lineage` reads its targets."""

raw: str = request.target.strip()
target: LineageTarget = resolve_lineage_target(all_keys=all_keys, target=raw)
is_move: bool = request.command == CliCommand.MV
_check_target(raw=raw, target=target, is_move=is_move)
model_name: str = target.name.partition(COLUMN_TARGET_SEPARATOR)[0]
if is_move:
return RefactorRequest(
operation=RefactorOperation.MOVE_MODEL,
model_name=model_name,
new_name="",
destination=request.destination or "",
)
new_name: str = (request.new_name or "").strip()
if target.column_name is None:
if request.cascade:
raise CliUserError("--cascade applies to column renames only", code="C954")
return RefactorRequest(
operation=RefactorOperation.RENAME_MODEL,
model_name=model_name,
new_name=new_name,
)
return RefactorRequest(
operation=RefactorOperation.RENAME_COLUMN,
model_name=model_name,
column_name=target.column_name,
new_name=new_name,
cascade=request.cascade,
)


def _check_target(*, raw: str, target: LineageTarget, is_move: bool) -> None:
verb: str = "move" if is_move else "rename"
usage: str = _MOVE_USAGE if is_move else _RENAME_USAGE
key: CompiledObjectKey | None = target.key
if not target.name or (target.column_name is not None and not target.column_name):
raise CliUserError(f"cannot {verb} '{raw}'", code="C954", help=usage)
if key is not None and key.resource_type != CompiledResourceType.MODEL:
raise CliUserError(
f"cannot {verb} '{raw}': {key.name} is a {key.resource_type}; "
f"{_MOVE_SCOPE if is_move else _RENAME_SCOPE}",
code="C954",
help=usage,
)
if key is not None and not target.kind_matches:
found: str = (
f"column: needs <model>.<column>, and {key.name} names a model"
if target.kind == COLUMN_TARGET_KIND
else f"{key.name} is a {key.resource_type}, not a {target.kind}"
)
raise CliUserError(f"cannot {verb} '{raw}': {found}", code="C954", help=usage)
if is_move and target.column_name is not None:
raise CliUserError(
f"sqb mv moves models; '{raw}' names a column",
code="C954",
help=f"rename a column with sqb rename {target.name} <new_name>",
)
1 change: 1 addition & 0 deletions src/sqlbuild/cli/commands/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
PATH_SEPARATOR: str = "/"
SELECTOR_EXPANSION_MARKER: str = "+"
COLUMN_TARGET_SEPARATOR: str = "."
COLUMN_TARGET_KIND: str = "column"
SUPPORTED_TYPED_SELECTOR_KINDS: frozenset[str] = frozenset(
{
SelectorKind.SEED,
Expand Down
11 changes: 11 additions & 0 deletions src/sqlbuild/cli/commands/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -764,6 +764,17 @@ class LineageSelectionAnchors:
retained: frozenset[CompiledObjectKey] = field(default_factory=frozenset)


@dataclass(frozen=True)
class LineageTarget:
"""One positional target, `[kind:]<name>` or `[kind:]<model>.<column>`, resolved by name."""

kind: str | None
name: str
key: CompiledObjectKey | None
column_name: str | None
kind_matches: bool


@dataclass(frozen=True)
class ParsedLineageSelector:
"""One parsed non-path lineage selector."""
Expand Down
2 changes: 1 addition & 1 deletion src/sqlbuild/compiler/planner/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@
SOURCE_INPUT_FUNCTION: str = "__source"
REF_INPUT_FUNCTION: str = "__ref"
MANUAL_RENAME_HINT: str = (
"matched by unchanged definition; sqb rename model:<old> <new> rewrites references and "
"matched by unchanged definition; sqb rename <old> <new> rewrites references and "
"declares migrate_from so the move does not depend on discovery"
)
HIDDEN_ORIGIN_REMEDY: str = (
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,12 @@


def find_model(*, project: CompiledProject, name: str) -> CompiledModel:
"""Return the compiled model a kind-qualified target names."""
"""Return the compiled model a target names."""

model: CompiledModel | None = next((item for item in project.models if item.name == name), None)
if model is None:
raise RefactorInputError(
f"{MODEL_KIND_PREFIX}{name} is not a model in this project",
f"no model named '{name}' in this project",
code="C951",
help="list models with sqb dag --json or sqb scope --browse models",
)
Expand Down
1 change: 0 additions & 1 deletion src/sqlbuild/compiler/refactoring/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@
)

MODEL_KIND_PREFIX: str = "model:"
COLUMN_KIND_PREFIX: str = "column:"
SQL_SUFFIX: str = ".sql"
PATH_SEPARATOR: str = "/"
IDENTIFIER_PATTERN: re.Pattern[str] = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,15 @@ class RefactorStatusMessagesTestCase:
new_name: str | None
destination: str | None
expected_lines: tuple[str, ...]


@dataclass(frozen=True)
class BareTargetRefactorTestCase:
description: str
command: str
target: str
new_name: str | None
destination: str | None
expected_path: str
expected_fragment: str
expected_missing_paths: tuple[str, ...]
Loading
Loading