Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 74 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
members = [
"crates/sqlbuild-core",
"crates/sqlbuild-sqltext",
"crates/sqlbuild-config",
"crates/sqlbuild-analysis",
"crates/sqlbuild-rules",
"crates/sqlbuild-python",
Expand All @@ -15,6 +16,7 @@ resolver = "2"
order = [
"sqlbuild-core",
"sqlbuild-sqltext",
"sqlbuild-config",
"sqlbuild-analysis",
"sqlbuild-rules",
"sqlbuild-python",
Expand Down Expand Up @@ -53,12 +55,14 @@ polyglot-sql-function-catalogs = { git = "https://github.com/chio-labs/polyglot.
] }
quote = "1"
rayon = "1"
saphyr-parser = "0.1"
regex = "1"
serde = { version = "1", features = ["derive"] }
# Every crate relies on insertion-ordered JSON objects, which the extension has always built with.
serde_json = { version = "1", features = ["preserve_order"] }
sha2 = "0.10"
sqlbuild-analysis = { path = "crates/sqlbuild-analysis" }
sqlbuild-config = { path = "crates/sqlbuild-config" }
sqlbuild-core = { path = "crates/sqlbuild-core" }
sqlbuild-rules = { path = "crates/sqlbuild-rules" }
sqlbuild-sqltext = { path = "crates/sqlbuild-sqltext" }
Expand All @@ -67,6 +71,8 @@ sqlparser = { version = "0.58", features = ["visitor"] }
syn = { version = "2", features = ["full", "visit", "extra-traits"] }
tempfile = "3"
toml = "0.9"
toml_edit = { version = "0.25", default-features = false, features = ["parse"] }
toml_parser = "1"
walkdir = "2"

[workspace.lints.rust]
Expand Down
9 changes: 7 additions & 2 deletions crates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ that can be tested and benchmarked with `cargo test` alone.

| Crate | Responsibility |
|---|---|
| `sqlbuild-core` | Constants shared by several crates and the panic boundary that turns an unwinding native failure into a named compiler error. |
| `sqlbuild-core` | Constants shared by several crates, the panic boundary that turns an unwinding native failure into a named compiler error, passive identity, diagnostic and SQL value types, text positions, and a JSON emitter whose output matches Python's `json.dumps` and `orjson` byte for byte. |
| `sqlbuild-sqltext` | Lexical SQL text without polyglot: comment, quote and parenthesis scanning, the rules quote policy, model header tokenization and matching, static variable substitution and static reference extraction. |
| `sqlbuild-config` | Configuration file reading without Python: `tomllib`-compatible TOML, PyYAML `safe_load`-compatible YAML 1.1, and typed project and local config readers for the fields discovery needs. Any error means "defer to Python", which re-parses for exact messages. |
| `sqlbuild-analysis` | SQL analysis over polyglot: SQL tokens, query analysis, the binding catalog, semantic validation and usage, column references, and SQL-test extraction, planning and rendering. |
| `sqlbuild-rules` | Built-in rules and the rules engine, the custom-rule host, SQL lint, quality checks and formatting, rules configuration and the request models. It also owns the build identity script. |
| `sqlbuild-python` | The only PyO3 crate: the `_native` module, its Python classes and functions, conversions from Python objects, and the process allocator. |
Expand All @@ -18,9 +19,13 @@ Dependencies point one way, from the top of this graph to the bottom:

```text
sqlbuild-python
-> sqlbuild-rules -> sqlbuild-analysis -> sqlbuild-sqltext -> sqlbuild-core
-> sqlbuild-rules -> sqlbuild-analysis -> sqlbuild-config -> sqlbuild-sqltext -> sqlbuild-core
```

`sqlbuild-config` does not depend on the crates below it today; its place in the order only
fixes which crates may use it. The JSON emitter lives in `sqlbuild-core` so that any layer can
produce text that must equal Python's `json.dumps` output.

Each crate may depend only on crates below it, and may also skip layers. The order is declared
in `[workspace.metadata.native-layers]` in the workspace `Cargo.toml`, and
`make check-native-layering` (part of `make check-ci`) fails when:
Expand Down
19 changes: 19 additions & 0 deletions crates/sqlbuild-config/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
[package]
name = "sqlbuild-config"
version.workspace = true
description = "Project configuration reading for SQLBuild: TOML and PyYAML-compatible YAML."
edition.workspace = true
license.workspace = true
publish.workspace = true
rust-version.workspace = true

[dependencies]
saphyr-parser.workspace = true
toml_edit.workspace = true
toml_parser.workspace = true

[dev-dependencies]
tempfile.workspace = true

[lints]
workspace = true
11 changes: 11 additions & 0 deletions crates/sqlbuild-config/src/constants.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
//! Calendar limits of Python's `datetime` types and characters shared by the readers.

pub(crate) const MIN_YEAR: i64 = 1;
pub(crate) const MAX_YEAR: i64 = 9999;
pub(crate) const MONTHS_PER_YEAR: i64 = 12;
pub(crate) const FEBRUARY: i64 = 2;
pub(crate) const MAX_HOUR: i64 = 23;
pub(crate) const MAX_MINUTE: i64 = 59;
pub(crate) const MAX_SECOND: i64 = 59;
pub(crate) const MAX_MICROSECOND: i64 = 999_999;
pub(crate) const BYTE_ORDER_MARK: char = '\u{feff}';
37 changes: 37 additions & 0 deletions crates/sqlbuild-config/src/errors.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
//! Structured failures of reading configuration files.

/// Why a configuration document could not be loaded natively.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum ConfigErrorKind {
/// The text is not a valid document; Python's parser rejects it too.
Syntax,
/// The document parses but a value cannot be built, as PyYAML's constructor or `tomllib` reports.
Construct,
/// The document uses a feature the native reader leaves to Python.
Unsupported,
/// A required file does not exist.
Missing,
/// A field SQLBuild reads has the wrong type or an invalid value.
InvalidField,
}

/// A configuration failure; the Python reader re-parses the file to report its exact message.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ConfigError {
pub kind: ConfigErrorKind,
pub message: String,
pub line: Option<usize>,
pub column: Option<usize>,
}

impl ConfigError {
/// An error without a position.
pub fn new(kind: ConfigErrorKind, message: impl Into<String>) -> Self {
Self {
kind,
message: message.into(),
line: None,
column: None,
}
}
}
10 changes: 10 additions & 0 deletions crates/sqlbuild-config/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
//! Project configuration reading: `tomllib`-compatible TOML and PyYAML-compatible YAML.

#![forbid(unsafe_code)]

pub(crate) mod constants;
pub mod errors;
pub mod models;
pub mod project;
pub mod toml;
pub mod yaml;
101 changes: 101 additions & 0 deletions crates/sqlbuild-config/src/models.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
//! Configuration values with the value domain of Python's `tomllib` and PyYAML `safe_load`.

use crate::constants::{
FEBRUARY, MAX_HOUR, MAX_MICROSECOND, MAX_MINUTE, MAX_SECOND, MAX_YEAR, MIN_YEAR,
MONTHS_PER_YEAR,
};

/// A calendar date, Python's `datetime.date`.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub struct ConfigDate {
pub year: u16,
pub month: u8,
pub day: u8,
}

/// A time of day with microseconds, Python's naive `datetime.time`.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub struct ConfigTime {
pub hour: u8,
pub minute: u8,
pub second: u8,
pub microsecond: u32,
}

/// A date and time, Python's `datetime.datetime`, with its UTC offset in seconds when aware.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub struct ConfigDateTime {
pub date: ConfigDate,
pub time: ConfigTime,
pub utc_offset_seconds: Option<i32>,
}

impl ConfigDate {
/// The date, or `None` where Python's `datetime.date` constructor raises.
pub fn checked(year: i64, month: i64, day: i64) -> Option<Self> {
let leap = year % 4 == 0 && (year % 100 != 0 || year % 400 == 0);
let month_days = match month {
FEBRUARY => 28 + i64::from(leap),
4 | 6 | 9 | 11 => 30,
_ => 31,
};
let valid = (MIN_YEAR..=MAX_YEAR).contains(&year)
&& (1..=MONTHS_PER_YEAR).contains(&month)
&& (1..=month_days).contains(&day);
valid.then_some(Self {
year: year as u16,
month: month as u8,
day: day as u8,
})
}
}

impl ConfigTime {
/// The time, or `None` where Python's `datetime.time` constructor raises.
pub fn checked(hour: i64, minute: i64, second: i64, microsecond: i64) -> Option<Self> {
let valid = (0..=MAX_HOUR).contains(&hour)
&& (0..=MAX_MINUTE).contains(&minute)
&& (0..=MAX_SECOND).contains(&second)
&& (0..=MAX_MICROSECOND).contains(&microsecond);
valid.then_some(Self {
hour: hour as u8,
minute: minute as u8,
second: second as u8,
microsecond: microsecond as u32,
})
}
}

/// One loaded configuration value; mappings keep document order and Python key equality.
#[derive(Clone, Debug, PartialEq)]
pub enum ConfigValue {
Null,
Bool(bool),
Integer(i64),
BigInteger(String),
Float(f64),
String(String),
Date(ConfigDate),
DateTime(ConfigDateTime),
Time(ConfigTime),
List(Vec<ConfigValue>),
Map(Vec<(ConfigValue, ConfigValue)>),
}

impl ConfigValue {
/// The entries of a mapping, or `None` for any other value.
pub fn as_map(&self) -> Option<&[(ConfigValue, ConfigValue)]> {
match self {
Self::Map(entries) => Some(entries),
_ => None,
}
}

/// The value of a string key in a mapping, as Python's `mapping.get(key)` finds it.
pub fn get(&self, key: &str) -> Option<&ConfigValue> {
self.as_map()?
.iter()
.find(|(candidate, _)| matches!(candidate, Self::String(text) if text == key))
.map(|(_, value)| value)
}
}
Loading
Loading