diff --git a/CHANGELOG.md b/CHANGELOG.md index 1930387..6f139c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ All notable changes to plx are recorded here. The format follows [Keep a Changelog](https://keepachangelog.com/), and plx uses the extension -version in `plx.control` (currently `1.0`). +version in `plx.control` (currently `2.0.0`). ## [2.0.1] - 2026-08-25 @@ -90,6 +90,10 @@ Each result interpolates a value that will become NULL when the value is NULL. Where the old rendering was wanted, make it explicit with `coalesce(x, '')` in the body. +The same applies to a dump taken from an earlier release and restored into +2.0.0: the restored functions keep the plpgsql stored in the dump, including +the empty-string rendering, until their DDL is run again. + ## [1.3.2] - 2026-08-24 Code-only patch release (no catalog changes) carrying a plxgo fix for diff --git a/README.md b/README.md index 59a2762..978d5c2 100644 --- a/README.md +++ b/README.md @@ -106,10 +106,10 @@ is plpgsql. See [doc/ARCHITECTURE.md](doc/ARCHITECTURE.md) and ## Example -One function, written in three dialects, each producing the same plpgsql: +One function, written in three dialects, each producing equivalent plpgsql: ```sql -CREATE FUNCTION grade(score int) RETURNS text LANGUAGE plxruby AS $$ +CREATE FUNCTION grade_rb(score int) RETURNS text LANGUAGE plxruby AS $$ grade #:: text if score >= 90 grade = "A" @@ -121,14 +121,14 @@ CREATE FUNCTION grade(score int) RETURNS text LANGUAGE plxruby AS $$ return grade $$; -CREATE FUNCTION grade(score int) RETURNS text LANGUAGE plxphp AS $$ +CREATE FUNCTION grade_php(score int) RETURNS text LANGUAGE plxphp AS $$ if ($score >= 90) { $grade = "A"; } elseif ($score >= 80) { $grade = "B"; } else { $grade = "F"; } return $grade; $$; -CREATE FUNCTION grade(score int) RETURNS text LANGUAGE plxjs AS $$ +CREATE FUNCTION grade_js(score int) RETURNS text LANGUAGE plxjs AS $$ let grade = "F"; if (score >= 90) { grade = "A"; } else if (score >= 80) { grade = "B"; } @@ -137,26 +137,41 @@ CREATE FUNCTION grade(score int) RETURNS text LANGUAGE plxjs AS $$ $$; ``` -The stored plpgsql (in `pg_proc.prosrc`) for each is: +The stored plpgsql (in `pg_proc.prosrc`) for the plxruby and plxphp versions is +shown below. The plxjs version is the same except that it initializes `grade` +in its `DECLARE` (`grade text := 'F';`). ```plpgsql +/*plx:v1:plxruby:b04416a1*/ DECLARE grade text; BEGIN - IF score >= 90 THEN grade := 'A'; - ELSIF score >= 80 THEN grade := 'B'; - ELSE grade := 'F'; + IF score >= 90 THEN + grade := 'A'; + ELSIF score >= 80 THEN + grade := 'B'; + ELSE + grade := 'F'; END IF; RETURN grade; END; +/*plx-orig:b64$...$plx-orig*/ ``` +The first line is a sentinel naming the dialect, and the last line is a +comment carrying your original source in base64, so it survives dump and +restore and can be recovered (see [doc/DEBUGGING.md](doc/DEBUGGING.md)). +Everything between them is the plpgsql that runs. + ## Performance -Because functions execute as plpgsql, the plx dialects match plpgsql (within about -11 percent across five workloads) and inherit its performance profile: several -times faster than the embedded-interpreter PLs on row iteration, and competitive -on arithmetic, branching, and call overhead. +Because functions execute as plpgsql, the plx dialects match plpgsql within +about 10 percent on most workloads (the worst cases measured are plxruby row +iteration at 1.17x and plxcobol's `EVALUATE` branch at 1.32x) and inherit its +performance profile: several times faster than the embedded-interpreter PLs on +row iteration, fastest on call overhead, and between the embedded PLs on +arithmetic. On a pure-CPU branching loop the embedded PLs are faster (native +PL/PHP runs it in 0.40x of plpgsql's time). The one place plpgsql is weak is building a string in a loop: `s := s || 'x'` is O(n^2), because text is immutable and each step copies the whole string. plx @@ -185,12 +200,23 @@ make PG_CONFIG=/path/to/pg_config make install PG_CONFIG=/path/to/pg_config ``` -Then, in a database: +Then, in a database (installing the extension requires a superuser): ```sql CREATE EXTENSION plx; ``` +To upgrade, install the new build and run, in each database: + +```sql +ALTER EXTENSION plx UPDATE; +``` + +Upgrading changes no existing function. plx transpiles at `CREATE FUNCTION` +time and stores the result, so a function keeps the plpgsql it was created with +until its DDL is run again. See [CHANGELOG.md](CHANGELOG.md) for the notes on +each release. + ## Documentation - Per-dialect chapters: [plxruby](doc/plxruby.md), [plxphp](doc/plxphp.md), @@ -218,7 +244,8 @@ CREATE EXTENSION plx; ## Layout ``` -plx.control, plx--1.0.sql extension control and install SQL +plx.control, plx--2.0.0.sql extension control and install SQL +plx--X--Y.sql upgrade scripts between releases Makefile PGXS build src/ C sources (transpiler, dialect surfaces, PL handler) doc/ per-dialect chapters, architecture, transpiler, parity @@ -231,11 +258,17 @@ bench/ benchmark harness and results ``` make installcheck # pg_regress suite +make differentialcheck # each dialect against a plpgsql reference +make metacheck # META.json against the PGXN spec (no server) python3 test/run_corpus.py # additional Ruby corpus runner +python3 test/fuzz.py # transpiler fuzzer ``` -`make installcheck` runs against an installed plx on a running server. The -expected output is in `test/expected/`. +`make installcheck` and `make differentialcheck` run against an installed plx on +a running server; run `differentialcheck` after `installcheck`, which creates +the `contrib_regression` database it uses. The expected output is in +`test/expected/`. CI runs installcheck, differentialcheck, and metacheck on +PostgreSQL 13 through 18. ## License diff --git a/bench/BENCHMARKS.md b/bench/BENCHMARKS.md index 841fa2e..de74d57 100644 --- a/bench/BENCHMARKS.md +++ b/bench/BENCHMARKS.md @@ -28,7 +28,8 @@ plx uses plx-prefixed language names, so it coexists with the native plruby, plphp, plperl, and plpython3u. plxruby, plxphp, plxjs, plxpython3, and plxcobol transpile to plpgsql at -`CREATE FUNCTION` time, so at run time they execute as plpgsql. plperl and +`CREATE FUNCTION` time, so at run time they execute as plpgsql. plxts, plxgo, +plxplsql, and plxtsql were not benchmarked; they also execute as plpgsql. plperl and plpython3u run their own embedded interpreters and retrieve rows through SPI. ## Results @@ -56,7 +57,7 @@ In the strbuild column, native plpgsql is the `s := s || 'x'` baseline and the plx dialects use the builder. This is the intended native-versus-plx comparison: the same accumulation idiom is 1743 ms in stock plpgsql and about 10 ms in the plx dialects. The plx builder is also faster than the native PLs' own in-language -append on this workload (native plruby 53 ms, native plphp 14 ms), because it +append on this workload (native plruby 53 ms, native plphp 15 ms), because it mutates one expanded-object buffer in place rather than materializing an interpreter string per step. These numbers are on PostgreSQL 18; the builder's amortized-O(1) append requires PostgreSQL 18 (see the version note under String @@ -65,9 +66,9 @@ building). On PostgreSQL 13 to 17 the plx strbuild column would match plpgsql ## Analysis -- The plx dialects match plpgsql within about 10 percent on the arith, iter, and - call workloads, because the stored function body is plpgsql. There is no - run-time translation cost; the translation happens once at `CREATE FUNCTION`. +- The plx dialects match plpgsql within about 10 percent on the arith and call + workloads, and within 17 percent on iter, because the stored function body is + plpgsql. There is no run-time translation cost; the translation happens once at `CREATE FUNCTION`. The small spread among the dialects is measurement noise. - plxcobol (branch): the one place plxcobol trails is the four-way branch @@ -78,7 +79,7 @@ building). On PostgreSQL 13 to 17 the plx strbuild column would match plpgsql counting loop lowers to an integer `FOR`, and `STRING-APPEND` lowers to the string builder). -- Row iteration (iter): plpgsql, and therefore the plx dialects, are 2.8x to 6.0x +- Row iteration (iter): plpgsql, and therefore the plx dialects, are 2.7x to 5.6x faster than the embedded PLs (plperl, plpython3u, native plruby, native plphp). plpgsql streams rows through a cursor and reads columns directly, while the embedded interpreters copy each row into their own data structures. @@ -91,21 +92,23 @@ building). On PostgreSQL 13 to 17 the plx strbuild column would match plpgsql amortized O(1), and the transpiler lowers the dialect append operators (`<<`, `.=`, `+=` on a string) onto it. The result is about 170x faster than the stock plpgsql idiom, and faster than the native PLs building the same - string with their own append (native plruby 53 ms, native plphp 14 ms), which + string with their own append (native plruby 53 ms, native plphp 15 ms), which each materialize a fresh interpreter string per step. In hand-written plpgsql the same win is available by calling `plx_sb_append`, or by assembling text in SQL with `string_agg` over a set. -- Arithmetic (arith): the embedded interpreters vary. plperl is fastest (native - integer arithmetic), plpython3u is slowest, and plpgsql with the plx dialects - sits between. +- Arithmetic (arith): the embedded interpreters vary. plperl is fastest (0.79x, + native integer arithmetic), native plruby is slowest (1.79x) followed by + plpython3u (1.31x), and plpgsql with the plx dialects sits between. -- Branching (branch): plpython3u is faster on this pure-CPU integer workload - (0.72x); plperl and plpgsql with the plx dialects are close to each other. +- Branching (branch): on this pure-CPU integer workload the embedded PLs are + faster: native plphp (0.40x), plpython3u (0.68x), native plruby (0.90x) and + plperl (0.95x). plpgsql and the plx dialects are at about 1.0x (plxcobol's + `EVALUATE` at 1.32x). - Call overhead (call): plpgsql and the plx dialects are fastest. The embedded PLs pay an SPI round trip per call and are slower even at one tenth the - iteration count (native plruby is the slowest here, at 2.97x). + iteration count (native plruby is the slowest here, at 3.03x). The transpile-to-plpgsql approach gives the plx dialects the performance profile of plpgsql: strong on set-oriented and SQL-bound work, competitive on procedural diff --git a/doc/ARCHITECTURE.md b/doc/ARCHITECTURE.md index 15505d3..87c6bab 100644 --- a/doc/ARCHITECTURE.md +++ b/doc/ARCHITECTURE.md @@ -18,7 +18,8 @@ dialect syntax and emits plpgsql; the execution engine is plpgsql. ## Catalog wiring -For each dialect, `plx--1.0.sql` creates a language whose parts are: +For each dialect, the install script (`plx--2.0.0.sql` for the current release) +creates a language whose parts are: - `HANDLER` is `plx_call_handler`, which is bound to plpgsql's exported `plpgsql_call_handler` symbol (`AS '$libdir/plpgsql', 'plpgsql_call_handler'`). @@ -120,7 +121,8 @@ per-dialect branching in the driver. ## Files ``` -plx.control, plx--1.0.sql extension control and install SQL +plx.control, plx--2.0.0.sql extension control and install SQL +plx--X--Y.sql upgrade scripts between releases src/plx.h public ABI (PlxDialect, PlxFuncMeta) src/plx_int.h internal ABI (PlxSurface, canonical keywords) src/plx_engine.h engine interface: PlxCtx, tokens, symtab, and the @@ -151,12 +153,15 @@ is a new `plx_dialect_X.c` (a surface with its `parse_body`, plus a few The plx languages are declared `TRUSTED`. A plx function executes as plpgsql through plpgsql's call handler, so it can do what a plpgsql function can do and -nothing more: no filesystem, no network, no arbitrary native code, and all SQL -runs with the caller's privileges. plx embeds no language runtime, which is the +nothing more: no filesystem, no network, no arbitrary native code, and SQL runs +with the caller's privileges (or the owner's, for a `SECURITY DEFINER` +function), exactly as in plpgsql. plx embeds no language runtime, which is the difference from the native PL/Ruby and PL/PHP, which are untrusted because they load a full interpreter into the backend. The transpiler is C that parses untrusted input at `CREATE FUNCTION` time; it has a recursion-depth limit and is -fuzzed (see `test/fuzz.py`). +fuzzed (see `test/fuzz.py`). See the user guide's +[Trust and security](USERGUIDE.md#trust-and-security) section for who can use +the languages and how to restrict them. ## Testing @@ -193,7 +198,8 @@ instead of passing over it. ## Related documents -- [TRANSPILER.md](TRANSPILER.md): the original transpiler design specification. +- [TRANSPILER.md](TRANSPILER.md): the original transpiler design specification + (historical; the dialect chapters are authoritative where they differ). - [PARITY.md](PARITY.md): the plpgsql construct parity matrix. - The per-dialect chapters: [plxruby](plxruby.md), [plxphp](plxphp.md), [plxjs](plxjs.md), [plxts](plxts.md), [plxpython3](plxpython3.md), diff --git a/doc/COMPATIBILITY.md b/doc/COMPATIBILITY.md index 39ed54d..b85bc9c 100644 --- a/doc/COMPATIBILITY.md +++ b/doc/COMPATIBILITY.md @@ -3,9 +3,12 @@ ## Supported versions plx supports PostgreSQL 13 through 18 (released), and builds and passes on the -19 and 20 development lines. The full pg_regress suite (plxruby, plxphp, plxjs, -plxpython3, plxgo, plxcobol, plxplsql, plxts, plxtsql, and the rejection tests) -passes on each of PostgreSQL 13 through 18, plus 19 and 20 built from source. +19 and 20 development lines. The full pg_regress suite (one file per dialect +plus plx_features, plx_output, plx_strbuild, and plx_errors) passes on each of +PostgreSQL 13 through 18. CI (`.github/workflows/ci.yml`) runs `make +installcheck`, `make differentialcheck`, and `make metacheck` on PostgreSQL 13 +through 18 from the PGDG packages on every push and pull request. The 19 and 20 +results were checked manually against source builds and are not part of CI. | PostgreSQL | Status | string builder | |------------|--------|----------------| @@ -15,8 +18,8 @@ passes on each of PostgreSQL 13 through 18, plus 19 and 20 built from source. | 16 | pass | correct, not accelerated | | 17 | pass | correct, not accelerated | | 18 | pass | accelerated (amortized O(1)) | -| 19 (beta) | pass (from source) | accelerated (amortized O(1)) | -| 20 (devel) | pass (from source) | accelerated (amortized O(1)) | +| 19 (beta) | pass (manual, from source) | accelerated (amortized O(1)) | +| 20 (devel) | pass (manual, from source) | accelerated (amortized O(1)) | PostgreSQL 19 and 20 compile with a C23 toolchain (for example gcc 15), where `pg_noreturn` becomes the standard `[[noreturn]]` attribute; plx writes it as the @@ -59,7 +62,9 @@ Two server APIs differ across the supported majors; plx handles both: ## Reproducing -`test/pg_matrix.sh` builds PostgreSQL 13 to 17 from source, builds plx against +CI is the reference: `.github/workflows/ci.yml` installs each PostgreSQL major +from PGDG, builds plx, and runs the suites listed above. To reproduce locally +from source, `test/pg_matrix.sh` builds PostgreSQL 13 to 17 from source, builds plx against each `pg_config`, and runs the regression suite per version. Run it as root in a build environment with the standard PostgreSQL build dependencies installed. diff --git a/doc/DEBUGGING.md b/doc/DEBUGGING.md index f133ec9..0bcd661 100644 --- a/doc/DEBUGGING.md +++ b/doc/DEBUGGING.md @@ -44,7 +44,7 @@ Then: SELECT plx_source('my_func(int)'); ``` -returns exactly the Ruby / PHP / JavaScript / Python / COBOL body you wrote. +returns exactly the dialect body you wrote. ## Why the error line differs from your source line diff --git a/doc/LIMITATIONS.md b/doc/LIMITATIONS.md index 76394b2..fe00e38 100644 --- a/doc/LIMITATIONS.md +++ b/doc/LIMITATIONS.md @@ -16,13 +16,19 @@ which dialect it was written in. These follow from running as plpgsql and are not specific to any one dialect. - **Trusted sandbox.** A plx function does what a plpgsql function can do and no - more: no filesystem access, no network access, no arbitrary native code. All - SQL runs with the privileges of the calling role. This is the point of the + more: no filesystem access, no network access, no arbitrary native code. SQL + runs with the privileges of the calling role, or of the function owner for a + `SECURITY DEFINER` function, exactly as in plpgsql. This is the point of the design, and the reason plx is a trusted language while the native PL/Ruby or - PL/PHP are not. -- **No transaction control.** A function runs inside the caller's transaction, so - `COMMIT`, `ROLLBACK`, `BEGIN TRANSACTION`, and savepoint management are not - available. This is a plpgsql function boundary, not a plx choice. + PL/PHP are not. Because plx is a syntax layer over plpgsql, USAGE on a plx + language is equivalent to USAGE on plpgsql; see + [Trust and security](USERGUIDE.md#trust-and-security). +- **Transaction control only in procedures.** A function runs inside the + caller's transaction, so `COMMIT` and `ROLLBACK` raise `invalid transaction + termination` there. In a `CREATE PROCEDURE` invoked with `CALL` (outside an + explicit transaction block), `commit()`/`rollback()` (COBOL `COMMIT`/`ROLLBACK`, + PL/SQL `COMMIT;`) work as in plpgsql. plxtsql and plxgo do not support + transaction control. Savepoint management is not available in any dialect. - **The signature is SQL.** The parameter and return types in `CREATE FUNCTION` are parsed by PostgreSQL before plx sees the body, so they must be PostgreSQL type names (`integer`, `numeric`, `text`, `timestamp`, ...). Dialect type names @@ -36,16 +42,26 @@ These follow from running as plpgsql and are not specific to any one dialect. so a comparison involving NULL is unknown rather than false, and `==`/`===` become `=`. Integer division and modulo truncate toward zero. Conditions must be boolean expressions; source-language "truthiness" (a bare value used as a - condition) is not emulated. Interpolating a NULL propagates it: the value is - concatenated as-is, so the whole string becomes NULL the way SQL `||` does. - Two exceptions. A message being built for `RAISE` keeps each interpolated - value as an empty string, so one NULL cannot swallow the message. And - `plxcobol`'s `STRING-APPEND` does not propagate, because it builds its string - through the `plx_strbuild` accumulator, whose append treats a NULL as nothing - to append by design. + condition) is not emulated. A non-boolean condition is not rejected at + `CREATE FUNCTION` time. When the function runs, plpgsql coerces it through + its text form: a value whose text is a boolean literal (`0`, `1`, `t`, `yes`, + `off`, ...) is silently accepted, and anything else raises `invalid input + syntax for type boolean`. Always write an explicit comparison. + Interpolating a NULL propagates it: the value is concatenated as-is, so the + whole string becomes NULL the way SQL `||` does. Two exceptions. A message + being built for `RAISE` keeps each interpolated value as an empty string, so + one NULL cannot swallow the message. And `plxcobol`'s `STRING-APPEND` does not + propagate, because it builds its string through the `plx_strbuild` + accumulator, whose append treats a NULL as nothing to append by design. +- **The message fallback covers the message text only.** An interpolated NULL + in a `RAISE` option such as `detail:` or `hint:` makes the `RAISE` itself fail + with `RAISE statement option cannot be null`, which hides the error you meant + to raise. A NULL in an `assert` message replaces your message with plpgsql's + generic `assertion failed`. Wrap such values in `coalesce(x, '')`. - **String concatenation with `+` is not string concatenation.** In every dialect `+` is SQL numeric addition. Use the dialect's string form: interpolation - (Ruby/PHP/Python/JS/TS), `||`, or `CONCAT(...)`. + (Ruby/PHP/Python/JS/TS), `fmt.Sprintf` (Go), `||` (PL/SQL, T-SQL), or + `CONCAT(...)`. In plxgo `||` is logical OR, and in plxcobol it is dropped. - **Type inference is limited.** Where a dialect infers a local's type from its initializer (`x = 5`, `x := 5`, `let x = ...`), plx infers only from a literal (int, decimal, string, boolean), an array literal, or a direct type @@ -56,17 +72,27 @@ These follow from running as plpgsql and are not specific to any one dialect. amortized-O(1) building. The in-place path relies on a PostgreSQL 18 planner hook; on 13 through 17 the result is correct but the append is O(n^2), the same as plain concatenation. See [COMPATIBILITY.md](COMPATIBILITY.md). +- **The string builder must be on the search path.** Functions that use the + append operators reference the extension's `plx_strbuild` type and + `plx_sb_append` function unqualified, so the extension's schema must be on the + function's `search_path` at run time, including any `SET search_path` clause + on the function. Otherwise the function is created but fails when called with + `type "plx_strbuild" does not exist`. Because the lookup goes through the + path, a `SECURITY DEFINER` function should pin it; see + [Trust and security](USERGUIDE.md#trust-and-security). - **Runtime errors point at the generated plpgsql.** plpgsql requires all declarations at the top, so line numbers in a runtime error refer to the transpiled body, not your dialect source. `plx_source()` recovers the embedded original, and the mapping is usually close. See [DEBUGGING.md](DEBUGGING.md). - **A depth cap guards pathological input.** Deeply nested expressions or - statements (hundreds of levels) are rejected at `CREATE FUNCTION` time rather - than risking a stack overflow. -- **What is always reachable.** Every plpgsql statement type is expressible from - every dialect (assignment, `IF`/`CASE`, all loops, `RETURN`/set-returning, - cursors, dynamic SQL, `RAISE`, exception handling, triggers, `GET - DIAGNOSTICS`). See [PARITY.md](PARITY.md) for the construct-by-construct matrix. + statements (hundreds of levels) are rejected at `CREATE FUNCTION` time with a + `nested too deeply` error rather than risking a stack overflow. +- **What is always reachable.** Every plpgsql statement type (assignment, + `IF`/`CASE`, all loops, `RETURN`/set-returning, cursors, dynamic SQL, `RAISE`, + exception handling, triggers, `GET DIAGNOSTICS`) is expressible from every + dialect, except that plxtsql has no cursors or transaction control and plxgo + has no transaction control. See + [PARITY.md](PARITY.md) for the construct-by-construct matrix. A dialect gives you its syntax over SQL semantics, not its standard library. With the exception of the `fmt`/`strings`/`math`/`strconv` subset plxgo maps, a @@ -78,14 +104,17 @@ library call you would reach for out of habit. ## What each dialect does not support -The per-dialect chapter is authoritative; this is a quick reference. +The per-dialect chapter is authoritative; this is a quick reference. Some +items are rejected at `CREATE FUNCTION` time; others (closures, literals of +unsupported types, library calls) are not translated, so they pass through and +fail when the function runs. ### plxruby ([chapter](plxruby.md)) - Method, class, and module definitions (`def`, `class`, `module`); gems. - Blocks and lambdas beyond the recognized `.each` forms. - Hash and array literals as general values. -- `||=`, `&&=`, and `and`/`or` in value position. +- `||=` and `&&=`. - `redo`, `retry`. - Predicate and bang methods (`x.zero?`, `arr.empty?`, `s.strip!`); use the SQL form (`x = 0`, `cardinality(arr) = 0`). @@ -95,20 +124,24 @@ The per-dialect chapter is authoritative; this is a quick reference. - Function, class, and namespace definitions; `use`, includes, closures. - Array and object literals as general values. - Non-counting `for`; only `for ($v = LO; $v < HI; $v++)` and `+= K` steps. -- `foreach` over a key/value pair (value iteration only). +- `foreach` over a key/value pair: the key is ignored (value iteration only). - `switch` fall-through (end each case with `break` or `return`). ### plxjs ([chapter](plxjs.md)) - Function, arrow-function, and class definitions; `import`. - Object and array literals as general values. -- `for...in`, and `for...of` over anything other than `query(...)` or an array. +- `for...in`. `for...of` treats anything other than `query(...)` as an array, + so a non-array fails when the function runs. +- Block scoping: `let`/`const` in an inner block assign the function-scoped + variable of the same name. - `switch` fall-through. ### plxts ([chapter](plxts.md)) - Everything plxjs rejects, plus type declarations (`interface`, `type`, `enum`), - generics beyond `T[]`, and type-only imports. + and type-only imports. Generic annotations (`Array`) and nested arrays + (`number[][]`) are not mapped; use `T[]` or a SQL type name. - Multiple annotated declarators in one statement (`let a: number = 1, b = 2;`); use one declaration per statement. - Annotations are not type-checked; they only set the plpgsql declaration type. @@ -119,6 +152,7 @@ The per-dialect chapter is authoritative; this is a quick reference. - `import`, modules, and the Python standard library. - Tuples, lists, dicts, and sets as general values. - `match`/`case`, and the conditional expression `a if c else b`. +- Floor division `//`; `/` on integers truncates toward zero. ### plxgo ([chapter](plxgo.md)) @@ -127,7 +161,9 @@ The per-dialect chapter is authoritative; this is a quick reference. - `map`, `chan`, `struct`, and `interface` types; declare data with SQL types or slices. - `switch` `fallthrough`. -- `+` for string concatenation (use `||` or build a slice and `array_to_string`). +- `+` for string concatenation. `||` is logical OR in plxgo; use + `fmt.Sprintf("%s%s", a, b)` or build a slice and `array_to_string`. +- `for i := range s` over a slice; use `for _, v := range s` or a counting loop. - Only a subset of `fmt`/`strings`/`math`/`strconv` is mapped; other calls pass through and must be valid PostgreSQL functions. - `fmt.Sprintf` renders every operand in its SQL text form. A `-` flag and a @@ -146,8 +182,12 @@ The per-dialect chapter is authoritative; this is a quick reference. - Group items (subordinate entries under an `01`) in `WORKING-STORAGE`; declare elementary items, or use a `TYPE` clause for a composite type. - Fixed-format source (column areas); use free format. -- Report Writer, screen sections, object orientation, and the standard COBOL - intrinsic function library (use SQL functions in expressions). +- Report Writer, screen sections, and object orientation. The standard COBOL + intrinsic function library is not translated (a `FUNCTION name(...)` call + passes through and fails when the function runs); use SQL functions in + expressions. +- The `||` operator and `::` casts inside `COMPUTE` and conditions; use + `CONCAT(...)` and `CAST(x AS type)`. ### plxplsql ([chapter](plxplsql.md)) @@ -161,6 +201,10 @@ The per-dialect chapter is authoritative; this is a quick reference. - Oracle spellings inside a `DBMS_OUTPUT.PUT_LINE`/`RAISE_APPLICATION_ERROR` argument, schema-qualified `seq.NEXTVAL`, nested-`DECLARE` type names, and parameterized cursors. +- Oracle exception names other than `NO_DATA_FOUND` and `TOO_MANY_ROWS`; use the + PostgreSQL condition name (`division_by_zero` for `ZERO_DIVIDE`, + `unique_violation` for `DUP_VAL_ON_INDEX`). +- `c%NOTFOUND`/`c%FOUND` cursor attributes; use `NOT FOUND`/`FOUND`. ### plxtsql ([chapter](plxtsql.md)) @@ -178,5 +222,6 @@ Because every dialect compiles to the same plpgsql, you can write the awkward part of a function in whichever dialect expresses it best, or drop to a SQL statement inside any dialect (all of them pass SQL text through to `query`, `execute`, `RETURN QUERY`, and the like). A construct a dialect rejects is -reported at `CREATE FUNCTION` time with a source line, so gaps surface -immediately rather than at run time. +reported at `CREATE FUNCTION` time with a source line, so those gaps surface +immediately. Library calls and expressions passed through to SQL are not +checked until the function runs. diff --git a/doc/MIGRATION.md b/doc/MIGRATION.md index aa1253d..8d907c7 100644 --- a/doc/MIGRATION.md +++ b/doc/MIGRATION.md @@ -111,9 +111,11 @@ END; $do$; ``` -`grade_pg` is a normal plpgsql function with no dependency on the extension. -Applied across your functions, this is an exit: you keep the plpgsql and drop -plx. The original dialect source is preserved in a trailing comment inside the +`grade_pg` is a normal plpgsql function. It depends on the extension only if +its body uses the string builder (`plx_strbuild` and `plx_sb_append`, emitted for +the string-append operators); rewrite those appends as `||` or `string_agg` +before dropping plx. Applied across your functions, this is an exit: you keep +the plpgsql and drop plx. The original dialect source is preserved in a trailing comment inside the generated body, so recreating it as plpgsql does not throw away the source either. See [debugging](DEBUGGING.md) for the helper that decodes it. diff --git a/doc/PARITY.md b/doc/PARITY.md index 33777d6..103ba6b 100644 --- a/doc/PARITY.md +++ b/doc/PARITY.md @@ -4,8 +4,8 @@ Target: every plpgsql capability is expressible in each plx dialect. The reference is the plpgsql statement set in `src/pl/plpgsql/src/plpgsql.h` (`PLpgSQL_stmt_*`) plus its declaration forms. -Every construct below is supported in all five column dialects — **plxruby**, -**plxphp**, **plxjs**, **plxpython3**, **plxcobol** — except where a footnote +Every construct below is supported in each column dialect (**plxruby**, +**plxphp**, **plxjs**, **plxpython3**, **plxcobol**) except where a footnote notes an equivalent form. The cells show the dialect spelling; full syntax and semantics are in the per-dialect chapters ([plxruby](plxruby.md), [plxphp](plxphp.md), [plxjs](plxjs.md), [plxpython3](plxpython3.md), @@ -27,16 +27,19 @@ IMMEDIATE`, `NVL`, `seq.NEXTVAL`, `NUMBER`/`VARCHAR2`, ...) on top. See `var`/`:=` declarations (with type inference) hoist into the `DECLARE` block, parenless `if`/`for` headers become `IF ... THEN`/`WHILE ... LOOP`, `for range` becomes `FOR`/`FOREACH`, the no-fallthrough `switch` becomes `IF`/`ELSIF`, and -`panic`/`fmt.Println` become `RAISE`. Every plpgsql construct is reachable -through Go surface syntax. See [plxgo.md](plxgo.md). +`panic`/`fmt.Println` become `RAISE`. Every plpgsql construct except +transaction control is reachable through Go surface syntax. See +[plxgo.md](plxgo.md). **plxtsql** (Transact-SQL, SQL Server) is not shown as a separate column because it restructures rather than renames: `@x` variables hoist to the `DECLARE` block, `IF`/`WHILE` bodies written as `BEGIN ... END` become `THEN ... END IF` / `LOOP ... END LOOP`, `SET @x = e` becomes `x := e`, `SELECT @x = c FROM t` becomes `SELECT c INTO x FROM t`, `PRINT`/`RAISERROR`/`THROW` become `RAISE`, and -`TRY`/`CATCH` becomes an `EXCEPTION` block. So every plpgsql construct is -reachable, but through T-SQL surface syntax rather than a one-to-one keyword map. +`TRY`/`CATCH` becomes an `EXCEPTION` block. So most plpgsql constructs are +reachable through T-SQL surface syntax rather than a one-to-one keyword map; +the exceptions are cursors and transaction control, which plxtsql does not +support. See [plxtsql.md](plxtsql.md). ## Control flow @@ -54,16 +57,16 @@ See [plxtsql.md](plxtsql.md). | `FOR` over dynamic query | same, non-literal SQL / binds | same | same | same | `… OVER "sql" USING …` | | `FOREACH` over array | `arr.each do \|v\|` | `foreach ($a as $v)` | `for (const v of arr)` | `for v in arr:` | `PERFORM v OVER ARRAY a` | | `EXIT` / `CONTINUE` | `break` / `next` | `break` / `continue` | `break` / `continue` | `break` / `continue` | `EXIT PERFORM` / `EXIT PERFORM CYCLE` | -| loop labels ² | `label:` + `break label` | `label:` + `break label` | `label:` (native) | — | — | +| loop labels ² | `label:` + `break label` | `label:` + `break label` | `label:` (native) | `label:` + `break label` | not offered | ¹ Python has no `match`/`case`, and PHP/JS `switch` is a simple `CASE` only, so a searched (arbitrary-condition) or Python multi-way branch uses `if`/`elif`(`else -if`)/`else` — the same control flow, lowered to plpgsql `IF`. Ruby `case` and +if`)/`else`, the same control flow, lowered to plpgsql `IF`. Ruby `case` and COBOL `EVALUATE` / `EVALUATE TRUE` express both simple and searched `CASE` directly. ² Loop labels (`label:` before a loop with `break label` / `next label`) are -native in JavaScript and a plx extension in Ruby and PHP; not offered in Python -or COBOL. +native in JavaScript and a plx extension in Ruby, PHP, and Python; not offered +in COBOL. ## Return, SQL, and data access @@ -77,7 +80,7 @@ or COBOL. | SQL / `SELECT INTO` | `fetch_one(sql)` · `fetch_one!` | `fetch_one(sql)` | `fetch_one(sql)` | `fetch_one(sql)` | `EXECUTE "sql" INTO v` | | cursors `OPEN`/`FETCH`/`MOVE`/`CLOSE` | `open_cursor` · `fetch_from` · `move_cursor` · `close_cursor` | (same call form) | (same call form) | (same call form) | `OPEN-` · `FETCH-` · `MOVE-` · `CLOSE-CURSOR` | | `CALL` procedure | `call("p", a)` | `call("p", $a)` | `call("p", a)` | `call("p", a)` | `CALL "p" USING a` | -| `COMMIT` / `ROLLBACK` | `commit()` / `rollback()` | (same) | (same) | (same) | `COMMIT` / `ROLLBACK` | +| `COMMIT` / `ROLLBACK` (procedures only) | `commit()` / `rollback()` | (same) | (same) | (same) | `COMMIT` / `ROLLBACK` | The call-form intrinsics (`perform`, `execute`, `fetch_one`, the `*_cursor` functions, `call`, `commit`, `rollback`, `return_next`, `return_query`) share the diff --git a/doc/TRANSPILER.md b/doc/TRANSPILER.md index fcb7779..00ec5e0 100644 --- a/doc/TRANSPILER.md +++ b/doc/TRANSPILER.md @@ -9,6 +9,15 @@ > `plx.so` rather than separate per-dialect modules. For the as-built > architecture and the per-dialect behavior, see ARCHITECTURE.md and the dialect > chapters. This file is kept as a design reference. +> +> Several semantic choices below were not adopted. `==`/`!=` lower to plain +> `=`/`<>` (not `IS [NOT] DISTINCT FROM`). Negation (`unless`, `!`, `not`) is +> `NOT` (not `IS NOT TRUE`). Value-position `||` is SQL concatenation (not +> `COALESCE`). Since 2.0.0 an interpolated NULL propagates, so interpolation is +> not wrapped in `COALESCE((e)::text, '')`; only a RAISE message keeps that +> empty-string fallback. The sentinel names the dialect as `plxruby`, not +> `plruby`. Where this file and the dialect chapters disagree, the chapters are +> authoritative. This is the specification for the Ruby surface. The transpiler runs at `CREATE FUNCTION` time inside `plx_generic_validator`, rewrites `pg_proc.prosrc` @@ -95,24 +104,24 @@ typedef struct { typedef struct { int line, col; char *msg; } PlxError; ``` -### 2.2 Pass 1 — byte-level lexer (`plx_lex`) +### 2.2 Pass 1: byte-level lexer (`plx_lex`) Skip inline whitespace (space/tab, **not** newline). Dispatch on current char: -- **letter/`_`** — scan `[A-Za-z_][A-Za-z0-9_]*` plus a trailing `?`/`!`. Keyword-table `bsearch` → `PLX_TOK_KEYWORD` with a `PlxKw`, else `PLX_TOK_IDENT`. -- **digit** — scan number. **Range guard:** on hitting `.`, peek `p[1]`; if it is also `.`, STOP the number (emit `INT`, let `..`/`...` become an OP). A single `.` before a digit is a float fraction. Underscores in numeric literals (`1_000`) are consumed and stripped at rewrite. Record whether the literal exceeds int4 range for the type-inference pass. -- **`'`** — single-quoted string. Only `\\` and `\'` are escapes; **everything else, including `#{`, is inert literal text.** One opaque `PLX_TOK_STRING` with `sq_string=true`, `interp_count=0`. -- **`"`** — double-quoted string. Escapes `\" \\ \n \t \r \0 \e \s \a \b \f \v \nnn \xHH \uXXXX`. **`\#{` (backslash before `#{`) suppresses interpolation** — the two chars `#{` become literal content. On an *unescaped* `#{`, push an interp context and scan a **balanced** sub-expression to the matching `}`: brace counter starts at 1, `++` on `{`, `--` on `}`, but only when not inside a nested string; nested `"..."`/`'...'` and nested `#{}` are honored. Record each interp `(start,len)` span. Empty/whitespace-only `#{}` is recorded with `len==0` (dropped later). One opaque `PLX_TOK_STRING`, `sq_string=false`, carrying its span list. -- **`%`** — percent-literal only in **value position** (prev significant token is OP/`(`/`,`/keyword/terminator) and `p[1]∈{q,Q,w,W,i,I}` with a delimiter following: `%q`=single semantics, `%Q`=double/interp, bracket delimiters `()[]{}<>` nest. `%w`/`%i` (word/symbol arrays) are lexed as an opaque STRING but flagged unsupported → rejected in lowering. Otherwise `%` is the modulo OP. -- **`<`** — `<<~IDENT` / `<<-IDENT` / `< + - * / & ? ~ ^` — scan the maximal run into one `PLX_TOK_OP` (`==`,`!=`,`&&`,`||`,`<=`,`>=`,`..`,`...`,`=>`,`<<`,`**` come through whole). -- **else** — record first error, `PLX_TOK_ERROR`. +- **letter/`_`**: scan `[A-Za-z_][A-Za-z0-9_]*` plus a trailing `?`/`!`. Keyword-table `bsearch` → `PLX_TOK_KEYWORD` with a `PlxKw`, else `PLX_TOK_IDENT`. +- **digit**: scan number. **Range guard:** on hitting `.`, peek `p[1]`; if it is also `.`, STOP the number (emit `INT`, let `..`/`...` become an OP). A single `.` before a digit is a float fraction. Underscores in numeric literals (`1_000`) are consumed and stripped at rewrite. Record whether the literal exceeds int4 range for the type-inference pass. +- **`'`**: single-quoted string. Only `\\` and `\'` are escapes; **everything else, including `#{`, is inert literal text.** One opaque `PLX_TOK_STRING` with `sq_string=true`, `interp_count=0`. +- **`"`**: double-quoted string. Escapes `\" \\ \n \t \r \0 \e \s \a \b \f \v \nnn \xHH \uXXXX`. **`\#{` (backslash before `#{`) suppresses interpolation**; the two chars `#{` become literal content. On an *unescaped* `#{`, push an interp context and scan a **balanced** sub-expression to the matching `}`: brace counter starts at 1, `++` on `{`, `--` on `}`, but only when not inside a nested string; nested `"..."`/`'...'` and nested `#{}` are honored. Record each interp `(start,len)` span. Empty/whitespace-only `#{}` is recorded with `len==0` (dropped later). One opaque `PLX_TOK_STRING`, `sq_string=false`, carrying its span list. +- **`%`**: percent-literal only in **value position** (prev significant token is OP/`(`/`,`/keyword/terminator) and `p[1]∈{q,Q,w,W,i,I}` with a delimiter following: `%q`=single semantics, `%Q`=double/interp, bracket delimiters `()[]{}<>` nest. `%w`/`%i` (word/symbol arrays) are lexed as an opaque STRING but flagged unsupported → rejected in lowering. Otherwise `%` is the modulo OP. +- **`<`**: `<<~IDENT` / `<<-IDENT` / `< + - * / & ? ~ ^`: scan the maximal run into one `PLX_TOK_OP` (`==`,`!=`,`&&`,`||`,`<=`,`>=`,`..`,`...`,`=>`,`<<`,`**` come through whole). +- **else**: record first error, `PLX_TOK_ERROR`. After each terminator, set `stmt_start=true` on the next significant token. **`stmt_start` bookkeeping is a first-class correctness surface** (opener vs modifier depends on it) and must be re-asserted correctly after a suppressed newline and after a heredoc body drain. @@ -126,7 +135,7 @@ A physical `\n` is DROPPED (statement continues) iff **any** of: Otherwise emit `PLX_TOK_NEWLINE`. `;` is always a separator. The cannot-end set is one `static const` classifier keyed on the OP token bytes plus a short `strcmp` for `..`/`...`/`=>`. -### 2.4 Pass 2 — block builder (`plx_build_blocks`) +### 2.4 Pass 2: block builder (`plx_build_blocks`) Push `BLK_TOPLEVEL`. Walk tokens, cutting statements at `NEWLINE`/`SEMI` in the current frame. Classify each statement's leading `stmt_start` token: @@ -140,7 +149,7 @@ Push `BLK_TOPLEVEL`. Walk tokens, cutting statements at `NEWLINE`/`SEMI` in the At EOF, a non-`[TOPLEVEL]` stack → `ereport` "unterminated `` block opened at line N"; a stray `end`/`}` → matching-close error at its own line. -Every token carries a 1-based dialect line; every block records `open_line` — feeding the plpgsql→dialect source map and line-accurate errors. +Every token carries a 1-based dialect line; every block records `open_line`; feeding the plpgsql→dialect source map and line-accurate errors. --- @@ -183,10 +192,10 @@ The parse ctx carries a `loop_depth` and a `handler_depth` counter (for `next`/` Emission mechanics: 2-space indent per level, body at level 1. Leaf: `;\n` (node supplies text sans `;`; emitter appends exactly one). Compound opener line ends in the plpgsql keyword (`THEN`/`LOOP`) with **no** `;`; children at indent+1; closers `END IF;`/`END LOOP;`/`END;`. Subquery aliases `__plx_fo_N`/`__plx_p_N` use a monotonic counter. -### PLX_ASSIGN — `LHS := RHS;` -`x = e` → `x := e';`. Compound `x += e` → `x := x + (e');` (same for `-= *= /= %=`). Multi-assign `a, b = e1, e2` → `SELECT e1', e2' INTO a, b;` (simultaneous → `a,b=b,a` swaps correctly). LHS must be a bare local or an OUT/TABLE-column param (from meta); assigning to `row.field` is rejected. **`||=`/`&&=` are rejected** (no faithful three-valued lowering) — message points at the explicit form, which itself lowers per the value-context `||` rule (§6). RHS-intrinsics (`fetch_one`, `query`, `emit`, …) are recognized **first** and routed to their own node; they are never assignments. +### PLX_ASSIGN: `LHS := RHS;` +`x = e` → `x := e';`. Compound `x += e` → `x := x + (e');` (same for `-= *= /= %=`). Multi-assign `a, b = e1, e2` → `SELECT e1', e2' INTO a, b;` (simultaneous → `a,b=b,a` swaps correctly). LHS must be a bare local or an OUT/TABLE-column param (from meta); assigning to `row.field` is rejected. **`||=`/`&&=` are rejected** (no faithful three-valued lowering); message points at the explicit form, which itself lowers per the value-context `||` rule (§6). RHS-intrinsics (`fetch_one`, `query`, `emit`, …) are recognized **first** and routed to their own node; they are never assignments. -### PLX_IF — if/elsif/else, unless, modifier-if +### PLX_IF: if/elsif/else, unless, modifier-if ``` IF C' THEN @@ -196,12 +205,12 @@ ELSE END IF; ``` -`unless C` → `IF (C') IS NOT TRUE THEN … END IF;` (**not** `NOT (C')` — matches Ruby falsey nil/false; see §6). Modifier `STMT if C` → `IF C' THEN STMT; END IF;`. Positive `if C` uses `C'` directly (NULL correctly skips the THEN arm). Condition must be boolean; **no truthiness coercion** — a bare non-boolean condition is a CREATE-time reject (§6/§7), not `IF `. +`unless C` → `IF (C') IS NOT TRUE THEN … END IF;` (**not** `NOT (C')`; matches Ruby falsey nil/false; see §6). Modifier `STMT if C` → `IF C' THEN STMT; END IF;`. Positive `if C` uses `C'` directly (NULL correctly skips the THEN arm). Condition must be boolean; **no truthiness coercion**; a bare non-boolean condition is a CREATE-time reject (§6/§7), not `IF `. -### PLX_WHILE — while/until +### PLX_WHILE: while/until `while C` → `WHILE C' LOOP … END LOOP;`. `until C` → `WHILE (C') IS NOT TRUE LOOP … END LOOP;`. Modifiers `STMT while C` / `STMT until C` expand to the full loop. `begin…end while` (do-while) is rejected. -### PLX_LOOP — `loop do … end` +### PLX_LOOP: `loop do … end` ``` LOOP … @@ -209,7 +218,7 @@ END LOOP; ``` Requires a reachable `break`/`return` (not statically enforced). -### PLX_FORINT — integer for +### PLX_FORINT: integer for Surfaces: `for v in LO..HI`, `for v in LO...HI`, `(LO..HI).each do |v|`, `(LO...HI).each`, `HI.downto(LO) do |v|`, `LO.upto(HI) do |v|`, `(LO..HI).step(k).each`, `LO.step(HI,k)`. ``` @@ -217,7 +226,7 @@ FOR v IN [REVERSE] LO'..HI' [BY k'] LOOP … END LOOP; ``` Fixes folded in: -- **Exclusive `...` is a property of the range node**, applied on every path: `LO'..(HI' - 1)`. So `(0...9).step(3)` → `FOR v IN 0..(9 - 1) BY 3 LOOP` (yields 0,3,6 — endpoint 9 correctly excluded). Guard the `(HI-1)` rewrite against INT_MIN underflow. +- **Exclusive `...` is a property of the range node**, applied on every path: `LO'..(HI' - 1)`. So `(0...9).step(3)` → `FOR v IN 0..(9 - 1) BY 3 LOOP` (yields 0,3,6; endpoint 9 correctly excluded). Guard the `(HI-1)` rewrite against INT_MIN underflow. - `downto` → `REVERSE HI'..LO'`. `upto` → ascending. - **`.step(k)` with a literal negative k** → `FOR v IN REVERSE LO'..HI' BY (-k) LOOP` (Numeric#step limit is inclusive; no `-1`). Never emit `BY `. - **`.step(k)` with a non-literal k** → emit ascending `BY k'` and **document that k must be `>0` at runtime** (plpgsql cannot pick direction dynamically); optionally a CREATE-time NOTICE. @@ -228,7 +237,7 @@ Fixes folded in: Loop var `v` is loop-local, plpgsql-declared, **not hoisted**; endpoints pass through to plpgsql. `v` referenced **outside** the loop body → reject `plruby: for-loop variable "v" is not visible after the loop` (Ruby's `for` leak is intentionally dropped and detected, not silently mis-emitted). -### PLX_FORQUERY — query iteration +### PLX_FORQUERY: query iteration `query(SQL[, args]).each do |row|` / `.each_with_index do |row, idx|`. **Static** (SQL is a string literal, no bind args). Interpolation `#{name}` inside the literal is spliced as a **bare plpgsql NAME reference** *only when it is in value position*: @@ -242,30 +251,30 @@ FOR row IN SELECT id, amount FROM orders WHERE cust = cid LOOP … END LOOP; FOR row IN EXECUTE USING a', b' LOOP … END LOOP; ``` -- **`row` is ALWAYS hoisted as `row RECORD;`** (mandatory — plpgsql auto-declares only *integer* FOR vars; a query FOR needs a pre-declared record var, else `"row" is not a known variable`). Register `row` in the active-loop-var stack for the body. +- **`row` is ALWAYS hoisted as `row RECORD;`** (mandatory: plpgsql auto-declares only *integer* FOR vars; a query FOR needs a pre-declared record var, else `"row" is not a known variable`). Register `row` in the active-loop-var stack for the body. - `each_with_index`: the index var is a **synthesized local** routed through the symtab (`idx integer;`), registered in the active-loop stack and checked against params. Emit `idx := -1;` immediately before the FOR and inject `idx := idx + 1;` as the **very first** body statement (above any `next if` guard) → 0-based, matching Ruby. - Field access `row.col`, `row[:col]`, `row['col']` all normalize to `row.col`. -- Nested/overlapping loops need distinct row names (checked against the active stack only — sequential sibling loops may reuse `row`, §5). +- Nested/overlapping loops need distinct row names (checked against the active stack only; sequential sibling loops may reuse `row`, §5). -### PLX_FETCH — `fetch_one` → SELECT INTO +### PLX_FETCH: `fetch_one` → SELECT INTO **Static:** wrap the user SQL so INTO needs no SQL parse: ``` SELECT * INTO [STRICT] FROM ( ) AS __plx_fo_1; ``` **Dynamic** (bind args / non-literal): `EXECUTE INTO [STRICT] USING args';`. -`fetch_one` non-STRICT (0 rows → all-NULL, extra rows → first); `fetch_one!` STRICT. Whole-row target hoists `RECORD`; scalar destructure targets need annotation/inference. Set-returning DML (`INSERT … RETURNING`) as the user SQL must use the dynamic/EXECUTE path (the subquery wrap fails on it). `.nil?` (row-presence) lowers to **`NOT FOUND`** evaluated right after the INTO — not `IS NULL` (which is true only when *every* column is NULL; §6). +`fetch_one` non-STRICT (0 rows → all-NULL, extra rows → first); `fetch_one!` STRICT. Whole-row target hoists `RECORD`; scalar destructure targets need annotation/inference. Set-returning DML (`INSERT … RETURNING`) as the user SQL must use the dynamic/EXECUTE path (the subquery wrap fails on it). `.nil?` (row-presence) lowers to **`NOT FOUND`** evaluated right after the INTO; not `IS NULL` (which is true only when *every* column is NULL; §6). -### PLX_PERFORM — run SQL, discard (static, literal, no binds) +### PLX_PERFORM: run SQL, discard (static, literal, no binds) DML (INSERT/UPDATE/DELETE/MERGE) → verbatim `…;`. Row-returning (SELECT/WITH/VALUES/TABLE) → `PERFORM * FROM ( ) AS __plx_p_1;`. Value-position interpolation → name reference (identifier position → reject/dynamic, as FORQUERY). -### PLX_EXECUTE — dynamic SQL, discard +### PLX_EXECUTE: dynamic SQL, discard `execute(SQL_expr[, a, b])` → `EXECUTE [USING a', b'];`. Always dynamic. Interpolation `#{e}` → runtime concat `|| COALESCE((e)::text,'')` (Ruby nil→''; also prevents `EXECUTE NULL`). User owns injection safety; recommend `USING`/`format(%I,%L)`. ### PLX_RETURN -`return` → `RETURN;`. `return e` → `RETURN e';`. In a SETOF function `return e` is rejected (use `emit`/`return_next`/`return_query`); bare `return` → `RETURN;`. `return a, b` with matching OUT params → `out1 := a'; out2 := b'; RETURN;`. **No implicit last-expression return** — a scalar function must end with explicit `return e`. +`return` → `RETURN;`. `return e` → `RETURN e';`. In a SETOF function `return e` is rejected (use `emit`/`return_next`/`return_query`); bare `return` → `RETURN;`. `return a, b` with matching OUT params → `out1 := a'; out2 := b'; RETURN;`. **No implicit last-expression return**; a scalar function must end with explicit `return e`. -### PLX_RETURN_NEXT — emit / return_next +### PLX_RETURN_NEXT: emit / return_next Bare `emit`/`return_next` → `RETURN NEXT;`. `emit(e)`/`return_next e` → `RETURN NEXT e';`. Requires `meta.retset`, else hard error. ### PLX_RETURN_QUERY @@ -274,9 +283,9 @@ Static: `RETURN QUERY ;`. Dynamic: `RETURN QUERY EXECUTE 0`), else hard error. `break `, labeled break, `redo`, `retry` rejected. -### PLX_EXCEPTION — begin/rescue/ensure +### PLX_EXCEPTION: begin/rescue/ensure Without `ensure`: ``` BEGIN @@ -348,7 +357,7 @@ typedef struct PlxSymtab { ``` ### 5.2 Seeding params (never hoist) -From `PlxFuncMeta->argnames[i]` for **all** argmodes (IN/OUT/INOUT/VARIADIC/TABLE). **Skip NULL/empty argnames** (`f(int, OUT numeric)` has unnamed args) — guard every `hash_search` key against NULL (a NULL key into a `HASH_STRINGS` dynahash segfaults). Case-fold with `downcase_truncate_identifier` (63-byte `NAMEDATALEN` truncation) to match plpgsql exactly. +From `PlxFuncMeta->argnames[i]` for **all** argmodes (IN/OUT/INOUT/VARIADIC/TABLE). **Skip NULL/empty argnames** (`f(int, OUT numeric)` has unnamed args); guard every `hash_search` key against NULL (a NULL key into a `HASH_STRINGS` dynahash segfaults). Case-fold with `downcase_truncate_identifier` (63-byte `NAMEDATALEN` truncation) to match plpgsql exactly. ### 5.3 Collection walk Single recursive descent. On a **loop node**: push its var(s) onto `active_loopvars`, recurse, pop on close. @@ -380,37 +389,37 @@ Empty symtab → no `DECLARE` keyword. Else `DECLARE\n` then one line per local ## 6. Expression rewrite rules & accepted divergences -`plx_rewrite_expr(char *src, ctx, boolean_context)` → `char *`, a **single left-to-right pass over source tokens** honoring string/`#{}`/bracket state. It **never re-scans its own output** (the `||`/`OR` invariant). Inside a **single-quoted** string, NOTHING is rewritten — no `#{}` splice, no operator/keyword rewrite, no `+`→`||`; only `'`→`''` for re-emission. `#{}` is an interpolation **only** in double-quoted strings/heredocs. +`plx_rewrite_expr(char *src, ctx, boolean_context)` → `char *`, a **single left-to-right pass over source tokens** honoring string/`#{}`/bracket state. It **never re-scans its own output** (the `||`/`OR` invariant). Inside a **single-quoted** string, NOTHING is rewritten; no `#{}` splice, no operator/keyword rewrite, no `+`→`||`; only `'`→`''` for re-emission. `#{}` is an interpolation **only** in double-quoted strings/heredocs. -### 6.1 String literals — decode then re-encode (never memcpy source bytes) +### 6.1 String literals: decode then re-encode (never memcpy source bytes) Ruby string escape alphabet ≠ plpgsql literal alphabet. Always **fully decode** the Ruby token to its abstract character sequence, then **re-encode** for the chosen target literal: - **Single-quoted** Ruby string: resolve `\\`→`\`, `\'`→`'`; all other bytes literal. Emit as plpgsql `'...'` with `'`→`''`. (`'a\\b'` = Ruby `a\b` → plpgsql `'a\b'`, one backslash.) -- **Double-quoted** Ruby string / bare-or-`"`-quoted heredoc: decode `\n \t \r \0 \e \\ \" \xHH \uXXXX …` to actual control bytes. If any control byte or backslash is present, emit an **escape string** `E'...'` (re-escaping `'`→`''` and `\`→`\\`); else a plain `'...'`. `standard_conforming_strings=on` means `'...\n...'` is a literal backslash-n — so `"line1\nline2"` **must** become `E'line1\nline2'`, not `'line1\nline2'`. `"C:\\tmp"` (Ruby value `C:\tmp`) → `E'C:\\tmp'`. +- **Double-quoted** Ruby string / bare-or-`"`-quoted heredoc: decode `\n \t \r \0 \e \\ \" \xHH \uXXXX …` to actual control bytes. If any control byte or backslash is present, emit an **escape string** `E'...'` (re-escaping `'`→`''` and `\`→`\\`); else a plain `'...'`. `standard_conforming_strings=on` means `'...\n...'` is a literal backslash-n; so `"line1\nline2"` **must** become `E'line1\nline2'`, not `'line1\nline2'`. `"C:\\tmp"` (Ruby value `C:\tmp`) → `E'C:\\tmp'`. ### 6.2 Interpolation `"lit#{e}lit2"` -Split into literal chunks + interp spans first. Each literal chunk is decoded/escaped per §6.1. Each interp expression is **recursively `plx_rewrite_expr`'d** (nested `"..."` inside `#{}` must themselves be lowered — never spliced verbatim) and wrapped to match Ruby nil→`''`: +Split into literal chunks + interp spans first. Each literal chunk is decoded/escaped per §6.1. Each interp expression is **recursively `plx_rewrite_expr`'d** (nested `"..."` inside `#{}` must themselves be lowered; never spliced verbatim) and wrapped to match Ruby nil→`''`: ``` 'lit' || COALESCE((e')::text, '') || 'lit2' ``` -A lone `#{e}` → `COALESCE((e')::text,'')`. **Empty/whitespace `#{}` → dropped** (never `()::text`). `\#{` → literal `#{` in the chunk. `'`→`''` and `%` are applied to literal chunks only, never to interp expressions. (Equivalently the whole interpolation may lower to `concat('lit', e', 'lit2')`, which ignores NULLs; pick one and be consistent.) **Documented divergence:** interpolation follows Ruby (nil→`''`), diverging from raw SQL `||` NULL-propagation; the RAISE `%`-path renders NULL as `` unless COALESCE'd — reconcile or document. +A lone `#{e}` → `COALESCE((e')::text,'')`. **Empty/whitespace `#{}` → dropped** (never `()::text`). `\#{` → literal `#{` in the chunk. `'`→`''` and `%` are applied to literal chunks only, never to interp expressions. (Equivalently the whole interpolation may lower to `concat('lit', e', 'lit2')`, which ignores NULLs; pick one and be consistent.) **Documented divergence:** interpolation follows Ruby (nil→`''`), diverging from raw SQL `||` NULL-propagation; the RAISE `%`-path renders NULL as `` unless COALESCE'd; reconcile or document. -### 6.3 Operators — boolean vs value context +### 6.3 Operators: boolean vs value context Context (condition slot vs value/RHS slot) is known to the emitter; the split is decidable. - `==` → **`IS NOT DISTINCT FROM`** (null-safe, matches Ruby `nil==nil`→true, `5==nil`→false). `!=` → **`IS DISTINCT FROM`**. **Literal-nil special case first** (token-level, before the generic rewrite): `E == nil` / `nil == E` → `((E') IS NULL)`; `E != nil` / `nil != E` → `((E') IS NOT NULL)`. (A perf-minded variant may keep `=`/`<>` when neither operand can be NULL, but null-safe is the default.) - `&&`/`||`/`!` and keyword `and`/`or`/`not`: - - **Boolean context** (if/unless/while/until tests, ternary test): `&&`/`and`→` AND `, `||`/`or`→` OR `, `!`/`not`→ negation. **Negation uses `(x) IS NOT TRUE`, not `NOT (x)`** — TRUE for both FALSE and NULL, matching Ruby falsey (`unless nil` runs). So `unless C`→`IF (C') IS NOT TRUE`, `next unless C`→`CONTINUE WHEN (C') IS NOT TRUE`, `!arr.include?(x)`→`(x = ANY(arr)) IS NOT TRUE`. + - **Boolean context** (if/unless/while/until tests, ternary test): `&&`/`and`→` AND `, `||`/`or`→` OR `, `!`/`not`→ negation. **Negation uses `(x) IS NOT TRUE`, not `NOT (x)`**; TRUE for both FALSE and NULL, matching Ruby falsey (`unless nil` runs). So `unless C`→`IF (C') IS NOT TRUE`, `next unless C`→`CONTINUE WHEN (C') IS NOT TRUE`, `!arr.include?(x)`→`(x = ANY(arr)) IS NOT TRUE`. - **Value/RHS context**: `a || b` → `COALESCE(a', b')` (Ruby nil-default; document that SQL can't distinguish stored FALSE from NULL, so `false || b` won't pick b). `a && b` value use → reject with a precise message pointing at an explicit ternary, or (boolean operands) `CASE WHEN (a') IS NOT TRUE THEN a' ELSE b' END`. Keyword `and`/`or` in value position → **reject** (their below-`=` precedence and `x or return` control-flow can't be faithfully lowered). This is why `||=`/`&&=` are rejected and the suggested `x = x || y` lowers to `COALESCE`. - `nil`→`NULL`, `true`/`false`→`TRUE`/`FALSE`. - Ternary `C ? A : B` → `CASE WHEN C' THEN A' ELSE B' END` (recursive). -- Ranges/membership (self-parenthesized): `(a..b).include?(x)`→`(x BETWEEN a' AND b')`; **`(a...b).include?(x)`→`(x >= a' AND x < b')`** (parenthesized — unparenthesized re-associates against outer operators); `arr.include?(x)`→`(x = ANY(arr'))`. +- Ranges/membership (self-parenthesized): `(a..b).include?(x)`→`(x BETWEEN a' AND b')`; **`(a...b).include?(x)`→`(x >= a' AND x < b')`** (parenthesized; unparenthesized re-associates against outer operators); `arr.include?(x)`→`(x = ANY(arr'))`. - `**`→`power(a,b)`; `1_000`→`1000`; casts `x.to_i/.to_s/.to_f`→`(x)::integer/::text/::double precision`. -- **`+` stays SQL numeric `+`** — the literal-adjacency `+`→`||` heuristic is **dropped** (it misfires both ways: `first + last` two text vars stay `+` and error at runtime; `amount + "0"` numeric becomes string concat). Interpolation is the sanctioned concatenation path. -- **`<<` is NOT blanket-rewritten** to `||` — PG `<<` is integer bit-shift, and the shovel's in-place mutation has no bare-statement plpgsql form. Leave `<<` verbatim (bit-shift) or reject; string/array building goes through interpolation or explicit `x = x || y`. +- **`+` stays SQL numeric `+`**: the literal-adjacency `+`→`||` heuristic is **dropped** (it misfires both ways: `first + last` two text vars stay `+` and error at runtime; `amount + "0"` numeric becomes string concat). Interpolation is the sanctioned concatenation path. +- **`<<` is NOT blanket-rewritten** to `||`: PG `<<` is integer bit-shift, and the shovel's in-place mutation has no bare-statement plpgsql form. Leave `<<` verbatim (bit-shift) or reject; string/array building goes through interpolation or explicit `x = x || y`. - `%` modulo, `::` casts, function calls, column refs, arithmetic → pass through verbatim to SQL. -### 6.4 Truthiness — no coercion, reject non-boolean conditions -Ruby treats `0`, `""`, `[]` as truthy — unrepresentable by any single SQL coercion. A bare non-boolean condition (`if rows_affected`, `return unless result`) is **rejected at CREATE**: `plruby: condition must be a boolean expression; Ruby truthiness is not emulated (0, "", [] are truthy in Ruby) — write an explicit comparison such as x != 0 or !x.nil?`. Do not pass `IF `/`NOT ()` to plpgsql. +### 6.4 Truthiness: no coercion, reject non-boolean conditions +Ruby treats `0`, `""`, `[]` as truthy; unrepresentable by any single SQL coercion. A bare non-boolean condition (`if rows_affected`, `return unless result`) is **rejected at CREATE**: `plruby: condition must be a boolean expression; Ruby truthiness is not emulated (0, "", [] are truthy in Ruby); write an explicit comparison such as x != 0 or !x.nil?`. Do not pass `IF `/`NOT ()` to plpgsql. ### 6.5 Accepted, documented divergences - Decimal literals → `numeric`, not `float8`. @@ -442,9 +451,9 @@ Hard-reject catalog (all at CREATE time, precise messages, real line numbers): ` ## 8. C file / function layout & validator integration ### 8.1 Files -- `plx_scan.h` / `plx_scan.c` — shared framework: enums, `PlxToken`/`PlxLexer`/`PlxBlock`/`PlxStmt`, `plx_lex`, `plx_build_blocks`. Dialect-neutral; consumes a `PlxSurface`. -- `plx_transpile.h` / `plx_transpile.c` — shared framework: `PlxNode`/`PlxArm`/`PlxLocal`/`PlxSymtab`, `plx_rewrite_expr`, `plx_infer_type`, the `plx_lower_*` builders, `plx_emit`, `plx_emit_declare`, and the public entry point `plx_transpile`. -- `plx_dialect_ruby.c` — the Ruby `PlxSurface` (keyword table, block-form matchers, intrinsic names, sigils `#{`/`#::`, exception-class map) and the thin `ruby_validator` wired into `CREATE FUNCTION`. +- `plx_scan.h` / `plx_scan.c`: shared framework: enums, `PlxToken`/`PlxLexer`/`PlxBlock`/`PlxStmt`, `plx_lex`, `plx_build_blocks`. Dialect-neutral; consumes a `PlxSurface`. +- `plx_transpile.h` / `plx_transpile.c`: shared framework: `PlxNode`/`PlxArm`/`PlxLocal`/`PlxSymtab`, `plx_rewrite_expr`, `plx_infer_type`, the `plx_lower_*` builders, `plx_emit`, `plx_emit_declare`, and the public entry point `plx_transpile`. +- `plx_dialect_ruby.c`: the Ruby `PlxSurface` (keyword table, block-form matchers, intrinsic names, sigils `#{`/`#::`, exception-class map) and the thin `ruby_validator` wired into `CREATE FUNCTION`. ### 8.2 Public API ```c @@ -498,12 +507,12 @@ if (!has_sentinel(rubysrc)) { ``` Because `lanplcallfoid` is `plpgsql_call_handler`, the stored (now-plpgsql) `prosrc` executes on stock plpgsql. Re-running `CREATE OR REPLACE` re-supplies Ruby; the sentinel check makes a second validator pass idempotent (never double-transpile already-plpgsql text). -### 8.4 Embedded original (Strategy A) — safe encoding +### 8.4 Embedded original (Strategy A): safe encoding The verbatim-splice of user bytes into a `/* … */` comment is unsafe: any `*/` in the body (in a string, `a */ b`, etc.) closes the comment early and corrupts `prosrc`; the `$plx-orig$` sentinel can likewise be forged. **Base64-encode the whole original body** and wrap: ``` /*plx-orig:b64$$plx-orig*/ ``` -Base64's alphabet contains no `*`, `/`, or `$`, so the terminator and sentinel are unforgeable. (Alternative: escape every `*/`→`*\/` and sentinel occurrences on write, reverse on read — base64 is simpler and total.) A recovery tool decodes this to show the user their original Ruby. +Base64's alphabet contains no `*`, `/`, or `$`, so the terminator and sentinel are unforgeable. (Alternative: escape every `*/`→`*\/` and sentinel occurrences on write, reverse on read; base64 is simpler and total.) A recovery tool decodes this to show the user their original Ruby. ### 8.5 Ruby `PlxSurface` (`plx_dialect_ruby.c`) Supplies keyword spellings (`if elsif unless while until for begin rescue ensure end next break return loop do case when then in and or not`), block-form matchers (`.each`, `.each_with_index`, `do |..|`, `loop do`, `{ |..| }`), intrinsics (`query fetch_one fetch_one! perform execute emit return_next return_query raise`), sigils `#{ }` and `#::`, and the exception-class→condition map. Nothing else is Ruby-specific. @@ -516,7 +525,7 @@ Supplies keyword spellings (`if elsif unless while until for begin rescue ensure **Dialect-specific (a new `plx_dialect_php.c` `PlxSurface`):** - keyword spellings (`if/elseif/else/endif` or brace-blocks, `while`, `foreach`, `function` reject, …); -- comment lexer + **type-annotation sigil** (PHP spelling `/*:: text */` instead of `#::`) — the shared framework consumes "a `::`-tagged type string in a trailing dialect comment"; the dialect supplies only its comment lexer; +- comment lexer + **type-annotation sigil** (PHP spelling `/*:: text */` instead of `#::`); the shared framework consumes "a `::`-tagged type string in a trailing dialect comment"; the dialect supplies only its comment lexer; - string/quote flavors (`"$var"` interpolation vs `'...'` literal), heredoc/nowdoc (`<< ", USER-ID) SQLSTATE "P0002" ``` +Note: `throw` in plxphp and plxjs and `raise ValueError(...)` in plxpython3 +raise SQLSTATE P0001 (`raise_exception`). Only the plxruby and plxcobol forms +set a specific SQLSTATE such as P0002 (`no_data_found`). + ## Anonymous code blocks plxruby: @@ -592,6 +601,52 @@ PROCEDURE DIVISION. $$; ``` +## Procedures and transaction control + +PL/pgSQL manual, "Transaction Management". A body can also be a procedure, +created with `CREATE PROCEDURE` and run with `CALL`. A procedure invoked outside +an explicit transaction block can commit and roll back; a function cannot. + +plpgsql: + +```sql +CREATE PROCEDURE load_batches(n int) AS $$ +BEGIN + FOR i IN 1..n LOOP + INSERT INTO batch_log VALUES (i); + COMMIT; + END LOOP; +END; +$$ LANGUAGE plpgsql; +``` + +plxruby: + +```sql +CREATE PROCEDURE load_batches(n int) LANGUAGE plxruby AS $$ +for i in 1..n + execute("INSERT INTO batch_log VALUES ($1)", i) + commit() +end +$$; +``` + +plxpython3: + +```sql +CREATE PROCEDURE load_batches(n int) LANGUAGE plxpython3 AS $$ +for i in range(1, n + 1): + execute("INSERT INTO batch_log VALUES ($1)", i) + commit() +$$; +``` + +plxphp, plxjs and plxts use the same `commit()` and `rollback()` calls, +plxcobol uses the `COMMIT` and `ROLLBACK` verbs, and plxplsql uses `COMMIT;` as +in PL/SQL. plxtsql and plxgo do not support transaction control. Calling +`commit()` from a function, or from a procedure called inside an explicit transaction block, +raises `invalid transaction termination`, as in plpgsql. + ## Trust and security The plx languages are declared `TRUSTED`. This is correct for the following @@ -599,30 +654,68 @@ reasons: - A plx function is transpiled to plpgsql and executed by plpgsql's call handler. It can do what a plpgsql function can do, and nothing more: no direct - filesystem access, no network access, no arbitrary native code. All SQL runs - with the privileges of the calling role. + filesystem access, no network access, no arbitrary native code. SQL runs with + the privileges of the calling role, or of the function owner for a + `SECURITY DEFINER` function, exactly as in plpgsql. - plx does not embed a language runtime. This is the difference from the native PL/PHP and PL/Ruby, which are untrusted because they load a full PHP or Ruby interpreter into the backend. -Two points to keep in mind: +Who can use the languages: + +- Installing plx (`CREATE EXTENSION plx`) requires a superuser. +- Once installed, every role can create functions in the plx languages, because + PostgreSQL grants USAGE on a trusted language to PUBLIC, as it does for + plpgsql. Restrict this per language, for example + `REVOKE USAGE ON LANGUAGE plxruby FROM PUBLIC`. +- plx is a syntax layer over plpgsql, so a role with USAGE on any plx language + can effectively run arbitrary plpgsql: a body that already begins with the + `/*plx:v1:...*/` sentinel is stored without transpiling, so that dump and + restore work, and plxplsql accepts a body that is already close to plpgsql. + Revoking USAGE on plpgsql therefore does not restrict a role that keeps USAGE + on a plx language: after `REVOKE USAGE ON LANGUAGE plpgsql FROM PUBLIC`, such + a role can still run plpgsql through, for example, `DO LANGUAGE plxplsql`. + To restrict plpgsql, revoke USAGE on plpgsql and on every plx language. + +Points to keep in mind: - The transpiler is C code that parses the function body at `CREATE FUNCTION` - time. It has a recursion-depth limit and has been fuzzed with mutation and - pathological inputs across all dialects (see `test/fuzz.py`) with no crashes or - hangs. Continued fuzzing is recommended before relying on it in a hostile - multi-tenant deployment. -- `query`, `perform`, `execute`, and `fetch_one` with interpolated values carry - the same SQL-injection responsibility as plpgsql `EXECUTE ... || value`. Use - bind parameters (`execute(sql, a, b)`) for untrusted input. + time. It limits expression and statement nesting depth, rejecting deeper + input with a `nested too deeply` error, and has been fuzzed with mutation and + pathological inputs across all dialects (see `test/fuzz.py`). Continued + fuzzing is recommended before relying on it in a hostile multi-tenant + deployment. +- `execute` with interpolated values builds the SQL string by concatenation and + carries the same SQL-injection responsibility as plpgsql + `EXECUTE ... || value`. Use bind parameters (`execute(sql, a, b)`) for + untrusted input. In `query`, `perform`, and `fetch_one` with no bind + arguments, an interpolated value is spliced as a variable reference, not as + text. Once such a call also passes bind arguments it becomes dynamic SQL, and + its interpolated values are concatenated exactly as in `execute`: do not mix + interpolation with binds. +- Functions that use the string-append operators reference the extension's + `plx_strbuild` type and `plx_sb_append` function unqualified, so both are + looked up through `search_path` each time the function runs. The extension's + schema must therefore be on the function's path, including any + `SET search_path` clause. It also means that a role able to create objects in + a schema that comes earlier on the path can put its own `plx_sb_append` there, + and a `SECURITY DEFINER` function would run it with the owner's privileges. + Pin the path on every `SECURITY DEFINER` function, listing only trusted + schemas, the extension's schema among them, and `pg_temp` last, as the + PostgreSQL manual advises: + `ALTER FUNCTION f(...) SET search_path = app, public, pg_temp`, where + `public` stands for the schema plx is installed in and is not writable by + untrusted roles. ## Further reading -- `README.md`: build and install. -- Per-dialect chapters: `doc/plxruby.md`, `doc/plxphp.md`, `doc/plxjs.md`, - `doc/plxpython3.md`, `doc/plxcobol.md`, `doc/plxplsql.md`, `doc/plxts.md`, - `doc/plxtsql.md`, `doc/plxgo.md` (full syntax, supported and rejected - constructs, semantic differences). -- `doc/PARITY.md`: the plpgsql construct parity matrix. -- `doc/ARCHITECTURE.md`: how plx maps to the plpgsql engine. -- `doc/TRANSPILER.md`: transpiler specification. +- [Home](index.md): build and install. +- Per-dialect chapters: [plxruby](plxruby.md), [plxphp](plxphp.md), + [plxjs](plxjs.md), [plxpython3](plxpython3.md), [plxcobol](plxcobol.md), + [plxplsql](plxplsql.md), [plxts](plxts.md), [plxtsql](plxtsql.md), + [plxgo](plxgo.md) (full syntax, supported and rejected constructs, semantic + differences). +- [PARITY.md](PARITY.md): the plpgsql construct parity matrix. +- [ARCHITECTURE.md](ARCHITECTURE.md): how plx maps to the plpgsql engine. +- [TRANSPILER.md](TRANSPILER.md): the original transpiler design + specification. diff --git a/doc/cookbook/plxcobol.md b/doc/cookbook/plxcobol.md index e0d2b9f..becdceb 100644 --- a/doc/cookbook/plxcobol.md +++ b/doc/cookbook/plxcobol.md @@ -1,6 +1,6 @@ # COBOL cookbook -Practical recipes for the plxcobol dialect (ISO/IEC 1989:2023). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxcobol chapter](plxcobol.md) for the full language reference. +Practical recipes for the plxcobol dialect (ISO/IEC 1989:2023). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxcobol chapter](../plxcobol.md) for the full language reference. ## Absolute value with IF / ELSE diff --git a/doc/cookbook/plxgo.md b/doc/cookbook/plxgo.md index ffcc03e..a18e1a4 100644 --- a/doc/cookbook/plxgo.md +++ b/doc/cookbook/plxgo.md @@ -1,6 +1,6 @@ # Go cookbook -Practical recipes for the plxgo dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxgo chapter](plxgo.md) for the full language reference. +Practical recipes for the plxgo dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxgo chapter](../plxgo.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxjs.md b/doc/cookbook/plxjs.md index 40ac494..82de895 100644 --- a/doc/cookbook/plxjs.md +++ b/doc/cookbook/plxjs.md @@ -1,6 +1,6 @@ # JavaScript cookbook -Practical recipes for the plxjs dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxjs chapter](plxjs.md) for the full language reference. +Practical recipes for the plxjs dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxjs chapter](../plxjs.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxphp.md b/doc/cookbook/plxphp.md index e7bed76..218b32c 100644 --- a/doc/cookbook/plxphp.md +++ b/doc/cookbook/plxphp.md @@ -1,6 +1,6 @@ # PHP cookbook -Practical recipes for the plxphp dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxphp chapter](plxphp.md) for the full language reference. +Practical recipes for the plxphp dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxphp chapter](../plxphp.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxplsql.md b/doc/cookbook/plxplsql.md index fce803e..786fb68 100644 --- a/doc/cookbook/plxplsql.md +++ b/doc/cookbook/plxplsql.md @@ -1,6 +1,6 @@ # Oracle PL/SQL cookbook -Practical recipes for the plxplsql dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. The function signature uses PostgreSQL types; the body is PL/SQL. See the [plxplsql chapter](plxplsql.md) for the full language reference. +Practical recipes for the plxplsql dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. The function signature uses PostgreSQL types; the body is PL/SQL. See the [plxplsql chapter](../plxplsql.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxpython3.md b/doc/cookbook/plxpython3.md index 4851f0b..244a950 100644 --- a/doc/cookbook/plxpython3.md +++ b/doc/cookbook/plxpython3.md @@ -1,6 +1,6 @@ # Python cookbook -Practical recipes for the plxpython3 dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxpython3 chapter](plxpython3.md) for the full language reference. +Practical recipes for the plxpython3 dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxpython3 chapter](../plxpython3.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxruby.md b/doc/cookbook/plxruby.md index 8b195a2..8c2ca1c 100644 --- a/doc/cookbook/plxruby.md +++ b/doc/cookbook/plxruby.md @@ -1,6 +1,6 @@ # Ruby cookbook -Practical recipes for the plxruby dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxruby chapter](plxruby.md) for the full language reference. +Practical recipes for the plxruby dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxruby chapter](../plxruby.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxts.md b/doc/cookbook/plxts.md index a6e11aa..15c3be5 100644 --- a/doc/cookbook/plxts.md +++ b/doc/cookbook/plxts.md @@ -1,6 +1,6 @@ # TypeScript cookbook -Practical recipes for the plxts dialect (plxjs plus type annotations). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxts chapter](plxts.md) for the full language reference. +Practical recipes for the plxts dialect (plxjs plus type annotations). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxts chapter](../plxts.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxtsql.md b/doc/cookbook/plxtsql.md index 707be43..f0d759f 100644 --- a/doc/cookbook/plxtsql.md +++ b/doc/cookbook/plxtsql.md @@ -1,6 +1,6 @@ # Transact-SQL cookbook -Practical recipes for the plxtsql dialect (SQL Server / Sybase T-SQL). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxtsql chapter](plxtsql.md) for the full language reference. +Practical recipes for the plxtsql dialect (SQL Server / Sybase T-SQL). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxtsql chapter](../plxtsql.md) for the full language reference. ## Scalar function with branching diff --git a/doc/index.md b/doc/index.md index bf2cae3..700fced 100644 --- a/doc/index.md +++ b/doc/index.md @@ -30,15 +30,15 @@ $$; The front end is dialect-pluggable, and the set is open-ended. The dialects available today: -- [**plxruby**](plxruby.md) — a Ruby dialect -- [**plxphp**](plxphp.md) — a PHP dialect -- [**plxjs**](plxjs.md) — a JavaScript dialect -- [**plxts**](plxts.md) — a TypeScript dialect (plxjs plus type annotations) -- [**plxpython3**](plxpython3.md) — a Python dialect -- [**plxgo**](plxgo.md) — a Go dialect -- [**plxcobol**](plxcobol.md) — a COBOL dialect (ISO/IEC 1989:2023) -- [**plxplsql**](plxplsql.md) — an Oracle PL/SQL dialect -- [**plxtsql**](plxtsql.md) — a Transact-SQL (SQL Server) dialect +- [**plxruby**](plxruby.md): a Ruby dialect +- [**plxphp**](plxphp.md): a PHP dialect +- [**plxjs**](plxjs.md): a JavaScript dialect +- [**plxts**](plxts.md): a TypeScript dialect (plxjs plus type annotations) +- [**plxpython3**](plxpython3.md): a Python dialect +- [**plxgo**](plxgo.md): a Go dialect +- [**plxcobol**](plxcobol.md): a COBOL dialect (ISO/IEC 1989:2023) +- [**plxplsql**](plxplsql.md): an Oracle PL/SQL dialect +- [**plxtsql**](plxtsql.md): a Transact-SQL (SQL Server) dialect The [user guide](USERGUIDE.md) shows the same worked examples across dialects, and [feature parity](PARITY.md) is the construct-by-construct matrix. @@ -69,18 +69,23 @@ Build from source against your PostgreSQL installation: make && make install ``` -Then, in the database: +Then, in the database (installing the extension requires a superuser): ```sql CREATE EXTENSION plx; ``` -Upgrading from an earlier install: +Upgrading from an earlier install, after installing the new build: ```sql -ALTER EXTENSION plx UPDATE TO '1.2'; +ALTER EXTENSION plx UPDATE; -- or UPDATE TO '2.0.0' ``` +Upgrading changes no existing function: plx transpiles at `CREATE FUNCTION` +time, so re-run a function's DDL to pick up new transpiler behaviour (see the +2.0.0 entry in the +[changelog](https://github.com/commandprompt/plx/blob/master/CHANGELOG.md)). + plx supports PostgreSQL 13 through 18; see [compatibility](COMPATIBILITY.md). ## More diff --git a/doc/plxcobol.md b/doc/plxcobol.md index ff6de2c..799e3e2 100644 --- a/doc/plxcobol.md +++ b/doc/plxcobol.md @@ -27,15 +27,18 @@ A function body is a free-format COBOL program fragment: an optional ```sql CREATE FUNCTION add(a int, b int) RETURNS int LANGUAGE plxcobol AS $$ +WORKING-STORAGE SECTION. +01 RESULT PIC 9(9). PROCEDURE DIVISION. COMPUTE RESULT = A + B GOBACK RETURNING RESULT. $$; ``` -Function arguments are referenced by name (case-insensitively). Any SQL -expression is valid inside `COMPUTE` and conditions, because expressions are -passed through to plpgsql and SQL. +Function arguments are referenced by name (case-insensitively). Most SQL +expressions are valid inside `COMPUTE` and conditions, because expressions are +passed through to plpgsql and SQL. The `||` operator and `::` casts are not: +the tokenizer drops them, so use `CONCAT(...)` and `CAST(x AS type)`. ### Local variables and types @@ -49,7 +52,7 @@ clause is mapped to a SQL type: | `PIC 9(n)`, larger | `numeric(n)` | | `PIC 9(i)V9(s)` (implied decimal) | `numeric(i+s, s)` | | `PIC X(n)`, `PIC A(n)` | `varchar(n)` | -| `USAGE COMP-1` / `COMP-2` | `real` / `double precision` | +| `PIC ... USAGE COMP-1` / `COMP-2` (a `PIC` clause is required) | `real` / `double precision` | A leading `S` (sign) is accepted and ignored for typing. A `VALUE` clause becomes the declaration's initializer: @@ -85,8 +88,8 @@ writes element `i` (1-based): ```sql 01 WS-ARR PIC 9(9) OCCURS 5 TIMES. ... -COMPUTE WS-ARR(WS-I) = WS-I * WS-I -- write element -COMPUTE WS-SUM = WS-SUM + WS-ARR(WS-I) -- read element +COMPUTE WS-ARR(WS-I) = WS-I * WS-I *> write element +COMPUTE WS-SUM = WS-SUM + WS-ARR(WS-I) *> read element ``` Iterate the whole table with `PERFORM v OVER ARRAY WS-ARR`. A subscript is @@ -336,8 +339,8 @@ $$; ``` `NEW`, `OLD`, and the `TG_` variables are available. Assigning to a record field -(`MOVE e TO NEW.COL`) works because a qualified name maps to `new.col`; a bare -`MOVE e TO NEW` is not supported. +(`MOVE e TO NEW.COL`) works because a qualified name maps to `new.col`; +`MOVE OLD TO NEW` replaces the whole row. ## Semantic differences @@ -346,7 +349,9 @@ These are intentional. plx pins semantics to SQL and plpgsql. - Decimal literals infer `numeric`, not a floating-point type. - Integer division and modulo follow SQL (truncate toward zero). - A condition must be a boolean expression; there is no COBOL 88-level or - class-condition abbreviation. + class-condition abbreviation. An 88-level entry is rejected. A class + condition such as `N IS NUMERIC` is not rejected but mistranslated (`IS` is + dropped), so write a SQL boolean expression. - Data names are function-scoped after mapping to plpgsql identifiers, so two names that differ only by `-` versus `_` collide. @@ -360,8 +365,11 @@ Rejected at `CREATE FUNCTION` time with a line number: - Group items (subordinate level entries under an `01`) in `WORKING-STORAGE`; declare elementary items, or use a `TYPE` clause for a composite plpgsql type. - Fixed-format source (columns, sequence area, indicator area). Use free format. -- Report Writer, screen sections, object orientation, and the standard COBOL - intrinsic function library. Use SQL functions inside expressions and SQL text. +- Report Writer, screen sections, and object orientation. + +Not translated: the standard COBOL intrinsic function library. A +`FUNCTION name(...)` call is passed through and fails when the function runs. +Use SQL functions inside expressions and SQL text. See [PARITY.md](PARITY.md) for the full plpgsql construct matrix and [ARCHITECTURE.md](ARCHITECTURE.md) for how plx maps to the plpgsql engine. diff --git a/doc/plxgo.md b/doc/plxgo.md index b643f1e..d37b08d 100644 --- a/doc/plxgo.md +++ b/doc/plxgo.md @@ -57,7 +57,7 @@ declarations are translated (see below). | `x += e` (also `-= *= /= %=`) | `x := x + (e);` | | `x++` / `x--` | `x := x + 1;` / `x := x - 1;` | | `a, b = x, y` | `SELECT x, y INTO a, b;` (parallel, so swaps work) | -| `a, b := f(), g()` | declares `a` and `b`, then assigns | +| `a, b := 1, "x"` | declares `a` (integer) and `b` (text), then assigns; each value must have an inferable type | `:=` infers the type from a literal (`int`/`float`/`string`/`bool`), an array literal, `len(...)`, or a type conversion `T(x)`. If the type cannot be inferred @@ -87,10 +87,12 @@ three-clause `for` lowers to a `WHILE` with the post statement at the end of the body; there, as in plpgsql, `continue` re-tests the condition without running the post statement, so prefer the counting form when you use `continue`. -In a `range` loop, a single variable is the index (Go semantics), and two -variables are `(index, value)`. The value variable's type is inferred from the -slice's declared element type; `for i, v := range` (both index and value) is not -supported, so use `for _, v := range` or `for i := range`. +In a `range` loop over an integer, the single variable is the index. Over a +slice, use `for _, v := range s`. The value variable's type is inferred from the +slice's declared element type; when `s` is a parameter, declare `v` with `var` +before the loop. `for i := range s` and `for i, v := range s` over a slice are +not supported; walk a slice by index with a counting loop +(`for i := 0; i < len(s); i++`). ### switch @@ -120,10 +122,9 @@ $$; | `nil` | `NULL` | | `"..."`, `` `...` ``, `'x'` | SQL string literal (escapes decoded) | | `[]int{1, 2, 3}` | `ARRAY[1, 2, 3]` | -| `len(x)` | `length(x)` (text) or `cardinality(x)` (a slice) | +| `len(x)` | `length(x)` (text) or `coalesce(cardinality(x), 0)` (a slice declared in the body) | | `append(s, x)` | `array_append(s, x)` | | `a[i]` (subscript) | `a[(i) + 1]` (see below) | -| `[]int{1, 2, 3}` (again) | `ARRAY[1, 2, 3]` | | `int(x)`, `int64(x)`, `float64(x)`, `string(x)` | `(x)::integer` / `::bigint` / `::double precision` / `::text` | | `panic(m)` | `RAISE EXCEPTION '%', m;` | | `fmt.Println(x)` / `fmt.Printf(f, x)` | `RAISE NOTICE '%', x;` | @@ -133,8 +134,10 @@ $$; A slice literal `[]T{...}` becomes a PostgreSQL `ARRAY[...]`. Go slices are 0-based while PostgreSQL arrays are 1-based, so a subscript `a[i]` is rewritten to `a[(i) + 1]`. Indexing therefore follows Go semantics: `a[0]` is the first -element, and `len(a)`, `for i := range a`, and `a[i]` all agree on 0-based -positions. Go slice expressions (`a[i:j]`) are not translated. +element, and `len(a)` and `a[i]` agree on 0-based positions, so +`for i := 0; i < len(a); i++` walks a slice by index. `len` and `range` +recognize only slices declared in the body; copy an array parameter into one +first (`var a []int = p`). Go slice expressions (`a[i:j]`) are not translated. `fmt.Println`/`fmt.Printf` used as a statement raise a `NOTICE` with one `%` placeholder per value argument (space-separated); there the format string's @@ -163,8 +166,9 @@ rather than rounding it to two decimal places. Verbs that change an operand's representation rather than its padding are affected the same way: `%x`, `%o`, `%b`, `%e` and `%q` all render the operand's text, so `fmt.Sprintf("%x", 255)` yields `255` and not `ff`. -A doubled `%%` stays a literal `%`, and a `%` that starts no directive is -passed through as a literal percent. +A doubled `%%` stays a literal `%`. A `%` that starts no directive is also +rendered as a literal percent, but it makes plx fall back to `format()`, where +a NULL operand renders as empty instead of propagating; write `%%` instead. ### Types (in declarations) @@ -186,7 +190,8 @@ An unrecognized type name is passed through, so PostgreSQL type names also work. A subset of the standard library is mapped: `strings.ToUpper`/`ToLower`/ `TrimSpace`/`ReplaceAll`/`Contains`, `math.Abs`/`Floor`/`Ceil`/`Sqrt`/`Pow`/ -`Max`/`Min`/`Mod`, `strconv.Itoa`/`Atoi`, and `time.Now`. Anything not mapped is +`Max`/`Min`/`Mod`, `strconv.Itoa` (integer to text) and `strconv.Atoi` (text +to integer), and `time.Now`. Anything not mapped is emitted as written, so a PostgreSQL function of the same name still works. ## SQL access @@ -197,7 +202,7 @@ plxgo provides small SQL intrinsics, mirroring the other dialects: | --- | --- | | `emit(x)` | `RETURN NEXT x;` (for a set-returning function) | | `execute("...")` | `EXECUTE '...';` | -| `perform("...")` | `PERFORM ...;` | +| `perform(expr)` | `PERFORM expr;` (write the call as Go, for example `perform(nextval("seq"))`; a quoted string is evaluated as a literal and does nothing) | | `for _, row := range query("...") { ... }` | `FOR row IN EXECUTE '...' LOOP ...` | ```sql @@ -215,10 +220,13 @@ $$; - `map`, `chan`, `struct`, and `interface` types (declare data with SQL types or arrays instead). - `switch` `fallthrough`. +- Transaction control. `commit()` and `rollback()` are accepted when a procedure + is created but fail when it runs (`function commit() does not exist`). - String concatenation with `+`. Go overloads `+` for numeric addition and string concatenation; plx cannot tell which is meant without type information, - so it leaves `+` as `+`. Use `||` (valid in an expression) or build the string - another way for text concatenation. + so it leaves `+` as `+`. `||` is logical OR in plxgo, so use + `fmt.Sprintf("%s%s", a, b)` for text concatenation (a NULL operand makes the + result NULL). - The full `fmt`/`strings`/`math`/`strconv` packages; only the subset above is translated. diff --git a/doc/plxjs.md b/doc/plxjs.md index daf5b7e..f0e1b02 100644 --- a/doc/plxjs.md +++ b/doc/plxjs.md @@ -41,6 +41,12 @@ let amount /*:: numeric */; amount = principal * rate; ``` +Known issue: a top-level first assignment of an interpolated string is folded +into the declaration, so it is evaluated before any earlier statement runs. +``let x = 1; x = 2; let m = `v ${x}`; return m;`` returns `v 1`. When an +interpolated initializer depends on earlier statements, declare the variable +first (`let m /*:: text */;`) and then assign it. + The annotation text is emitted verbatim, so `%TYPE` and `%ROWTYPE` work: ```sql @@ -97,7 +103,7 @@ while (n > 0) { n = n - 1; } ```sql outer: for (let i = 1; i <= n; i++) { for (let j = 1; j <= n; j++) { - if (i + j >= limit) { break outer; } + if (i + j >= lim) { break outer; } } } ``` @@ -113,7 +119,8 @@ for (const row of query(`SELECT id, amount FROM orders WHERE grp = ${g}`)) { ``` The row variable is a record. Field access is `row.col`. For bind parameters or -non-literal SQL, pass extra arguments: `query(sql, a) of ...`. +non-literal SQL, pass extra arguments: `for (const row of query(sql, a))`, which +becomes `FOR row IN EXECUTE sql USING a`. ### Iterating an array @@ -164,7 +171,7 @@ close_cursor(c); ```sql let rc = row_count(); // GET DIAGNOSTICS rc = ROW_COUNT -found() // the FOUND variable +if (found()) { return rc; } // the FOUND variable ``` ### Set-returning functions @@ -190,20 +197,23 @@ The call form `raise("notice", "message")` emits a `RAISE` at the named level ### Handling ```sql +let r = 0; try { - return 100 / d; + r = 100 / d; } catch (e) { raise("notice", `caught: ${e.message}`); - return -1; + r = -1; } finally { perform("INSERT INTO log(msg) VALUES ('done')"); } +return r; ``` `catch (e)` maps to `WHEN OTHERS`. Accessors: `e.message` to `SQLERRM`, `e.sqlstate` to `SQLSTATE`, and `e.detail`, `e.hint`, `e.constraint`, `e.column`, `e.table`, `e.schema`, `e.datatype` to the matching `GET STACKED DIAGNOSTICS` -fields. +fields. A `return` inside the `try` or `catch` block leaves the function without +running `finally`; set a variable and return after the block instead. ### Assertions @@ -217,7 +227,10 @@ assert(n > 0, "must be positive"); `'...'` strings do not interpolate. Use template literals for concatenation. - Comparison: `==`, `===`, `!=`, `!==` map to `=` and `<>`. `x === null` becomes `x IS NULL`. -- Boolean: `&&`, `||`, `!` map to `AND`, `OR`, `NOT`. +- Boolean: `&&` and `!` map to `AND` and `NOT`. `||` maps to `OR` only in an + `if`/`while` condition or a ternary test. Elsewhere (assignments, `return`) + it passes through as SQL `||` (string concatenation), so write `OR` there: + `let ok = a OR b /*:: boolean */;`. - Ternary: `c ? a : b` becomes `CASE WHEN c THEN a ELSE b END`. - `null` and `undefined` become `NULL`. @@ -272,23 +285,35 @@ These are intentional. plx pins semantics to SQL and plpgsql. - `+` is numeric addition; use template literals for string building. - Interpolating a NULL in a template literal propagates it, so the whole string is NULL. It is neither the JavaScript string `"null"` nor an empty string. In - a message being built for `throw` the value falls back to an empty string - instead, so one NULL cannot swallow the message. + a `throw` or `raise` message, including the `raise("level", ...)` call form, + the value falls back to an empty string instead, so one NULL cannot swallow + the message. An `assert` message gets no fallback: a NULL there replaces your + message with plpgsql's generic `assertion failed`. - Integer division and modulo follow SQL (truncate toward zero). - JavaScript truthiness is not emulated. A condition must be a boolean - expression. A non-boolean condition is an error reported by plpgsql when the - function runs. -- Locals are function-scoped, matching JavaScript `var` scope. + expression. A non-boolean condition is not rejected at CREATE time. When the function runs, plpgsql coerces it + through its text form: a value whose text is a boolean literal (`0`, `1`, + `t`, `yes`, `off`, ...) is silently accepted, and anything else raises + `invalid input syntax for type boolean`. +- Locals are function-scoped, matching JavaScript `var` scope. `let` and + `const` inside a block are not block-scoped: a redeclaration in an inner block + assigns the outer variable. `const` is not enforced. ## Not supported Rejected at `CREATE FUNCTION` time with a line number: -- Function definitions, arrow functions, classes, `import`. -- Object and array literals as general values. -- `for...in`, and `for...of` over anything other than `query(...)` or an array. +- Function definitions, classes, `import`. +- `for...in`. - `switch` fall-through (end each case with `break` or `return`). -- Per-block local `DECLARE` (locals are function-scoped by design). + +Not translated: + +- Arrow functions and object and array literals as general values. Unannotated, + they fail at `CREATE FUNCTION` time with `cannot infer a PostgreSQL type`; + annotated, they pass through and fail when the function runs. +- `for...of` treats anything other than `query(...)` as an array + (`FOREACH ... IN ARRAY`); a non-array fails when the function runs. See [PARITY.md](PARITY.md) for the full plpgsql construct matrix and [ARCHITECTURE.md](ARCHITECTURE.md) for how plx maps to the plpgsql engine. diff --git a/doc/plxphp.md b/doc/plxphp.md index 8bbd8be..0e2de37 100644 --- a/doc/plxphp.md +++ b/doc/plxphp.md @@ -40,6 +40,12 @@ $amount /*:: numeric */; $amount = $principal * $rate; ``` +Known issue: a top-level first assignment of an interpolated string is folded +into the declaration, so it is evaluated before any earlier statement runs. The +body `$x = 1; $x = 2; $m = "v {$x}"; return $m;` returns `v 1`. When an +interpolated initializer depends on earlier statements, declare the variable +first (`$m /*:: text */;`) and then assign it. + The annotation text is emitted verbatim, so `%TYPE` and `%ROWTYPE` work: ```sql @@ -98,7 +104,7 @@ while ($n > 0) { $n = $n - 1; } ```sql outer: for ($i = 1; $i <= $n; $i++) { for ($j = 1; $j <= $n; $j++) { - if ($i + $j >= $limit) { break outer; } + if ($i + $j >= $lim) { break outer; } } } ``` @@ -166,7 +172,7 @@ close_cursor($c); ```sql $rc = row_count(); // GET DIAGNOSTICS $rc = ROW_COUNT -found() // the FOUND variable +if (found()) { return $rc; } // the FOUND variable ``` ### Set-returning functions @@ -207,6 +213,8 @@ which maps to `unique_violation`. Accessors: `$e->message` to `SQLERRM`, `$e->sqlstate` to `SQLSTATE`, and `$e->detail`, `$e->hint`, `$e->constraint`, `$e->column`, `$e->table`, `$e->schema`, `$e->datatype` to the matching `GET STACKED DIAGNOSTICS` fields. +A `return` inside the `try` or `catch` block leaves the function without +running `finally`; set a variable and return after the block instead. ### Assertions @@ -221,7 +229,10 @@ assert($n > 0, "must be positive"); - Concatenation: `$a . $b` maps to `$a || $b`. - Comparison: `==`, `===`, `!=`, `!==` map to `=` and `<>`. `$x == null` becomes `$x IS NULL`. -- Boolean: `&&`, `||`, `!` map to `AND`, `OR`, `NOT`. +- Boolean: `&&` and `!` map to `AND` and `NOT`. `||` maps to `OR` only in an + `if`/`while` condition or a ternary test. Elsewhere (assignments, `return`) + it passes through as SQL `||` (string concatenation), so write `OR` there: + `$ok = $a OR $b /*:: boolean */;`. - Ternary: `$c ? $a : $b` becomes `CASE WHEN $c THEN $a ELSE $b END`. - `null` becomes `NULL`. @@ -273,12 +284,17 @@ These are intentional. plx pins semantics to SQL and plpgsql. `==` and `!=` and use SQL type resolution. `"1" === 1` is false in PHP but compares as equal here. Compare like-typed values. - Interpolating a NULL propagates it, so the whole string becomes NULL rather - than rendering the value as an empty string. In a message being built for - `throw` the value falls back to an empty string instead, so one NULL cannot - swallow the message. + than rendering the value as an empty string. In a `throw` or `raise` message, + including the `raise("level", ...)` call form, the value falls back to an + empty string instead, so one NULL cannot swallow the message. An `assert` + message gets no fallback: a NULL there replaces your message with plpgsql's + generic `assertion failed`. - Integer division and modulo follow SQL (truncate toward zero). - PHP truthiness is not emulated. A condition must be a boolean expression. A - non-boolean condition is an error reported by plpgsql when the function runs. + non-boolean condition is not rejected at CREATE time. When the function runs, plpgsql coerces it + through its text form: a value whose text is a boolean literal (`0`, `1`, + `t`, `yes`, `off`, ...) is silently accepted, and anything else raises + `invalid input syntax for type boolean`. - Locals are function-scoped, matching PHP function scope. ## Not supported @@ -286,13 +302,18 @@ These are intentional. plx pins semantics to SQL and plpgsql. Rejected at `CREATE FUNCTION` time with a line number: - Function or class definitions, namespaces, `use`, includes. -- Closures and anonymous functions. -- Array and object literals as general values. - Non-counting `for`; only `for ($v = LO; $v < HI; $v++)` and `for (...; $v += K)` are supported. -- `foreach` over an array key/value pair (only value iteration). - `switch` fall-through (end each case with `break` or `return`). -- Per-block local `DECLARE` (locals are function-scoped by design). + +Not translated: + +- Closures, anonymous functions, and array and object literals as general + values. An unannotated assignment of one fails at `CREATE FUNCTION` time + (`cannot infer a PostgreSQL type`); an annotated one passes through and fails + when the function runs. +- `foreach ($a as $k => $v)`: the key is ignored; only value iteration is + supported. See [PARITY.md](PARITY.md) for the full plpgsql construct matrix and [ARCHITECTURE.md](ARCHITECTURE.md) for how plx maps to the plpgsql engine. diff --git a/doc/plxplsql.md b/doc/plxplsql.md index cb890b6..789ba52 100644 --- a/doc/plxplsql.md +++ b/doc/plxplsql.md @@ -60,7 +60,7 @@ Everything not listed here is emitted verbatim (it is already valid plpgsql). | `PLS_INTEGER`, `BINARY_INTEGER`, `SIMPLE_INTEGER` | `integer` | | `BINARY_FLOAT` / `BINARY_DOUBLE` | `real` / `double precision` | | `CLOB`, `NCLOB`, `LONG` | `text` | -| `BLOB`, `RAW` | `bytea` | +| `BLOB`, `RAW` | `bytea` (`RAW(n)` becomes the invalid `bytea(n)`, so declare `RAW` without a length) | ### Statements and functions @@ -86,7 +86,7 @@ CURSOR c IS SELECT v FROM t ORDER BY v; OPEN c; LOOP FETCH c INTO r; - EXIT WHEN NOT FOUND; -- or c%NOTFOUND, see notes + EXIT WHEN NOT FOUND; -- c%NOTFOUND is not translated; use NOT FOUND ... END LOOP; CLOSE c; @@ -110,9 +110,11 @@ EXCEPTION RETURN -1; ``` -Condition names such as `NO_DATA_FOUND`, `TOO_MANY_ROWS`, `DUP_VAL_ON_INDEX`, -and `ZERO_DIVIDE` are recognized by plpgsql as well; where a name differs, use -the SQLSTATE condition name. +The condition names `NO_DATA_FOUND` and `TOO_MANY_ROWS` are recognized by +plpgsql as well. Other Oracle names are not translated: use the PostgreSQL +condition name (`division_by_zero` for `ZERO_DIVIDE`, `unique_violation` for +`DUP_VAL_ON_INDEX`). An unrecognized name fails when the function is first +called with `unrecognized exception condition`. ## Dynamic SQL @@ -139,7 +141,8 @@ These are intentional. plx pins semantics to SQL and plpgsql. ## Not supported -Rejected or unchanged (and therefore likely to error) at `CREATE FUNCTION` time: +Not translated. These pass through unchanged, so `CREATE FUNCTION` succeeds and +the function fails with a plpgsql error when it is first called: - Packages (`CREATE PACKAGE`), package-qualified calls other than the `DBMS_OUTPUT` forms above, and the wider `DBMS_*` / `UTL_*` library. diff --git a/doc/plxpython3.md b/doc/plxpython3.md index 71bcb48..d24d827 100644 --- a/doc/plxpython3.md +++ b/doc/plxpython3.md @@ -41,6 +41,12 @@ amount #:: numeric amount = principal * rate ``` +Known issue: a top-level first assignment of an interpolated string is folded +into the declaration, so it is evaluated before any earlier statement runs. The +body `x = 1`, `x = 2`, `m = f"v {x}"`, `return m` returns `v 1`. When an +interpolated initializer depends on earlier statements, declare the variable +first (`m #:: text`) and then assign it. + The annotation text is emitted verbatim, so `%TYPE` and `%ROWTYPE` work: ```sql @@ -87,7 +93,7 @@ while n > 0: ```sql outer: for i in range(n): for j in range(n): - if i + j >= limit: + if i + j >= lim: break outer ``` @@ -150,7 +156,8 @@ close_cursor(c) ```sql rc = row_count() # GET DIAGNOSTICS rc = ROW_COUNT -found() # the FOUND variable +if found(): # the FOUND variable + return rc ``` ### Set-returning functions @@ -191,6 +198,8 @@ name it with the `PG::` spelling (`except PG::UniqueViolation as e`), which maps to `unique_violation`. Accessors: `e.message` to `SQLERRM`, `e.sqlstate` to `SQLSTATE`, and `e.detail`, `e.hint`, `e.constraint`, `e.column`, `e.table`, `e.schema`, `e.datatype` to the matching `GET STACKED DIAGNOSTICS` fields. +A `return` inside the `try` or `except` block leaves the function without +running `finally`; set a variable and return after the block instead. ### Assertions @@ -251,29 +260,43 @@ These are intentional. plx pins semantics to SQL and plpgsql. a comparison involving NULL is unknown. Use `is None` / `is not None` (which become `IS NULL` / `IS NOT NULL`) to test for NULL. A positive `if`/`while` condition treats NULL as false. -- Integer division follows SQL rules, not Python's `/` (float) and `//` (floor). - Division and modulo truncate toward zero: `-7 // 2` is `-4` in Python but the - SQL result here is `-3`, and `-7 % 2` is `1` in Python but `-1` here. +- Integer division follows SQL rules, not Python's. `/` on integers truncates + toward zero (`-7 / 2` is `-3`, where Python gives `-3.5`), and `%` takes the + sign of the dividend (`-7 % 2` is `-1`, where Python gives `1`). Floor + division `//` is not supported: it is passed through and fails when the + function runs with `operator does not exist: integer // integer`. - Interpolating a NULL in an f-string propagates it, so the whole string is NULL. It is neither the Python string `"None"` nor an empty string. In a - message being built for `raise` the value falls back to an empty string - instead, so one NULL cannot swallow the message. + `raise` message, including the `raise("level", ...)` call form, the value + falls back to an empty string instead, so one NULL cannot swallow the + message. An `assert` message gets no fallback: a NULL there replaces your + message with plpgsql's generic `assertion failed`. - Comparisons use SQL type resolution. `1 == "1"` compares an integer to a string literal, which SQL coerces and treats as equal; in Python it is false. - Python truthiness is not emulated. A condition must be a boolean expression. A - non-boolean condition is an error reported by plpgsql when the function runs. + non-boolean condition is not rejected at CREATE time. When the function runs, plpgsql coerces it + through its text form: a value whose text is a boolean literal (`0`, `1`, + `t`, `yes`, `off`, ...) is silently accepted, and anything else raises + `invalid input syntax for type boolean`. - Locals are function-scoped, matching Python function scope. ## Not supported Rejected at `CREATE FUNCTION` time with a line number: -- `def`, `class`, `lambda`, decorators, comprehensions, generators. -- `import`, modules, and the Python standard library. -- Tuples, lists, dicts, and sets as general values. +- `def`, `class` (reported as `def is not supported`), decorators, `yield`. +- `import`, `from`, and `with`. - `match`/`case`. - The conditional expression `a if c else b`. -- Per-block local scope (locals are function-scoped by design). + +Not translated: + +- `lambda`, comprehensions, generator expressions, and tuple, list, dict, and + set literals. Unannotated, they fail at `CREATE FUNCTION` time + (`cannot infer a PostgreSQL type`); annotated, they pass through and fail when + the function runs. +- The Python standard library: a call passes through and must name a + PostgreSQL function. See [PARITY.md](PARITY.md) for the full plpgsql construct matrix and [ARCHITECTURE.md](ARCHITECTURE.md) for how plx maps to the plpgsql engine. diff --git a/doc/plxruby.md b/doc/plxruby.md index 22a2c79..c49bf83 100644 --- a/doc/plxruby.md +++ b/doc/plxruby.md @@ -33,8 +33,8 @@ infers it from the first value when it is an integer, numeric, text, or boolean literal. Otherwise annotate the variable with `#:: type`. ```sql -n = 42 -- inferred integer -label = "count" -- inferred text +n = 42 # inferred integer +label = "count" # inferred text amount #:: numeric amount = principal * rate ``` @@ -42,6 +42,12 @@ amount = principal * rate A first assignment of a constant literal at the top level is folded into the declaration, so `n = 42` becomes `DECLARE n integer := 42;`. +Known issue: a top-level first assignment of an interpolated string is also +folded into the declaration, so it is evaluated before any earlier statement +runs. The body `x = 1`, `x = 2`, `m = "v #{x}"`, `return m` returns `v 1`. When +an interpolated initializer depends on earlier statements, declare the variable +first (`m #:: text`) and then assign it. + The annotation text is emitted verbatim as the plpgsql type, so `%TYPE` and `%ROWTYPE` work: @@ -160,7 +166,7 @@ one row. ```sql perform("UPDATE counters SET n = n + 1 WHERE id = #{cid}") execute("INSERT INTO t(msg) VALUES ($1)", note) -return_query("SELECT id FROM vip ORDER BY id") -- in a set-returning function +return_query("SELECT id FROM vip ORDER BY id") # in a set-returning function ``` `perform` runs a literal statement. `execute` runs dynamic SQL with optional bind @@ -183,8 +189,10 @@ close_cursor(c) ### Diagnostics ```sql -rc = row_count() -- GET DIAGNOSTICS rc = ROW_COUNT -found? -- the FOUND variable +rc = row_count() # GET DIAGNOSTICS rc = ROW_COUNT +if found? # the FOUND variable + return rc +end ``` ### Set-returning functions @@ -205,8 +213,8 @@ $$; ### Raising ```sql -raise "bad value" -- EXCEPTION -raise notice: "processed #{n} rows" -- NOTICE +raise "bad value" # EXCEPTION +raise notice: "processed #{n} rows" # NOTICE raise exception: "negative: #{v}", errcode: "22023" ``` @@ -244,7 +252,10 @@ assert(n > 0, "must be positive") - Interpolation: `"total is #{amount}"` becomes `'total is ' || (amount)::text`. Single-quoted strings do not interpolate. - Comparison: `==` and `!=` map to `=` and `<>`. `x == nil` becomes `x IS NULL`. -- Boolean: `&&`, `||`, `!`, and `and`, `or`, `not` map to `AND`, `OR`, `NOT`. +- Boolean: `&&`, `!`, `and`, `or`, and `not` map to `AND`, `NOT`, `AND`, `OR`, + and `NOT`. `||` maps to `OR` only in a condition (`if`, `unless`, `while`, + `until`, or a ternary test); elsewhere it passes through as SQL `||` + (string concatenation), so write `or` there. - Ternary: `c ? a : b` becomes `CASE WHEN c THEN a ELSE b END`. - Casts: `x.to_i`, `x.to_s`, `x.to_f` become `::integer`, `::text`, `::double precision`. `nil` becomes `NULL`. @@ -297,7 +308,11 @@ These are intentional. plx pins semantics to SQL and plpgsql. - Interpolating a NULL propagates it, so `"x=#{v}"` is NULL when `v` is NULL, rather than the `'x='` that Ruby's `nil.to_s` would give. In a message being built for `raise` the value falls back to an empty string instead, so one - NULL cannot swallow the message. + NULL cannot swallow the message. The fallback covers the message text only: + a NULL interpolated into a `detail:` or `hint:` option makes the `raise` + fail with `RAISE statement option cannot be null`, and a NULL in an `assert` + message replaces it with `assertion failed`. Wrap such values in + `coalesce(x, '')`. - Comparisons use SQL type resolution, not Ruby's. `1 == "1"` compares an integer to a string literal, which SQL coerces and treats as equal; in Ruby it is false. Compare like-typed values. @@ -305,8 +320,11 @@ These are intentional. plx pins semantics to SQL and plpgsql. and `-7 % 2` is `-1`, where Ruby gives `-4` and `1`. - Locals are function-scoped, matching Ruby method scope. - Ruby truthiness is not emulated. A condition must be a boolean expression; - write `x != 0` or `!x.nil?` rather than a bare `x`. A non-boolean condition is - an error reported by plpgsql when the function runs, not at CREATE time. + write `x != 0` or `x != nil` rather than a bare `x`. A non-boolean condition + is not rejected at CREATE time. When the function runs, plpgsql coerces it + through its text form: a value whose text is a boolean literal (`0`, `1`, + `t`, `yes`, `off`, ...) is silently accepted, and anything else raises + `invalid input syntax for type boolean`. ## Not supported @@ -315,7 +333,7 @@ Rejected at `CREATE FUNCTION` time with a line number: - Method or class definitions (`def`, classes, modules), gems. - Blocks and lambdas beyond the recognized `.each` forms. - Hash and array literals as general values. -- `||=`, `&&=`, and `and`/`or` in value position. +- `||=` and `&&=`. - `redo`, `retry`. - Predicate/bang methods (`x.zero?`, `arr.empty?`, `s.strip!`): the trailing `?`/`!` is not a SQL operator. Use the SQL form (`x = 0`, diff --git a/doc/plxts.md b/doc/plxts.md index 0a995f9..3df72d8 100644 --- a/doc/plxts.md +++ b/doc/plxts.md @@ -91,17 +91,21 @@ These are intentional and inherited from plxjs (see [plxjs.md](plxjs.md)). - The type system is not enforced at compile time: annotations only set the plpgsql declaration type. Structural/interface types, generics beyond `T[]`, and complex unions are not modeled; use a SQL type name for anything the table - above does not cover. + above does not cover. A union of two non-null types uses the first member + (`number | string` becomes `numeric`). ## Not supported -Everything plxjs rejects (function/class/arrow definitions, object/array -literals as general values, non-`query`/array `for...of`, `switch` -fall-through), plus: +Everything plxjs rejects or does not translate (see [plxjs.md](plxjs.md)), +plus: -- TypeScript type declarations (`interface`, `type`, `enum`), generics, and - type-only imports. +- TypeScript type declarations (`interface`, `type`, `enum`) and type-only + imports, which are rejected at `CREATE FUNCTION` time. +- Generic annotations (`Array`, `Map`) and nested arrays (`number[][]`) + are not mapped. They are emitted verbatim and fail with `invalid type name` + the first time the function runs; use `T[]` or a SQL type name. - Multiple annotated declarators in one statement - (`let a: number = 1, b: string = 'x';`); use one declaration per statement. + (`let a: number = 1, b: string = 'x';`), reported as a type-inference error; + use one declaration per statement. See [PARITY.md](PARITY.md) and [plxjs.md](plxjs.md). diff --git a/doc/plxtsql.md b/doc/plxtsql.md index a0a3a31..366fd5a 100644 --- a/doc/plxtsql.md +++ b/doc/plxtsql.md @@ -46,8 +46,8 @@ unwrapped (the declarations inside still hoist). The **body** is T-SQL, but the function **signature** (the parameter and return types in `CREATE FUNCTION`) is parsed by PostgreSQL before plx sees the body, so -it must use PostgreSQL type names: write `RETURNS int`, not `RETURNS INT` inside -a T-SQL `CREATE PROCEDURE`. Parameters are referenced in the body with the `@` +it must use PostgreSQL type names: write `RETURNS timestamp`, not +`RETURNS DATETIME`. Parameters are referenced in the body with the `@` sigil (`@score` above binds to the parameter `score`). Inside the body, T-SQL type names in `DECLARE` are translated (see below). @@ -196,8 +196,10 @@ session option. - Table variables (`DECLARE @t TABLE (...)`) and T-SQL cursors (`DECLARE c CURSOR`). - `GOTO`, `WAITFOR`, and label targets. -- Transaction control (`BEGIN TRAN`, `COMMIT`, `ROLLBACK`): a plpgsql function - runs inside the caller's transaction and cannot manage transactions. +- Transaction control (`BEGIN TRAN`, `COMMIT`, `ROLLBACK`), in a function or a + procedure; plxtsql rejects them everywhere. plxruby, plxphp, plxjs, plxts, + plxpython3, plxcobol and plxplsql support `COMMIT` and `ROLLBACK` in a + procedure; plxgo does not. - Calling a stored procedure by name (`EXEC procname`); only `EXEC('')` dynamic SQL is supported. - String concatenation with `+`. T-SQL overloads `+` for both numeric addition diff --git a/mkdocs.yml b/mkdocs.yml index 91047ac..5b4052f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -81,7 +81,7 @@ nav: - Gaps and limitations: LIMITATIONS.md - Compatibility: COMPATIBILITY.md - Architecture: ARCHITECTURE.md - - Transpiler: TRANSPILER.md + - Transpiler design (historical): TRANSPILER.md - Debugging: DEBUGGING.md extra: