From 102c40940ea80c448fa59744c1257ab09be75ccf Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Mon, 9 Nov 2026 18:55:01 +0000 Subject: [PATCH 1/2] Docs audit: fix factual errors, stale text, broken links, and em-dashes Every changed behavioural claim was re-run against PostgreSQL 18 with plx 2.0.0 installed. Factual errors: - Non-boolean conditions are coerced through text at run time (0, 1, 'yes' accepted), not always an error; drop the broken `!x.nil?` advice. - RAISE detail/hint options and assert messages get no NULL fallback; a NULL there fails the RAISE or loses the message. - Recreating a plx body as plpgsql still depends on plx when it uses the string builder. - COMMIT/ROLLBACK work in procedures (all dialects but plxtsql); add a procedures section to the user guide. - README example: distinct function names, real stored plpgsql including the sentinel and base64 trailer; benchmark summary matches the table. - USERGUIDE: note where dialect forms differ (WHEN OTHERS, SQLSTATE P0001); injection guidance applies to execute(), not query/perform/fetch_one. - SECURITY DEFINER caveat on "runs with the caller's privileges". - Pasteable snippets: comment syntax, undeclared COBOL RESULT, bare found(), reserved `limit`, a finally example that never ran. - Per-dialect corrections: value-position `||`, Python `//`, plxgo range over slices, len, perform, Sprintf lone `%`, multi-assign; plxplsql condition names, RAW(n), c%NOTFOUND, run-time (not CREATE-time) failures; plxcobol `||`/`::`, COMP-1/COMP-2, MOVE OLD TO NEW, class conditions, intrinsic functions; plxtsql signature example; plxts generics; "not supported" lists split into rejected vs not translated; Ruby `and`/`or` work in value position; Python loop labels exist. - Document the known issue that an interpolated top-level initializer is hoisted into DECLARE and evaluated early. - TRANSPILER.md banner lists the design choices that were not adopted. Stale or inconsistent: - Versions and file names (CHANGELOG header, index upgrade command, plx--2.0.0.sql header, install script names in README/ARCHITECTURE). - Test commands (differentialcheck, metacheck, fuzz), CI matrix, suite list, dialect lists in DEBUGGING and BENCHMARKS, PARITY plxtsql reachability. Broken links: cookbook pages linked to themselves instead of the chapter. Gaps: security model (superuser install, PUBLIC USAGE, plx USAGE implies plpgsql, nesting limits), search_path dependency of the string builder, upgrade instructions, dump/restore under 2.0.0. Style: remove all remaining em-dashes from doc/*.md; rename the Transpiler nav entry to mark it historical. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 6 +- README.md | 63 +++++++++++++----- bench/BENCHMARKS.md | 13 ++-- doc/ARCHITECTURE.md | 18 +++-- doc/COMPATIBILITY.md | 17 +++-- doc/DEBUGGING.md | 2 +- doc/LIMITATIONS.md | 102 +++++++++++++++++++--------- doc/MIGRATION.md | 8 ++- doc/PARITY.md | 20 +++--- doc/TRANSPILER.md | 123 ++++++++++++++++++---------------- doc/USERGUIDE.md | 132 ++++++++++++++++++++++++++++++------- doc/cookbook/plxcobol.md | 2 +- doc/cookbook/plxgo.md | 2 +- doc/cookbook/plxjs.md | 2 +- doc/cookbook/plxphp.md | 2 +- doc/cookbook/plxplsql.md | 2 +- doc/cookbook/plxpython3.md | 2 +- doc/cookbook/plxruby.md | 2 +- doc/cookbook/plxts.md | 2 +- doc/cookbook/plxtsql.md | 2 +- doc/index.md | 29 ++++---- doc/plxcobol.md | 30 +++++---- doc/plxgo.md | 36 +++++----- doc/plxjs.md | 57 +++++++++++----- doc/plxphp.md | 43 ++++++++---- doc/plxplsql.md | 15 +++-- doc/plxpython3.md | 47 +++++++++---- doc/plxruby.md | 42 ++++++++---- doc/plxts.md | 18 +++-- doc/plxtsql.md | 9 +-- mkdocs.yml | 2 +- plx--2.0.0.sql | 2 +- 32 files changed, 576 insertions(+), 276 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1930387..6f139c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ All notable changes to plx are recorded here. The format follows [Keep a Changelog](https://keepachangelog.com/), and plx uses the extension -version in `plx.control` (currently `1.0`). +version in `plx.control` (currently `2.0.0`). ## [2.0.1] - 2026-08-25 @@ -90,6 +90,10 @@ Each result interpolates a value that will become NULL when the value is NULL. Where the old rendering was wanted, make it explicit with `coalesce(x, '')` in the body. +The same applies to a dump taken from an earlier release and restored into +2.0.0: the restored functions keep the plpgsql stored in the dump, including +the empty-string rendering, until their DDL is run again. + ## [1.3.2] - 2026-08-24 Code-only patch release (no catalog changes) carrying a plxgo fix for diff --git a/README.md b/README.md index 59a2762..68ebc7c 100644 --- a/README.md +++ b/README.md @@ -106,10 +106,10 @@ is plpgsql. See [doc/ARCHITECTURE.md](doc/ARCHITECTURE.md) and ## Example -One function, written in three dialects, each producing the same plpgsql: +One function, written in three dialects, each producing equivalent plpgsql: ```sql -CREATE FUNCTION grade(score int) RETURNS text LANGUAGE plxruby AS $$ +CREATE FUNCTION grade_rb(score int) RETURNS text LANGUAGE plxruby AS $$ grade #:: text if score >= 90 grade = "A" @@ -121,14 +121,14 @@ CREATE FUNCTION grade(score int) RETURNS text LANGUAGE plxruby AS $$ return grade $$; -CREATE FUNCTION grade(score int) RETURNS text LANGUAGE plxphp AS $$ +CREATE FUNCTION grade_php(score int) RETURNS text LANGUAGE plxphp AS $$ if ($score >= 90) { $grade = "A"; } elseif ($score >= 80) { $grade = "B"; } else { $grade = "F"; } return $grade; $$; -CREATE FUNCTION grade(score int) RETURNS text LANGUAGE plxjs AS $$ +CREATE FUNCTION grade_js(score int) RETURNS text LANGUAGE plxjs AS $$ let grade = "F"; if (score >= 90) { grade = "A"; } else if (score >= 80) { grade = "B"; } @@ -137,26 +137,39 @@ CREATE FUNCTION grade(score int) RETURNS text LANGUAGE plxjs AS $$ $$; ``` -The stored plpgsql (in `pg_proc.prosrc`) for each is: +The stored plpgsql (in `pg_proc.prosrc`) for the plxruby and plxphp versions is +shown below. The plxjs version is the same except that it initializes `grade` +in its `DECLARE` (`grade text := 'F';`). ```plpgsql +/*plx:v1:plxruby:b04416a1*/ DECLARE grade text; BEGIN - IF score >= 90 THEN grade := 'A'; - ELSIF score >= 80 THEN grade := 'B'; - ELSE grade := 'F'; + IF score >= 90 THEN + grade := 'A'; + ELSIF score >= 80 THEN + grade := 'B'; + ELSE + grade := 'F'; END IF; RETURN grade; END; +/*plx-orig:b64$...$plx-orig*/ ``` +The first line is a sentinel naming the dialect, and the last line is a +comment carrying your original source in base64, so it survives dump and +restore and can be recovered (see [doc/DEBUGGING.md](doc/DEBUGGING.md)). +Everything between them is the plpgsql that runs. + ## Performance -Because functions execute as plpgsql, the plx dialects match plpgsql (within about -11 percent across five workloads) and inherit its performance profile: several -times faster than the embedded-interpreter PLs on row iteration, and competitive -on arithmetic, branching, and call overhead. +Because functions execute as plpgsql, the plx dialects match plpgsql within +about 10 percent on most workloads (the worst cases measured are plxruby row +iteration at 1.17x and plxcobol's `EVALUATE` branch at 1.32x) and inherit its +performance profile: several times faster than the embedded-interpreter PLs on +row iteration, and competitive on arithmetic, branching, and call overhead. The one place plpgsql is weak is building a string in a loop: `s := s || 'x'` is O(n^2), because text is immutable and each step copies the whole string. plx @@ -185,12 +198,23 @@ make PG_CONFIG=/path/to/pg_config make install PG_CONFIG=/path/to/pg_config ``` -Then, in a database: +Then, in a database (installing the extension requires a superuser): ```sql CREATE EXTENSION plx; ``` +To upgrade, install the new build and run, in each database: + +```sql +ALTER EXTENSION plx UPDATE; +``` + +Upgrading changes no existing function. plx transpiles at `CREATE FUNCTION` +time and stores the result, so a function keeps the plpgsql it was created with +until its DDL is run again. See [CHANGELOG.md](CHANGELOG.md) for the notes on +each release. + ## Documentation - Per-dialect chapters: [plxruby](doc/plxruby.md), [plxphp](doc/plxphp.md), @@ -218,7 +242,8 @@ CREATE EXTENSION plx; ## Layout ``` -plx.control, plx--1.0.sql extension control and install SQL +plx.control, plx--2.0.0.sql extension control and install SQL +plx--X--Y.sql upgrade scripts between releases Makefile PGXS build src/ C sources (transpiler, dialect surfaces, PL handler) doc/ per-dialect chapters, architecture, transpiler, parity @@ -231,11 +256,17 @@ bench/ benchmark harness and results ``` make installcheck # pg_regress suite +make differentialcheck # each dialect against a plpgsql reference +make metacheck # META.json against the PGXN spec (no server) python3 test/run_corpus.py # additional Ruby corpus runner +python3 test/fuzz.py # transpiler fuzzer ``` -`make installcheck` runs against an installed plx on a running server. The -expected output is in `test/expected/`. +`make installcheck` and `make differentialcheck` run against an installed plx on +a running server; run `differentialcheck` after `installcheck`, which creates +the `contrib_regression` database it uses. The expected output is in +`test/expected/`. CI runs installcheck, differentialcheck, and metacheck on +PostgreSQL 13 through 18. ## License diff --git a/bench/BENCHMARKS.md b/bench/BENCHMARKS.md index 841fa2e..996705f 100644 --- a/bench/BENCHMARKS.md +++ b/bench/BENCHMARKS.md @@ -28,7 +28,8 @@ plx uses plx-prefixed language names, so it coexists with the native plruby, plphp, plperl, and plpython3u. plxruby, plxphp, plxjs, plxpython3, and plxcobol transpile to plpgsql at -`CREATE FUNCTION` time, so at run time they execute as plpgsql. plperl and +`CREATE FUNCTION` time, so at run time they execute as plpgsql. plxts, plxgo, +plxplsql, and plxtsql were not benchmarked; they also execute as plpgsql. plperl and plpython3u run their own embedded interpreters and retrieve rows through SPI. ## Results @@ -56,7 +57,7 @@ In the strbuild column, native plpgsql is the `s := s || 'x'` baseline and the plx dialects use the builder. This is the intended native-versus-plx comparison: the same accumulation idiom is 1743 ms in stock plpgsql and about 10 ms in the plx dialects. The plx builder is also faster than the native PLs' own in-language -append on this workload (native plruby 53 ms, native plphp 14 ms), because it +append on this workload (native plruby 53 ms, native plphp 15 ms), because it mutates one expanded-object buffer in place rather than materializing an interpreter string per step. These numbers are on PostgreSQL 18; the builder's amortized-O(1) append requires PostgreSQL 18 (see the version note under String @@ -65,9 +66,9 @@ building). On PostgreSQL 13 to 17 the plx strbuild column would match plpgsql ## Analysis -- The plx dialects match plpgsql within about 10 percent on the arith, iter, and - call workloads, because the stored function body is plpgsql. There is no - run-time translation cost; the translation happens once at `CREATE FUNCTION`. +- The plx dialects match plpgsql within about 10 percent on the arith and call + workloads, and within 17 percent on iter, because the stored function body is + plpgsql. There is no run-time translation cost; the translation happens once at `CREATE FUNCTION`. The small spread among the dialects is measurement noise. - plxcobol (branch): the one place plxcobol trails is the four-way branch @@ -91,7 +92,7 @@ building). On PostgreSQL 13 to 17 the plx strbuild column would match plpgsql amortized O(1), and the transpiler lowers the dialect append operators (`<<`, `.=`, `+=` on a string) onto it. The result is about 170x faster than the stock plpgsql idiom, and faster than the native PLs building the same - string with their own append (native plruby 53 ms, native plphp 14 ms), which + string with their own append (native plruby 53 ms, native plphp 15 ms), which each materialize a fresh interpreter string per step. In hand-written plpgsql the same win is available by calling `plx_sb_append`, or by assembling text in SQL with `string_agg` over a set. diff --git a/doc/ARCHITECTURE.md b/doc/ARCHITECTURE.md index 15505d3..87c6bab 100644 --- a/doc/ARCHITECTURE.md +++ b/doc/ARCHITECTURE.md @@ -18,7 +18,8 @@ dialect syntax and emits plpgsql; the execution engine is plpgsql. ## Catalog wiring -For each dialect, `plx--1.0.sql` creates a language whose parts are: +For each dialect, the install script (`plx--2.0.0.sql` for the current release) +creates a language whose parts are: - `HANDLER` is `plx_call_handler`, which is bound to plpgsql's exported `plpgsql_call_handler` symbol (`AS '$libdir/plpgsql', 'plpgsql_call_handler'`). @@ -120,7 +121,8 @@ per-dialect branching in the driver. ## Files ``` -plx.control, plx--1.0.sql extension control and install SQL +plx.control, plx--2.0.0.sql extension control and install SQL +plx--X--Y.sql upgrade scripts between releases src/plx.h public ABI (PlxDialect, PlxFuncMeta) src/plx_int.h internal ABI (PlxSurface, canonical keywords) src/plx_engine.h engine interface: PlxCtx, tokens, symtab, and the @@ -151,12 +153,15 @@ is a new `plx_dialect_X.c` (a surface with its `parse_body`, plus a few The plx languages are declared `TRUSTED`. A plx function executes as plpgsql through plpgsql's call handler, so it can do what a plpgsql function can do and -nothing more: no filesystem, no network, no arbitrary native code, and all SQL -runs with the caller's privileges. plx embeds no language runtime, which is the +nothing more: no filesystem, no network, no arbitrary native code, and SQL runs +with the caller's privileges (or the owner's, for a `SECURITY DEFINER` +function), exactly as in plpgsql. plx embeds no language runtime, which is the difference from the native PL/Ruby and PL/PHP, which are untrusted because they load a full interpreter into the backend. The transpiler is C that parses untrusted input at `CREATE FUNCTION` time; it has a recursion-depth limit and is -fuzzed (see `test/fuzz.py`). +fuzzed (see `test/fuzz.py`). See the user guide's +[Trust and security](USERGUIDE.md#trust-and-security) section for who can use +the languages and how to restrict them. ## Testing @@ -193,7 +198,8 @@ instead of passing over it. ## Related documents -- [TRANSPILER.md](TRANSPILER.md): the original transpiler design specification. +- [TRANSPILER.md](TRANSPILER.md): the original transpiler design specification + (historical; the dialect chapters are authoritative where they differ). - [PARITY.md](PARITY.md): the plpgsql construct parity matrix. - The per-dialect chapters: [plxruby](plxruby.md), [plxphp](plxphp.md), [plxjs](plxjs.md), [plxts](plxts.md), [plxpython3](plxpython3.md), diff --git a/doc/COMPATIBILITY.md b/doc/COMPATIBILITY.md index 39ed54d..d24477f 100644 --- a/doc/COMPATIBILITY.md +++ b/doc/COMPATIBILITY.md @@ -3,9 +3,12 @@ ## Supported versions plx supports PostgreSQL 13 through 18 (released), and builds and passes on the -19 and 20 development lines. The full pg_regress suite (plxruby, plxphp, plxjs, -plxpython3, plxgo, plxcobol, plxplsql, plxts, plxtsql, and the rejection tests) -passes on each of PostgreSQL 13 through 18, plus 19 and 20 built from source. +19 and 20 development lines. The full pg_regress suite (the nine dialect files +plus plx_features, plx_output, plx_strbuild, and plx_errors) passes on each of +PostgreSQL 13 through 18. CI (`.github/workflows/ci.yml`) runs `make +installcheck`, `make differentialcheck`, and `make metacheck` on PostgreSQL 13 +through 18 from the PGDG packages on every push and pull request. The 19 and 20 +results were checked manually against source builds and are not part of CI. | PostgreSQL | Status | string builder | |------------|--------|----------------| @@ -15,8 +18,8 @@ passes on each of PostgreSQL 13 through 18, plus 19 and 20 built from source. | 16 | pass | correct, not accelerated | | 17 | pass | correct, not accelerated | | 18 | pass | accelerated (amortized O(1)) | -| 19 (beta) | pass (from source) | accelerated (amortized O(1)) | -| 20 (devel) | pass (from source) | accelerated (amortized O(1)) | +| 19 (beta) | pass (manual, from source) | accelerated (amortized O(1)) | +| 20 (devel) | pass (manual, from source) | accelerated (amortized O(1)) | PostgreSQL 19 and 20 compile with a C23 toolchain (for example gcc 15), where `pg_noreturn` becomes the standard `[[noreturn]]` attribute; plx writes it as the @@ -59,7 +62,9 @@ Two server APIs differ across the supported majors; plx handles both: ## Reproducing -`test/pg_matrix.sh` builds PostgreSQL 13 to 17 from source, builds plx against +CI is the reference: `.github/workflows/ci.yml` installs each PostgreSQL major +from PGDG, builds plx, and runs the suites listed above. To reproduce locally +from source, `test/pg_matrix.sh` builds PostgreSQL 13 to 17 from source, builds plx against each `pg_config`, and runs the regression suite per version. Run it as root in a build environment with the standard PostgreSQL build dependencies installed. diff --git a/doc/DEBUGGING.md b/doc/DEBUGGING.md index f133ec9..0bcd661 100644 --- a/doc/DEBUGGING.md +++ b/doc/DEBUGGING.md @@ -44,7 +44,7 @@ Then: SELECT plx_source('my_func(int)'); ``` -returns exactly the Ruby / PHP / JavaScript / Python / COBOL body you wrote. +returns exactly the dialect body you wrote. ## Why the error line differs from your source line diff --git a/doc/LIMITATIONS.md b/doc/LIMITATIONS.md index 76394b2..04f5f59 100644 --- a/doc/LIMITATIONS.md +++ b/doc/LIMITATIONS.md @@ -16,13 +16,19 @@ which dialect it was written in. These follow from running as plpgsql and are not specific to any one dialect. - **Trusted sandbox.** A plx function does what a plpgsql function can do and no - more: no filesystem access, no network access, no arbitrary native code. All - SQL runs with the privileges of the calling role. This is the point of the + more: no filesystem access, no network access, no arbitrary native code. SQL + runs with the privileges of the calling role, or of the function owner for a + `SECURITY DEFINER` function, exactly as in plpgsql. This is the point of the design, and the reason plx is a trusted language while the native PL/Ruby or - PL/PHP are not. -- **No transaction control.** A function runs inside the caller's transaction, so - `COMMIT`, `ROLLBACK`, `BEGIN TRANSACTION`, and savepoint management are not - available. This is a plpgsql function boundary, not a plx choice. + PL/PHP are not. Because plx is a syntax layer over plpgsql, USAGE on a plx + language is equivalent to USAGE on plpgsql; see + [Trust and security](USERGUIDE.md#trust-and-security). +- **Transaction control only in procedures.** A function runs inside the + caller's transaction, so `COMMIT` and `ROLLBACK` raise `invalid transaction + termination` there. In a `CREATE PROCEDURE` invoked with `CALL` (outside an + explicit transaction block), `commit()`/`rollback()` (COBOL `COMMIT`/`ROLLBACK`, + PL/SQL `COMMIT;`) work as in plpgsql. plxtsql does not support transaction + control. Savepoint management is not available in any dialect. - **The signature is SQL.** The parameter and return types in `CREATE FUNCTION` are parsed by PostgreSQL before plx sees the body, so they must be PostgreSQL type names (`integer`, `numeric`, `text`, `timestamp`, ...). Dialect type names @@ -36,16 +42,26 @@ These follow from running as plpgsql and are not specific to any one dialect. so a comparison involving NULL is unknown rather than false, and `==`/`===` become `=`. Integer division and modulo truncate toward zero. Conditions must be boolean expressions; source-language "truthiness" (a bare value used as a - condition) is not emulated. Interpolating a NULL propagates it: the value is - concatenated as-is, so the whole string becomes NULL the way SQL `||` does. - Two exceptions. A message being built for `RAISE` keeps each interpolated - value as an empty string, so one NULL cannot swallow the message. And - `plxcobol`'s `STRING-APPEND` does not propagate, because it builds its string - through the `plx_strbuild` accumulator, whose append treats a NULL as nothing - to append by design. + condition) is not emulated. A non-boolean condition is not rejected at + `CREATE FUNCTION` time. When the function runs, plpgsql coerces it through + its text form: a value whose text is a boolean literal (`0`, `1`, `t`, `yes`, + `off`, ...) is silently accepted, and anything else raises `invalid input + syntax for type boolean`. Always write an explicit comparison. + Interpolating a NULL propagates it: the value is concatenated as-is, so the + whole string becomes NULL the way SQL `||` does. Two exceptions. A message + being built for `RAISE` keeps each interpolated value as an empty string, so + one NULL cannot swallow the message. And `plxcobol`'s `STRING-APPEND` does not + propagate, because it builds its string through the `plx_strbuild` + accumulator, whose append treats a NULL as nothing to append by design. +- **The message fallback covers the message text only.** An interpolated NULL + in a `RAISE` option such as `detail:` or `hint:` makes the `RAISE` itself fail + with `RAISE statement option cannot be null`, which hides the error you meant + to raise. A NULL in an `assert` message replaces your message with plpgsql's + generic `assertion failed`. Wrap such values in `coalesce(x, '')`. - **String concatenation with `+` is not string concatenation.** In every dialect `+` is SQL numeric addition. Use the dialect's string form: interpolation - (Ruby/PHP/Python/JS/TS), `||`, or `CONCAT(...)`. + (Ruby/PHP/Python/JS/TS), `fmt.Sprintf` (Go), `||` (PL/SQL, T-SQL), or + `CONCAT(...)`. In plxgo `||` is logical OR, and in plxcobol it is dropped. - **Type inference is limited.** Where a dialect infers a local's type from its initializer (`x = 5`, `x := 5`, `let x = ...`), plx infers only from a literal (int, decimal, string, boolean), an array literal, or a direct type @@ -56,17 +72,24 @@ These follow from running as plpgsql and are not specific to any one dialect. amortized-O(1) building. The in-place path relies on a PostgreSQL 18 planner hook; on 13 through 17 the result is correct but the append is O(n^2), the same as plain concatenation. See [COMPATIBILITY.md](COMPATIBILITY.md). +- **The string builder must be on the search path.** Functions that use the + append operators reference the extension's `plx_strbuild` type and + `plx_sb_append` function unqualified, so the extension's schema must be on the + function's `search_path` at run time, including any `SET search_path` clause + on the function. Otherwise the function is created but fails when called with + `type "plx_strbuild" does not exist`. - **Runtime errors point at the generated plpgsql.** plpgsql requires all declarations at the top, so line numbers in a runtime error refer to the transpiled body, not your dialect source. `plx_source()` recovers the embedded original, and the mapping is usually close. See [DEBUGGING.md](DEBUGGING.md). - **A depth cap guards pathological input.** Deeply nested expressions or - statements (hundreds of levels) are rejected at `CREATE FUNCTION` time rather - than risking a stack overflow. -- **What is always reachable.** Every plpgsql statement type is expressible from - every dialect (assignment, `IF`/`CASE`, all loops, `RETURN`/set-returning, - cursors, dynamic SQL, `RAISE`, exception handling, triggers, `GET - DIAGNOSTICS`). See [PARITY.md](PARITY.md) for the construct-by-construct matrix. + statements (hundreds of levels) are rejected at `CREATE FUNCTION` time with a + `nested too deeply` error rather than risking a stack overflow. +- **What is always reachable.** Every plpgsql statement type (assignment, + `IF`/`CASE`, all loops, `RETURN`/set-returning, cursors, dynamic SQL, `RAISE`, + exception handling, triggers, `GET DIAGNOSTICS`) is expressible from every + dialect, except that plxtsql has no cursors or transaction control. See + [PARITY.md](PARITY.md) for the construct-by-construct matrix. A dialect gives you its syntax over SQL semantics, not its standard library. With the exception of the `fmt`/`strings`/`math`/`strconv` subset plxgo maps, a @@ -78,14 +101,17 @@ library call you would reach for out of habit. ## What each dialect does not support -The per-dialect chapter is authoritative; this is a quick reference. +The per-dialect chapter is authoritative; this is a quick reference. Some +items are rejected at `CREATE FUNCTION` time; others (closures, literals of +unsupported types, library calls) are not translated, so they pass through and +fail when the function runs. ### plxruby ([chapter](plxruby.md)) - Method, class, and module definitions (`def`, `class`, `module`); gems. - Blocks and lambdas beyond the recognized `.each` forms. - Hash and array literals as general values. -- `||=`, `&&=`, and `and`/`or` in value position. +- `||=` and `&&=`. - `redo`, `retry`. - Predicate and bang methods (`x.zero?`, `arr.empty?`, `s.strip!`); use the SQL form (`x = 0`, `cardinality(arr) = 0`). @@ -95,20 +121,24 @@ The per-dialect chapter is authoritative; this is a quick reference. - Function, class, and namespace definitions; `use`, includes, closures. - Array and object literals as general values. - Non-counting `for`; only `for ($v = LO; $v < HI; $v++)` and `+= K` steps. -- `foreach` over a key/value pair (value iteration only). +- `foreach` over a key/value pair: the key is ignored (value iteration only). - `switch` fall-through (end each case with `break` or `return`). ### plxjs ([chapter](plxjs.md)) - Function, arrow-function, and class definitions; `import`. - Object and array literals as general values. -- `for...in`, and `for...of` over anything other than `query(...)` or an array. +- `for...in`. `for...of` treats anything other than `query(...)` as an array, + so a non-array fails when the function runs. +- Block scoping: `let`/`const` in an inner block assign the function-scoped + variable of the same name. - `switch` fall-through. ### plxts ([chapter](plxts.md)) - Everything plxjs rejects, plus type declarations (`interface`, `type`, `enum`), - generics beyond `T[]`, and type-only imports. + and type-only imports. Generic annotations (`Array`) and nested arrays + (`number[][]`) are not mapped; use `T[]` or a SQL type name. - Multiple annotated declarators in one statement (`let a: number = 1, b = 2;`); use one declaration per statement. - Annotations are not type-checked; they only set the plpgsql declaration type. @@ -119,6 +149,7 @@ The per-dialect chapter is authoritative; this is a quick reference. - `import`, modules, and the Python standard library. - Tuples, lists, dicts, and sets as general values. - `match`/`case`, and the conditional expression `a if c else b`. +- Floor division `//`; `/` on integers truncates toward zero. ### plxgo ([chapter](plxgo.md)) @@ -127,7 +158,9 @@ The per-dialect chapter is authoritative; this is a quick reference. - `map`, `chan`, `struct`, and `interface` types; declare data with SQL types or slices. - `switch` `fallthrough`. -- `+` for string concatenation (use `||` or build a slice and `array_to_string`). +- `+` for string concatenation. `||` is logical OR in plxgo; use + `fmt.Sprintf("%s%s", a, b)` or build a slice and `array_to_string`. +- `for i := range s` over a slice; use `for _, v := range s` or a counting loop. - Only a subset of `fmt`/`strings`/`math`/`strconv` is mapped; other calls pass through and must be valid PostgreSQL functions. - `fmt.Sprintf` renders every operand in its SQL text form. A `-` flag and a @@ -146,8 +179,12 @@ The per-dialect chapter is authoritative; this is a quick reference. - Group items (subordinate entries under an `01`) in `WORKING-STORAGE`; declare elementary items, or use a `TYPE` clause for a composite type. - Fixed-format source (column areas); use free format. -- Report Writer, screen sections, object orientation, and the standard COBOL - intrinsic function library (use SQL functions in expressions). +- Report Writer, screen sections, and object orientation. The standard COBOL + intrinsic function library is not translated (a `FUNCTION name(...)` call + passes through and fails when the function runs); use SQL functions in + expressions. +- The `||` operator and `::` casts inside `COMPUTE` and conditions; use + `CONCAT(...)` and `CAST(x AS type)`. ### plxplsql ([chapter](plxplsql.md)) @@ -161,6 +198,10 @@ The per-dialect chapter is authoritative; this is a quick reference. - Oracle spellings inside a `DBMS_OUTPUT.PUT_LINE`/`RAISE_APPLICATION_ERROR` argument, schema-qualified `seq.NEXTVAL`, nested-`DECLARE` type names, and parameterized cursors. +- Oracle exception names other than `NO_DATA_FOUND` and `TOO_MANY_ROWS`; use the + PostgreSQL condition name (`division_by_zero` for `ZERO_DIVIDE`, + `unique_violation` for `DUP_VAL_ON_INDEX`). +- `c%NOTFOUND`/`c%FOUND` cursor attributes; use `NOT FOUND`/`FOUND`. ### plxtsql ([chapter](plxtsql.md)) @@ -178,5 +219,6 @@ Because every dialect compiles to the same plpgsql, you can write the awkward part of a function in whichever dialect expresses it best, or drop to a SQL statement inside any dialect (all of them pass SQL text through to `query`, `execute`, `RETURN QUERY`, and the like). A construct a dialect rejects is -reported at `CREATE FUNCTION` time with a source line, so gaps surface -immediately rather than at run time. +reported at `CREATE FUNCTION` time with a source line, so those gaps surface +immediately. Library calls and expressions passed through to SQL are not +checked until the function runs. diff --git a/doc/MIGRATION.md b/doc/MIGRATION.md index aa1253d..8d907c7 100644 --- a/doc/MIGRATION.md +++ b/doc/MIGRATION.md @@ -111,9 +111,11 @@ END; $do$; ``` -`grade_pg` is a normal plpgsql function with no dependency on the extension. -Applied across your functions, this is an exit: you keep the plpgsql and drop -plx. The original dialect source is preserved in a trailing comment inside the +`grade_pg` is a normal plpgsql function. It depends on the extension only if +its body uses the string builder (`plx_strbuild` and `plx_sb_append`, emitted for +the string-append operators); rewrite those appends as `||` or `string_agg` +before dropping plx. Applied across your functions, this is an exit: you keep +the plpgsql and drop plx. The original dialect source is preserved in a trailing comment inside the generated body, so recreating it as plpgsql does not throw away the source either. See [debugging](DEBUGGING.md) for the helper that decodes it. diff --git a/doc/PARITY.md b/doc/PARITY.md index 33777d6..f1f40f4 100644 --- a/doc/PARITY.md +++ b/doc/PARITY.md @@ -4,8 +4,8 @@ Target: every plpgsql capability is expressible in each plx dialect. The reference is the plpgsql statement set in `src/pl/plpgsql/src/plpgsql.h` (`PLpgSQL_stmt_*`) plus its declaration forms. -Every construct below is supported in all five column dialects — **plxruby**, -**plxphp**, **plxjs**, **plxpython3**, **plxcobol** — except where a footnote +Every construct below is supported in all five column dialects (**plxruby**, +**plxphp**, **plxjs**, **plxpython3**, **plxcobol**) except where a footnote notes an equivalent form. The cells show the dialect spelling; full syntax and semantics are in the per-dialect chapters ([plxruby](plxruby.md), [plxphp](plxphp.md), [plxjs](plxjs.md), [plxpython3](plxpython3.md), @@ -35,8 +35,10 @@ it restructures rather than renames: `@x` variables hoist to the `DECLARE` block `IF`/`WHILE` bodies written as `BEGIN ... END` become `THEN ... END IF` / `LOOP ... END LOOP`, `SET @x = e` becomes `x := e`, `SELECT @x = c FROM t` becomes `SELECT c INTO x FROM t`, `PRINT`/`RAISERROR`/`THROW` become `RAISE`, and -`TRY`/`CATCH` becomes an `EXCEPTION` block. So every plpgsql construct is -reachable, but through T-SQL surface syntax rather than a one-to-one keyword map. +`TRY`/`CATCH` becomes an `EXCEPTION` block. So most plpgsql constructs are +reachable through T-SQL surface syntax rather than a one-to-one keyword map; +the exceptions are cursors and transaction control, which plxtsql does not +support. See [plxtsql.md](plxtsql.md). ## Control flow @@ -54,16 +56,16 @@ See [plxtsql.md](plxtsql.md). | `FOR` over dynamic query | same, non-literal SQL / binds | same | same | same | `… OVER "sql" USING …` | | `FOREACH` over array | `arr.each do \|v\|` | `foreach ($a as $v)` | `for (const v of arr)` | `for v in arr:` | `PERFORM v OVER ARRAY a` | | `EXIT` / `CONTINUE` | `break` / `next` | `break` / `continue` | `break` / `continue` | `break` / `continue` | `EXIT PERFORM` / `EXIT PERFORM CYCLE` | -| loop labels ² | `label:` + `break label` | `label:` + `break label` | `label:` (native) | — | — | +| loop labels ² | `label:` + `break label` | `label:` + `break label` | `label:` (native) | `label:` + `break label` | not offered | ¹ Python has no `match`/`case`, and PHP/JS `switch` is a simple `CASE` only, so a searched (arbitrary-condition) or Python multi-way branch uses `if`/`elif`(`else -if`)/`else` — the same control flow, lowered to plpgsql `IF`. Ruby `case` and +if`)/`else`, the same control flow, lowered to plpgsql `IF`. Ruby `case` and COBOL `EVALUATE` / `EVALUATE TRUE` express both simple and searched `CASE` directly. ² Loop labels (`label:` before a loop with `break label` / `next label`) are -native in JavaScript and a plx extension in Ruby and PHP; not offered in Python -or COBOL. +native in JavaScript and a plx extension in Ruby, PHP, and Python; not offered +in COBOL. ## Return, SQL, and data access @@ -77,7 +79,7 @@ or COBOL. | SQL / `SELECT INTO` | `fetch_one(sql)` · `fetch_one!` | `fetch_one(sql)` | `fetch_one(sql)` | `fetch_one(sql)` | `EXECUTE "sql" INTO v` | | cursors `OPEN`/`FETCH`/`MOVE`/`CLOSE` | `open_cursor` · `fetch_from` · `move_cursor` · `close_cursor` | (same call form) | (same call form) | (same call form) | `OPEN-` · `FETCH-` · `MOVE-` · `CLOSE-CURSOR` | | `CALL` procedure | `call("p", a)` | `call("p", $a)` | `call("p", a)` | `call("p", a)` | `CALL "p" USING a` | -| `COMMIT` / `ROLLBACK` | `commit()` / `rollback()` | (same) | (same) | (same) | `COMMIT` / `ROLLBACK` | +| `COMMIT` / `ROLLBACK` (procedures only) | `commit()` / `rollback()` | (same) | (same) | (same) | `COMMIT` / `ROLLBACK` | The call-form intrinsics (`perform`, `execute`, `fetch_one`, the `*_cursor` functions, `call`, `commit`, `rollback`, `return_next`, `return_query`) share the diff --git a/doc/TRANSPILER.md b/doc/TRANSPILER.md index fcb7779..00ec5e0 100644 --- a/doc/TRANSPILER.md +++ b/doc/TRANSPILER.md @@ -9,6 +9,15 @@ > `plx.so` rather than separate per-dialect modules. For the as-built > architecture and the per-dialect behavior, see ARCHITECTURE.md and the dialect > chapters. This file is kept as a design reference. +> +> Several semantic choices below were not adopted. `==`/`!=` lower to plain +> `=`/`<>` (not `IS [NOT] DISTINCT FROM`). Negation (`unless`, `!`, `not`) is +> `NOT` (not `IS NOT TRUE`). Value-position `||` is SQL concatenation (not +> `COALESCE`). Since 2.0.0 an interpolated NULL propagates, so interpolation is +> not wrapped in `COALESCE((e)::text, '')`; only a RAISE message keeps that +> empty-string fallback. The sentinel names the dialect as `plxruby`, not +> `plruby`. Where this file and the dialect chapters disagree, the chapters are +> authoritative. This is the specification for the Ruby surface. The transpiler runs at `CREATE FUNCTION` time inside `plx_generic_validator`, rewrites `pg_proc.prosrc` @@ -95,24 +104,24 @@ typedef struct { typedef struct { int line, col; char *msg; } PlxError; ``` -### 2.2 Pass 1 — byte-level lexer (`plx_lex`) +### 2.2 Pass 1: byte-level lexer (`plx_lex`) Skip inline whitespace (space/tab, **not** newline). Dispatch on current char: -- **letter/`_`** — scan `[A-Za-z_][A-Za-z0-9_]*` plus a trailing `?`/`!`. Keyword-table `bsearch` → `PLX_TOK_KEYWORD` with a `PlxKw`, else `PLX_TOK_IDENT`. -- **digit** — scan number. **Range guard:** on hitting `.`, peek `p[1]`; if it is also `.`, STOP the number (emit `INT`, let `..`/`...` become an OP). A single `.` before a digit is a float fraction. Underscores in numeric literals (`1_000`) are consumed and stripped at rewrite. Record whether the literal exceeds int4 range for the type-inference pass. -- **`'`** — single-quoted string. Only `\\` and `\'` are escapes; **everything else, including `#{`, is inert literal text.** One opaque `PLX_TOK_STRING` with `sq_string=true`, `interp_count=0`. -- **`"`** — double-quoted string. Escapes `\" \\ \n \t \r \0 \e \s \a \b \f \v \nnn \xHH \uXXXX`. **`\#{` (backslash before `#{`) suppresses interpolation** — the two chars `#{` become literal content. On an *unescaped* `#{`, push an interp context and scan a **balanced** sub-expression to the matching `}`: brace counter starts at 1, `++` on `{`, `--` on `}`, but only when not inside a nested string; nested `"..."`/`'...'` and nested `#{}` are honored. Record each interp `(start,len)` span. Empty/whitespace-only `#{}` is recorded with `len==0` (dropped later). One opaque `PLX_TOK_STRING`, `sq_string=false`, carrying its span list. -- **`%`** — percent-literal only in **value position** (prev significant token is OP/`(`/`,`/keyword/terminator) and `p[1]∈{q,Q,w,W,i,I}` with a delimiter following: `%q`=single semantics, `%Q`=double/interp, bracket delimiters `()[]{}<>` nest. `%w`/`%i` (word/symbol arrays) are lexed as an opaque STRING but flagged unsupported → rejected in lowering. Otherwise `%` is the modulo OP. -- **`<`** — `<<~IDENT` / `<<-IDENT` / `< + - * / & ? ~ ^` — scan the maximal run into one `PLX_TOK_OP` (`==`,`!=`,`&&`,`||`,`<=`,`>=`,`..`,`...`,`=>`,`<<`,`**` come through whole). -- **else** — record first error, `PLX_TOK_ERROR`. +- **letter/`_`**: scan `[A-Za-z_][A-Za-z0-9_]*` plus a trailing `?`/`!`. Keyword-table `bsearch` → `PLX_TOK_KEYWORD` with a `PlxKw`, else `PLX_TOK_IDENT`. +- **digit**: scan number. **Range guard:** on hitting `.`, peek `p[1]`; if it is also `.`, STOP the number (emit `INT`, let `..`/`...` become an OP). A single `.` before a digit is a float fraction. Underscores in numeric literals (`1_000`) are consumed and stripped at rewrite. Record whether the literal exceeds int4 range for the type-inference pass. +- **`'`**: single-quoted string. Only `\\` and `\'` are escapes; **everything else, including `#{`, is inert literal text.** One opaque `PLX_TOK_STRING` with `sq_string=true`, `interp_count=0`. +- **`"`**: double-quoted string. Escapes `\" \\ \n \t \r \0 \e \s \a \b \f \v \nnn \xHH \uXXXX`. **`\#{` (backslash before `#{`) suppresses interpolation**; the two chars `#{` become literal content. On an *unescaped* `#{`, push an interp context and scan a **balanced** sub-expression to the matching `}`: brace counter starts at 1, `++` on `{`, `--` on `}`, but only when not inside a nested string; nested `"..."`/`'...'` and nested `#{}` are honored. Record each interp `(start,len)` span. Empty/whitespace-only `#{}` is recorded with `len==0` (dropped later). One opaque `PLX_TOK_STRING`, `sq_string=false`, carrying its span list. +- **`%`**: percent-literal only in **value position** (prev significant token is OP/`(`/`,`/keyword/terminator) and `p[1]∈{q,Q,w,W,i,I}` with a delimiter following: `%q`=single semantics, `%Q`=double/interp, bracket delimiters `()[]{}<>` nest. `%w`/`%i` (word/symbol arrays) are lexed as an opaque STRING but flagged unsupported → rejected in lowering. Otherwise `%` is the modulo OP. +- **`<`**: `<<~IDENT` / `<<-IDENT` / `< + - * / & ? ~ ^`: scan the maximal run into one `PLX_TOK_OP` (`==`,`!=`,`&&`,`||`,`<=`,`>=`,`..`,`...`,`=>`,`<<`,`**` come through whole). +- **else**: record first error, `PLX_TOK_ERROR`. After each terminator, set `stmt_start=true` on the next significant token. **`stmt_start` bookkeeping is a first-class correctness surface** (opener vs modifier depends on it) and must be re-asserted correctly after a suppressed newline and after a heredoc body drain. @@ -126,7 +135,7 @@ A physical `\n` is DROPPED (statement continues) iff **any** of: Otherwise emit `PLX_TOK_NEWLINE`. `;` is always a separator. The cannot-end set is one `static const` classifier keyed on the OP token bytes plus a short `strcmp` for `..`/`...`/`=>`. -### 2.4 Pass 2 — block builder (`plx_build_blocks`) +### 2.4 Pass 2: block builder (`plx_build_blocks`) Push `BLK_TOPLEVEL`. Walk tokens, cutting statements at `NEWLINE`/`SEMI` in the current frame. Classify each statement's leading `stmt_start` token: @@ -140,7 +149,7 @@ Push `BLK_TOPLEVEL`. Walk tokens, cutting statements at `NEWLINE`/`SEMI` in the At EOF, a non-`[TOPLEVEL]` stack → `ereport` "unterminated `` block opened at line N"; a stray `end`/`}` → matching-close error at its own line. -Every token carries a 1-based dialect line; every block records `open_line` — feeding the plpgsql→dialect source map and line-accurate errors. +Every token carries a 1-based dialect line; every block records `open_line`; feeding the plpgsql→dialect source map and line-accurate errors. --- @@ -183,10 +192,10 @@ The parse ctx carries a `loop_depth` and a `handler_depth` counter (for `next`/` Emission mechanics: 2-space indent per level, body at level 1. Leaf: `;\n` (node supplies text sans `;`; emitter appends exactly one). Compound opener line ends in the plpgsql keyword (`THEN`/`LOOP`) with **no** `;`; children at indent+1; closers `END IF;`/`END LOOP;`/`END;`. Subquery aliases `__plx_fo_N`/`__plx_p_N` use a monotonic counter. -### PLX_ASSIGN — `LHS := RHS;` -`x = e` → `x := e';`. Compound `x += e` → `x := x + (e');` (same for `-= *= /= %=`). Multi-assign `a, b = e1, e2` → `SELECT e1', e2' INTO a, b;` (simultaneous → `a,b=b,a` swaps correctly). LHS must be a bare local or an OUT/TABLE-column param (from meta); assigning to `row.field` is rejected. **`||=`/`&&=` are rejected** (no faithful three-valued lowering) — message points at the explicit form, which itself lowers per the value-context `||` rule (§6). RHS-intrinsics (`fetch_one`, `query`, `emit`, …) are recognized **first** and routed to their own node; they are never assignments. +### PLX_ASSIGN: `LHS := RHS;` +`x = e` → `x := e';`. Compound `x += e` → `x := x + (e');` (same for `-= *= /= %=`). Multi-assign `a, b = e1, e2` → `SELECT e1', e2' INTO a, b;` (simultaneous → `a,b=b,a` swaps correctly). LHS must be a bare local or an OUT/TABLE-column param (from meta); assigning to `row.field` is rejected. **`||=`/`&&=` are rejected** (no faithful three-valued lowering); message points at the explicit form, which itself lowers per the value-context `||` rule (§6). RHS-intrinsics (`fetch_one`, `query`, `emit`, …) are recognized **first** and routed to their own node; they are never assignments. -### PLX_IF — if/elsif/else, unless, modifier-if +### PLX_IF: if/elsif/else, unless, modifier-if ``` IF C' THEN @@ -196,12 +205,12 @@ ELSE END IF; ``` -`unless C` → `IF (C') IS NOT TRUE THEN … END IF;` (**not** `NOT (C')` — matches Ruby falsey nil/false; see §6). Modifier `STMT if C` → `IF C' THEN STMT; END IF;`. Positive `if C` uses `C'` directly (NULL correctly skips the THEN arm). Condition must be boolean; **no truthiness coercion** — a bare non-boolean condition is a CREATE-time reject (§6/§7), not `IF `. +`unless C` → `IF (C') IS NOT TRUE THEN … END IF;` (**not** `NOT (C')`; matches Ruby falsey nil/false; see §6). Modifier `STMT if C` → `IF C' THEN STMT; END IF;`. Positive `if C` uses `C'` directly (NULL correctly skips the THEN arm). Condition must be boolean; **no truthiness coercion**; a bare non-boolean condition is a CREATE-time reject (§6/§7), not `IF `. -### PLX_WHILE — while/until +### PLX_WHILE: while/until `while C` → `WHILE C' LOOP … END LOOP;`. `until C` → `WHILE (C') IS NOT TRUE LOOP … END LOOP;`. Modifiers `STMT while C` / `STMT until C` expand to the full loop. `begin…end while` (do-while) is rejected. -### PLX_LOOP — `loop do … end` +### PLX_LOOP: `loop do … end` ``` LOOP … @@ -209,7 +218,7 @@ END LOOP; ``` Requires a reachable `break`/`return` (not statically enforced). -### PLX_FORINT — integer for +### PLX_FORINT: integer for Surfaces: `for v in LO..HI`, `for v in LO...HI`, `(LO..HI).each do |v|`, `(LO...HI).each`, `HI.downto(LO) do |v|`, `LO.upto(HI) do |v|`, `(LO..HI).step(k).each`, `LO.step(HI,k)`. ``` @@ -217,7 +226,7 @@ FOR v IN [REVERSE] LO'..HI' [BY k'] LOOP … END LOOP; ``` Fixes folded in: -- **Exclusive `...` is a property of the range node**, applied on every path: `LO'..(HI' - 1)`. So `(0...9).step(3)` → `FOR v IN 0..(9 - 1) BY 3 LOOP` (yields 0,3,6 — endpoint 9 correctly excluded). Guard the `(HI-1)` rewrite against INT_MIN underflow. +- **Exclusive `...` is a property of the range node**, applied on every path: `LO'..(HI' - 1)`. So `(0...9).step(3)` → `FOR v IN 0..(9 - 1) BY 3 LOOP` (yields 0,3,6; endpoint 9 correctly excluded). Guard the `(HI-1)` rewrite against INT_MIN underflow. - `downto` → `REVERSE HI'..LO'`. `upto` → ascending. - **`.step(k)` with a literal negative k** → `FOR v IN REVERSE LO'..HI' BY (-k) LOOP` (Numeric#step limit is inclusive; no `-1`). Never emit `BY `. - **`.step(k)` with a non-literal k** → emit ascending `BY k'` and **document that k must be `>0` at runtime** (plpgsql cannot pick direction dynamically); optionally a CREATE-time NOTICE. @@ -228,7 +237,7 @@ Fixes folded in: Loop var `v` is loop-local, plpgsql-declared, **not hoisted**; endpoints pass through to plpgsql. `v` referenced **outside** the loop body → reject `plruby: for-loop variable "v" is not visible after the loop` (Ruby's `for` leak is intentionally dropped and detected, not silently mis-emitted). -### PLX_FORQUERY — query iteration +### PLX_FORQUERY: query iteration `query(SQL[, args]).each do |row|` / `.each_with_index do |row, idx|`. **Static** (SQL is a string literal, no bind args). Interpolation `#{name}` inside the literal is spliced as a **bare plpgsql NAME reference** *only when it is in value position*: @@ -242,30 +251,30 @@ FOR row IN SELECT id, amount FROM orders WHERE cust = cid LOOP … END LOOP; FOR row IN EXECUTE USING a', b' LOOP … END LOOP; ``` -- **`row` is ALWAYS hoisted as `row RECORD;`** (mandatory — plpgsql auto-declares only *integer* FOR vars; a query FOR needs a pre-declared record var, else `"row" is not a known variable`). Register `row` in the active-loop-var stack for the body. +- **`row` is ALWAYS hoisted as `row RECORD;`** (mandatory: plpgsql auto-declares only *integer* FOR vars; a query FOR needs a pre-declared record var, else `"row" is not a known variable`). Register `row` in the active-loop-var stack for the body. - `each_with_index`: the index var is a **synthesized local** routed through the symtab (`idx integer;`), registered in the active-loop stack and checked against params. Emit `idx := -1;` immediately before the FOR and inject `idx := idx + 1;` as the **very first** body statement (above any `next if` guard) → 0-based, matching Ruby. - Field access `row.col`, `row[:col]`, `row['col']` all normalize to `row.col`. -- Nested/overlapping loops need distinct row names (checked against the active stack only — sequential sibling loops may reuse `row`, §5). +- Nested/overlapping loops need distinct row names (checked against the active stack only; sequential sibling loops may reuse `row`, §5). -### PLX_FETCH — `fetch_one` → SELECT INTO +### PLX_FETCH: `fetch_one` → SELECT INTO **Static:** wrap the user SQL so INTO needs no SQL parse: ``` SELECT * INTO [STRICT] FROM ( ) AS __plx_fo_1; ``` **Dynamic** (bind args / non-literal): `EXECUTE INTO [STRICT] USING args';`. -`fetch_one` non-STRICT (0 rows → all-NULL, extra rows → first); `fetch_one!` STRICT. Whole-row target hoists `RECORD`; scalar destructure targets need annotation/inference. Set-returning DML (`INSERT … RETURNING`) as the user SQL must use the dynamic/EXECUTE path (the subquery wrap fails on it). `.nil?` (row-presence) lowers to **`NOT FOUND`** evaluated right after the INTO — not `IS NULL` (which is true only when *every* column is NULL; §6). +`fetch_one` non-STRICT (0 rows → all-NULL, extra rows → first); `fetch_one!` STRICT. Whole-row target hoists `RECORD`; scalar destructure targets need annotation/inference. Set-returning DML (`INSERT … RETURNING`) as the user SQL must use the dynamic/EXECUTE path (the subquery wrap fails on it). `.nil?` (row-presence) lowers to **`NOT FOUND`** evaluated right after the INTO; not `IS NULL` (which is true only when *every* column is NULL; §6). -### PLX_PERFORM — run SQL, discard (static, literal, no binds) +### PLX_PERFORM: run SQL, discard (static, literal, no binds) DML (INSERT/UPDATE/DELETE/MERGE) → verbatim `…;`. Row-returning (SELECT/WITH/VALUES/TABLE) → `PERFORM * FROM ( ) AS __plx_p_1;`. Value-position interpolation → name reference (identifier position → reject/dynamic, as FORQUERY). -### PLX_EXECUTE — dynamic SQL, discard +### PLX_EXECUTE: dynamic SQL, discard `execute(SQL_expr[, a, b])` → `EXECUTE [USING a', b'];`. Always dynamic. Interpolation `#{e}` → runtime concat `|| COALESCE((e)::text,'')` (Ruby nil→''; also prevents `EXECUTE NULL`). User owns injection safety; recommend `USING`/`format(%I,%L)`. ### PLX_RETURN -`return` → `RETURN;`. `return e` → `RETURN e';`. In a SETOF function `return e` is rejected (use `emit`/`return_next`/`return_query`); bare `return` → `RETURN;`. `return a, b` with matching OUT params → `out1 := a'; out2 := b'; RETURN;`. **No implicit last-expression return** — a scalar function must end with explicit `return e`. +`return` → `RETURN;`. `return e` → `RETURN e';`. In a SETOF function `return e` is rejected (use `emit`/`return_next`/`return_query`); bare `return` → `RETURN;`. `return a, b` with matching OUT params → `out1 := a'; out2 := b'; RETURN;`. **No implicit last-expression return**; a scalar function must end with explicit `return e`. -### PLX_RETURN_NEXT — emit / return_next +### PLX_RETURN_NEXT: emit / return_next Bare `emit`/`return_next` → `RETURN NEXT;`. `emit(e)`/`return_next e` → `RETURN NEXT e';`. Requires `meta.retset`, else hard error. ### PLX_RETURN_QUERY @@ -274,9 +283,9 @@ Static: `RETURN QUERY ;`. Dynamic: `RETURN QUERY EXECUTE 0`), else hard error. `break `, labeled break, `redo`, `retry` rejected. -### PLX_EXCEPTION — begin/rescue/ensure +### PLX_EXCEPTION: begin/rescue/ensure Without `ensure`: ``` BEGIN @@ -348,7 +357,7 @@ typedef struct PlxSymtab { ``` ### 5.2 Seeding params (never hoist) -From `PlxFuncMeta->argnames[i]` for **all** argmodes (IN/OUT/INOUT/VARIADIC/TABLE). **Skip NULL/empty argnames** (`f(int, OUT numeric)` has unnamed args) — guard every `hash_search` key against NULL (a NULL key into a `HASH_STRINGS` dynahash segfaults). Case-fold with `downcase_truncate_identifier` (63-byte `NAMEDATALEN` truncation) to match plpgsql exactly. +From `PlxFuncMeta->argnames[i]` for **all** argmodes (IN/OUT/INOUT/VARIADIC/TABLE). **Skip NULL/empty argnames** (`f(int, OUT numeric)` has unnamed args); guard every `hash_search` key against NULL (a NULL key into a `HASH_STRINGS` dynahash segfaults). Case-fold with `downcase_truncate_identifier` (63-byte `NAMEDATALEN` truncation) to match plpgsql exactly. ### 5.3 Collection walk Single recursive descent. On a **loop node**: push its var(s) onto `active_loopvars`, recurse, pop on close. @@ -380,37 +389,37 @@ Empty symtab → no `DECLARE` keyword. Else `DECLARE\n` then one line per local ## 6. Expression rewrite rules & accepted divergences -`plx_rewrite_expr(char *src, ctx, boolean_context)` → `char *`, a **single left-to-right pass over source tokens** honoring string/`#{}`/bracket state. It **never re-scans its own output** (the `||`/`OR` invariant). Inside a **single-quoted** string, NOTHING is rewritten — no `#{}` splice, no operator/keyword rewrite, no `+`→`||`; only `'`→`''` for re-emission. `#{}` is an interpolation **only** in double-quoted strings/heredocs. +`plx_rewrite_expr(char *src, ctx, boolean_context)` → `char *`, a **single left-to-right pass over source tokens** honoring string/`#{}`/bracket state. It **never re-scans its own output** (the `||`/`OR` invariant). Inside a **single-quoted** string, NOTHING is rewritten; no `#{}` splice, no operator/keyword rewrite, no `+`→`||`; only `'`→`''` for re-emission. `#{}` is an interpolation **only** in double-quoted strings/heredocs. -### 6.1 String literals — decode then re-encode (never memcpy source bytes) +### 6.1 String literals: decode then re-encode (never memcpy source bytes) Ruby string escape alphabet ≠ plpgsql literal alphabet. Always **fully decode** the Ruby token to its abstract character sequence, then **re-encode** for the chosen target literal: - **Single-quoted** Ruby string: resolve `\\`→`\`, `\'`→`'`; all other bytes literal. Emit as plpgsql `'...'` with `'`→`''`. (`'a\\b'` = Ruby `a\b` → plpgsql `'a\b'`, one backslash.) -- **Double-quoted** Ruby string / bare-or-`"`-quoted heredoc: decode `\n \t \r \0 \e \\ \" \xHH \uXXXX …` to actual control bytes. If any control byte or backslash is present, emit an **escape string** `E'...'` (re-escaping `'`→`''` and `\`→`\\`); else a plain `'...'`. `standard_conforming_strings=on` means `'...\n...'` is a literal backslash-n — so `"line1\nline2"` **must** become `E'line1\nline2'`, not `'line1\nline2'`. `"C:\\tmp"` (Ruby value `C:\tmp`) → `E'C:\\tmp'`. +- **Double-quoted** Ruby string / bare-or-`"`-quoted heredoc: decode `\n \t \r \0 \e \\ \" \xHH \uXXXX …` to actual control bytes. If any control byte or backslash is present, emit an **escape string** `E'...'` (re-escaping `'`→`''` and `\`→`\\`); else a plain `'...'`. `standard_conforming_strings=on` means `'...\n...'` is a literal backslash-n; so `"line1\nline2"` **must** become `E'line1\nline2'`, not `'line1\nline2'`. `"C:\\tmp"` (Ruby value `C:\tmp`) → `E'C:\\tmp'`. ### 6.2 Interpolation `"lit#{e}lit2"` -Split into literal chunks + interp spans first. Each literal chunk is decoded/escaped per §6.1. Each interp expression is **recursively `plx_rewrite_expr`'d** (nested `"..."` inside `#{}` must themselves be lowered — never spliced verbatim) and wrapped to match Ruby nil→`''`: +Split into literal chunks + interp spans first. Each literal chunk is decoded/escaped per §6.1. Each interp expression is **recursively `plx_rewrite_expr`'d** (nested `"..."` inside `#{}` must themselves be lowered; never spliced verbatim) and wrapped to match Ruby nil→`''`: ``` 'lit' || COALESCE((e')::text, '') || 'lit2' ``` -A lone `#{e}` → `COALESCE((e')::text,'')`. **Empty/whitespace `#{}` → dropped** (never `()::text`). `\#{` → literal `#{` in the chunk. `'`→`''` and `%` are applied to literal chunks only, never to interp expressions. (Equivalently the whole interpolation may lower to `concat('lit', e', 'lit2')`, which ignores NULLs; pick one and be consistent.) **Documented divergence:** interpolation follows Ruby (nil→`''`), diverging from raw SQL `||` NULL-propagation; the RAISE `%`-path renders NULL as `` unless COALESCE'd — reconcile or document. +A lone `#{e}` → `COALESCE((e')::text,'')`. **Empty/whitespace `#{}` → dropped** (never `()::text`). `\#{` → literal `#{` in the chunk. `'`→`''` and `%` are applied to literal chunks only, never to interp expressions. (Equivalently the whole interpolation may lower to `concat('lit', e', 'lit2')`, which ignores NULLs; pick one and be consistent.) **Documented divergence:** interpolation follows Ruby (nil→`''`), diverging from raw SQL `||` NULL-propagation; the RAISE `%`-path renders NULL as `` unless COALESCE'd; reconcile or document. -### 6.3 Operators — boolean vs value context +### 6.3 Operators: boolean vs value context Context (condition slot vs value/RHS slot) is known to the emitter; the split is decidable. - `==` → **`IS NOT DISTINCT FROM`** (null-safe, matches Ruby `nil==nil`→true, `5==nil`→false). `!=` → **`IS DISTINCT FROM`**. **Literal-nil special case first** (token-level, before the generic rewrite): `E == nil` / `nil == E` → `((E') IS NULL)`; `E != nil` / `nil != E` → `((E') IS NOT NULL)`. (A perf-minded variant may keep `=`/`<>` when neither operand can be NULL, but null-safe is the default.) - `&&`/`||`/`!` and keyword `and`/`or`/`not`: - - **Boolean context** (if/unless/while/until tests, ternary test): `&&`/`and`→` AND `, `||`/`or`→` OR `, `!`/`not`→ negation. **Negation uses `(x) IS NOT TRUE`, not `NOT (x)`** — TRUE for both FALSE and NULL, matching Ruby falsey (`unless nil` runs). So `unless C`→`IF (C') IS NOT TRUE`, `next unless C`→`CONTINUE WHEN (C') IS NOT TRUE`, `!arr.include?(x)`→`(x = ANY(arr)) IS NOT TRUE`. + - **Boolean context** (if/unless/while/until tests, ternary test): `&&`/`and`→` AND `, `||`/`or`→` OR `, `!`/`not`→ negation. **Negation uses `(x) IS NOT TRUE`, not `NOT (x)`**; TRUE for both FALSE and NULL, matching Ruby falsey (`unless nil` runs). So `unless C`→`IF (C') IS NOT TRUE`, `next unless C`→`CONTINUE WHEN (C') IS NOT TRUE`, `!arr.include?(x)`→`(x = ANY(arr)) IS NOT TRUE`. - **Value/RHS context**: `a || b` → `COALESCE(a', b')` (Ruby nil-default; document that SQL can't distinguish stored FALSE from NULL, so `false || b` won't pick b). `a && b` value use → reject with a precise message pointing at an explicit ternary, or (boolean operands) `CASE WHEN (a') IS NOT TRUE THEN a' ELSE b' END`. Keyword `and`/`or` in value position → **reject** (their below-`=` precedence and `x or return` control-flow can't be faithfully lowered). This is why `||=`/`&&=` are rejected and the suggested `x = x || y` lowers to `COALESCE`. - `nil`→`NULL`, `true`/`false`→`TRUE`/`FALSE`. - Ternary `C ? A : B` → `CASE WHEN C' THEN A' ELSE B' END` (recursive). -- Ranges/membership (self-parenthesized): `(a..b).include?(x)`→`(x BETWEEN a' AND b')`; **`(a...b).include?(x)`→`(x >= a' AND x < b')`** (parenthesized — unparenthesized re-associates against outer operators); `arr.include?(x)`→`(x = ANY(arr'))`. +- Ranges/membership (self-parenthesized): `(a..b).include?(x)`→`(x BETWEEN a' AND b')`; **`(a...b).include?(x)`→`(x >= a' AND x < b')`** (parenthesized; unparenthesized re-associates against outer operators); `arr.include?(x)`→`(x = ANY(arr'))`. - `**`→`power(a,b)`; `1_000`→`1000`; casts `x.to_i/.to_s/.to_f`→`(x)::integer/::text/::double precision`. -- **`+` stays SQL numeric `+`** — the literal-adjacency `+`→`||` heuristic is **dropped** (it misfires both ways: `first + last` two text vars stay `+` and error at runtime; `amount + "0"` numeric becomes string concat). Interpolation is the sanctioned concatenation path. -- **`<<` is NOT blanket-rewritten** to `||` — PG `<<` is integer bit-shift, and the shovel's in-place mutation has no bare-statement plpgsql form. Leave `<<` verbatim (bit-shift) or reject; string/array building goes through interpolation or explicit `x = x || y`. +- **`+` stays SQL numeric `+`**: the literal-adjacency `+`→`||` heuristic is **dropped** (it misfires both ways: `first + last` two text vars stay `+` and error at runtime; `amount + "0"` numeric becomes string concat). Interpolation is the sanctioned concatenation path. +- **`<<` is NOT blanket-rewritten** to `||`: PG `<<` is integer bit-shift, and the shovel's in-place mutation has no bare-statement plpgsql form. Leave `<<` verbatim (bit-shift) or reject; string/array building goes through interpolation or explicit `x = x || y`. - `%` modulo, `::` casts, function calls, column refs, arithmetic → pass through verbatim to SQL. -### 6.4 Truthiness — no coercion, reject non-boolean conditions -Ruby treats `0`, `""`, `[]` as truthy — unrepresentable by any single SQL coercion. A bare non-boolean condition (`if rows_affected`, `return unless result`) is **rejected at CREATE**: `plruby: condition must be a boolean expression; Ruby truthiness is not emulated (0, "", [] are truthy in Ruby) — write an explicit comparison such as x != 0 or !x.nil?`. Do not pass `IF `/`NOT ()` to plpgsql. +### 6.4 Truthiness: no coercion, reject non-boolean conditions +Ruby treats `0`, `""`, `[]` as truthy; unrepresentable by any single SQL coercion. A bare non-boolean condition (`if rows_affected`, `return unless result`) is **rejected at CREATE**: `plruby: condition must be a boolean expression; Ruby truthiness is not emulated (0, "", [] are truthy in Ruby); write an explicit comparison such as x != 0 or !x.nil?`. Do not pass `IF `/`NOT ()` to plpgsql. ### 6.5 Accepted, documented divergences - Decimal literals → `numeric`, not `float8`. @@ -442,9 +451,9 @@ Hard-reject catalog (all at CREATE time, precise messages, real line numbers): ` ## 8. C file / function layout & validator integration ### 8.1 Files -- `plx_scan.h` / `plx_scan.c` — shared framework: enums, `PlxToken`/`PlxLexer`/`PlxBlock`/`PlxStmt`, `plx_lex`, `plx_build_blocks`. Dialect-neutral; consumes a `PlxSurface`. -- `plx_transpile.h` / `plx_transpile.c` — shared framework: `PlxNode`/`PlxArm`/`PlxLocal`/`PlxSymtab`, `plx_rewrite_expr`, `plx_infer_type`, the `plx_lower_*` builders, `plx_emit`, `plx_emit_declare`, and the public entry point `plx_transpile`. -- `plx_dialect_ruby.c` — the Ruby `PlxSurface` (keyword table, block-form matchers, intrinsic names, sigils `#{`/`#::`, exception-class map) and the thin `ruby_validator` wired into `CREATE FUNCTION`. +- `plx_scan.h` / `plx_scan.c`: shared framework: enums, `PlxToken`/`PlxLexer`/`PlxBlock`/`PlxStmt`, `plx_lex`, `plx_build_blocks`. Dialect-neutral; consumes a `PlxSurface`. +- `plx_transpile.h` / `plx_transpile.c`: shared framework: `PlxNode`/`PlxArm`/`PlxLocal`/`PlxSymtab`, `plx_rewrite_expr`, `plx_infer_type`, the `plx_lower_*` builders, `plx_emit`, `plx_emit_declare`, and the public entry point `plx_transpile`. +- `plx_dialect_ruby.c`: the Ruby `PlxSurface` (keyword table, block-form matchers, intrinsic names, sigils `#{`/`#::`, exception-class map) and the thin `ruby_validator` wired into `CREATE FUNCTION`. ### 8.2 Public API ```c @@ -498,12 +507,12 @@ if (!has_sentinel(rubysrc)) { ``` Because `lanplcallfoid` is `plpgsql_call_handler`, the stored (now-plpgsql) `prosrc` executes on stock plpgsql. Re-running `CREATE OR REPLACE` re-supplies Ruby; the sentinel check makes a second validator pass idempotent (never double-transpile already-plpgsql text). -### 8.4 Embedded original (Strategy A) — safe encoding +### 8.4 Embedded original (Strategy A): safe encoding The verbatim-splice of user bytes into a `/* … */` comment is unsafe: any `*/` in the body (in a string, `a */ b`, etc.) closes the comment early and corrupts `prosrc`; the `$plx-orig$` sentinel can likewise be forged. **Base64-encode the whole original body** and wrap: ``` /*plx-orig:b64$$plx-orig*/ ``` -Base64's alphabet contains no `*`, `/`, or `$`, so the terminator and sentinel are unforgeable. (Alternative: escape every `*/`→`*\/` and sentinel occurrences on write, reverse on read — base64 is simpler and total.) A recovery tool decodes this to show the user their original Ruby. +Base64's alphabet contains no `*`, `/`, or `$`, so the terminator and sentinel are unforgeable. (Alternative: escape every `*/`→`*\/` and sentinel occurrences on write, reverse on read; base64 is simpler and total.) A recovery tool decodes this to show the user their original Ruby. ### 8.5 Ruby `PlxSurface` (`plx_dialect_ruby.c`) Supplies keyword spellings (`if elsif unless while until for begin rescue ensure end next break return loop do case when then in and or not`), block-form matchers (`.each`, `.each_with_index`, `do |..|`, `loop do`, `{ |..| }`), intrinsics (`query fetch_one fetch_one! perform execute emit return_next return_query raise`), sigils `#{ }` and `#::`, and the exception-class→condition map. Nothing else is Ruby-specific. @@ -516,7 +525,7 @@ Supplies keyword spellings (`if elsif unless while until for begin rescue ensure **Dialect-specific (a new `plx_dialect_php.c` `PlxSurface`):** - keyword spellings (`if/elseif/else/endif` or brace-blocks, `while`, `foreach`, `function` reject, …); -- comment lexer + **type-annotation sigil** (PHP spelling `/*:: text */` instead of `#::`) — the shared framework consumes "a `::`-tagged type string in a trailing dialect comment"; the dialect supplies only its comment lexer; +- comment lexer + **type-annotation sigil** (PHP spelling `/*:: text */` instead of `#::`); the shared framework consumes "a `::`-tagged type string in a trailing dialect comment"; the dialect supplies only its comment lexer; - string/quote flavors (`"$var"` interpolation vs `'...'` literal), heredoc/nowdoc (`<< ", USER-ID) SQLSTATE "P0002" ``` +Note: `throw` in plxphp and plxjs and `raise ValueError(...)` in plxpython3 +raise SQLSTATE P0001 (`raise_exception`). Only the plxruby and plxcobol forms +set a specific SQLSTATE such as P0002 (`no_data_found`). + ## Anonymous code blocks plxruby: @@ -592,6 +601,52 @@ PROCEDURE DIVISION. $$; ``` +## Procedures and transaction control + +PL/pgSQL manual, "Transaction Management". A body can also be a procedure, +created with `CREATE PROCEDURE` and run with `CALL`. A procedure invoked outside +an explicit transaction block can commit and roll back; a function cannot. + +plpgsql: + +```sql +CREATE PROCEDURE load_batches(n int) AS $$ +BEGIN + FOR i IN 1..n LOOP + INSERT INTO batch_log VALUES (i); + COMMIT; + END LOOP; +END; +$$ LANGUAGE plpgsql; +``` + +plxruby: + +```sql +CREATE PROCEDURE load_batches(n int) LANGUAGE plxruby AS $$ +for i in 1..n + execute("INSERT INTO batch_log VALUES ($1)", i) + commit() +end +$$; +``` + +plxpython3: + +```sql +CREATE PROCEDURE load_batches(n int) LANGUAGE plxpython3 AS $$ +for i in range(1, n + 1): + execute("INSERT INTO batch_log VALUES ($1)", i) + commit() +$$; +``` + +plxphp and plxjs use the same `commit()` and `rollback()` calls, plxcobol uses +the `COMMIT` and `ROLLBACK` verbs, and plxplsql uses `COMMIT;` as in PL/SQL. +plxtsql does not support transaction control. Calling `commit()` from a +function, or from a procedure called inside an explicit transaction block, +raises `invalid transaction termination`, as in plpgsql. + ## Trust and security The plx languages are declared `TRUSTED`. This is correct for the following @@ -599,30 +654,59 @@ reasons: - A plx function is transpiled to plpgsql and executed by plpgsql's call handler. It can do what a plpgsql function can do, and nothing more: no direct - filesystem access, no network access, no arbitrary native code. All SQL runs - with the privileges of the calling role. + filesystem access, no network access, no arbitrary native code. SQL runs with + the privileges of the calling role, or of the function owner for a + `SECURITY DEFINER` function, exactly as in plpgsql. - plx does not embed a language runtime. This is the difference from the native PL/PHP and PL/Ruby, which are untrusted because they load a full PHP or Ruby interpreter into the backend. -Two points to keep in mind: +Who can use the languages: + +- Installing plx (`CREATE EXTENSION plx`) requires a superuser. +- Once installed, every role can create functions in the plx languages, because + PostgreSQL grants USAGE on a trusted language to PUBLIC, as it does for + plpgsql. Restrict this per language, for example + `REVOKE USAGE ON LANGUAGE plxruby FROM PUBLIC`. +- plx is a syntax layer over plpgsql, so a role with USAGE on any plx language + can effectively run arbitrary plpgsql: a body that already begins with the + `/*plx:v1:...*/` sentinel is stored without transpiling, so that dump and + restore work, and plxplsql accepts a body that is already close to plpgsql. + Revoking USAGE on plpgsql therefore does not restrict a role that keeps USAGE + on a plx language: after `REVOKE USAGE ON LANGUAGE plpgsql FROM PUBLIC`, such + a role can still run plpgsql through, for example, `DO LANGUAGE plxplsql`. + To restrict plpgsql, revoke USAGE on plpgsql and on every plx language. + +Points to keep in mind: - The transpiler is C code that parses the function body at `CREATE FUNCTION` - time. It has a recursion-depth limit and has been fuzzed with mutation and - pathological inputs across all dialects (see `test/fuzz.py`) with no crashes or - hangs. Continued fuzzing is recommended before relying on it in a hostile - multi-tenant deployment. -- `query`, `perform`, `execute`, and `fetch_one` with interpolated values carry - the same SQL-injection responsibility as plpgsql `EXECUTE ... || value`. Use - bind parameters (`execute(sql, a, b)`) for untrusted input. + time. It limits expression and statement nesting depth, rejecting deeper + input with a `nested too deeply` error, and has been fuzzed with mutation and + pathological inputs across all dialects (see `test/fuzz.py`). Continued + fuzzing is recommended before relying on it in a hostile multi-tenant + deployment. +- `execute` with interpolated values builds the SQL string by concatenation and + carries the same SQL-injection responsibility as plpgsql + `EXECUTE ... || value`. Use bind parameters (`execute(sql, a, b)`) for + untrusted input. In `query`, `perform`, and `fetch_one` with no bind + arguments, an interpolated value is spliced as a variable reference, not as + text. Once such a call also passes bind arguments it becomes dynamic SQL, and + its interpolated values are concatenated exactly as in `execute`: do not mix + interpolation with binds. +- Functions that use the string-append operators reference the extension's + `plx_strbuild` type and `plx_sb_append` function unqualified, so the + extension's schema must be on the function's `search_path` at run time, + including any `SET search_path` clause on a `SECURITY DEFINER` function. ## Further reading -- `README.md`: build and install. -- Per-dialect chapters: `doc/plxruby.md`, `doc/plxphp.md`, `doc/plxjs.md`, - `doc/plxpython3.md`, `doc/plxcobol.md`, `doc/plxplsql.md`, `doc/plxts.md`, - `doc/plxtsql.md`, `doc/plxgo.md` (full syntax, supported and rejected - constructs, semantic differences). -- `doc/PARITY.md`: the plpgsql construct parity matrix. -- `doc/ARCHITECTURE.md`: how plx maps to the plpgsql engine. -- `doc/TRANSPILER.md`: transpiler specification. +- [Home](index.md): build and install. +- Per-dialect chapters: [plxruby](plxruby.md), [plxphp](plxphp.md), + [plxjs](plxjs.md), [plxpython3](plxpython3.md), [plxcobol](plxcobol.md), + [plxplsql](plxplsql.md), [plxts](plxts.md), [plxtsql](plxtsql.md), + [plxgo](plxgo.md) (full syntax, supported and rejected constructs, semantic + differences). +- [PARITY.md](PARITY.md): the plpgsql construct parity matrix. +- [ARCHITECTURE.md](ARCHITECTURE.md): how plx maps to the plpgsql engine. +- [TRANSPILER.md](TRANSPILER.md): the original transpiler design + specification. diff --git a/doc/cookbook/plxcobol.md b/doc/cookbook/plxcobol.md index e0d2b9f..becdceb 100644 --- a/doc/cookbook/plxcobol.md +++ b/doc/cookbook/plxcobol.md @@ -1,6 +1,6 @@ # COBOL cookbook -Practical recipes for the plxcobol dialect (ISO/IEC 1989:2023). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxcobol chapter](plxcobol.md) for the full language reference. +Practical recipes for the plxcobol dialect (ISO/IEC 1989:2023). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxcobol chapter](../plxcobol.md) for the full language reference. ## Absolute value with IF / ELSE diff --git a/doc/cookbook/plxgo.md b/doc/cookbook/plxgo.md index ffcc03e..a18e1a4 100644 --- a/doc/cookbook/plxgo.md +++ b/doc/cookbook/plxgo.md @@ -1,6 +1,6 @@ # Go cookbook -Practical recipes for the plxgo dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxgo chapter](plxgo.md) for the full language reference. +Practical recipes for the plxgo dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxgo chapter](../plxgo.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxjs.md b/doc/cookbook/plxjs.md index 40ac494..82de895 100644 --- a/doc/cookbook/plxjs.md +++ b/doc/cookbook/plxjs.md @@ -1,6 +1,6 @@ # JavaScript cookbook -Practical recipes for the plxjs dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxjs chapter](plxjs.md) for the full language reference. +Practical recipes for the plxjs dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxjs chapter](../plxjs.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxphp.md b/doc/cookbook/plxphp.md index e7bed76..218b32c 100644 --- a/doc/cookbook/plxphp.md +++ b/doc/cookbook/plxphp.md @@ -1,6 +1,6 @@ # PHP cookbook -Practical recipes for the plxphp dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxphp chapter](plxphp.md) for the full language reference. +Practical recipes for the plxphp dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxphp chapter](../plxphp.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxplsql.md b/doc/cookbook/plxplsql.md index fce803e..786fb68 100644 --- a/doc/cookbook/plxplsql.md +++ b/doc/cookbook/plxplsql.md @@ -1,6 +1,6 @@ # Oracle PL/SQL cookbook -Practical recipes for the plxplsql dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. The function signature uses PostgreSQL types; the body is PL/SQL. See the [plxplsql chapter](plxplsql.md) for the full language reference. +Practical recipes for the plxplsql dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. The function signature uses PostgreSQL types; the body is PL/SQL. See the [plxplsql chapter](../plxplsql.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxpython3.md b/doc/cookbook/plxpython3.md index 4851f0b..244a950 100644 --- a/doc/cookbook/plxpython3.md +++ b/doc/cookbook/plxpython3.md @@ -1,6 +1,6 @@ # Python cookbook -Practical recipes for the plxpython3 dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxpython3 chapter](plxpython3.md) for the full language reference. +Practical recipes for the plxpython3 dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxpython3 chapter](../plxpython3.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxruby.md b/doc/cookbook/plxruby.md index 8b195a2..8c2ca1c 100644 --- a/doc/cookbook/plxruby.md +++ b/doc/cookbook/plxruby.md @@ -1,6 +1,6 @@ # Ruby cookbook -Practical recipes for the plxruby dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxruby chapter](plxruby.md) for the full language reference. +Practical recipes for the plxruby dialect. Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxruby chapter](../plxruby.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxts.md b/doc/cookbook/plxts.md index a6e11aa..15c3be5 100644 --- a/doc/cookbook/plxts.md +++ b/doc/cookbook/plxts.md @@ -1,6 +1,6 @@ # TypeScript cookbook -Practical recipes for the plxts dialect (plxjs plus type annotations). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxts chapter](plxts.md) for the full language reference. +Practical recipes for the plxts dialect (plxjs plus type annotations). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxts chapter](../plxts.md) for the full language reference. ## Scalar function with branching diff --git a/doc/cookbook/plxtsql.md b/doc/cookbook/plxtsql.md index 707be43..f0d759f 100644 --- a/doc/cookbook/plxtsql.md +++ b/doc/cookbook/plxtsql.md @@ -1,6 +1,6 @@ # Transact-SQL cookbook -Practical recipes for the plxtsql dialect (SQL Server / Sybase T-SQL). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxtsql chapter](plxtsql.md) for the full language reference. +Practical recipes for the plxtsql dialect (SQL Server / Sybase T-SQL). Every recipe here was run on PostgreSQL; plx transpiles the body to plpgsql and the standard interpreter executes it. See the [plxtsql chapter](../plxtsql.md) for the full language reference. ## Scalar function with branching diff --git a/doc/index.md b/doc/index.md index bf2cae3..700fced 100644 --- a/doc/index.md +++ b/doc/index.md @@ -30,15 +30,15 @@ $$; The front end is dialect-pluggable, and the set is open-ended. The dialects available today: -- [**plxruby**](plxruby.md) — a Ruby dialect -- [**plxphp**](plxphp.md) — a PHP dialect -- [**plxjs**](plxjs.md) — a JavaScript dialect -- [**plxts**](plxts.md) — a TypeScript dialect (plxjs plus type annotations) -- [**plxpython3**](plxpython3.md) — a Python dialect -- [**plxgo**](plxgo.md) — a Go dialect -- [**plxcobol**](plxcobol.md) — a COBOL dialect (ISO/IEC 1989:2023) -- [**plxplsql**](plxplsql.md) — an Oracle PL/SQL dialect -- [**plxtsql**](plxtsql.md) — a Transact-SQL (SQL Server) dialect +- [**plxruby**](plxruby.md): a Ruby dialect +- [**plxphp**](plxphp.md): a PHP dialect +- [**plxjs**](plxjs.md): a JavaScript dialect +- [**plxts**](plxts.md): a TypeScript dialect (plxjs plus type annotations) +- [**plxpython3**](plxpython3.md): a Python dialect +- [**plxgo**](plxgo.md): a Go dialect +- [**plxcobol**](plxcobol.md): a COBOL dialect (ISO/IEC 1989:2023) +- [**plxplsql**](plxplsql.md): an Oracle PL/SQL dialect +- [**plxtsql**](plxtsql.md): a Transact-SQL (SQL Server) dialect The [user guide](USERGUIDE.md) shows the same worked examples across dialects, and [feature parity](PARITY.md) is the construct-by-construct matrix. @@ -69,18 +69,23 @@ Build from source against your PostgreSQL installation: make && make install ``` -Then, in the database: +Then, in the database (installing the extension requires a superuser): ```sql CREATE EXTENSION plx; ``` -Upgrading from an earlier install: +Upgrading from an earlier install, after installing the new build: ```sql -ALTER EXTENSION plx UPDATE TO '1.2'; +ALTER EXTENSION plx UPDATE; -- or UPDATE TO '2.0.0' ``` +Upgrading changes no existing function: plx transpiles at `CREATE FUNCTION` +time, so re-run a function's DDL to pick up new transpiler behaviour (see the +2.0.0 entry in the +[changelog](https://github.com/commandprompt/plx/blob/master/CHANGELOG.md)). + plx supports PostgreSQL 13 through 18; see [compatibility](COMPATIBILITY.md). ## More diff --git a/doc/plxcobol.md b/doc/plxcobol.md index ff6de2c..799e3e2 100644 --- a/doc/plxcobol.md +++ b/doc/plxcobol.md @@ -27,15 +27,18 @@ A function body is a free-format COBOL program fragment: an optional ```sql CREATE FUNCTION add(a int, b int) RETURNS int LANGUAGE plxcobol AS $$ +WORKING-STORAGE SECTION. +01 RESULT PIC 9(9). PROCEDURE DIVISION. COMPUTE RESULT = A + B GOBACK RETURNING RESULT. $$; ``` -Function arguments are referenced by name (case-insensitively). Any SQL -expression is valid inside `COMPUTE` and conditions, because expressions are -passed through to plpgsql and SQL. +Function arguments are referenced by name (case-insensitively). Most SQL +expressions are valid inside `COMPUTE` and conditions, because expressions are +passed through to plpgsql and SQL. The `||` operator and `::` casts are not: +the tokenizer drops them, so use `CONCAT(...)` and `CAST(x AS type)`. ### Local variables and types @@ -49,7 +52,7 @@ clause is mapped to a SQL type: | `PIC 9(n)`, larger | `numeric(n)` | | `PIC 9(i)V9(s)` (implied decimal) | `numeric(i+s, s)` | | `PIC X(n)`, `PIC A(n)` | `varchar(n)` | -| `USAGE COMP-1` / `COMP-2` | `real` / `double precision` | +| `PIC ... USAGE COMP-1` / `COMP-2` (a `PIC` clause is required) | `real` / `double precision` | A leading `S` (sign) is accepted and ignored for typing. A `VALUE` clause becomes the declaration's initializer: @@ -85,8 +88,8 @@ writes element `i` (1-based): ```sql 01 WS-ARR PIC 9(9) OCCURS 5 TIMES. ... -COMPUTE WS-ARR(WS-I) = WS-I * WS-I -- write element -COMPUTE WS-SUM = WS-SUM + WS-ARR(WS-I) -- read element +COMPUTE WS-ARR(WS-I) = WS-I * WS-I *> write element +COMPUTE WS-SUM = WS-SUM + WS-ARR(WS-I) *> read element ``` Iterate the whole table with `PERFORM v OVER ARRAY WS-ARR`. A subscript is @@ -336,8 +339,8 @@ $$; ``` `NEW`, `OLD`, and the `TG_` variables are available. Assigning to a record field -(`MOVE e TO NEW.COL`) works because a qualified name maps to `new.col`; a bare -`MOVE e TO NEW` is not supported. +(`MOVE e TO NEW.COL`) works because a qualified name maps to `new.col`; +`MOVE OLD TO NEW` replaces the whole row. ## Semantic differences @@ -346,7 +349,9 @@ These are intentional. plx pins semantics to SQL and plpgsql. - Decimal literals infer `numeric`, not a floating-point type. - Integer division and modulo follow SQL (truncate toward zero). - A condition must be a boolean expression; there is no COBOL 88-level or - class-condition abbreviation. + class-condition abbreviation. An 88-level entry is rejected. A class + condition such as `N IS NUMERIC` is not rejected but mistranslated (`IS` is + dropped), so write a SQL boolean expression. - Data names are function-scoped after mapping to plpgsql identifiers, so two names that differ only by `-` versus `_` collide. @@ -360,8 +365,11 @@ Rejected at `CREATE FUNCTION` time with a line number: - Group items (subordinate level entries under an `01`) in `WORKING-STORAGE`; declare elementary items, or use a `TYPE` clause for a composite plpgsql type. - Fixed-format source (columns, sequence area, indicator area). Use free format. -- Report Writer, screen sections, object orientation, and the standard COBOL - intrinsic function library. Use SQL functions inside expressions and SQL text. +- Report Writer, screen sections, and object orientation. + +Not translated: the standard COBOL intrinsic function library. A +`FUNCTION name(...)` call is passed through and fails when the function runs. +Use SQL functions inside expressions and SQL text. See [PARITY.md](PARITY.md) for the full plpgsql construct matrix and [ARCHITECTURE.md](ARCHITECTURE.md) for how plx maps to the plpgsql engine. diff --git a/doc/plxgo.md b/doc/plxgo.md index b643f1e..1733379 100644 --- a/doc/plxgo.md +++ b/doc/plxgo.md @@ -57,7 +57,7 @@ declarations are translated (see below). | `x += e` (also `-= *= /= %=`) | `x := x + (e);` | | `x++` / `x--` | `x := x + 1;` / `x := x - 1;` | | `a, b = x, y` | `SELECT x, y INTO a, b;` (parallel, so swaps work) | -| `a, b := f(), g()` | declares `a` and `b`, then assigns | +| `a, b := 1, "x"` | declares `a` (integer) and `b` (text), then assigns; each value must have an inferable type | `:=` infers the type from a literal (`int`/`float`/`string`/`bool`), an array literal, `len(...)`, or a type conversion `T(x)`. If the type cannot be inferred @@ -87,10 +87,12 @@ three-clause `for` lowers to a `WHILE` with the post statement at the end of the body; there, as in plpgsql, `continue` re-tests the condition without running the post statement, so prefer the counting form when you use `continue`. -In a `range` loop, a single variable is the index (Go semantics), and two -variables are `(index, value)`. The value variable's type is inferred from the -slice's declared element type; `for i, v := range` (both index and value) is not -supported, so use `for _, v := range` or `for i := range`. +In a `range` loop over an integer, the single variable is the index. Over a +slice, use `for _, v := range s`. The value variable's type is inferred from the +slice's declared element type; when `s` is a parameter, declare `v` with `var` +before the loop. `for i := range s` and `for i, v := range s` over a slice are +not supported; walk a slice by index with a counting loop +(`for i := 0; i < len(s); i++`). ### switch @@ -120,10 +122,9 @@ $$; | `nil` | `NULL` | | `"..."`, `` `...` ``, `'x'` | SQL string literal (escapes decoded) | | `[]int{1, 2, 3}` | `ARRAY[1, 2, 3]` | -| `len(x)` | `length(x)` (text) or `cardinality(x)` (a slice) | +| `len(x)` | `length(x)` (text) or `coalesce(cardinality(x), 0)` (a slice declared in the body) | | `append(s, x)` | `array_append(s, x)` | | `a[i]` (subscript) | `a[(i) + 1]` (see below) | -| `[]int{1, 2, 3}` (again) | `ARRAY[1, 2, 3]` | | `int(x)`, `int64(x)`, `float64(x)`, `string(x)` | `(x)::integer` / `::bigint` / `::double precision` / `::text` | | `panic(m)` | `RAISE EXCEPTION '%', m;` | | `fmt.Println(x)` / `fmt.Printf(f, x)` | `RAISE NOTICE '%', x;` | @@ -133,8 +134,10 @@ $$; A slice literal `[]T{...}` becomes a PostgreSQL `ARRAY[...]`. Go slices are 0-based while PostgreSQL arrays are 1-based, so a subscript `a[i]` is rewritten to `a[(i) + 1]`. Indexing therefore follows Go semantics: `a[0]` is the first -element, and `len(a)`, `for i := range a`, and `a[i]` all agree on 0-based -positions. Go slice expressions (`a[i:j]`) are not translated. +element, and `len(a)` and `a[i]` agree on 0-based positions, so +`for i := 0; i < len(a); i++` walks a slice by index. `len` and `range` +recognize only slices declared in the body; copy an array parameter into one +first (`var a []int = p`). Go slice expressions (`a[i:j]`) are not translated. `fmt.Println`/`fmt.Printf` used as a statement raise a `NOTICE` with one `%` placeholder per value argument (space-separated); there the format string's @@ -163,8 +166,9 @@ rather than rounding it to two decimal places. Verbs that change an operand's representation rather than its padding are affected the same way: `%x`, `%o`, `%b`, `%e` and `%q` all render the operand's text, so `fmt.Sprintf("%x", 255)` yields `255` and not `ff`. -A doubled `%%` stays a literal `%`, and a `%` that starts no directive is -passed through as a literal percent. +A doubled `%%` stays a literal `%`. A `%` that starts no directive is also +rendered as a literal percent, but it makes plx fall back to `format()`, where +a NULL operand renders as empty instead of propagating; write `%%` instead. ### Types (in declarations) @@ -186,7 +190,8 @@ An unrecognized type name is passed through, so PostgreSQL type names also work. A subset of the standard library is mapped: `strings.ToUpper`/`ToLower`/ `TrimSpace`/`ReplaceAll`/`Contains`, `math.Abs`/`Floor`/`Ceil`/`Sqrt`/`Pow`/ -`Max`/`Min`/`Mod`, `strconv.Itoa`/`Atoi`, and `time.Now`. Anything not mapped is +`Max`/`Min`/`Mod`, `strconv.Itoa` (integer to text) and `strconv.Atoi` (text +to integer), and `time.Now`. Anything not mapped is emitted as written, so a PostgreSQL function of the same name still works. ## SQL access @@ -197,7 +202,7 @@ plxgo provides small SQL intrinsics, mirroring the other dialects: | --- | --- | | `emit(x)` | `RETURN NEXT x;` (for a set-returning function) | | `execute("...")` | `EXECUTE '...';` | -| `perform("...")` | `PERFORM ...;` | +| `perform(expr)` | `PERFORM expr;` (write the call as Go, for example `perform(nextval("seq"))`; a quoted string is evaluated as a literal and does nothing) | | `for _, row := range query("...") { ... }` | `FOR row IN EXECUTE '...' LOOP ...` | ```sql @@ -217,8 +222,9 @@ $$; - `switch` `fallthrough`. - String concatenation with `+`. Go overloads `+` for numeric addition and string concatenation; plx cannot tell which is meant without type information, - so it leaves `+` as `+`. Use `||` (valid in an expression) or build the string - another way for text concatenation. + so it leaves `+` as `+`. `||` is logical OR in plxgo, so use + `fmt.Sprintf("%s%s", a, b)` for text concatenation (a NULL operand makes the + result NULL). - The full `fmt`/`strings`/`math`/`strconv` packages; only the subset above is translated. diff --git a/doc/plxjs.md b/doc/plxjs.md index daf5b7e..f0e1b02 100644 --- a/doc/plxjs.md +++ b/doc/plxjs.md @@ -41,6 +41,12 @@ let amount /*:: numeric */; amount = principal * rate; ``` +Known issue: a top-level first assignment of an interpolated string is folded +into the declaration, so it is evaluated before any earlier statement runs. +``let x = 1; x = 2; let m = `v ${x}`; return m;`` returns `v 1`. When an +interpolated initializer depends on earlier statements, declare the variable +first (`let m /*:: text */;`) and then assign it. + The annotation text is emitted verbatim, so `%TYPE` and `%ROWTYPE` work: ```sql @@ -97,7 +103,7 @@ while (n > 0) { n = n - 1; } ```sql outer: for (let i = 1; i <= n; i++) { for (let j = 1; j <= n; j++) { - if (i + j >= limit) { break outer; } + if (i + j >= lim) { break outer; } } } ``` @@ -113,7 +119,8 @@ for (const row of query(`SELECT id, amount FROM orders WHERE grp = ${g}`)) { ``` The row variable is a record. Field access is `row.col`. For bind parameters or -non-literal SQL, pass extra arguments: `query(sql, a) of ...`. +non-literal SQL, pass extra arguments: `for (const row of query(sql, a))`, which +becomes `FOR row IN EXECUTE sql USING a`. ### Iterating an array @@ -164,7 +171,7 @@ close_cursor(c); ```sql let rc = row_count(); // GET DIAGNOSTICS rc = ROW_COUNT -found() // the FOUND variable +if (found()) { return rc; } // the FOUND variable ``` ### Set-returning functions @@ -190,20 +197,23 @@ The call form `raise("notice", "message")` emits a `RAISE` at the named level ### Handling ```sql +let r = 0; try { - return 100 / d; + r = 100 / d; } catch (e) { raise("notice", `caught: ${e.message}`); - return -1; + r = -1; } finally { perform("INSERT INTO log(msg) VALUES ('done')"); } +return r; ``` `catch (e)` maps to `WHEN OTHERS`. Accessors: `e.message` to `SQLERRM`, `e.sqlstate` to `SQLSTATE`, and `e.detail`, `e.hint`, `e.constraint`, `e.column`, `e.table`, `e.schema`, `e.datatype` to the matching `GET STACKED DIAGNOSTICS` -fields. +fields. A `return` inside the `try` or `catch` block leaves the function without +running `finally`; set a variable and return after the block instead. ### Assertions @@ -217,7 +227,10 @@ assert(n > 0, "must be positive"); `'...'` strings do not interpolate. Use template literals for concatenation. - Comparison: `==`, `===`, `!=`, `!==` map to `=` and `<>`. `x === null` becomes `x IS NULL`. -- Boolean: `&&`, `||`, `!` map to `AND`, `OR`, `NOT`. +- Boolean: `&&` and `!` map to `AND` and `NOT`. `||` maps to `OR` only in an + `if`/`while` condition or a ternary test. Elsewhere (assignments, `return`) + it passes through as SQL `||` (string concatenation), so write `OR` there: + `let ok = a OR b /*:: boolean */;`. - Ternary: `c ? a : b` becomes `CASE WHEN c THEN a ELSE b END`. - `null` and `undefined` become `NULL`. @@ -272,23 +285,35 @@ These are intentional. plx pins semantics to SQL and plpgsql. - `+` is numeric addition; use template literals for string building. - Interpolating a NULL in a template literal propagates it, so the whole string is NULL. It is neither the JavaScript string `"null"` nor an empty string. In - a message being built for `throw` the value falls back to an empty string - instead, so one NULL cannot swallow the message. + a `throw` or `raise` message, including the `raise("level", ...)` call form, + the value falls back to an empty string instead, so one NULL cannot swallow + the message. An `assert` message gets no fallback: a NULL there replaces your + message with plpgsql's generic `assertion failed`. - Integer division and modulo follow SQL (truncate toward zero). - JavaScript truthiness is not emulated. A condition must be a boolean - expression. A non-boolean condition is an error reported by plpgsql when the - function runs. -- Locals are function-scoped, matching JavaScript `var` scope. + expression. A non-boolean condition is not rejected at CREATE time. When the function runs, plpgsql coerces it + through its text form: a value whose text is a boolean literal (`0`, `1`, + `t`, `yes`, `off`, ...) is silently accepted, and anything else raises + `invalid input syntax for type boolean`. +- Locals are function-scoped, matching JavaScript `var` scope. `let` and + `const` inside a block are not block-scoped: a redeclaration in an inner block + assigns the outer variable. `const` is not enforced. ## Not supported Rejected at `CREATE FUNCTION` time with a line number: -- Function definitions, arrow functions, classes, `import`. -- Object and array literals as general values. -- `for...in`, and `for...of` over anything other than `query(...)` or an array. +- Function definitions, classes, `import`. +- `for...in`. - `switch` fall-through (end each case with `break` or `return`). -- Per-block local `DECLARE` (locals are function-scoped by design). + +Not translated: + +- Arrow functions and object and array literals as general values. Unannotated, + they fail at `CREATE FUNCTION` time with `cannot infer a PostgreSQL type`; + annotated, they pass through and fail when the function runs. +- `for...of` treats anything other than `query(...)` as an array + (`FOREACH ... IN ARRAY`); a non-array fails when the function runs. See [PARITY.md](PARITY.md) for the full plpgsql construct matrix and [ARCHITECTURE.md](ARCHITECTURE.md) for how plx maps to the plpgsql engine. diff --git a/doc/plxphp.md b/doc/plxphp.md index 8bbd8be..0e2de37 100644 --- a/doc/plxphp.md +++ b/doc/plxphp.md @@ -40,6 +40,12 @@ $amount /*:: numeric */; $amount = $principal * $rate; ``` +Known issue: a top-level first assignment of an interpolated string is folded +into the declaration, so it is evaluated before any earlier statement runs. The +body `$x = 1; $x = 2; $m = "v {$x}"; return $m;` returns `v 1`. When an +interpolated initializer depends on earlier statements, declare the variable +first (`$m /*:: text */;`) and then assign it. + The annotation text is emitted verbatim, so `%TYPE` and `%ROWTYPE` work: ```sql @@ -98,7 +104,7 @@ while ($n > 0) { $n = $n - 1; } ```sql outer: for ($i = 1; $i <= $n; $i++) { for ($j = 1; $j <= $n; $j++) { - if ($i + $j >= $limit) { break outer; } + if ($i + $j >= $lim) { break outer; } } } ``` @@ -166,7 +172,7 @@ close_cursor($c); ```sql $rc = row_count(); // GET DIAGNOSTICS $rc = ROW_COUNT -found() // the FOUND variable +if (found()) { return $rc; } // the FOUND variable ``` ### Set-returning functions @@ -207,6 +213,8 @@ which maps to `unique_violation`. Accessors: `$e->message` to `SQLERRM`, `$e->sqlstate` to `SQLSTATE`, and `$e->detail`, `$e->hint`, `$e->constraint`, `$e->column`, `$e->table`, `$e->schema`, `$e->datatype` to the matching `GET STACKED DIAGNOSTICS` fields. +A `return` inside the `try` or `catch` block leaves the function without +running `finally`; set a variable and return after the block instead. ### Assertions @@ -221,7 +229,10 @@ assert($n > 0, "must be positive"); - Concatenation: `$a . $b` maps to `$a || $b`. - Comparison: `==`, `===`, `!=`, `!==` map to `=` and `<>`. `$x == null` becomes `$x IS NULL`. -- Boolean: `&&`, `||`, `!` map to `AND`, `OR`, `NOT`. +- Boolean: `&&` and `!` map to `AND` and `NOT`. `||` maps to `OR` only in an + `if`/`while` condition or a ternary test. Elsewhere (assignments, `return`) + it passes through as SQL `||` (string concatenation), so write `OR` there: + `$ok = $a OR $b /*:: boolean */;`. - Ternary: `$c ? $a : $b` becomes `CASE WHEN $c THEN $a ELSE $b END`. - `null` becomes `NULL`. @@ -273,12 +284,17 @@ These are intentional. plx pins semantics to SQL and plpgsql. `==` and `!=` and use SQL type resolution. `"1" === 1` is false in PHP but compares as equal here. Compare like-typed values. - Interpolating a NULL propagates it, so the whole string becomes NULL rather - than rendering the value as an empty string. In a message being built for - `throw` the value falls back to an empty string instead, so one NULL cannot - swallow the message. + than rendering the value as an empty string. In a `throw` or `raise` message, + including the `raise("level", ...)` call form, the value falls back to an + empty string instead, so one NULL cannot swallow the message. An `assert` + message gets no fallback: a NULL there replaces your message with plpgsql's + generic `assertion failed`. - Integer division and modulo follow SQL (truncate toward zero). - PHP truthiness is not emulated. A condition must be a boolean expression. A - non-boolean condition is an error reported by plpgsql when the function runs. + non-boolean condition is not rejected at CREATE time. When the function runs, plpgsql coerces it + through its text form: a value whose text is a boolean literal (`0`, `1`, + `t`, `yes`, `off`, ...) is silently accepted, and anything else raises + `invalid input syntax for type boolean`. - Locals are function-scoped, matching PHP function scope. ## Not supported @@ -286,13 +302,18 @@ These are intentional. plx pins semantics to SQL and plpgsql. Rejected at `CREATE FUNCTION` time with a line number: - Function or class definitions, namespaces, `use`, includes. -- Closures and anonymous functions. -- Array and object literals as general values. - Non-counting `for`; only `for ($v = LO; $v < HI; $v++)` and `for (...; $v += K)` are supported. -- `foreach` over an array key/value pair (only value iteration). - `switch` fall-through (end each case with `break` or `return`). -- Per-block local `DECLARE` (locals are function-scoped by design). + +Not translated: + +- Closures, anonymous functions, and array and object literals as general + values. An unannotated assignment of one fails at `CREATE FUNCTION` time + (`cannot infer a PostgreSQL type`); an annotated one passes through and fails + when the function runs. +- `foreach ($a as $k => $v)`: the key is ignored; only value iteration is + supported. See [PARITY.md](PARITY.md) for the full plpgsql construct matrix and [ARCHITECTURE.md](ARCHITECTURE.md) for how plx maps to the plpgsql engine. diff --git a/doc/plxplsql.md b/doc/plxplsql.md index cb890b6..789ba52 100644 --- a/doc/plxplsql.md +++ b/doc/plxplsql.md @@ -60,7 +60,7 @@ Everything not listed here is emitted verbatim (it is already valid plpgsql). | `PLS_INTEGER`, `BINARY_INTEGER`, `SIMPLE_INTEGER` | `integer` | | `BINARY_FLOAT` / `BINARY_DOUBLE` | `real` / `double precision` | | `CLOB`, `NCLOB`, `LONG` | `text` | -| `BLOB`, `RAW` | `bytea` | +| `BLOB`, `RAW` | `bytea` (`RAW(n)` becomes the invalid `bytea(n)`, so declare `RAW` without a length) | ### Statements and functions @@ -86,7 +86,7 @@ CURSOR c IS SELECT v FROM t ORDER BY v; OPEN c; LOOP FETCH c INTO r; - EXIT WHEN NOT FOUND; -- or c%NOTFOUND, see notes + EXIT WHEN NOT FOUND; -- c%NOTFOUND is not translated; use NOT FOUND ... END LOOP; CLOSE c; @@ -110,9 +110,11 @@ EXCEPTION RETURN -1; ``` -Condition names such as `NO_DATA_FOUND`, `TOO_MANY_ROWS`, `DUP_VAL_ON_INDEX`, -and `ZERO_DIVIDE` are recognized by plpgsql as well; where a name differs, use -the SQLSTATE condition name. +The condition names `NO_DATA_FOUND` and `TOO_MANY_ROWS` are recognized by +plpgsql as well. Other Oracle names are not translated: use the PostgreSQL +condition name (`division_by_zero` for `ZERO_DIVIDE`, `unique_violation` for +`DUP_VAL_ON_INDEX`). An unrecognized name fails when the function is first +called with `unrecognized exception condition`. ## Dynamic SQL @@ -139,7 +141,8 @@ These are intentional. plx pins semantics to SQL and plpgsql. ## Not supported -Rejected or unchanged (and therefore likely to error) at `CREATE FUNCTION` time: +Not translated. These pass through unchanged, so `CREATE FUNCTION` succeeds and +the function fails with a plpgsql error when it is first called: - Packages (`CREATE PACKAGE`), package-qualified calls other than the `DBMS_OUTPUT` forms above, and the wider `DBMS_*` / `UTL_*` library. diff --git a/doc/plxpython3.md b/doc/plxpython3.md index 71bcb48..d24d827 100644 --- a/doc/plxpython3.md +++ b/doc/plxpython3.md @@ -41,6 +41,12 @@ amount #:: numeric amount = principal * rate ``` +Known issue: a top-level first assignment of an interpolated string is folded +into the declaration, so it is evaluated before any earlier statement runs. The +body `x = 1`, `x = 2`, `m = f"v {x}"`, `return m` returns `v 1`. When an +interpolated initializer depends on earlier statements, declare the variable +first (`m #:: text`) and then assign it. + The annotation text is emitted verbatim, so `%TYPE` and `%ROWTYPE` work: ```sql @@ -87,7 +93,7 @@ while n > 0: ```sql outer: for i in range(n): for j in range(n): - if i + j >= limit: + if i + j >= lim: break outer ``` @@ -150,7 +156,8 @@ close_cursor(c) ```sql rc = row_count() # GET DIAGNOSTICS rc = ROW_COUNT -found() # the FOUND variable +if found(): # the FOUND variable + return rc ``` ### Set-returning functions @@ -191,6 +198,8 @@ name it with the `PG::` spelling (`except PG::UniqueViolation as e`), which maps to `unique_violation`. Accessors: `e.message` to `SQLERRM`, `e.sqlstate` to `SQLSTATE`, and `e.detail`, `e.hint`, `e.constraint`, `e.column`, `e.table`, `e.schema`, `e.datatype` to the matching `GET STACKED DIAGNOSTICS` fields. +A `return` inside the `try` or `except` block leaves the function without +running `finally`; set a variable and return after the block instead. ### Assertions @@ -251,29 +260,43 @@ These are intentional. plx pins semantics to SQL and plpgsql. a comparison involving NULL is unknown. Use `is None` / `is not None` (which become `IS NULL` / `IS NOT NULL`) to test for NULL. A positive `if`/`while` condition treats NULL as false. -- Integer division follows SQL rules, not Python's `/` (float) and `//` (floor). - Division and modulo truncate toward zero: `-7 // 2` is `-4` in Python but the - SQL result here is `-3`, and `-7 % 2` is `1` in Python but `-1` here. +- Integer division follows SQL rules, not Python's. `/` on integers truncates + toward zero (`-7 / 2` is `-3`, where Python gives `-3.5`), and `%` takes the + sign of the dividend (`-7 % 2` is `-1`, where Python gives `1`). Floor + division `//` is not supported: it is passed through and fails when the + function runs with `operator does not exist: integer // integer`. - Interpolating a NULL in an f-string propagates it, so the whole string is NULL. It is neither the Python string `"None"` nor an empty string. In a - message being built for `raise` the value falls back to an empty string - instead, so one NULL cannot swallow the message. + `raise` message, including the `raise("level", ...)` call form, the value + falls back to an empty string instead, so one NULL cannot swallow the + message. An `assert` message gets no fallback: a NULL there replaces your + message with plpgsql's generic `assertion failed`. - Comparisons use SQL type resolution. `1 == "1"` compares an integer to a string literal, which SQL coerces and treats as equal; in Python it is false. - Python truthiness is not emulated. A condition must be a boolean expression. A - non-boolean condition is an error reported by plpgsql when the function runs. + non-boolean condition is not rejected at CREATE time. When the function runs, plpgsql coerces it + through its text form: a value whose text is a boolean literal (`0`, `1`, + `t`, `yes`, `off`, ...) is silently accepted, and anything else raises + `invalid input syntax for type boolean`. - Locals are function-scoped, matching Python function scope. ## Not supported Rejected at `CREATE FUNCTION` time with a line number: -- `def`, `class`, `lambda`, decorators, comprehensions, generators. -- `import`, modules, and the Python standard library. -- Tuples, lists, dicts, and sets as general values. +- `def`, `class` (reported as `def is not supported`), decorators, `yield`. +- `import`, `from`, and `with`. - `match`/`case`. - The conditional expression `a if c else b`. -- Per-block local scope (locals are function-scoped by design). + +Not translated: + +- `lambda`, comprehensions, generator expressions, and tuple, list, dict, and + set literals. Unannotated, they fail at `CREATE FUNCTION` time + (`cannot infer a PostgreSQL type`); annotated, they pass through and fail when + the function runs. +- The Python standard library: a call passes through and must name a + PostgreSQL function. See [PARITY.md](PARITY.md) for the full plpgsql construct matrix and [ARCHITECTURE.md](ARCHITECTURE.md) for how plx maps to the plpgsql engine. diff --git a/doc/plxruby.md b/doc/plxruby.md index 22a2c79..c49bf83 100644 --- a/doc/plxruby.md +++ b/doc/plxruby.md @@ -33,8 +33,8 @@ infers it from the first value when it is an integer, numeric, text, or boolean literal. Otherwise annotate the variable with `#:: type`. ```sql -n = 42 -- inferred integer -label = "count" -- inferred text +n = 42 # inferred integer +label = "count" # inferred text amount #:: numeric amount = principal * rate ``` @@ -42,6 +42,12 @@ amount = principal * rate A first assignment of a constant literal at the top level is folded into the declaration, so `n = 42` becomes `DECLARE n integer := 42;`. +Known issue: a top-level first assignment of an interpolated string is also +folded into the declaration, so it is evaluated before any earlier statement +runs. The body `x = 1`, `x = 2`, `m = "v #{x}"`, `return m` returns `v 1`. When +an interpolated initializer depends on earlier statements, declare the variable +first (`m #:: text`) and then assign it. + The annotation text is emitted verbatim as the plpgsql type, so `%TYPE` and `%ROWTYPE` work: @@ -160,7 +166,7 @@ one row. ```sql perform("UPDATE counters SET n = n + 1 WHERE id = #{cid}") execute("INSERT INTO t(msg) VALUES ($1)", note) -return_query("SELECT id FROM vip ORDER BY id") -- in a set-returning function +return_query("SELECT id FROM vip ORDER BY id") # in a set-returning function ``` `perform` runs a literal statement. `execute` runs dynamic SQL with optional bind @@ -183,8 +189,10 @@ close_cursor(c) ### Diagnostics ```sql -rc = row_count() -- GET DIAGNOSTICS rc = ROW_COUNT -found? -- the FOUND variable +rc = row_count() # GET DIAGNOSTICS rc = ROW_COUNT +if found? # the FOUND variable + return rc +end ``` ### Set-returning functions @@ -205,8 +213,8 @@ $$; ### Raising ```sql -raise "bad value" -- EXCEPTION -raise notice: "processed #{n} rows" -- NOTICE +raise "bad value" # EXCEPTION +raise notice: "processed #{n} rows" # NOTICE raise exception: "negative: #{v}", errcode: "22023" ``` @@ -244,7 +252,10 @@ assert(n > 0, "must be positive") - Interpolation: `"total is #{amount}"` becomes `'total is ' || (amount)::text`. Single-quoted strings do not interpolate. - Comparison: `==` and `!=` map to `=` and `<>`. `x == nil` becomes `x IS NULL`. -- Boolean: `&&`, `||`, `!`, and `and`, `or`, `not` map to `AND`, `OR`, `NOT`. +- Boolean: `&&`, `!`, `and`, `or`, and `not` map to `AND`, `NOT`, `AND`, `OR`, + and `NOT`. `||` maps to `OR` only in a condition (`if`, `unless`, `while`, + `until`, or a ternary test); elsewhere it passes through as SQL `||` + (string concatenation), so write `or` there. - Ternary: `c ? a : b` becomes `CASE WHEN c THEN a ELSE b END`. - Casts: `x.to_i`, `x.to_s`, `x.to_f` become `::integer`, `::text`, `::double precision`. `nil` becomes `NULL`. @@ -297,7 +308,11 @@ These are intentional. plx pins semantics to SQL and plpgsql. - Interpolating a NULL propagates it, so `"x=#{v}"` is NULL when `v` is NULL, rather than the `'x='` that Ruby's `nil.to_s` would give. In a message being built for `raise` the value falls back to an empty string instead, so one - NULL cannot swallow the message. + NULL cannot swallow the message. The fallback covers the message text only: + a NULL interpolated into a `detail:` or `hint:` option makes the `raise` + fail with `RAISE statement option cannot be null`, and a NULL in an `assert` + message replaces it with `assertion failed`. Wrap such values in + `coalesce(x, '')`. - Comparisons use SQL type resolution, not Ruby's. `1 == "1"` compares an integer to a string literal, which SQL coerces and treats as equal; in Ruby it is false. Compare like-typed values. @@ -305,8 +320,11 @@ These are intentional. plx pins semantics to SQL and plpgsql. and `-7 % 2` is `-1`, where Ruby gives `-4` and `1`. - Locals are function-scoped, matching Ruby method scope. - Ruby truthiness is not emulated. A condition must be a boolean expression; - write `x != 0` or `!x.nil?` rather than a bare `x`. A non-boolean condition is - an error reported by plpgsql when the function runs, not at CREATE time. + write `x != 0` or `x != nil` rather than a bare `x`. A non-boolean condition + is not rejected at CREATE time. When the function runs, plpgsql coerces it + through its text form: a value whose text is a boolean literal (`0`, `1`, + `t`, `yes`, `off`, ...) is silently accepted, and anything else raises + `invalid input syntax for type boolean`. ## Not supported @@ -315,7 +333,7 @@ Rejected at `CREATE FUNCTION` time with a line number: - Method or class definitions (`def`, classes, modules), gems. - Blocks and lambdas beyond the recognized `.each` forms. - Hash and array literals as general values. -- `||=`, `&&=`, and `and`/`or` in value position. +- `||=` and `&&=`. - `redo`, `retry`. - Predicate/bang methods (`x.zero?`, `arr.empty?`, `s.strip!`): the trailing `?`/`!` is not a SQL operator. Use the SQL form (`x = 0`, diff --git a/doc/plxts.md b/doc/plxts.md index 0a995f9..3df72d8 100644 --- a/doc/plxts.md +++ b/doc/plxts.md @@ -91,17 +91,21 @@ These are intentional and inherited from plxjs (see [plxjs.md](plxjs.md)). - The type system is not enforced at compile time: annotations only set the plpgsql declaration type. Structural/interface types, generics beyond `T[]`, and complex unions are not modeled; use a SQL type name for anything the table - above does not cover. + above does not cover. A union of two non-null types uses the first member + (`number | string` becomes `numeric`). ## Not supported -Everything plxjs rejects (function/class/arrow definitions, object/array -literals as general values, non-`query`/array `for...of`, `switch` -fall-through), plus: +Everything plxjs rejects or does not translate (see [plxjs.md](plxjs.md)), +plus: -- TypeScript type declarations (`interface`, `type`, `enum`), generics, and - type-only imports. +- TypeScript type declarations (`interface`, `type`, `enum`) and type-only + imports, which are rejected at `CREATE FUNCTION` time. +- Generic annotations (`Array`, `Map`) and nested arrays (`number[][]`) + are not mapped. They are emitted verbatim and fail with `invalid type name` + the first time the function runs; use `T[]` or a SQL type name. - Multiple annotated declarators in one statement - (`let a: number = 1, b: string = 'x';`); use one declaration per statement. + (`let a: number = 1, b: string = 'x';`), reported as a type-inference error; + use one declaration per statement. See [PARITY.md](PARITY.md) and [plxjs.md](plxjs.md). diff --git a/doc/plxtsql.md b/doc/plxtsql.md index a0a3a31..8fc7d4e 100644 --- a/doc/plxtsql.md +++ b/doc/plxtsql.md @@ -46,8 +46,8 @@ unwrapped (the declarations inside still hoist). The **body** is T-SQL, but the function **signature** (the parameter and return types in `CREATE FUNCTION`) is parsed by PostgreSQL before plx sees the body, so -it must use PostgreSQL type names: write `RETURNS int`, not `RETURNS INT` inside -a T-SQL `CREATE PROCEDURE`. Parameters are referenced in the body with the `@` +it must use PostgreSQL type names: write `RETURNS timestamp`, not +`RETURNS DATETIME`. Parameters are referenced in the body with the `@` sigil (`@score` above binds to the parameter `score`). Inside the body, T-SQL type names in `DECLARE` are translated (see below). @@ -196,8 +196,9 @@ session option. - Table variables (`DECLARE @t TABLE (...)`) and T-SQL cursors (`DECLARE c CURSOR`). - `GOTO`, `WAITFOR`, and label targets. -- Transaction control (`BEGIN TRAN`, `COMMIT`, `ROLLBACK`): a plpgsql function - runs inside the caller's transaction and cannot manage transactions. +- Transaction control (`BEGIN TRAN`, `COMMIT`, `ROLLBACK`), in a function or a + procedure. The other dialects support `COMMIT` and `ROLLBACK` in a procedure; + plxtsql rejects them everywhere. - Calling a stored procedure by name (`EXEC procname`); only `EXEC('')` dynamic SQL is supported. - String concatenation with `+`. T-SQL overloads `+` for both numeric addition diff --git a/mkdocs.yml b/mkdocs.yml index 91047ac..5b4052f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -81,7 +81,7 @@ nav: - Gaps and limitations: LIMITATIONS.md - Compatibility: COMPATIBILITY.md - Architecture: ARCHITECTURE.md - - Transpiler: TRANSPILER.md + - Transpiler design (historical): TRANSPILER.md - Debugging: DEBUGGING.md extra: diff --git a/plx--2.0.0.sql b/plx--2.0.0.sql index c0d8fc6..67d8416 100644 --- a/plx--2.0.0.sql +++ b/plx--2.0.0.sql @@ -1,4 +1,4 @@ -/* plx 1.0: registers the plx dialect languages and the string-builder type */ +/* plx 2.0.0: registers the nine plx dialect languages and the string-builder type */ \echo Use "CREATE EXTENSION plx" to load this file. \quit From f6c3d365a5b61c7e56fca8f1bd2df77b77608559 Mon Sep 17 00:00:00 2001 From: "Joshua D. Drake" Date: Mon, 9 Nov 2026 20:05:25 +0000 Subject: [PATCH 2/2] Address review: transaction control in plxgo, counts, benchmarks, search_path - plxgo has no transaction control: commit() is accepted at CREATE time and fails at CALL (function commit() does not exist). Name the dialects that support it in plxtsql.md, USERGUIDE and LIMITATIONS, list it under plxgo.md "Not supported", and qualify the PARITY claim for plxgo. - Drop hardcoded dialect counts (COMPATIBILITY "nine dialect files", PARITY "all five column dialects"). - Revert the comment edit to the released plx--2.0.0.sql install script. - bench/BENCHMARKS.md prose now matches its table (iter range, arith slowest, branch fastest, plruby call 3.03x), and the README no longer calls plpgsql competitive on branching, where plphp runs in 0.40x. - Trust and security: the unqualified plx_strbuild/plx_sb_append lookup is also an attack surface for SECURITY DEFINER functions (a planted plx_sb_append earlier on the path); advise pinning search_path with trusted schemas and pg_temp last, as the PostgreSQL manual does. Co-Authored-By: Claude Opus 5.5 --- README.md | 4 +++- bench/BENCHMARKS.md | 16 +++++++++------- doc/COMPATIBILITY.md | 2 +- doc/LIMITATIONS.md | 11 +++++++---- doc/PARITY.md | 7 ++++--- doc/USERGUIDE.md | 23 ++++++++++++++++------- doc/plxgo.md | 2 ++ doc/plxtsql.md | 5 +++-- plx--2.0.0.sql | 2 +- 9 files changed, 46 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 68ebc7c..978d5c2 100644 --- a/README.md +++ b/README.md @@ -169,7 +169,9 @@ Because functions execute as plpgsql, the plx dialects match plpgsql within about 10 percent on most workloads (the worst cases measured are plxruby row iteration at 1.17x and plxcobol's `EVALUATE` branch at 1.32x) and inherit its performance profile: several times faster than the embedded-interpreter PLs on -row iteration, and competitive on arithmetic, branching, and call overhead. +row iteration, fastest on call overhead, and between the embedded PLs on +arithmetic. On a pure-CPU branching loop the embedded PLs are faster (native +PL/PHP runs it in 0.40x of plpgsql's time). The one place plpgsql is weak is building a string in a loop: `s := s || 'x'` is O(n^2), because text is immutable and each step copies the whole string. plx diff --git a/bench/BENCHMARKS.md b/bench/BENCHMARKS.md index 996705f..de74d57 100644 --- a/bench/BENCHMARKS.md +++ b/bench/BENCHMARKS.md @@ -79,7 +79,7 @@ building). On PostgreSQL 13 to 17 the plx strbuild column would match plpgsql counting loop lowers to an integer `FOR`, and `STRING-APPEND` lowers to the string builder). -- Row iteration (iter): plpgsql, and therefore the plx dialects, are 2.8x to 6.0x +- Row iteration (iter): plpgsql, and therefore the plx dialects, are 2.7x to 5.6x faster than the embedded PLs (plperl, plpython3u, native plruby, native plphp). plpgsql streams rows through a cursor and reads columns directly, while the embedded interpreters copy each row into their own data structures. @@ -97,16 +97,18 @@ building). On PostgreSQL 13 to 17 the plx strbuild column would match plpgsql the same win is available by calling `plx_sb_append`, or by assembling text in SQL with `string_agg` over a set. -- Arithmetic (arith): the embedded interpreters vary. plperl is fastest (native - integer arithmetic), plpython3u is slowest, and plpgsql with the plx dialects - sits between. +- Arithmetic (arith): the embedded interpreters vary. plperl is fastest (0.79x, + native integer arithmetic), native plruby is slowest (1.79x) followed by + plpython3u (1.31x), and plpgsql with the plx dialects sits between. -- Branching (branch): plpython3u is faster on this pure-CPU integer workload - (0.72x); plperl and plpgsql with the plx dialects are close to each other. +- Branching (branch): on this pure-CPU integer workload the embedded PLs are + faster: native plphp (0.40x), plpython3u (0.68x), native plruby (0.90x) and + plperl (0.95x). plpgsql and the plx dialects are at about 1.0x (plxcobol's + `EVALUATE` at 1.32x). - Call overhead (call): plpgsql and the plx dialects are fastest. The embedded PLs pay an SPI round trip per call and are slower even at one tenth the - iteration count (native plruby is the slowest here, at 2.97x). + iteration count (native plruby is the slowest here, at 3.03x). The transpile-to-plpgsql approach gives the plx dialects the performance profile of plpgsql: strong on set-oriented and SQL-bound work, competitive on procedural diff --git a/doc/COMPATIBILITY.md b/doc/COMPATIBILITY.md index d24477f..b85bc9c 100644 --- a/doc/COMPATIBILITY.md +++ b/doc/COMPATIBILITY.md @@ -3,7 +3,7 @@ ## Supported versions plx supports PostgreSQL 13 through 18 (released), and builds and passes on the -19 and 20 development lines. The full pg_regress suite (the nine dialect files +19 and 20 development lines. The full pg_regress suite (one file per dialect plus plx_features, plx_output, plx_strbuild, and plx_errors) passes on each of PostgreSQL 13 through 18. CI (`.github/workflows/ci.yml`) runs `make installcheck`, `make differentialcheck`, and `make metacheck` on PostgreSQL 13 diff --git a/doc/LIMITATIONS.md b/doc/LIMITATIONS.md index 04f5f59..fe00e38 100644 --- a/doc/LIMITATIONS.md +++ b/doc/LIMITATIONS.md @@ -27,8 +27,8 @@ These follow from running as plpgsql and are not specific to any one dialect. caller's transaction, so `COMMIT` and `ROLLBACK` raise `invalid transaction termination` there. In a `CREATE PROCEDURE` invoked with `CALL` (outside an explicit transaction block), `commit()`/`rollback()` (COBOL `COMMIT`/`ROLLBACK`, - PL/SQL `COMMIT;`) work as in plpgsql. plxtsql does not support transaction - control. Savepoint management is not available in any dialect. + PL/SQL `COMMIT;`) work as in plpgsql. plxtsql and plxgo do not support + transaction control. Savepoint management is not available in any dialect. - **The signature is SQL.** The parameter and return types in `CREATE FUNCTION` are parsed by PostgreSQL before plx sees the body, so they must be PostgreSQL type names (`integer`, `numeric`, `text`, `timestamp`, ...). Dialect type names @@ -77,7 +77,9 @@ These follow from running as plpgsql and are not specific to any one dialect. `plx_sb_append` function unqualified, so the extension's schema must be on the function's `search_path` at run time, including any `SET search_path` clause on the function. Otherwise the function is created but fails when called with - `type "plx_strbuild" does not exist`. + `type "plx_strbuild" does not exist`. Because the lookup goes through the + path, a `SECURITY DEFINER` function should pin it; see + [Trust and security](USERGUIDE.md#trust-and-security). - **Runtime errors point at the generated plpgsql.** plpgsql requires all declarations at the top, so line numbers in a runtime error refer to the transpiled body, not your dialect source. `plx_source()` recovers the embedded @@ -88,7 +90,8 @@ These follow from running as plpgsql and are not specific to any one dialect. - **What is always reachable.** Every plpgsql statement type (assignment, `IF`/`CASE`, all loops, `RETURN`/set-returning, cursors, dynamic SQL, `RAISE`, exception handling, triggers, `GET DIAGNOSTICS`) is expressible from every - dialect, except that plxtsql has no cursors or transaction control. See + dialect, except that plxtsql has no cursors or transaction control and plxgo + has no transaction control. See [PARITY.md](PARITY.md) for the construct-by-construct matrix. A dialect gives you its syntax over SQL semantics, not its standard library. diff --git a/doc/PARITY.md b/doc/PARITY.md index f1f40f4..103ba6b 100644 --- a/doc/PARITY.md +++ b/doc/PARITY.md @@ -4,7 +4,7 @@ Target: every plpgsql capability is expressible in each plx dialect. The reference is the plpgsql statement set in `src/pl/plpgsql/src/plpgsql.h` (`PLpgSQL_stmt_*`) plus its declaration forms. -Every construct below is supported in all five column dialects (**plxruby**, +Every construct below is supported in each column dialect (**plxruby**, **plxphp**, **plxjs**, **plxpython3**, **plxcobol**) except where a footnote notes an equivalent form. The cells show the dialect spelling; full syntax and semantics are in the per-dialect chapters ([plxruby](plxruby.md), @@ -27,8 +27,9 @@ IMMEDIATE`, `NVL`, `seq.NEXTVAL`, `NUMBER`/`VARCHAR2`, ...) on top. See `var`/`:=` declarations (with type inference) hoist into the `DECLARE` block, parenless `if`/`for` headers become `IF ... THEN`/`WHILE ... LOOP`, `for range` becomes `FOR`/`FOREACH`, the no-fallthrough `switch` becomes `IF`/`ELSIF`, and -`panic`/`fmt.Println` become `RAISE`. Every plpgsql construct is reachable -through Go surface syntax. See [plxgo.md](plxgo.md). +`panic`/`fmt.Println` become `RAISE`. Every plpgsql construct except +transaction control is reachable through Go surface syntax. See +[plxgo.md](plxgo.md). **plxtsql** (Transact-SQL, SQL Server) is not shown as a separate column because it restructures rather than renames: `@x` variables hoist to the `DECLARE` block, diff --git a/doc/USERGUIDE.md b/doc/USERGUIDE.md index 8fec64f..4b3d024 100644 --- a/doc/USERGUIDE.md +++ b/doc/USERGUIDE.md @@ -641,10 +641,10 @@ for i in range(1, n + 1): $$; ``` -plxphp and plxjs use the same `commit()` and `rollback()` calls, plxcobol uses -the `COMMIT` and `ROLLBACK` verbs, and plxplsql uses `COMMIT;` as in PL/SQL. -plxtsql does not support transaction control. Calling `commit()` from a -function, or from a procedure called inside an explicit transaction block, +plxphp, plxjs and plxts use the same `commit()` and `rollback()` calls, +plxcobol uses the `COMMIT` and `ROLLBACK` verbs, and plxplsql uses `COMMIT;` as +in PL/SQL. plxtsql and plxgo do not support transaction control. Calling +`commit()` from a function, or from a procedure called inside an explicit transaction block, raises `invalid transaction termination`, as in plpgsql. ## Trust and security @@ -694,9 +694,18 @@ Points to keep in mind: its interpolated values are concatenated exactly as in `execute`: do not mix interpolation with binds. - Functions that use the string-append operators reference the extension's - `plx_strbuild` type and `plx_sb_append` function unqualified, so the - extension's schema must be on the function's `search_path` at run time, - including any `SET search_path` clause on a `SECURITY DEFINER` function. + `plx_strbuild` type and `plx_sb_append` function unqualified, so both are + looked up through `search_path` each time the function runs. The extension's + schema must therefore be on the function's path, including any + `SET search_path` clause. It also means that a role able to create objects in + a schema that comes earlier on the path can put its own `plx_sb_append` there, + and a `SECURITY DEFINER` function would run it with the owner's privileges. + Pin the path on every `SECURITY DEFINER` function, listing only trusted + schemas, the extension's schema among them, and `pg_temp` last, as the + PostgreSQL manual advises: + `ALTER FUNCTION f(...) SET search_path = app, public, pg_temp`, where + `public` stands for the schema plx is installed in and is not writable by + untrusted roles. ## Further reading diff --git a/doc/plxgo.md b/doc/plxgo.md index 1733379..d37b08d 100644 --- a/doc/plxgo.md +++ b/doc/plxgo.md @@ -220,6 +220,8 @@ $$; - `map`, `chan`, `struct`, and `interface` types (declare data with SQL types or arrays instead). - `switch` `fallthrough`. +- Transaction control. `commit()` and `rollback()` are accepted when a procedure + is created but fail when it runs (`function commit() does not exist`). - String concatenation with `+`. Go overloads `+` for numeric addition and string concatenation; plx cannot tell which is meant without type information, so it leaves `+` as `+`. `||` is logical OR in plxgo, so use diff --git a/doc/plxtsql.md b/doc/plxtsql.md index 8fc7d4e..366fd5a 100644 --- a/doc/plxtsql.md +++ b/doc/plxtsql.md @@ -197,8 +197,9 @@ session option. (`DECLARE c CURSOR`). - `GOTO`, `WAITFOR`, and label targets. - Transaction control (`BEGIN TRAN`, `COMMIT`, `ROLLBACK`), in a function or a - procedure. The other dialects support `COMMIT` and `ROLLBACK` in a procedure; - plxtsql rejects them everywhere. + procedure; plxtsql rejects them everywhere. plxruby, plxphp, plxjs, plxts, + plxpython3, plxcobol and plxplsql support `COMMIT` and `ROLLBACK` in a + procedure; plxgo does not. - Calling a stored procedure by name (`EXEC procname`); only `EXEC('')` dynamic SQL is supported. - String concatenation with `+`. T-SQL overloads `+` for both numeric addition diff --git a/plx--2.0.0.sql b/plx--2.0.0.sql index 67d8416..c0d8fc6 100644 --- a/plx--2.0.0.sql +++ b/plx--2.0.0.sql @@ -1,4 +1,4 @@ -/* plx 2.0.0: registers the nine plx dialect languages and the string-builder type */ +/* plx 1.0: registers the plx dialect languages and the string-builder type */ \echo Use "CREATE EXTENSION plx" to load this file. \quit