Repository navigation
feat(faustwp): add an opt-in versioned document store with conflict detection #2674
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
+3,227
−5
Closed
Changes from all commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
b9d8bfd
feat(faustwp): add an opt-in versioned document store with conflict d…
josephfusco ffe318f
fix(faustwp): return empty objects in documents as objects
josephfusco 835d32c
test(faustwp): cover concurrent creates in the document store
josephfusco 0fbc7cc
docs: show the full conflict response for document writes
josephfusco 138c61e
ci(faustwp): give PHPStan 2 GB of memory
josephfusco 630bca7
docs(examples): add a document store example that runs WordPress locally
josephfusco ad9248a
docs(examples): show WordPress errors that aren't JSON in the documen…
josephfusco 13dd94e
docs(examples): explain how to work on the plugin with the document s…
josephfusco 8528b1a
fix(examples): hide the expected errors while the document store setu…
josephfusco 3d7ec21
feat(faustwp): record when and by whom each document was last saved
josephfusco 53a0a1c
feat(faustwp): list stored documents under Tools → Faust Documents
josephfusco 3ab72b9
feat(examples): add a three-way merge helper to the document store ex…
josephfusco 39b9a89
feat(examples): merge concurrent edits in the document store example
josephfusco 57587c2
docs(examples): describe merging and field-level conflicts in the doc…
josephfusco 64ac670
fix(examples): label conflicting fields with their form labels
josephfusco e284813
fix(examples): point the document store example's WordPress at its ow…
josephfusco 6bcbcb2
fix(faustwp): show document JSON without per-line code backgrounds
josephfusco 84d5414
docs(faustwp): mention the Tools screen in the document post type doc…
josephfusco fe4f36a
perf(faustwp): list documents without a slug lookup per row
josephfusco 46516e3
refactor(faustwp): store documents in a dedicated table instead of posts
josephfusco 3456726
fix(faustwp): retry a failed documents table install and drop the tab…
josephfusco 087677c
docs(faustwp): clarify the documents schema version and shared owner …
josephfusco aea1486
refactor(faustwp): organize documents into collections with IDs
josephfusco 5c28245
feat(faustwp): list, add and delete documents, with timestamps and sa…
josephfusco 178c0fc
docs: describe listing, adding and deleting documents and the documen…
josephfusco 0510d71
feat(examples): add a waitlist page that adds, lists and removes docu…
josephfusco 3a9d0ef
fix(faustwp): never report a 1970 creation date after a racing delete
josephfusco bea7d06
refactor: use the faust prefix for the document store's hooks, routes…
josephfusco 85707b1
fix(faustwp): never reuse document versions, and only take identifier…
josephfusco ee5d2e7
fix(examples): keep the document store example's data and edits safe
josephfusco File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| # Your WordPress site URL | ||
| NEXT_PUBLIC_WORDPRESS_URL=http://localhost:8090 | ||
|
|
||
| # A WordPress user that can read and write documents, and an Application | ||
| # Password for it. `npm run wordpress:setup` creates both and prints the values. | ||
| # These stay on the server: never prefix them with NEXT_PUBLIC_. | ||
| WP_SERVICE_USER=document-app | ||
| WP_SERVICE_APP_PASSWORD= |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| # Document store example | ||
|
|
||
| A small Next.js app that uses the [FaustWP document store](https://faustjs.org/docs/how-to/write-data-to-wordpress/). The **Team notes** page edits a shared document and shows what happens when two people save at the same time. The **Waitlist** page adds, lists and removes documents in a collection. | ||
|
|
||
| The app's API route talks to WordPress with an [Application Password](https://make.wordpress.org/core/2020/11/05/application-passwords-integration-guide/), so the credentials stay on the server. It doesn't use Faust.js login: every visitor edits the document as the same WordPress user. | ||
|
|
||
| ## Run it locally | ||
|
|
||
| You need Docker and Node.js. | ||
|
|
||
| 1. Start WordPress with FaustWP and the document store enabled: | ||
|
|
||
| ```bash | ||
| npm run wordpress:setup | ||
| ``` | ||
|
|
||
| This runs WordPress on `http://localhost:8090` (log in with `admin` / `password`), creates a user that can only read and write documents, and prints the values for the next step. Set `WP_PORT` or `DB_PORT` to use other ports, or `WP_VERSION` to pick a WordPress version. | ||
|
|
||
| 2. Copy `.env.local.sample` to `.env.local` and paste in the printed values. | ||
|
|
||
| 3. Install and start the app: | ||
|
|
||
| ```bash | ||
| npm install | ||
| npm run dev | ||
| ``` | ||
|
|
||
| 4. Open `http://localhost:3000` in two tabs and try a few conflicts. The first save in either tab always works. The second is rejected by WordPress with `409 Conflict`, because it's based on a version that's no longer current, and the page decides what to do: | ||
| - **No edits in the second tab:** it loads the newer version. | ||
| - **The tabs changed different fields** (for example, notes in one and status in the other): it merges both changes and saves a new version. | ||
| - **The tabs changed the same field:** it lists just those fields with the earlier value, theirs and yours. Choose what to keep for each and click **Save merged**. Everything else is already merged. | ||
|
|
||
| 5. Open `http://localhost:3000/waitlist` and add a few emails. Each one becomes a document with an ID WordPress generates, the list shows five at a time, newest first, and **Remove** deletes a signup. | ||
|
|
||
| Stop WordPress with `npm run wordpress:stop`. Add `-- --volumes` to also delete its database. | ||
|
|
||
| ## Working on the document store | ||
|
|
||
| WordPress loads FaustWP straight from `plugins/faustwp` in this repository, so changes to the plugin take effect on the next request without restarting anything. To run WP-CLI against the site, for example to inspect a document: | ||
|
|
||
| ```bash | ||
| docker compose --project-name faust-document-store exec wordpress wp --allow-root eval 'global $wpdb; print_r( $wpdb->get_results( "SELECT collection, document_id, version, modified_gmt FROM {$wpdb->prefix}faust_documents" ) );' | ||
| ``` | ||
|
|
||
| ## How it works | ||
|
|
||
| - `mu-plugins/enable-document-store.php` is a must-use plugin that turns on the document store with the `faust_enable_document_store` filter. | ||
| - `lib/wordpress.js` forwards requests to `/?rest_route=/faust/v1/...` with the Application Password, and passes WordPress's status code back: `404` before a document exists, `201` when one is created, `200` when it's updated or deleted and `409` for a stale write or delete. | ||
| - `pages/api/collections/[collection]/documents/[id].js` handles `GET`, `PUT` and `DELETE` for one document, and `pages/api/collections/[collection]/documents/index.js` handles `GET` (list) and `POST` (add with a generated ID) for a collection. | ||
| - `pages/waitlist.js` adds signups with `POST`, pages through the list with `next_cursor`, and removes a signup by sending the version it was listed at. | ||
| - `pages/index.js` keeps the version it last read and sends it with every save. On a `409`, the response includes the current document, and the page merges it with the local edits. | ||
| - `lib/mergeDocuments.mjs` is the three-way merge: it compares the version the tab started from with theirs and yours, field by field, and reports fields both sides changed differently. Run its tests with `npm test`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Used with plugins/faustwp/docker-compose.yml by scripts/setup-wordpress.sh. | ||
| # Paths are relative to plugins/faustwp, the directory of the first compose file. | ||
| services: | ||
| wordpress: | ||
| environment: | ||
| # WordPress only allows Application Passwords over HTTPS or on local sites. | ||
| WP_ENVIRONMENT_TYPE: local | ||
| volumes: | ||
| - ../../examples/next/document-store/mu-plugins/enable-document-store.php:/var/www/html/wp-content/mu-plugins/enable-document-store.php | ||
| db: | ||
| volumes: | ||
| # Keep the database when `npm run wordpress:stop` removes the container. | ||
| - db:/var/lib/mysql | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| // Three-way merge for plain JSON objects. | ||
| // | ||
| // `base` is the version both sides started from, `theirs` is what's stored now, | ||
| // and `yours` is the local edit. Fields only one side changed are merged | ||
| // automatically. Fields both sides changed to different values are returned as | ||
| // conflicts, with the merged result keeping theirs until someone decides. | ||
| // | ||
| // Nested plain objects are merged field by field. Arrays and other values are | ||
| // compared as a whole. | ||
|
|
||
| function isPlainObject(value) { | ||
| return ( | ||
| value !== null && | ||
| typeof value === 'object' && | ||
| !Array.isArray(value) && | ||
| Object.getPrototypeOf(value) === Object.prototype | ||
| ); | ||
| } | ||
|
|
||
| export function deepEqual(a, b) { | ||
| if (a === b) { | ||
| return true; | ||
| } | ||
| if (Array.isArray(a) && Array.isArray(b)) { | ||
| return a.length === b.length && a.every((item, i) => deepEqual(item, b[i])); | ||
| } | ||
| if (isPlainObject(a) && isPlainObject(b)) { | ||
| const keys = Object.keys(a); | ||
| return ( | ||
| keys.length === Object.keys(b).length && | ||
| keys.every((key) => Object.hasOwn(b, key) && deepEqual(a[key], b[key])) | ||
| ); | ||
| } | ||
| return false; | ||
| } | ||
|
|
||
| // A missing key and a key set to undefined both mean "not there". | ||
| const MISSING = undefined; | ||
|
|
||
| function mergeValue(base, theirs, yours, path, conflicts) { | ||
| const theyChanged = !deepEqual(base, theirs); | ||
| const youChanged = !deepEqual(base, yours); | ||
|
|
||
| if (!youChanged) { | ||
| return theirs; | ||
| } | ||
| if (!theyChanged || deepEqual(theirs, yours)) { | ||
| return yours; | ||
| } | ||
| if (isPlainObject(base) && isPlainObject(theirs) && isPlainObject(yours)) { | ||
| return mergeObjects(base, theirs, yours, path, conflicts); | ||
| } | ||
|
|
||
| conflicts.push({ path, base, theirs, yours }); | ||
| return theirs; | ||
| } | ||
|
|
||
| function mergeObjects(base, theirs, yours, path, conflicts) { | ||
| const merged = {}; | ||
| const keys = new Set([ | ||
| ...Object.keys(base), | ||
| ...Object.keys(theirs), | ||
| ...Object.keys(yours), | ||
| ]); | ||
|
|
||
| for (const key of keys) { | ||
| const value = mergeValue( | ||
| Object.hasOwn(base, key) ? base[key] : MISSING, | ||
| Object.hasOwn(theirs, key) ? theirs[key] : MISSING, | ||
| Object.hasOwn(yours, key) ? yours[key] : MISSING, | ||
| [...path, key], | ||
| conflicts, | ||
| ); | ||
| if (value !== MISSING) { | ||
| // defineProperty rather than assignment, so a "__proto__" key from | ||
| // the JSON becomes a field instead of changing the prototype. | ||
| Object.defineProperty(merged, key, { | ||
| value, | ||
| enumerable: true, | ||
| configurable: true, | ||
| writable: true, | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| return merged; | ||
| } | ||
|
|
||
| /** | ||
| * @param {object} base The document both sides started from. | ||
| * @param {object} theirs The document as it's stored now. | ||
| * @param {object} yours The local edit. | ||
| * @returns {{ merged: object, conflicts: Array<{ path: string[], base: any, theirs: any, yours: any }> }} | ||
| */ | ||
| export function mergeDocuments(base, theirs, yours) { | ||
| const conflicts = []; | ||
| const merged = mergeObjects(base ?? {}, theirs ?? {}, yours ?? {}, [], conflicts); | ||
|
|
||
| return { merged, conflicts }; | ||
| } | ||
|
|
||
| /** | ||
| * Return a copy of `object` with the value at `path` replaced. | ||
| * | ||
| * @param {object} object | ||
| * @param {string[]} path | ||
| * @param {any} value Use undefined to remove the key. | ||
| * @returns {object} | ||
| */ | ||
| export function setAtPath(object, path, value) { | ||
| const [key, ...rest] = path; | ||
| // A computed key defines an own property, even for "__proto__", so the | ||
| // assignments below write a field instead of changing the prototype. | ||
| const copy = { ...object, [key]: undefined }; | ||
| if (rest.length) { | ||
| copy[key] = setAtPath(isPlainObject(object[key]) ? object[key] : {}, rest, value); | ||
| } else if (value === MISSING) { | ||
| delete copy[key]; | ||
| } else { | ||
| copy[key] = value; | ||
| } | ||
| return copy; | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.