Skip to content

About

Deterministic layout reconstruction for messy documents. It reads PDF geometry (text operators, vector paths, the struct tree) and rebuilds columns, tables, figures, math and reading order into **GxDoc**, a typed intermediate structure that apps edit and export

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Ginexys Layout Engine

Deterministic layout reconstruction for messy documents. It reads PDF geometry (text operators, vector paths, the struct tree) and rebuilds columns, tables, figures, math and reading order into GxDoc, a typed intermediate structure that apps edit and export. It runs in the browser or a worker: no upload, no required model weights, same input gives the same output.

This is the engine behind Document Workbench. The Workbench app consumes it as a package and contains no extraction code.

Install

npm install @ginexys/layout-engine pdfjs-dist@4

pdfjs-dist 4.x is a peer dependency. onnxruntime-web is an optional peer, needed only for the layout-model worker.

Quick start

Tell the engine where the pdf.js worker is served from, once, at startup. With Vite:

import pdfWorkerUrl from 'pdfjs-dist/build/pdf.worker.mjs?url';
import { setPdfWorkerSrc } from '@ginexys/layout-engine/extraction/util/assetBase';

setPdfWorkerSrc(pdfWorkerUrl);

Reconstruct a PDF in the geometry worker. Results stream back one page at a time:

import GeometryWorker from '@ginexys/layout-engine/workers/geometryWorker?worker';
import { pdfWorkerSrc } from '@ginexys/layout-engine/extraction/util/assetBase';

const worker = new GeometryWorker();
worker.onmessage = ({ data }) => {
  if (data.type === 'page') console.log(data.page, data.regions, data.html);
  if (data.type === 'complete') console.log(`${data.pageCount} pages`);
  if (data.type === 'error') console.error(data.error);
};
const bytes = new Uint8Array(await file.arrayBuffer());
worker.postMessage({ type: 'process', bytes, pdfWorkerSrc: pdfWorkerSrc() });

Work with the result as GxDoc:

import { validateDoc } from '@ginexys/layout-engine/gxdoc';
import { htmlToGxDoc } from '@ginexys/layout-engine/ir/htmlToGxDoc';
import { gxDocToHtml } from '@ginexys/layout-engine/ir/gxDocToHtml';

With other bundlers, create the worker from @ginexys/layout-engine/workers/geometryWorker the way that bundler handles module workers.

From a CDN

The modules are plain ES modules, but they import pdfjs-dist, katex and @canwork/boxwood by name, so load them through a CDN that rewrites those imports, such as esm.sh:

<script type="module">
  import { validateDoc } from 'https://esm.sh/@ginexys/layout-engine@0.1.0/gxdoc';
  import { gxDocToHtml } from 'https://esm.sh/@ginexys/layout-engine@0.1.0/ir/gxDocToHtml';
</script>

Browsers only start workers from the page's own origin, so for the geometry worker, install the package or serve its files yourself.

What is in here

Path Role
src/extraction/vector/ Path reconciler, column detection, table reconstruction, page assembly, math
src/extraction/forensics/ Document forensics: altered regions, duplicate pages, OCR confidence, reading order
src/extraction/util/ Engine utilities (asset base, image store, math render, OCR grid and scale)
src/ir/ GxDoc, the output contract, plus importers and renderers
src/workers/ Geometry and layout workers

Boundary rule

The engine never imports from an app. Apps import the engine only through the entry points in package.json exports:

import { validateDoc } from '@ginexys/layout-engine/gxdoc';
import { gxDocToHtml } from '@ginexys/layout-engine/ir/gxDocToHtml';
import { analyseDocument } from '@ginexys/layout-engine/forensics';

GxDoc is versioned (gx-doc/1). A breaking schema change is a major version.

Benchmark

Vector layer, scored on the 4,073 born-digital pages of the DocLayNet v1.2 test split (CDLA-Permissive-1.0, human-labelled, never used for tuning): 0 crashes, identical output on repeat runs, about 22 ms per page (median). Scanned and image-backed pages (926) are reported separately once the DocForensics path is complete. Accuracy, the method and the known gaps it exposed are in bench/README.md.

Development

npm install
npm test

License

AGPL-3.0-or-later, with a commercial license available from Canwork LLC (legal@ginexys.com). Code from v0.1.0-mit and earlier remains MIT. See NOTICE. Outside contributions require a contributor license agreement so the dual license stays possible.

About

Deterministic layout reconstruction for messy documents. It reads PDF geometry (text operators, vector paths, the struct tree) and rebuilds columns, tables, figures, math and reading order into **GxDoc**, a typed intermediate structure that apps edit and export

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages