Skip to content

javaabu/exports v2.0: streaming exports, queued exports, exports UI - #2

Open
n4ff4h wants to merge 13 commits into
Javaabu:mainfrom
n4ff4h:next-gen
Open

javaabu/exports v2.0: streaming exports, queued exports, exports UI#2
n4ff4h wants to merge 13 commits into
Javaabu:mainfrom
n4ff4h:next-gen

Conversation

@n4ff4h

@n4ff4h n4ff4h commented Nov 4, 2026

Copy link
Copy Markdown

javaabu/exports v2.0

The next-gen major of exports, built on the fork n4ff4h/exports (next-gen). Tracked in n4ff4h#13.

What's new

  • Streaming with OpenSpout, no more maatwebsite/excel. CSV, XLSX and ODS are written row by row, and rows are read with lazyById() (or lazy() for ordered queries) with eager loads per chunk. A 1M-row benchmark runs in CI: +4.8 MB peak for CSV and +3.6 MB for XLSX.
  • Smarter ModelExport defaults.
    • Hidden attributes and secrets are left out, and foreign keys are replaced by their relation.
    • Headings come from validation attribute names, the model, or a translated headline.
    • Enums show their labels, booleans show Yes/No, and dates are in the app timezone.
    • columns() takes dot notation and Column definitions.
  • Queued exports tracked by an Export model (uuid, user morph, status, progress, file, expiry).
    • ProcessExport saves progress, can be cancelled, writes in the locale the export was queued in, and stores to any disk.
    • The user is notified with a signed, expiring download link.
    • exports:prune deletes expired exports.
  • Exports page and JSON API.
    • Route::exports() lets the app choose the prefix, name, guard and middleware.
    • The controller can be overridden through config, the macro or the container.
    • Exports::register() lets the page start an export by name.
    • The pages ship in Blade (javaabu/forms v2) and Inertia React (TypeScript, @javaabu/forms + @javaabu/onu, delivered through javaabu/manifest).
  • Live progress. With laravel/reverb installed, progress is broadcast on private channels; without it, pages poll. Reverb is suggested, not required.
  • Permissions. ExportPolicy plus view_exports, view_others_exports, delete_exports and delete_others_exports, registered through javaabu/permissions v3 and overridable. Tested with two guards and two user models.
  • Dhivehi. Every string is translated, XLSX sheets are RTL in Dhivehi, CSV files carry a BOM, and the views use logical utilities only.
  • Safety. Strings are never written as formulas, and CSV formula injection is escaped.
  • Docs and tooling. Full docs in docs/, plus llms.txt, a Laravel Boost guideline and skill, and AGENTS.md.

Upgrade notes (full guide in docs/upgrading.md)

  • PHP 8.4+ and Laravel 12/13. maatwebsite/excel can't be installed on PHP 8.5 (PhpSpreadsheet caps it), which is part of why it was dropped.
  • Replace Excel::download(new UsersExport, …) with UsersExport::make()->download(…), and the same for store().
    • maatwebsite's queue() becomes queue($user, …), which returns an Export.
    • maatwebsite concerns (WithStyles, WithEvents, …) aren't supported.
  • v1 hooks still work: modelClass, relationsToInclude, allowedAttributes, formatValue, headings, map.
  • The output changes in a few places: Yes/No instead of True/False, string dates, ID headings, FK columns replaced by relations, , separators.
  • Run php artisan migrate (the exports table) and php artisan permissions:sync.

Dependencies on unreleased packages

This branch requires javaabu/permissions ^3.0@dev, javaabu/forms ^2.0@dev and javaabu/manifest ^1.0@dev through VCS repositories. It also uses @javaabu/forms from github:Javaabu/react-forms#next-gen for the React pages. Before tagging v2.0:

  1. Release those packages.
  2. Switch the constraints to stable.
  3. Remove the repositories entries.

For the reviewer

  • CODECOV_TOKEN must be available to Javaabu/exports, or coverage uploads are skipped (CI doesn't fail on that).
  • The Dhivehi strings listed in docs/advanced-usage/translations.md#strings-for-review need a native speaker's check.
  • Register the docs with docs.javaabu.com (Register with Javaabu Docs (docs.javaabu.com) n4ff4h/exports#12).

🤖 Generated with Claude Code

n4ff4h and others added 11 commits November 4, 2026 13:17
@n4ff4h @claude
Foundations: PHP 8.4, Laravel 12/13, Pint, CI matrix, Codecov; write …
903d41e 
…files with OpenSpout

maatwebsite/excel can't be installed on PHP 8.5 (PhpSpreadsheet 1.30 caps PHP
below 8.5), so it's replaced here rather than later: exports are written with
OpenSpout (CSV, XLSX, ODS) through SpreadsheetWriter, and ModelExport gets
download(), store() and writeTo() from the WritesFiles trait. The v1 ModelExport
API and output are unchanged for now.

Closes #1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Stream model exports in chunks with smarter defaults
7b5da35 
Rows are read with lazyById() (lazy() when the query is ordered) and eager
loaded per chunk, so memory stays flat: the 1M-row benchmark (composer
benchmark, and its own CI job) peaks at +4.4 MB for CSV and +3.3 MB for XLSX.

ModelExport now picks readable defaults: no hidden attributes or secrets,
foreign keys replaced by their BelongsTo relation, headings from the column,
the model's exportHeadings(), validation attribute names, or a translated
headline; enum labels, Yes/No, dates in the app timezone, JSON for arrays,
related models by name. columns() takes dot notation and Column definitions.
The v1 hooks (allowedAttributes, relationsToInclude, formatValue, headings,
map) still work.

Closes #2
Closes #3

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Queue exports with an Export model, progress, notifications and pruning
686466b 
queue() creates an Export (uuid, user morph, format, status, rows, disk,
path, size, expiry) and dispatches ProcessExport. The job streams the whole
file in one go (XLSX and ODS can't be appended to from parallel jobs), saves
progress every chunk, stops when the export is cancelled, moves the file to
the configured disk under exports/{uuid}/, and notifies the user with a signed
link that expires with the file. Builders passed to an export are serialized
as SQL, bindings, order and eager loads (closures included) and rebuilt in
the job; models held by an export are serialized as identifiers.

exports:prune deletes expired exports, and stuck ones older than the expiry
period, with their files.

Closes #4
Refs #5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Add exports routes, an overridable controller, the policy and permiss…
6e3283e 
…ions

Route::exports() registers index, store, show, download, cancel and destroy
under the app's prefix, name, guard and middleware, plus a signed download
route for notification links. ExportsController answers JSON (ExportResource
with progress, abilities, links and broadcast info) or renders pages through
ViewRenderer, and is replaceable from config, the macro or the container.

Permissions view_exports, view_others_exports, delete_exports and
delete_others_exports are registered with javaabu/permissions, with
ExportPolicy as the default policy (overridable from permissions.policies or
Gate::policy()). Users see their own exports; *_others_* see everyone's.
Tested with two guards and two user models.

Closes #11

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Broadcast export progress over Reverb, with a polling fallback
db44748 
Export events implement ShouldBroadcastNow and broadcast only when
Broadcasting::enabled(): forced by exports.broadcasting.enabled, or on 'auto'
when laravel/reverb is installed and the default broadcaster isn't null or
log. They go to private-exports.{uuid} (authorized with the policy's view
ability) and private-exports.owner.{type}.{id} (the owner only) as
export.started, export.progressed, export.completed, export.failed and
export.cancelled, with a small progress payload.

Without broadcasting, the JSON meta says broadcasting: false and gives the
poll interval, so pages poll the show route. laravel/reverb is suggested and
installed for tests only.

Closes #6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Add English and Dhivehi translations for every user-facing string
511b04a 
lang/en.json and lang/dv.json cover statuses, formats, messages,
notifications, permission descriptions and the common default headings.
A test fails when a __() string in src or the views has no Dhivehi line.

Refs #9

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Document v2 for people and agents; add the Boost guideline and skill
5b966e3 
Docs for installing, model exports, downloading and storing, queued exports,
the exports page and API, performance (with the benchmark), permissions and
policies with multiple guards, live progress, translations, configuration,
an API reference, AI assistants, and upgrading from v1 with the reasons for
dropping maatwebsite/excel and how to migrate. llms.txt indexes every page.
AGENTS.md and CLAUDE.md for agents working on the repo.

Queued exports now write the file in the locale active when queue() was
called. Broadcasting's enabled flag accepts boolean strings.

The pages for the views come with the views.

Closes #8
Refs #7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Add the exports pages in Blade and Inertia React
026b716 
Blade pages (exports::exports.index and .show) built with javaabu/forms v2
components: a form to start a registered export, a status filter, a table
with status, a progress bar, rows, owner and actions, and pagination. They
extend the app's layout (exports.layout) and load exports::progress through
javaabu/manifest, which follows unfinished rows over Echo or by polling.

Inertia pages (exports::Exports/Index and /Show) in TypeScript with
@javaabu/forms and @javaabu/onu's ProgressBar, registered with manifest's
registerAssets() (pages, @javaabu/exports alias, npm dependencies). They get
their translated text in a strings prop and follow progress with
useExportProgress(). Both stacks use logical utilities only, and a test
rejects physical ones.

Tests: Blade rendering (English and Dhivehi, permissions, flash messages,
required assets), Inertia props, manifest registration, and Vitest for the
hook, the progress component and the Blade script. CI gets a JS job.

Closes #10
Closes #9
Closes #5
Closes #7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Fix the exports table with the latest forms, and the JS CI install
544dfc4 
forms next-gen now honours no-bulk, but its table template then drops the
whole table (Javaabu/forms#17). The index doesn't pass a bulk form, so it
leaves no-bulk off.

Install pnpm packages with the store outside node_modules in CI, so the git
build of @javaabu/forms works (Javaabu/react-forms issue filed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Authenticate Composer with the job token in CI; don't cache Node
ca16f25 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Point the README badges at Javaabu/exports main, for after the merge
d6baa50 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h n4ff4h mentioned this pull request Nov 4, 2026
Open
12 tasks done
n4ff4h and others added 2 commits November 6, 2026 06:27
@n4ff4h @claude
Declare the React pages' npm packages in manifest's pages scope
ab1c8ab 
@javaabu/forms, @javaabu/onu, react, react-dom and @inertiajs/react are only
imported by the Inertia pages and the @javaabu/exports alias; the Blade
progress script imports no npm packages. Moving them into
npm(..., for: NpmScopes::PAGES) (Javaabu/manifest#17) means a Blade-only
app's manifest:check asks exports for nothing but vite and
laravel-vite-plugin.

NpmRequirementsTest runs manifest:check against a Blade-only and an Inertia
package.json fixture. Docs, the Boost skill and AGENTS.md say which packages
each stack needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@n4ff4h @claude
Check only exports' own npm requirements in the Blade-only test
e143624 
javaabu/permissions now registers React pages in the assets scope (it hasn't
adopted Javaabu/manifest#17 yet), so the whole-app check listed react. The
test now judges only the rows exports is needed by.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@n4ff4h