Skip to content

perf(admin): push file list filters, sort and pagination into SQL - #552

Closed
jcbl1 wants to merge 1 commit into
vastsa:masterfrom
jcbl1:upstream/05-admin-sql
Closed

jcbl1 wants to merge 1 commit into
vastsa:masterfrom
jcbl1:upstream/05-admin-sql

Conversation

@jcbl1

@jcbl1 jcbl1 commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

FileService.list_files no longer loads FileCodes.all() (including every text payload) and filters/sorts/paginates in Python. Status, type, health and keyword predicates are translated to SQL, ordering and offset/limit run in SQLite, and only the returned page (<=100 rows) is materialized.

build_file_health_summary now uses SQL counts instead of walking every row; healthy/attention are derived from the danger/warning/success partition.

Equivalence is covered by tests/test_admin_query_pushdown.py, which compares every filter/sort result against the previous per-item Python rules.

10k-row benchmark: list_files 1629ms/29.6MB -> 24.6ms/0.1MB; dashboard health summary 1483ms/29MB -> 13.8ms/<0.1MB.

FileService.list_files no longer loads FileCodes.all() (including every text
payload) and filters/sorts/paginates in Python. Status, type, health and
keyword predicates are translated to SQL, ordering and offset/limit run in
SQLite, and only the returned page (<=100 rows) is materialized.

build_file_health_summary now uses SQL counts instead of walking every row;
healthy/attention are derived from the danger/warning/success partition.

Equivalence is covered by tests/test_admin_query_pushdown.py, which compares
every filter/sort result against the previous per-item Python rules.

10k-row benchmark: list_files 1629ms/29.6MB -> 24.6ms/0.1MB;
dashboard health summary 1483ms/29MB -> 13.8ms/<0.1MB.
@vercel

vercel Bot commented Oct 24, 2026

Copy link
Copy Markdown

@jcbl1 is attempting to deploy a commit to the vastsa's projects Team on Vercel.

A member of the Team first needs to authorize it.

@jcbl1

jcbl1 commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

closed as stale

1 similar comment
@jcbl1

jcbl1 commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

closed as stale

@jcbl1 jcbl1 closed this Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant