Skip to content

refactor(services): use the shared request-logging filter and SQL-state constant - #7796

Open
aglinxinyuan wants to merge 1 commit into
apache:mainfrom
aglinxinyuan:refactor/shared-filter-and-sqlstate
Open

refactor(services): use the shared request-logging filter and SQL-state constant#7796
aglinxinyuan wants to merge 1 commit into
apache:mainfrom
aglinxinyuan:refactor/shared-filter-and-sqlstate

Conversation

@aglinxinyuan

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Two places where a shared helper already existed but a local copy or literal was used instead. 9 insertions, 27 deletions.

1. WorkflowCompilingService now calls the shared RequestLoggingFilter. It was the only Dropwizard-4 service still inlining its own anonymous request-logging filter; the other five (access-control, computing-unit-managing, config, file, notebook-migration) all call RequestLoggingFilter.register(environment.getApplicationContext) at exactly this point.

The two implementations were compared line by line before swapping, not assumed equivalent. Normalising fully-qualified prefixes and the logger's local name, every observable matches: logger name "org.eclipse.jetty.server.RequestLog", the isInfoEnabled guard, the same access-log format string, url pattern "/*", and EnumSet.allOf(classOf[DispatcherType]). The inline copy also sat immediately after RoleAnnotationEnforcer.enforce, which is exactly where the siblings place the register call — so this is a body-only replacement.

2. Four sites now use SqlStates.UNIQUE_VIOLATION instead of the literal "23505".

Site Was
PveResource.scala:110, :148 e.sqlState() == "23505"
DatasetResource.scala:1644 if (e.sqlState() == "23505")
NotebookMigrationResource.scala:331 e.sqlState == "23505"

Each module's dependsOn was checked first: NotebookMigrationService has a direct DAO dependency, and the other two reach it transitively (WorkflowCompiler → WorkflowOperator → WorkflowCore → DAO, and WorkflowCore → DAO). No build change was needed, and no module was given one. The constant was already in use by ExternalAuthProvisioner and LocalAuthProvisioner.

The remaining 23505 occurrences are CSV test fixtures, a yarn.lock checksum, SqlStates.scala itself, and two comments that explain what the code means — those are left alone.

amber's two inline copies are deliberately untouched

Worth recording so they are not "fixed" later: amber's remaining inline filters cannot use the shared one. They are javax.servlet, because amber pins Jetty 9.4.20, while RequestLoggingFilter is jakarta.servlet. Their existing TODO — replace once Dropwizard is upgraded — is accurate, and this PR does not touch them.

Verification

  • WorkflowCompilingService/test: 2 suites, 13 tests, 0 failures. This matters because WorkflowCompilingServiceRunSpec has cases asserting the filter's registration, its dispatch set, and that it forwards the request while logging one access line. They read the filter back out of the captured FilterHolder rather than by class, so they survived the swap unchanged — and they are what confirms the shared filter behaves identically in place.
  • NotebookMigrationResourceSpec: 28 passed. It drives a real unique-constraint violation against embedded Postgres, so it exercises the changed comparison.
  • FileService/compile succeeds. DatasetResourceSpec extends MockLakeFS (testcontainers) and cannot run without Docker, which is unavailable here — for that site the evidence is compilation plus the provable value identity of the constant.
  • PveResourceSpec fails 6 tests both with and without this change (37 succeeded / 6 failed / 1 canceled either way, verified by stashing the PveResource.scala edit and re-running). Pre-existing on this machine, unrelated to this PR.
  • WorkflowCompilingService/scalafmtCheck and Test/scalafmtCheck pass; no imports were left orphaned (FilterHolder removed, RequestLoggingFilter added).

Any related issues, documentation, discussions?

Closes #7792

How was this PR tested?

sbt "WorkflowCompilingService/test" "FileService/compile" "NotebookMigrationService/testOnly org.apache.texera.service.resource.NotebookMigrationResourceSpec"
[info] Suites: completed 2, aborted 0
[info] Tests: succeeded 13, failed 0, canceled 0, ignored 0, pending 0
[info] Tests: succeeded 28, failed 0, canceled 0, ignored 0, pending 0

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 5)

Copilot AI lite review requested due to automatic review settings August 19, 2026 23:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added engine refactor Refactor the code platform Non-amber Scala service paths labels Aug 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @SarahAsad23, @tanishqgandhi1908, @zyratlo
    You can notify them by mentioning @SarahAsad23, @tanishqgandhi1908, @zyratlo in a comment.

@codecov-commenter

codecov-commenter commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 40.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.22%. Comparing base (c927890) to head (c06291b).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...esource/pythonvirtualenvironment/PveResource.scala 0.00% 0 Missing and 2 partials ⚠️
...ache/texera/service/resource/DatasetResource.scala 0.00% 0 Missing and 1 partial ⚠️

❌ Your patch status has failed because the patch coverage (40.00%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #7796      +/-   ##
============================================
- Coverage     91.25%   91.22%   -0.04%     
- Complexity     4471     4472       +1     
============================================
  Files          1171     1171              
  Lines         47124    47160      +36     
  Branches       5286     5304      +18     
============================================
+ Hits          43003    43020      +17     
- Misses         2447     2462      +15     
- Partials       1674     1678       +4     
Flag Coverage Δ *Carryforward flag
access-control-service 81.00% <ø> (ø)
agent-service 98.62% <ø> (ø) Carriedforward from c927890
amber 88.02% <0.00%> (ø)
computing-unit-managing-service 73.67% <ø> (ø)
config-service 86.73% <ø> (ø)
file-service 68.90% <0.00%> (ø)
frontend 92.63% <ø> (ø) Carriedforward from c927890
notebook-migration-service 79.11% <100.00%> (-4.63%) ⬇️
pyamber 97.57% <ø> (ø) Carriedforward from c927890
workflow-compiling-service 72.34% <100.00%> (-4.86%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 4 worse · ⚪ 9 noise (<±5%) · 0 without baseline

Compared against main ee06e4b benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 433 0.264 21,735/30,643/30,643 us 🔴 +10.0% / 🔴 +96.5%
🟢 bs=100 sw=10 sl=64 955 0.583 104,153/120,668/120,668 us 🟢 -5.8% / 🔴 +13.0%
bs=1000 sw=10 sl=64 1,109 0.677 898,265/999,996/999,996 us ⚪ within ±5% / 🟢 -7.5%
Baseline details

Latest main ee06e4b from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 433 tuples/sec 479 tuples/sec 771.02 tuples/sec -9.6% -43.8%
bs=10 sw=10 sl=64 MB/s 0.264 MB/s 0.292 MB/s 0.471 MB/s -9.6% -43.9%
bs=10 sw=10 sl=64 p50 21,735 us 21,743 us 12,749 us -0.0% +70.5%
bs=10 sw=10 sl=64 p95 30,643 us 27,848 us 15,594 us +10.0% +96.5%
bs=10 sw=10 sl=64 p99 30,643 us 27,848 us 19,320 us +10.0% +58.6%
bs=100 sw=10 sl=64 throughput 955 tuples/sec 976 tuples/sec 997.05 tuples/sec -2.2% -4.2%
bs=100 sw=10 sl=64 MB/s 0.583 MB/s 0.596 MB/s 0.609 MB/s -2.2% -4.2%
bs=100 sw=10 sl=64 p50 104,153 us 101,566 us 100,339 us +2.5% +3.8%
bs=100 sw=10 sl=64 p95 120,668 us 128,099 us 106,781 us -5.8% +13.0%
bs=100 sw=10 sl=64 p99 120,668 us 128,099 us 113,101 us -5.8% +6.7%
bs=1000 sw=10 sl=64 throughput 1,109 tuples/sec 1,093 tuples/sec 1,037 tuples/sec +1.5% +6.9%
bs=1000 sw=10 sl=64 MB/s 0.677 MB/s 0.667 MB/s 0.633 MB/s +1.5% +6.9%
bs=1000 sw=10 sl=64 p50 898,265 us 918,148 us 971,388 us -2.2% -7.5%
bs=1000 sw=10 sl=64 p95 999,996 us 958,892 us 1,013,249 us +4.3% -1.3%
bs=1000 sw=10 sl=64 p99 999,996 us 958,892 us 1,039,879 us +4.3% -3.8%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,461.79,200,128000,433,0.264,21735.29,30643.05,30643.05
1,100,10,64,20,2094.06,2000,1280000,955,0.583,104152.51,120667.70,120667.70
2,1000,10,64,20,18034.55,20000,12800000,1109,0.677,898265.13,999995.50,999995.50

@aglinxinyuan
aglinxinyuan requested a review from mengw15 August 20, 2026 05:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine platform Non-amber Scala service paths refactor Refactor the code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use the shared request-logging filter and SQL-state constant instead of local copies

3 participants