refactor(services): use the shared request-logging filter and SQL-state constant - #7796
refactor(services): use the shared request-logging filter and SQL-state constant#7796aglinxinyuan wants to merge 1 commit into
Conversation
Automated Reviewer SuggestionsBased on the
|
Codecov Report❌ Patch coverage is ❌ Your patch status has failed because the patch coverage (40.00%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #7796 +/- ##
============================================
- Coverage 91.25% 91.22% -0.04%
- Complexity 4471 4472 +1
============================================
Files 1171 1171
Lines 47124 47160 +36
Branches 5286 5304 +18
============================================
+ Hits 43003 43020 +17
- Misses 2447 2462 +15
- Partials 1674 1678 +4
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 433 | 0.264 | 21,735/30,643/30,643 us | 🔴 +10.0% / 🔴 +96.5% |
| 🟢 | bs=100 sw=10 sl=64 | 955 | 0.583 | 104,153/120,668/120,668 us | 🟢 -5.8% / 🔴 +13.0% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,109 | 0.677 | 898,265/999,996/999,996 us | ⚪ within ±5% / 🟢 -7.5% |
Baseline details
Latest main ee06e4b from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 433 tuples/sec | 479 tuples/sec | 771.02 tuples/sec | -9.6% | -43.8% |
| bs=10 sw=10 sl=64 | MB/s | 0.264 MB/s | 0.292 MB/s | 0.471 MB/s | -9.6% | -43.9% |
| bs=10 sw=10 sl=64 | p50 | 21,735 us | 21,743 us | 12,749 us | -0.0% | +70.5% |
| bs=10 sw=10 sl=64 | p95 | 30,643 us | 27,848 us | 15,594 us | +10.0% | +96.5% |
| bs=10 sw=10 sl=64 | p99 | 30,643 us | 27,848 us | 19,320 us | +10.0% | +58.6% |
| bs=100 sw=10 sl=64 | throughput | 955 tuples/sec | 976 tuples/sec | 997.05 tuples/sec | -2.2% | -4.2% |
| bs=100 sw=10 sl=64 | MB/s | 0.583 MB/s | 0.596 MB/s | 0.609 MB/s | -2.2% | -4.2% |
| bs=100 sw=10 sl=64 | p50 | 104,153 us | 101,566 us | 100,339 us | +2.5% | +3.8% |
| bs=100 sw=10 sl=64 | p95 | 120,668 us | 128,099 us | 106,781 us | -5.8% | +13.0% |
| bs=100 sw=10 sl=64 | p99 | 120,668 us | 128,099 us | 113,101 us | -5.8% | +6.7% |
| bs=1000 sw=10 sl=64 | throughput | 1,109 tuples/sec | 1,093 tuples/sec | 1,037 tuples/sec | +1.5% | +6.9% |
| bs=1000 sw=10 sl=64 | MB/s | 0.677 MB/s | 0.667 MB/s | 0.633 MB/s | +1.5% | +6.9% |
| bs=1000 sw=10 sl=64 | p50 | 898,265 us | 918,148 us | 971,388 us | -2.2% | -7.5% |
| bs=1000 sw=10 sl=64 | p95 | 999,996 us | 958,892 us | 1,013,249 us | +4.3% | -1.3% |
| bs=1000 sw=10 sl=64 | p99 | 999,996 us | 958,892 us | 1,039,879 us | +4.3% | -3.8% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,461.79,200,128000,433,0.264,21735.29,30643.05,30643.05
1,100,10,64,20,2094.06,2000,1280000,955,0.583,104152.51,120667.70,120667.70
2,1000,10,64,20,18034.55,20000,12800000,1109,0.677,898265.13,999995.50,999995.50
What changes were proposed in this PR?
Two places where a shared helper already existed but a local copy or literal was used instead. 9 insertions, 27 deletions.
1.
WorkflowCompilingServicenow calls the sharedRequestLoggingFilter. It was the only Dropwizard-4 service still inlining its own anonymous request-logging filter; the other five (access-control, computing-unit-managing, config, file, notebook-migration) all callRequestLoggingFilter.register(environment.getApplicationContext)at exactly this point.The two implementations were compared line by line before swapping, not assumed equivalent. Normalising fully-qualified prefixes and the logger's local name, every observable matches: logger name
"org.eclipse.jetty.server.RequestLog", theisInfoEnabledguard, the same access-log format string, url pattern"/*", andEnumSet.allOf(classOf[DispatcherType]). The inline copy also sat immediately afterRoleAnnotationEnforcer.enforce, which is exactly where the siblings place theregistercall — so this is a body-only replacement.2. Four sites now use
SqlStates.UNIQUE_VIOLATIONinstead of the literal"23505".PveResource.scala:110,:148e.sqlState() == "23505"DatasetResource.scala:1644if (e.sqlState() == "23505")NotebookMigrationResource.scala:331e.sqlState == "23505"Each module's
dependsOnwas checked first:NotebookMigrationServicehas a direct DAO dependency, and the other two reach it transitively (WorkflowCompiler → WorkflowOperator → WorkflowCore → DAO, andWorkflowCore → DAO). No build change was needed, and no module was given one. The constant was already in use byExternalAuthProvisionerandLocalAuthProvisioner.The remaining
23505occurrences are CSV test fixtures, ayarn.lockchecksum,SqlStates.scalaitself, and two comments that explain what the code means — those are left alone.amber's two inline copies are deliberately untouched
Worth recording so they are not "fixed" later: amber's remaining inline filters cannot use the shared one. They are
javax.servlet, because amber pins Jetty 9.4.20, whileRequestLoggingFilterisjakarta.servlet. Their existing TODO — replace once Dropwizard is upgraded — is accurate, and this PR does not touch them.Verification
WorkflowCompilingService/test: 2 suites, 13 tests, 0 failures. This matters becauseWorkflowCompilingServiceRunSpechas cases asserting the filter's registration, its dispatch set, and that it forwards the request while logging one access line. They read the filter back out of the capturedFilterHolderrather than by class, so they survived the swap unchanged — and they are what confirms the shared filter behaves identically in place.NotebookMigrationResourceSpec: 28 passed. It drives a real unique-constraint violation against embedded Postgres, so it exercises the changed comparison.FileService/compilesucceeds.DatasetResourceSpecextendsMockLakeFS(testcontainers) and cannot run without Docker, which is unavailable here — for that site the evidence is compilation plus the provable value identity of the constant.PveResourceSpecfails 6 tests both with and without this change (37 succeeded / 6 failed / 1 canceled either way, verified by stashing thePveResource.scalaedit and re-running). Pre-existing on this machine, unrelated to this PR.WorkflowCompilingService/scalafmtCheckandTest/scalafmtCheckpass; no imports were left orphaned (FilterHolderremoved,RequestLoggingFilteradded).Any related issues, documentation, discussions?
Closes #7792
How was this PR tested?
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 5)