feat(amber): report the owner's email in dataset search - #7875
Open
aglinxinyuan wants to merge 1 commit into
Open
feat(amber): report the owner's email in dataset search#7875aglinxinyuan wants to merge 1 commit into
aglinxinyuan wants to merge 1 commit into
Conversation
Contributor
Backport auto-label reportThis
|
Contributor
Automated Reviewer SuggestionsBased on the
|
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 626 | 0.382 | 15,612/22,013/22,013 us | 🟢 -10.5% / 🔴 +53.1% |
| 🔴 | bs=100 sw=10 sl=64 | 1,324 | 0.808 | 74,271/103,562/103,562 us | 🔴 +9.3% / 🟢 +22.2% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,520 | 0.928 | 660,235/697,220/697,220 us | ⚪ within ±5% / 🟢 +35.9% |
Baseline details
Latest main 649a27d from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 626 tuples/sec | 671 tuples/sec | 833.79 tuples/sec | -6.7% | -24.9% |
| bs=10 sw=10 sl=64 | MB/s | 0.382 MB/s | 0.409 MB/s | 0.509 MB/s | -6.6% | -24.9% |
| bs=10 sw=10 sl=64 | p50 | 15,612 us | 14,676 us | 11,864 us | +6.4% | +31.6% |
| bs=10 sw=10 sl=64 | p95 | 22,013 us | 24,605 us | 14,381 us | -10.5% | +53.1% |
| bs=10 sw=10 sl=64 | p99 | 22,013 us | 24,605 us | 18,035 us | -10.5% | +22.1% |
| bs=100 sw=10 sl=64 | throughput | 1,324 tuples/sec | 1,336 tuples/sec | 1,083 tuples/sec | -0.9% | +22.2% |
| bs=100 sw=10 sl=64 | MB/s | 0.808 MB/s | 0.815 MB/s | 0.661 MB/s | -0.9% | +22.2% |
| bs=100 sw=10 sl=64 | p50 | 74,271 us | 74,865 us | 93,077 us | -0.8% | -20.2% |
| bs=100 sw=10 sl=64 | p95 | 103,562 us | 94,779 us | 99,553 us | +9.3% | +4.0% |
| bs=100 sw=10 sl=64 | p99 | 103,562 us | 94,779 us | 108,604 us | +9.3% | -4.6% |
| bs=1000 sw=10 sl=64 | throughput | 1,520 tuples/sec | 1,532 tuples/sec | 1,119 tuples/sec | -0.8% | +35.9% |
| bs=1000 sw=10 sl=64 | MB/s | 0.928 MB/s | 0.935 MB/s | 0.683 MB/s | -0.7% | +35.9% |
| bs=1000 sw=10 sl=64 | p50 | 660,235 us | 651,860 us | 909,247 us | +1.3% | -27.4% |
| bs=1000 sw=10 sl=64 | p95 | 697,220 us | 681,929 us | 952,561 us | +2.2% | -26.8% |
| bs=1000 sw=10 sl=64 | p99 | 697,220 us | 681,929 us | 985,186 us | +2.2% | -29.2% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,319.57,200,128000,626,0.382,15612.12,22013.36,22013.36
1,100,10,64,20,1510.01,2000,1280000,1324,0.808,74270.98,103561.89,103561.89
2,1000,10,64,20,13153.66,20000,12800000,1520,0.928,660235.20,697219.54,697219.54`DatasetSearchQueryBuilder` joins USER on the dataset's owner but never
projected a USER column: `UnifiedResourceSchema` defaults `userEmail` to
`DSL.inline("")` and the dataset schema left the slot alone, so the query
rendered `'' as email`. `translateRecord` then dedupes the four
`inline("")` defaults into one field, so the translated record carried no
`email` at all, `record.into(USER)` yielded an empty POJO, and
`DashboardDataset.ownerEmail` was null on every row — the owner email has
never actually been reported by dataset search.
Name the slot `userEmail = USER.EMAIL`, matching the workflow sibling,
which already opts into the USER column it reads.
The suite gains the owner-email pair that kills the
`record.into(USER).into(classOf[User])` -> `new User` mutant apache#7855
recorded as surviving, plus a projection assertion and the first test to
execute the three-way `unionAll` the dashboard's default view builds.
aglinxinyuan
force-pushed
the
fix/dataset-search-owner-email
branch
from
August 23, 2026 06:48
c1b76f5 to
efeb472
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
Why it was never populated.
DatasetSearchQueryBuilderjoins the owner row and then never projects a column from it.constructFromClausewrites the join, but the select list comes entirely frommappedResourceSchema, andUnifiedResourceSchema.applydefaultsuserEmailtoDSL.inline("")(UnifiedResourceSchema:74) — a slot the dataset schema never named. SotoEntryImplreadsowner.getEmailout of a record that has no owner in it.Two steps turn the missing projection into a
null:Change. Name the slot. One line in the schema;
toEntryImplis untouched.Before → after, on the rendered projection and the entry:
'' as emailtexera_db.user.email as emailDashboardDataset.ownerEmailnull, every rowleftJoin(USER)WorkflowSearchQueryBuilderis the sibling that shows the step that was missed: it opts into the USER column it reads (userName = USER.NAME, plusUSER.NAMEingetGroupByFields), and it already filters onUSER.EMAILfor theownersquery param — the email was reachable through the join all along, only the projection was absent.HubResource:316and file-service's/dataset/listboth populate the same field correctly; dataset search was the one producer that did not.Scope of the impact, stated precisely because it is narrower than it looks:
DashboardEntry.ownerEmail(dashboard-entry.ts:132) receives the null, and no frontend code reads that field for datasets today, so no screen was visibly wrong. It was a trap rather than a broken page —dataset-selection-modal.component.ts:127builds the storage logical path/${ResourceType.Dataset}/${ownerEmail}/${name}/${version}out of aDashboardDataset, and only escapes/dataset/null/...because it lists via file-service.Any related issues, documentation, discussions?
Closes #7874
How was this PR tested?
DatasetSearchQueryBuilderSpecgoes from 18 tests to 21, reusing the fixture and lakeFS loopback stub #7855 built.carry the owner's email addresstake the email from the dataset's owner, not from the callerotherUid, who reaches the dataset only because it is public, so a lookup that echoed the signed-in caller back would failproject every dataset column under the alias its schema slot names(extended)user.email as emailin the SELECT, so a slot dropped back to a literal fails at the projection instead of as a null three layers downstay union-compatible with the workflow and project branchesThe mutation #7855 recorded as surviving now dies. Replacing
record.into(USER).into(classOf[User])with a freshUser:new Usermutantsucceeded 19, failed 2— both owner-email testsVerified in order, one sbt JVM each:
succeeded 18, failed 3, canceled 0— projection missinguser.email, plusnull was not equal to "dataset_search_owner@mail.com"twicesucceeded 21, failed 0, canceled 0new Usermutantsucceeded 19, failed 2, canceled 0Suites: completed 7, Tests: succeeded 108, failed 0, canceled 0scalafmtCheckAll+Test/scalafixAll --checksbt "WorkflowExecutionService/testOnly org.apache.texera.web.resource.dashboard.DatasetSearchQueryBuilderSpec"Two notes for a reviewer:
The union test earns its place. This change swaps a column in one branch of the three-way
unionAllthatDashboardResource.searchAllResourcesbuilds for the dashboard's default view, from an untyped''literal tovarchar. Nothing in the repo executed that union, so a type incompatibility would have shipped as a runtime failure on the default view with nothing failing to compile. The new test executes it.canceledis the failure mode to watch, and it moved. The stub binds the configured port (localhost:8000) rather than an ephemeral one, becauseLakeFSStorageClient.apiClientis alazy valcapturingStorageConfig.lakefsEndpointonce per JVM and amber runs every suite in one unforked JVM. If something else holds that port — a localbin/local-dev.sh up— the stub-dependent tests cancel, and that count goes from 4 to 6 with the owner-email pair added. A cancel is quiet: sbt printsAll tests passedat exit 0. Measured here, with the port held andsizemutated to0L, the run reportssucceeded 15, failed 0, canceled 6and still exits green — so the owner-email pair is disarmed alongside the rest of thetoEntryhalf. The class comment carries these numbers and I updated them. CI has no lakeFS in this job, so it is deterministic there.Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-opus-5)