Skip to content

feat(amber): report the owner's email in dataset search - #7875

Open
aglinxinyuan wants to merge 1 commit into
apache:mainfrom
aglinxinyuan:fix/dataset-search-owner-email
Open

feat(amber): report the owner's email in dataset search#7875
aglinxinyuan wants to merge 1 commit into
apache:mainfrom
aglinxinyuan:fix/dataset-search-owner-email

Conversation

@aglinxinyuan

@aglinxinyuan aglinxinyuan commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Why it was never populated. DatasetSearchQueryBuilder joins the owner row and then never projects a column from it. constructFromClause writes the join, but the select list comes entirely from mappedResourceSchema, and UnifiedResourceSchema.apply defaults userEmail to DSL.inline("") (UnifiedResourceSchema:74) — a slot the dataset schema never named. So toEntryImpl reads owner.getEmail out of a record that has no owner in it.

Two steps turn the missing projection into a null:

schema slot  userEmail = DSL.inline("")        <- default, never overridden
                 |
SQL          '' as email                       <- USER is joined, never read
                 |
translateRecord  dedupes by ORIGINAL field, and jOOQ compares fields by
                 rendered SQL, so projectsOfWorkflow / userName / userEmail /
                 projectColor -- all DSL.inline("") -- collapse to ONE entry
                 keyed on the first of them
                 |
record       no `email` column at all  ->  record.into(USER) = empty User
                 |
entry        DashboardDataset.ownerEmail = null      (not "")

Change. Name the slot. One line in the schema; toEntryImpl is untouched.

ownerId = DATASET.OWNER_UID,
userEmail = USER.EMAIL,
did = DATASET.DID,

Before → after, on the rendered projection and the entry:

before after
projection '' as email texera_db.user.email as email
DashboardDataset.ownerEmail null, every row the dataset owner's address
leftJoin(USER) joined, selected from, never read read

WorkflowSearchQueryBuilder is the sibling that shows the step that was missed: it opts into the USER column it reads (userName = USER.NAME, plus USER.NAME in getGroupByFields), and it already filters on USER.EMAIL for the owners query param — the email was reachable through the join all along, only the projection was absent. HubResource:316 and file-service's /dataset/list both populate the same field correctly; dataset search was the one producer that did not.

Scope of the impact, stated precisely because it is narrower than it looks: DashboardEntry.ownerEmail (dashboard-entry.ts:132) receives the null, and no frontend code reads that field for datasets today, so no screen was visibly wrong. It was a trap rather than a broken page — dataset-selection-modal.component.ts:127 builds the storage logical path /${ResourceType.Dataset}/${ownerEmail}/${name}/${version} out of a DashboardDataset, and only escapes /dataset/null/... because it lists via file-service.

Any related issues, documentation, discussions?

Closes #7874

How was this PR tested?

DatasetSearchQueryBuilderSpec goes from 18 tests to 21, reusing the fixture and lakeFS loopback stub #7855 built.

test what it pins
carry the owner's email address the value on the entry — the assertion that kills the mutant below
take the email from the dataset's owner, not from the caller fetched as otherUid, who reaches the dataset only because it is public, so a lookup that echoed the signed-in caller back would fail
project every dataset column under the alias its schema slot names (extended) user.email as email in the SELECT, so a slot dropped back to a literal fails at the projection instead of as a null three layers down
stay union-compatible with the workflow and project branches new — see below

The mutation #7855 recorded as surviving now dies. Replacing record.into(USER).into(classOf[User]) with a fresh User:

before this PR after
new User mutant survives the whole suite (equivalent mutant, given the defect) succeeded 19, failed 2 — both owner-email tests

Verified in order, one sbt JVM each:

run result
new tests against the builder as it stands on main succeeded 18, failed 3, canceled 0 — projection missing user.email, plus null was not equal to "dataset_search_owner@mail.com" twice
with the change succeeded 21, failed 0, canceled 0
change + new User mutant succeeded 19, failed 2, canceled 0
the 7 dashboard suites together Suites: completed 7, Tests: succeeded 108, failed 0, canceled 0
scalafmtCheckAll + Test/scalafixAll --check clean
sbt "WorkflowExecutionService/testOnly org.apache.texera.web.resource.dashboard.DatasetSearchQueryBuilderSpec"

Two notes for a reviewer:

The union test earns its place. This change swaps a column in one branch of the three-way unionAll that DashboardResource.searchAllResources builds for the dashboard's default view, from an untyped '' literal to varchar. Nothing in the repo executed that union, so a type incompatibility would have shipped as a runtime failure on the default view with nothing failing to compile. The new test executes it.

canceled is the failure mode to watch, and it moved. The stub binds the configured port (localhost:8000) rather than an ephemeral one, because LakeFSStorageClient.apiClient is a lazy val capturing StorageConfig.lakefsEndpoint once per JVM and amber runs every suite in one unforked JVM. If something else holds that port — a local bin/local-dev.sh up — the stub-dependent tests cancel, and that count goes from 4 to 6 with the owner-email pair added. A cancel is quiet: sbt prints All tests passed at exit 0. Measured here, with the port held and size mutated to 0L, the run reports succeeded 15, failed 0, canceled 6 and still exits green — so the owner-email pair is disarmed alongside the rest of the toEntry half. The class comment carries these numbers and I updated them. CI has no lakeFS in this job, so it is deterministic there.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (claude-opus-5)

Copilot AI lite review requested due to automatic review settings August 23, 2026 05:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Yicong-Huang Yicong-Huang added the release/v1.2 back porting to release/v1.2 label Aug 23, 2026
@github-actions
github-actions Bot requested a review from xuang7 August 23, 2026 05:34
@github-actions

Copy link
Copy Markdown
Contributor

Backport auto-label report

This fix: PR was checked against each actively-supported release branch. release/* labels drive the post-merge backport, so add or remove one to change where this fix lands.

Release branch Analysis
release/v1.2 Change detected on this branch — label added; this fix is queued to backport here. Requested review from @xuang7.

Auto-label run.

@github-actions github-actions Bot added the fix label Aug 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @Mrudhulraj
    You can notify them by mentioning @Mrudhulraj in a comment.

@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 5 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main 649a27d benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 626 0.382 15,612/22,013/22,013 us 🟢 -10.5% / 🔴 +53.1%
🔴 bs=100 sw=10 sl=64 1,324 0.808 74,271/103,562/103,562 us 🔴 +9.3% / 🟢 +22.2%
bs=1000 sw=10 sl=64 1,520 0.928 660,235/697,220/697,220 us ⚪ within ±5% / 🟢 +35.9%
Baseline details

Latest main 649a27d from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 626 tuples/sec 671 tuples/sec 833.79 tuples/sec -6.7% -24.9%
bs=10 sw=10 sl=64 MB/s 0.382 MB/s 0.409 MB/s 0.509 MB/s -6.6% -24.9%
bs=10 sw=10 sl=64 p50 15,612 us 14,676 us 11,864 us +6.4% +31.6%
bs=10 sw=10 sl=64 p95 22,013 us 24,605 us 14,381 us -10.5% +53.1%
bs=10 sw=10 sl=64 p99 22,013 us 24,605 us 18,035 us -10.5% +22.1%
bs=100 sw=10 sl=64 throughput 1,324 tuples/sec 1,336 tuples/sec 1,083 tuples/sec -0.9% +22.2%
bs=100 sw=10 sl=64 MB/s 0.808 MB/s 0.815 MB/s 0.661 MB/s -0.9% +22.2%
bs=100 sw=10 sl=64 p50 74,271 us 74,865 us 93,077 us -0.8% -20.2%
bs=100 sw=10 sl=64 p95 103,562 us 94,779 us 99,553 us +9.3% +4.0%
bs=100 sw=10 sl=64 p99 103,562 us 94,779 us 108,604 us +9.3% -4.6%
bs=1000 sw=10 sl=64 throughput 1,520 tuples/sec 1,532 tuples/sec 1,119 tuples/sec -0.8% +35.9%
bs=1000 sw=10 sl=64 MB/s 0.928 MB/s 0.935 MB/s 0.683 MB/s -0.7% +35.9%
bs=1000 sw=10 sl=64 p50 660,235 us 651,860 us 909,247 us +1.3% -27.4%
bs=1000 sw=10 sl=64 p95 697,220 us 681,929 us 952,561 us +2.2% -26.8%
bs=1000 sw=10 sl=64 p99 697,220 us 681,929 us 985,186 us +2.2% -29.2%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,319.57,200,128000,626,0.382,15612.12,22013.36,22013.36
1,100,10,64,20,1510.01,2000,1280000,1324,0.808,74270.98,103561.89,103561.89
2,1000,10,64,20,13153.66,20000,12800000,1520,0.928,660235.20,697219.54,697219.54

@aglinxinyuan aglinxinyuan added backend Anything related to backend services and removed fix release/v1.2 back porting to release/v1.2 engine labels Aug 23, 2026
@aglinxinyuan
aglinxinyuan requested review from mengw15 and removed request for xuang7 August 23, 2026 06:47
`DatasetSearchQueryBuilder` joins USER on the dataset's owner but never
projected a USER column: `UnifiedResourceSchema` defaults `userEmail` to
`DSL.inline("")` and the dataset schema left the slot alone, so the query
rendered `'' as email`. `translateRecord` then dedupes the four
`inline("")` defaults into one field, so the translated record carried no
`email` at all, `record.into(USER)` yielded an empty POJO, and
`DashboardDataset.ownerEmail` was null on every row — the owner email has
never actually been reported by dataset search.

Name the slot `userEmail = USER.EMAIL`, matching the workflow sibling,
which already opts into the USER column it reads.

The suite gains the owner-email pair that kills the
`record.into(USER).into(classOf[User])` -> `new User` mutant apache#7855
recorded as surviving, plus a projection assertion and the first test to
execute the three-way `unionAll` the dashboard's default view builds.
@aglinxinyuan
aglinxinyuan force-pushed the fix/dataset-search-owner-email branch from c1b76f5 to efeb472 Compare August 23, 2026 06:48
@aglinxinyuan aglinxinyuan changed the title fix(amber): dataset search reports a null owner email feat(amber): report the owner's email in dataset search Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend Anything related to backend services engine fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dataset search leaves the owner email null on every entry

3 participants