Skip to content

fix(dataset): derive virtual dataset schema from SQL Lab query - #43313

Draft
sadpandajoe wants to merge 1 commit into
masterfrom
fix-sqllab-virtual-dataset-schema-mismatch
Draft

fix(dataset): derive virtual dataset schema from SQL Lab query#43313
sadpandajoe wants to merge 1 commit into
masterfrom
fix-sqllab-virtual-dataset-schema-mismatch

Conversation

@sadpandajoe

Copy link
Copy Markdown
Member

SUMMARY

When a virtual dataset is created from SQL Lab's "Save & Explore" action, the saved
dataset's schema is always set to whatever schema is selected in the SQL Lab
dropdown — even when the query itself explicitly references a different schema. For
example, selecting information_schema in the dropdown and then running
select * from public."Vehicle Sales" creates a dataset with schema=information_schema,
not schema=public. This has been reported since 2021 (#16791) and affects dataset
metadata correctness, RLS resolution, and permission checks that read the dataset's
stored schema.

This PR derives the schema from the query itself when it is unambiguous, and falls
back to today's dropdown-based behavior in every other case:

  • The query is parsed with the existing Jinja-aware SQL parser (process_jinja_sql,
    the same one already used by the access-control check on this path) — no new SQL
    parsing is introduced.
  • The dropdown schema is replaced with the query's schema only when the query
    references exactly one (catalog, schema) location, every referenced table is
    explicitly schema-qualified (zero unqualified references), the statement doesn't
    mutate data, doesn't change the default schema mid-script, doesn't contain a
    metadata (SHOW ...) statement, and doesn't reference any quoted catalog/schema
    identifier (conservative, since quoting can carry case-sensitive identity on some
    engines).
  • Any other case — ambiguous, multi-schema, or unparseable queries — keeps today's
    behavior unchanged. This is an intentional, documented limitation, not an oversight.
  • Any failure in the optional parsing/derivation step (including malformed
    template_params or a Jinja rendering error) safely falls back to the submitted
    schema rather than blocking dataset creation.

Out of scope for this PR (see RCA.md for details): the broader multi-schema dataset
model proposed in community discussion, and permission/RLS enforcement based on the
derived schema — both would be separate, larger changes.

RCA.md in this PR documents the full investigation: root cause, why it wasn't
caught, the exact safety conditions and why each is needed, a verified matrix of
parser behavior across ~20 query shapes, and known limitations of this fix.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Backend-only change (dataset creation logic); no UI change. Behavior before/after for
the reported repro:

  • Before: dropdown schema information_schema + query select * from public."Vehicle Sales" → saved dataset schema = information_schema (wrong)
  • After: same inputs → saved dataset schema = public (matches the query)

TESTING INSTRUCTIONS

  1. pytest tests/unit_tests/commands/dataset/test_create.py — covers the primary
    repro case plus every fallback condition (zero-schema, multi-schema, unqualified
    reference, mutating statement, metadata statement, quoted identifier, unparseable/
    template-error SQL, Jinja partition-macro tables, template-parameter-driven schema
    selection, and positive catalog+schema derivation).
  2. Manual repro: in SQL Lab, select a database and a schema in the dropdowns, run a
    query that explicitly references a different schema (e.g.
    select * from <other_schema>.<table>), click Explore → name the dataset → Save &
    Explore. Confirm the saved dataset's schema matches the query, not the dropdown.

ADDITIONAL INFORMATION

Derive an unambiguous catalog and schema from the Jinja-aware parsed query while preserving submitted defaults for ambiguous, mutating, metadata, or unparseable SQL. Include template parameters and conservative guards for quoted identifiers and incomplete parser results.

Addresses #16791.
@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.09%. Comparing base (34cd50c) to head (a322256).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
superset/commands/dataset/create.py 36.36% 13 Missing and 8 partials ⚠️
superset/sql/parse.py 76.47% 3 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #43313      +/-   ##
==========================================
- Coverage   66.67%   66.09%   -0.58%     
==========================================
  Files        2876     2876              
  Lines      164007   164056      +49     
  Branches    37834    37848      +14     
==========================================
- Hits       109347   108436     -911     
- Misses      52514    53464     +950     
- Partials     2146     2156      +10     
Flag Coverage Δ
hive 38.09% <16.00%> (-0.02%) ⬇️
mysql 57.75% <50.00%> (-0.01%) ⬇️
postgres 57.78% <50.00%> (-0.01%) ⬇️
presto 40.02% <16.00%> (-0.02%) ⬇️
python 58.02% <50.00%> (-1.16%) ⬇️
sqlite 57.42% <50.00%> (-0.01%) ⬇️
unit ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[SQL LAB]Virtual dataset created from SQLab with incorrect schema

1 participant