Skip to content

Datasets step: show the datasets the region filter hides, instead of dropping them #97

Closed

Description

@mosoriob

Question

Build it. The rule is decided (below). This ticket does the work.

A dataset with no location metadata is invisible in the thread wizard. No control
shows it. At TACC this hides 11 of 33 annotated packages, for every region.

The problem

Four constraints narrow the Datasets step. They disagree on what missing
metadata means.

Constraint Applied Dataset with no such metadata
Standard variable client, after #94 dropped — correct, it declares no variable
Datatype client, MintDatasets.tsx:404 relaxable today — "Show N datasets…" link
Dates client, overlapsDateRange kept — the code says so
Region server, ext_bbox dropped, and silently

Region is the only constraint on the server. It is also the only one where
unknown means invisible.

ext_bbox drops every package with no spatial field. This is not a filter on
location. It is a filter on having a location.

Measured against TACC, 2026-08-09

33 annotated packages.

  • 11 have no spatial extent.
  • 15 have no temporal extent.
  • 7 have neither.
  • 22 have a spatial extent. 16 of those are inside Texas.

The date rule already forgives its 15. They show today. The region rule forgives
nothing.

The decision

Two rules, not one toggle. "No location" and "wrong location" are different
claims.

  1. Unknown extent — show it, and mark it. Badge the row no location or
    no dates. This is the rule dates already follow.
  2. Known extent, outside the region — hide it, and count it. Use the same
    "Show N datasets…" link that the datatype filter already has
    (MintDatasets.tsx:638).
Datasets for "Weather data"

  * Edwards Aquifer Recharge 2020    Texas      2019-2021
  * TWDB GAM model files             ! no location
  * Pumping records                  Texas      ! no dates
  ------------------------------------------------------
  Show 6 datasets outside this region

What to build

Move the region filter to the client. Remove ext_bbox from findDatasets.
#94 already fetches the whole catalog and filters it here, so this costs no extra
request.

Write a real bounding-box extractor first. packageSpatialCoverage reads
coordinates[0] only. It handles a bare Polygon. TACC also holds 5
FeatureCollection, 1 Feature, 1 Point and 1 MultiPolygon. A client-side
filter without this drops those 8 without a word. That is the #94 fault again.

Report the reason, not only the result. findDatasets must tell the UI why a
dataset is out. The UI cannot badge or count what it cannot see.

Scope

Client-only. No change at TACC. No Hasura metadata change.

It does not block a thread run (#92). Texas returns 16 datasets, which is
enough to walk one run.

It does block the flip (#77), by decision: TACC must see a correct Datasets
step the first time they open a thread, not an empty one they have to ask about.

Activity

  1. added
    wayfinder:taskWayfinder ticket: manual work that unblocks a decision
    on Aug 9, 2026
  2. self-assigned this
    on Aug 10, 2026
  3. mosoriob commented on Aug 10, 2026

    @mosoriob
    ContributorAuthor

    Built and verified live against TACC. Two faults, opposite in direction — and the ticket named only one.

    Code in PR #113, 883 tests green.

    The second fault

    ext_bbox hid the datasets with no extent, as measured. But the wizard never passed the region down at all — MintThread rendered DatasetsStep without regionGeometry. So the step showed every annotated dataset in the catalog, Alaska included, under a Texas framing. The two failures are opposite, and one rule had to settle both.

    The rule, as built

    1. Unknown extent — shown and badged ! no location, as the date rule already does.
    2. Known extent, outside — hidden behind a counted link, Show N datasets outside this region.

    findDatasets no longer drops anything for the region. It labels every dataset region_match: inside | outside | unknown and the UI decides. This costs no request: #94 already reads the whole catalog.

    Measured live, annotated packages

    Region Offered of which no location Behind the link Before
    Texas 29 11 4 16
    California (the default framing) 11 11 22 0
    Alaska 15 11 18 —

    California is the one that matters for the map's "reads as an empty deployment" note: it now shows 11 badged rows instead of nothing.

    ext_bbox is not a server-side version of the same test

    The ticket assumed it filtered on location and additionally dropped the no-location rows. It does more than that. ckanext-spatial indexes bare geometries only, so ext_bbox also drops a package whose spatial is a Feature or a FeatureCollection. Two of TACC's annotated packages are exactly that — setx_climate_data and test-sunday-speaker-series-links — and both sit squarely inside Texas. Moving the filter to the client recovers them. That is why Texas goes 16 → 18 in-region, not 16 → 16.

    The extractor

    Written first, as the ticket asked, and it replaces two partial ones rather than one. packageSpatialCoverage read coordinates[0] and knew a bare Polygon. calculateBoundingBox — the region map's — recursed coordinates and geometries but not Feature.geometry or FeatureCollection.features. TACC holds Polygon ×103, FeatureCollection ×5, MultiPolygon, Point and Feature, so 8 packages sat outside the union of the two. One extractor now serves both surfaces (lib/geo/bbox.ts), so the region picker gained the same coverage for free. The Feature/FeatureCollection/Point assertions were checked to fail against both old versions.

    Two things this changed that the ticket did not ask for

    • GetThread now selects region { geometries }. region_id names the region without saying where it is, so there was nothing to filter on. Confirmed readable under the anonymous role at TACC, so it does not depend on a token.
    • The region chip states when a region carries no extent, rather than implying a filter that never ran.

    Two traps found, neither fixed here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

wayfinder:taskWayfinder ticket: manual work that unblocks a decision

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions