diff --git a/docs/design/2026-11-20-flow-setting-schemas.md b/docs/design/2026-11-20-flow-setting-schemas.md new file mode 100644 index 00000000..443aea06 --- /dev/null +++ b/docs/design/2026-11-20-flow-setting-schemas.md @@ -0,0 +1,390 @@ +# Flow settings with choices, explanations and author defaults + +## Status + +Approved (2026-11-20). Implementing. + +Checked against `origin/develop` at the time of writing, and against the dev catalog +(`mintdevgraphql`) with read-only queries. Paths are relative to the monorepo root unless they +name the `Semantic-Sensemaking` library. Items marked **[unverified]** were not confirmed in code +or data. + +## Objective + +Make a flow's settings as clear to configure as a Tapis app's `app.json` arguments. Each setting +gets a readable title, an explanation, a type, a default, and, when it has a fixed set of +values, a dropdown where every choice carries its own explanation. The flow's author, a data +scientist, sets the flow's own defaults. The author can also fix a value so runners cannot +change it, and keep rarely changed settings out of the way. The mechanism must work for any +piece and any flow, not only Semantic Sensemaking. + +## User need + +- **Primary user:** a domain scientist running a flow from a thread. Knows the subject, does not + know the pieces' code. +- **Secondary user:** the flow's author, a data scientist, who builds the flow from pieces and + decides its defaults. +- **Current pain, seen on dev:** + - The thread's settings form shows every flow setting as a free-text box labelled with the raw + name, for example "match mode". + - The choices are hidden in a sentence ("How terms are counted: auto, word_boundary or + longest_match.") and are checked only after the user types a value. + - No setting on the three Sensemaking flows has a default. Every parameter row has + `has_default_value` and `has_accepted_values` set to null. + - The meaning of each choice is written nowhere. +- **Definition of success:** + - A runner opening a Sensemaking flow sees readable titles, and dropdowns for `match_mode`, + `mapping_method`, `unmapped_groups`, `embedding_kind`, `chat_kind` and `grounding`, each + choice explained. + - Each setting is filled with the flow's default. + - Settings marked advanced are hidden behind an "Advanced settings" toggle. + - An author can change a default or fix a value in the flow builder without code. + +## Current code/system summary + +- **Pieces.** + - A piece (`modelcatalog_etl_process`) describes its settings as JSON Schema in + `runtime_json.parameters_schema_json`. + - On dev each property has only `type` and `description`. For example, map-and-project has + `match_mode`, `output_uri`, `taxonomy_uri`, `mapping_method` and `unmapped_groups`. + - The schema is produced by `Semantic-Sensemaking/src/semantic_sensemaking/pieces/flow_catalog.py` + from a `{name: description}` map (`SCORING`, `EMBEDDING`, `CHAT`, and per-piece arguments). + - A piece is never changed in place. A changed declaration is registered under a new id + (`REVISION` in `flow_catalog.py`). +- **The adapter's piece summary.** `_piece_summary` in `svo-adapter-service/app/main.py` (around + line 2786) passes `type`, `default` and `description` of each setting to the UI as + `setting_details`. It drops every other schema key. +- **The flow builder.** + - `ui-react/src/components/flows/StepCard.tsx` lists a step's settings as "the flow's + defaults; a runner can change them", one text box each, with the piece default as the + placeholder. + - Values are saved as the step's `settings_json` (`ui-react/src/lib/flow-authoring.ts`). + - Each distinct setting name becomes one flow parameter row named + `{configurationId}_setting_{name}` (`docs/design/2026-11-20-flow-authoring-in-the-ui.md`). +- **Catalog parameter rows.** `modelcatalog_parameter` has `has_data_type`, `has_default_value`, + `has_fixed_value`, `has_accepted_values` (a string list), `has_minimum_accepted_value` and + `has_maximum_accepted_value`. It has no field for a title separate from `label`, which is the + key a run binds to, and none for per-choice explanations. +- **The run form** (`ui-react/src/pages/modeling/thread/MintParameters.tsx`): + - It renders text inputs only. + - `has_accepted_values` is used only to reject a value ("Accepted values: …"). + - Min, max and default show as a grey hint. + - Parameters with a fixed value appear in a read-only table, "Expert modeler has selected the + following parameters". So locking already has a display. + - It has two parameter tables: catalog parameters, and adapter-plan parameters. **[unverified]** + which of the two renders a flow's settings in every case. The implementation must confirm + this and make both read the same setting description. + +## Proposed design + +### 1. A setting description in the piece's schema (no new catalog column) + +Each property in `parameters_schema_json` may carry these keys. All of them are standard JSON +Schema except `x-advanced`. + +| Key | Meaning | `app.json` equivalent | +|---|---|---| +| `title` | readable name shown instead of the key | `notes.label` | +| `description` | what the setting does | `description` | +| `type` | `string`, `integer`, `number`, `boolean` | `inputMode`/type | +| `default` | the piece's default | `arg` default | +| `oneOf: [{const, title, description}]` | the choices, each explained | `notes.enum_values` | +| `minimum`, `maximum` | numeric bounds | — | +| `x-advanced: true` | hidden behind "Advanced settings" unless set | — | + +A plain `enum` without `oneOf` stays valid. It renders as a dropdown without per-choice +explanations. + +### 2. The library describes its settings properly + +`flow_catalog.py` replaces the prose maps with structured specs. The titles, defaults, choices +and choice meanings are taken from the library code, never guessed. Each must be traced to the +function that reads the argument (for example, how `match_mode` is applied in `score_lexicon`). +The pieces are registered again as new revisions, because their declaration changes. + +### 3. The adapter passes the description through + +`_piece_summary` keeps `title`, `oneOf`, `enum`, `minimum`, `maximum` and `x-advanced` in +`setting_details`, alongside the keys it keeps today. The adapter's flow-plan parameter list, +which the run form reads, carries the same keys. **[unverified]** where that list is built; the +implementation finds it first. + +### 4. The flow's author sets defaults, fixes values, and chooses what runners see + +Each setting in the flow builder's step card renders from the description: a dropdown with +explanations, or a number box with bounds. For each setting the author gives a **value** and a +**visibility**. + +The value is one of: + +- **Default:** the runner sees this value and can change it. It is saved in the step's + `settings_json` (as today) and copied to the flow parameter row's `has_default_value`. +- **Fixed:** the runner cannot change it. It is copied to `has_fixed_value`. + +The visibility is one of: + +- **Shown:** in the main settings list. +- **Advanced:** inside "Advanced settings". The piece's `x-advanced` gives the initial choice. +- **Hidden:** not shown to the runner at all. The run uses the fixed value, else the author's + default, else the piece's default. A hidden setting with none of these, and required by the + piece, is refused by the flow check. + +Visibility is stored per flow in a new column, `modelcatalog_configuration_flow.settings_json` +(jsonb, default `{}`), as `{"": {"visibility": "shown" | "advanced" | "hidden"}}`. +It sits on the flow, not on a step, because one setting name used by several steps is one +flow parameter. `parameter_type` is not reused: it already holds `adjustment`, `model_param`, +`secret` and `standard`. + +When a run starts, a setting takes the first value found in this order: + +1. a fixed value from the flow's author; +2. the runner's value (refused for a fixed or hidden setting); +3. the flow author's default; +4. the piece's default. + +The refusal in step 2 is enforced by the adapter's flow-run check, not only by the form. +Otherwise fixed and hidden settings could still be overridden through the API. The adapter +already rejects a run value for a setting wired from an earlier step, and that rule stays. + +### 5. The run form renders the description + +In `MintParameters.tsx`, for flows: + +- A setting with choices is a ` change(event.target.value)} + className={cn(base, 'w-full bg-background')} + > + {/* A value the choices do not hold (an old one) stays visible rather than + silently becoming the first choice. */} + {(current === '' || !known) && } + {choices.map((choice: SettingChoice) => ( + + ))} + + {chosen?.description && ( +

+ {chosen.description} +

+ )} + + ); + } + + if (kind === 'boolean') { + return ( + change(String(event.target.checked))} + className={className} + /> + ); + } + + return ( + change(event.target.value)} + className={cn(base, 'w-full', className)} + /> + ); +} diff --git a/ui-react/src/components/common/__tests__/SettingInput.test.tsx b/ui-react/src/components/common/__tests__/SettingInput.test.tsx new file mode 100644 index 00000000..60c778c4 --- /dev/null +++ b/ui-react/src/components/common/__tests__/SettingInput.test.tsx @@ -0,0 +1,93 @@ +import { fireEvent, render, screen } from '@testing-library/react'; +import { describe, expect, it, vi } from 'vitest'; + +import { settingInputKind } from '@/lib/flow-settings'; + +import { SettingInput } from '../SettingInput'; + +const choices = [ + { value: 'auto', title: 'Automatic', description: 'Words and phrases.' }, + { value: 'longest', title: 'Longest match', description: 'No double counting.' }, +]; + +describe('SettingInput', () => { + it('picks the input from the description', () => { + expect(settingInputKind({ choices })).toBe('choice'); + expect(settingInputKind({ type: 'integer' })).toBe('number'); + expect(settingInputKind({ type: 'number' })).toBe('number'); + expect(settingInputKind({ type: 'boolean' })).toBe('boolean'); + expect(settingInputKind({ type: 'string' })).toBe('text'); + expect(settingInputKind(undefined)).toBe('text'); + }); + + it('shows choices by their titles and explains the chosen one', () => { + render(); + const select = screen.getByTestId('setting-input-mode') as HTMLSelectElement; + expect(select.value).toBe('auto'); + expect(Array.from(select.options).map((option) => option.textContent)).toEqual([ + 'Automatic', + 'Longest match', + ]); + expect(screen.getByTestId('setting-input-mode-explanation')).toHaveTextContent( + 'Words and phrases.', + ); + fireEvent.change(select, { target: { value: 'longest' } }); + expect(screen.getByTestId('setting-input-mode-explanation')).toHaveTextContent( + 'No double counting.', + ); + }); + + it('shows plain choices (an enum) with no explanation', () => { + render(); + expect(screen.queryByTestId('setting-input-c-explanation')).not.toBeInTheDocument(); + expect(screen.getByRole('option', { name: 'green' })).toBeInTheDocument(); + }); + + it('keeps a value the choices do not hold, and says what an empty dropdown means', () => { + const { rerender } = render(); + expect(screen.getByRole('option', { name: 'old' })).toBeInTheDocument(); + rerender(); + expect(screen.getByRole('option', { name: 'Piece default' })).toBeInTheDocument(); + }); + + it('gives a number box its bounds, and whole steps for an integer', () => { + render( + , + ); + const input = screen.getByTestId('setting-input-n'); + expect(input).toHaveAttribute('type', 'number'); + expect(input).toHaveAttribute('min', '1'); + expect(input).toHaveAttribute('max', '500'); + expect(input).toHaveAttribute('step', '1'); + expect(input).toHaveAttribute('placeholder', '200'); + }); + + it('is a checkbox for a yes or no', () => { + const onChange = vi.fn(); + render( + , + ); + const box = screen.getByTestId('setting-input-b') as HTMLInputElement; + expect(box.type).toBe('checkbox'); + expect(box.checked).toBe(false); + fireEvent.click(box); + expect(onChange).toHaveBeenCalledWith('true'); + }); + + it('is text otherwise, and reports what is typed', () => { + const onChange = vi.fn(); + render(); + fireEvent.change(screen.getByTestId('setting-input-t'), { target: { value: 'x' } }); + expect(onChange).toHaveBeenCalledWith('x'); + expect((screen.getByTestId('setting-input-t') as HTMLInputElement).value).toBe('x'); + }); + + it('follows `value` when it is given', () => { + const { rerender } = render(); + rerender(); + expect((screen.getByTestId('setting-input-t') as HTMLInputElement).value).toBe('b'); + }); +}); diff --git a/ui-react/src/components/flows/FlowAuthoringForm.tsx b/ui-react/src/components/flows/FlowAuthoringForm.tsx index bad6bfb3..d2970406 100644 --- a/ui-react/src/components/flows/FlowAuthoringForm.tsx +++ b/ui-react/src/components/flows/FlowAuthoringForm.tsx @@ -39,6 +39,7 @@ import { setInputWire, setPause, sortProblems, + setSettingRule, updateStep, type BindingRole, type DraftOutput, @@ -108,7 +109,16 @@ export function FlowAuthoringForm({ }, [pieces]); // --- the live check --------------------------------------------------------- - const body = useMemo(() => buildCheckBody(draft), [draft]); + // The setting names the pieces have, once every step's piece has been read: a decision + // about a setting whose step was removed is then left out rather than failing the check. + const knownSettings = useMemo( + () => + draft.steps.every((step) => summaries[step.pieceId]) + ? new Set(draft.steps.flatMap((step) => summaries[step.pieceId]?.settings ?? [])) + : undefined, + [draft.steps, summaries], + ); + const body = useMemo(() => buildCheckBody(draft, knownSettings), [draft, knownSettings]); const bodyKey = JSON.stringify(body); useEffect(() => { if (!token || draft.steps.length === 0) return; @@ -141,7 +151,7 @@ export function FlowAuthoringForm({ const ask = async (slot: SlotKey, purpose: Purpose, snapshot: FlowDraft) => { setSuggestion({ slot, result: 'loading' }); const result = token - ? await proposeForFlow(buildCheckBody(snapshot), purpose, token) + ? await proposeForFlow(buildCheckBody(snapshot, knownSettings), purpose, token) : { error: 'Sign in to get proposals.' }; setSuggestion({ slot, result }); }; @@ -202,7 +212,7 @@ export function FlowAuthoringForm({ const save = async () => { if (!token || !canSave) return; setSaving(true); - setSaved(await saveFlow(buildSaveBody(draft), token)); + setSaved(await saveFlow(buildSaveBody(draft, knownSettings), token)); setSaving(false); }; @@ -527,6 +537,8 @@ export function FlowAuthoringForm({ updateStep(draft, index, { settings: { ...step.settings, [name]: value } }), ) } + rules={draft.settings} + onRule={(name, rule) => change(setSettingRule(draft, name, rule))} onRemoveWire={(wireIndex) => change(removeWire(draft, index, wireIndex))} onPause={(pause) => change(setPause(draft, index, pause))} editable={editableChoice(draft, position)} diff --git a/ui-react/src/components/flows/StepCard.tsx b/ui-react/src/components/flows/StepCard.tsx index a3608d4e..8621b0af 100644 --- a/ui-react/src/components/flows/StepCard.tsx +++ b/ui-react/src/components/flows/StepCard.tsx @@ -11,6 +11,7 @@ import { Trash2 } from 'lucide-react'; import { Button } from '@/components/ui/button'; import { Input } from '@/components/ui/input'; +import { SettingInput } from '@/components/common/SettingInput'; import type { PieceSummary } from '@/lib/flow-authoring-api'; import { EDIT_CHECKS, @@ -20,6 +21,13 @@ import { type DraftStep, type FlowProblem, } from '@/lib/flow-authoring'; +import { + settingText, + settingTitle, + visibilityOf, + type SettingRule, + type Visibility, +} from '@/lib/flow-settings'; import { describeWire, parseWires, type FlowLabels } from '@/lib/flow-steps'; import { ProposalList, type ProposalListProps } from './ProposalList'; @@ -45,6 +53,9 @@ export interface StepCardProps { onSuggestInput: (inputPosition: number) => void; onSuggestNext: (outputPosition: number) => void; onSetting: (name: string, value: string) => void; + /** What the author decided about each setting of the flow, by name. */ + rules: Record; + onRule: (name: string, rule: SettingRule) => void; onRemoveWire: (wireIndex: number) => void; onPause: (pause: boolean) => void; /** The file this step's pause makes editable, and the change of it (`null` clears). */ @@ -202,31 +213,82 @@ export function StepCard(props: StepCardProps) { {summary && summary.settings.length > 0 && (
- Settings (the flow's defaults; a runner can change them, so the model that runs - this flow must accept them) + Settings. The value is the flow's default, which a runner can change, unless you + fix it. The model that runs this flow must accept a default. {summary.settings.map((name) => { const details = summary.setting_details[name]; const wired = settingWired.has(name); + const rule = props.rules[name]; + const fixed = rule?.fixed === true; + const shown = visibilityOf(rule, details); + const title = settingTitle(name, details); return ( -
- - props.onSetting(name, event.target.value)} - /> - {wired && ( - - linked to an earlier step; it cannot also have a default - +
+
+ +
+ props.onSetting(name, value)} + disabled={wired} + unsetLabel={ + details?.default === undefined + ? 'Not set' + : `Piece default: ${settingText(details.default)}` + } + className="h-8" + /> +
+ {wired ? ( + + linked to an earlier step; it cannot also have a default + + ) : ( + <> + + + {shown === 'hidden' && ( + + Hidden means not shown, not secret: anyone who can read this flow can see + the value. Use a stored credential for anything private. + + )} + + )} +
+ {details?.description && ( +

{details.description}

)}
); diff --git a/ui-react/src/components/flows/__tests__/FlowAuthoringForm.test.tsx b/ui-react/src/components/flows/__tests__/FlowAuthoringForm.test.tsx index 053559cc..b613f7d7 100644 --- a/ui-react/src/components/flows/__tests__/FlowAuthoringForm.test.tsx +++ b/ui-react/src/components/flows/__tests__/FlowAuthoringForm.test.tsx @@ -1,7 +1,7 @@ import userEvent from '@testing-library/user-event'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; -import { draftFromStoredFlow, type StoredFlowSource } from '@/lib/flow-authoring'; +import { draftFromStoredFlow, emptyDraft, type StoredFlowSource } from '@/lib/flow-authoring'; import { renderWithProviders, screen, waitFor, within } from '@/test/utils/render'; import { setMintConfig } from '@/test/utils/mint-config'; @@ -27,6 +27,7 @@ vi.mock('@/components/autocomplete/StandardVariableCombobox', () => ({ const PIECES = [ { id: 'piece-a', label: 'Ingest' }, { id: 'piece-b', label: 'Score' }, + { id: 'piece-c', label: 'Count' }, ]; const SUMMARIES = { 'piece-a': { @@ -38,6 +39,27 @@ const SUMMARIES = { output_contracts: { '1': { standard_variable_uri: 'https://x.org/clean' } }, setting_details: { top_n: { type: 'string', default: '10' } }, }, + 'piece-c': { + inputs: [1], + outputs: [1], + settings: ['match_mode', 'max_files'], + values: [], + input_contracts: { '1': { standard_variable_uri: 'https://x.org/corpus' } }, + output_contracts: { '1': {} }, + setting_details: { + match_mode: { + type: 'string', + title: 'How terms are counted', + description: 'The counting rule.', + default: 'auto', + choices: [ + { value: 'auto', title: 'Automatic', description: 'Words and phrases.' }, + { value: 'longest', title: 'Longest match', description: 'No double counting.' }, + ], + }, + max_files: { type: 'integer', title: 'Most files', minimum: 1, maximum: 500, advanced: true }, + }, + }, 'piece-b': { inputs: [1], outputs: [1], @@ -155,7 +177,7 @@ describe('FlowAuthoringForm', () => { await addStepOf(user, 'Ingest'); const step = screen.getByTestId('flow-form-step-1'); expect(await within(step).findByLabelText('Source of input 1 of step 1')).toBeInTheDocument(); - expect(within(step).getByLabelText('top_n')).toBeInTheDocument(); + expect(within(step).getByLabelText('top n')).toBeInTheDocument(); expect(await within(step).findByText('input 1 of step 1 is not linked')).toBeInTheDocument(); expect(screen.getByRole('button', { name: 'Save flow' })).toBeDisabled(); }); @@ -205,7 +227,7 @@ describe('FlowAuthoringForm', () => { expect(proposal.url).toBe('http://em.test/v1/adapter/flows/propose'); expect(proposal.body.purpose).toEqual({ step: 2, input: 1 }); - await user.type(within(screen.getByTestId('flow-form-step-1')).getByLabelText('top_n'), '5'); + await user.type(within(screen.getByTestId('flow-form-step-1')).getByLabelText('top n'), '5'); const save = screen.getByRole('button', { name: 'Save flow' }); await waitFor(() => expect(save).toBeEnabled()); await user.click(save); @@ -239,6 +261,113 @@ describe('FlowAuthoringForm', () => { ); }); + describe('settings described by their piece', () => { + const counting = () => ({ + ...emptyDraft(), + software: { id: '', label: 'Sensemaking' }, + version: { id: '', label: '1' }, + configuration: { label: 'My flow', description: '' }, + inputs: [ + { + key: 'input1', + label: 'Docs', + standardVariable: 'sv:corpus', + standardVariableLabel: 'Corpus', + format: '', + optional: false, + }, + ], + steps: [ + { + pieceId: 'piece-c', + settings: {}, + wires: [{ to: { input: 1 }, from: { flow_input: 'input1' } }], + pauseAfter: false, + }, + ], + }); + + it('shows titles, a dropdown with explanations and a bounded number box', async () => { + fakeServer(); + const user = userEvent.setup(); + render({ initial: counting() }); + const step = await screen.findByTestId('flow-form-step-1'); + const mode = await within(step).findByLabelText('How terms are counted'); + expect(mode.tagName).toBe('SELECT'); + await user.selectOptions(mode, 'Longest match'); + expect(within(step).getByText('No double counting.')).toBeInTheDocument(); + const files = within(step).getByLabelText('Most files'); + expect(files).toHaveAttribute('type', 'number'); + expect(files).toHaveAttribute('max', '500'); + // The piece suggests this one is advanced, and the author may change that. + expect(within(step).getByLabelText('Who sees Most files')).toHaveValue('advanced'); + expect(within(step).getByLabelText('Who sees How terms are counted')).toHaveValue('shown'); + }); + + it('saves the value, whether it is fixed and who sees it, in the check and the save', async () => { + fakeServer(); + const user = userEvent.setup(); + render({ initial: counting() }); + const step = await screen.findByTestId('flow-form-step-1'); + await user.selectOptions( + await within(step).findByLabelText('How terms are counted'), + 'Longest match', + ); + await user.selectOptions( + within(step).getByLabelText('Is the value of How terms are counted a default or fixed'), + 'fixed', + ); + await user.selectOptions( + within(step).getByLabelText('Who sees How terms are counted'), + 'hidden', + ); + const save = screen.getByRole('button', { name: 'Save flow' }); + await waitFor(() => expect(save).toBeEnabled()); + await user.click(save); + await waitFor(() => + expect(calls.some((c) => c.url === 'http://catalog.test/v2.0.0/flows')).toBe(true), + ); + const sent = calls.find((c) => c.url === 'http://catalog.test/v2.0.0/flows')!; + expect(sent.body.steps[0].settings).toEqual({ match_mode: 'longest' }); + expect(sent.body.settings).toEqual({ match_mode: { fixed: true, visibility: 'hidden' } }); + const check = [...calls].reverse().find((c) => c.url.endsWith('/flows/check'))!; + expect(check.body.settings).toEqual({ match_mode: { fixed: true, visibility: 'hidden' } }); + }); + + it('says that hidden is not secret, once a setting is hidden', async () => { + fakeServer(); + const user = userEvent.setup(); + render({ initial: counting() }); + const step = await screen.findByTestId('flow-form-step-1'); + expect(within(step).queryByText(/not shown, not secret/)).not.toBeInTheDocument(); + await user.selectOptions( + await within(step).findByLabelText('Who sees How terms are counted'), + 'hidden', + ); + expect(within(step).getByText(/not shown, not secret/)).toBeInTheDocument(); + }); + + it('sends no decisions when the author made none', async () => { + fakeServer(); + render({ initial: counting() }); + await screen.findByTestId('flow-form-step-1'); + await waitFor(() => expect(calls.some((c) => c.url.endsWith('/flows/check'))).toBe(true)); + expect(calls.find((c) => c.url.endsWith('/flows/check'))!.body.settings).toEqual({}); + }); + + it('leaves out a decision about a setting no step has any more', async () => { + fakeServer(); + const draft = { ...counting(), settings: { gone: { visibility: 'hidden' as const } } }; + render({ initial: draft }); + await screen.findByLabelText('How terms are counted'); + await waitFor(() => + expect(calls.filter((c) => c.url.endsWith('/flows/check')).at(-1)?.body.settings).toEqual( + {}, + ), + ); + }); + }); + it('adds a proposed next step wired from the output it was asked about', async () => { const user = userEvent.setup(); render(); @@ -437,7 +566,7 @@ describe('FlowAuthoringForm', () => { ).toBeInTheDocument(); expect(screen.getByLabelText('Input 1 takes a collection of files')).toBeChecked(); expect(screen.getByLabelText('Region is filled in by')).toHaveValue('input:Corpus'); - expect(await screen.findByLabelText('top_n')).toHaveValue('25'); + expect(await screen.findByLabelText('top n')).toHaveValue('25'); const save = screen.getByRole('button', { name: 'Save flow' }); await waitFor(() => expect(save).toBeEnabled()); diff --git a/ui-react/src/graphql/queries/flow-authoring.ts b/ui-react/src/graphql/queries/flow-authoring.ts index 499fc4bf..0d76d315 100644 --- a/ui-react/src/graphql/queries/flow-authoring.ts +++ b/ui-react/src/graphql/queries/flow-authoring.ts @@ -94,6 +94,7 @@ export const GET_FLOW_FOR_COPY = gql` flow { thread_bindings_json outputs_json + settings_json steps(order_by: { position: asc }) { position etl_process_id diff --git a/ui-react/src/hooks/useFlowSettings.ts b/ui-react/src/hooks/useFlowSettings.ts new file mode 100644 index 00000000..35f7353f --- /dev/null +++ b/ui-react/src/hooks/useFlowSettings.ts @@ -0,0 +1,30 @@ +import { useEffect, useState } from 'react'; + +import { useAuth } from '@/lib/auth/useAuth'; +import { fetchFlowSettings, type FlowSettings } from '@/lib/flow-settings'; + +/** + * The settings of each flow in `flowIds` as a runner meets them, by flow id. A flow + * whose settings could not be read is absent, so the form falls back to the catalog's + * parameter rows and stays usable. + */ +export function useFlowSettings(flowIds: string[]): Record { + const { isAuthenticated, accessToken } = useAuth(); + const [found, setFound] = useState>({}); + const wanted = flowIds.join('\n'); + + useEffect(() => { + if (!wanted || !isAuthenticated || !accessToken) return; + const controller = new AbortController(); + for (const id of wanted.split('\n')) { + void fetchFlowSettings(id, accessToken, controller.signal).then((settings) => { + if (settings && !controller.signal.aborted) { + setFound((previous) => ({ ...previous, [id]: settings })); + } + }); + } + return () => controller.abort(); + }, [wanted, isAuthenticated, accessToken]); + + return found; +} diff --git a/ui-react/src/lib/__tests__/flow-authoring.test.ts b/ui-react/src/lib/__tests__/flow-authoring.test.ts index 0ec26de3..25164fc4 100644 --- a/ui-react/src/lib/__tests__/flow-authoring.test.ts +++ b/ui-react/src/lib/__tests__/flow-authoring.test.ts @@ -12,6 +12,8 @@ import { removeInput, removeStep, setInputWire, + setSettingRule, + settingRulesJson, sortProblems, type FlowDraft, type StoredFlowSource, @@ -285,3 +287,66 @@ describe('editing as a copy', () => { expect(draft.bindings.collections).toEqual([]); }); }); + +describe('what the author decides about a setting', () => { + it('records a decision, and removes a decision that decides nothing', () => { + let draft = setSettingRule(emptyDraft(), 'match_mode', { visibility: 'hidden', fixed: true }); + expect(draft.settings).toEqual({ match_mode: { visibility: 'hidden', fixed: true } }); + draft = setSettingRule(draft, 'match_mode', { fixed: false }); + expect(draft.settings).toEqual({}); + }); + + it('sends only decisions, and only for settings a step has once that is known', () => { + const draft: FlowDraft = { + ...emptyDraft(), + settings: { + a: { visibility: 'advanced' }, + b: { fixed: true }, + c: { fixed: false }, + gone: { visibility: 'hidden' }, + }, + }; + expect(settingRulesJson(draft)).toEqual({ + a: { visibility: 'advanced' }, + b: { fixed: true }, + gone: { visibility: 'hidden' }, + }); + expect(settingRulesJson(draft, new Set(['a', 'b', 'c']))).toEqual({ + a: { visibility: 'advanced' }, + b: { fixed: true }, + }); + expect(buildCheckBody(draft).settings).toEqual(settingRulesJson(draft)); + expect(buildSaveBody(draft, new Set(['a'])).settings).toEqual({ + a: { visibility: 'advanced' }, + }); + }); + + it('tells the check which settings the thread fills in', () => { + const draft: FlowDraft = { + ...emptyDraft(), + bindings: { ...emptyDraft().bindings, start_date: 'setting:begin', region: 'input:corpus' }, + }; + expect(buildCheckBody(draft).thread_settings).toEqual(['begin']); + }); + + it('is read back from a stored flow, ignoring what it cannot read', () => { + const base: StoredFlowSource = { + configuration: { label: 'F', inputs: [], outputs: [], parameters: [] }, + flow: { + steps: [], + settings_json: { + a: { visibility: 'hidden', fixed: true }, + b: { visibility: 'sideways' }, + c: 'nope', + }, + }, + }; + expect(draftFromStoredFlow(base).settings).toEqual({ + a: { visibility: 'hidden', fixed: true }, + b: {}, + }); + expect( + draftFromStoredFlow({ ...base, flow: { steps: [], settings_json: 7 } }).settings, + ).toEqual({}); + }); +}); diff --git a/ui-react/src/lib/__tests__/flow-errors.test.ts b/ui-react/src/lib/__tests__/flow-errors.test.ts index ce508894..ce095d55 100644 --- a/ui-react/src/lib/__tests__/flow-errors.test.ts +++ b/ui-react/src/lib/__tests__/flow-errors.test.ts @@ -95,6 +95,14 @@ describe('flowErrorMessage', () => { expect(fixed?.lines).toContain('title'); }); + it('names the settings the author fixed or hid, and which', () => { + const refused = refuse('FLOW_SETTING_REFUSED', { fixed: ['match_mode'], hidden: ['internal'] }); + expect(refused?.title).toBe('A setting is fixed or hidden by the flow'); + expect(refused?.lines).toEqual( + expect.arrayContaining(['match_mode (fixed)', 'internal (hidden)']), + ); + }); + it('lists the pieces that are not approved and the problems with their steps', () => { const stored = refuse('STORED_SOURCE_REFUSED', { problems: [{ code: 'NOT_APPROVED', message: 'piece p1 is not approved', step: 'clean' }], diff --git a/ui-react/src/lib/__tests__/flow-settings.test.ts b/ui-react/src/lib/__tests__/flow-settings.test.ts new file mode 100644 index 00000000..0036c0eb --- /dev/null +++ b/ui-react/src/lib/__tests__/flow-settings.test.ts @@ -0,0 +1,86 @@ +import { afterEach, describe, expect, it, vi } from 'vitest'; + +import { + choiceLabel, + fetchFlowSettings, + settingText, + settingTitle, + visibilityOf, +} from '../flow-settings'; + +const SETTING = { + name: 'match_mode', + parameter_id: 'cfg_setting_match_mode', + visibility: 'shown', + title: 'How terms are counted', +}; + +afterEach(() => vi.unstubAllGlobals()); + +describe('naming and showing a setting', () => { + it('prefers the piece title and falls back to the name with spaces', () => { + expect(settingTitle('match_mode', { title: 'How terms are counted' })).toBe( + 'How terms are counted', + ); + expect(settingTitle('match_mode', {})).toBe('match mode'); + expect(settingTitle('match_mode', { title: ' ' })).toBe('match mode'); + expect(settingTitle('top_n')).toBe('top n'); + }); + + it('labels a choice by its title, else its value', () => { + expect(choiceLabel({ value: 'a', title: 'Automatic' })).toBe('Automatic'); + expect(choiceLabel({ value: 5 })).toBe('5'); + }); + + it('writes a value as the text an input shows', () => { + expect(settingText('x')).toBe('x'); + expect(settingText(25)).toBe('25'); + expect(settingText(false)).toBe('false'); + expect(settingText(undefined)).toBe(''); + expect(settingText(null)).toBe(''); + }); + + it('takes the authors choice, else the pieces hint, else shown', () => { + expect(visibilityOf({ visibility: 'hidden' }, { advanced: true })).toBe('hidden'); + expect(visibilityOf({ fixed: true }, { advanced: true })).toBe('advanced'); + expect(visibilityOf(undefined, {})).toBe('shown'); + expect(visibilityOf({ visibility: 'bogus' as never }, undefined)).toBe('shown'); + }); +}); + +describe('fetchFlowSettings', () => { + const answers = (body: unknown, ok = true) => { + const fetchMock = vi.fn().mockResolvedValue({ ok, json: async () => body }); + vi.stubGlobal('fetch', fetchMock); + return fetchMock; + }; + + it('reads the settings of a flow with the callers token', async () => { + const fetchMock = answers({ + settings: [SETTING, { name: 'broken' }], + thread_bound: ['start', 3], + }); + const found = await fetchFlowSettings('https://w3id.org/okn/i/mint/dict', 'tok'); + expect(found?.settings).toEqual([SETTING]); + expect(found?.thread_bound).toEqual(['start']); + const [url, init] = fetchMock.mock.calls[0]!; + expect(String(url)).toContain( + `/flows/${encodeURIComponent('https://w3id.org/okn/i/mint/dict')}/settings`, + ); + expect((init as RequestInit).headers).toEqual({ Authorization: 'Bearer tok' }); + }); + + it.each([ + ['no token', null, { settings: [] }, true], + ['a refusal', 'tok', { settings: [] }, false], + ['a malformed answer', 'tok', { nope: 1 }, true], + ])('is null for %s', async (_name, token, body, ok) => { + answers(body, ok); + expect(await fetchFlowSettings('f', token)).toBeNull(); + }); + + it('is null when the adapter cannot be reached', async () => { + vi.stubGlobal('fetch', vi.fn().mockRejectedValue(new Error('down'))); + expect(await fetchFlowSettings('f', 'tok')).toBeNull(); + }); +}); diff --git a/ui-react/src/lib/__tests__/spatial-boundaries.test.ts b/ui-react/src/lib/__tests__/spatial-boundaries.test.ts new file mode 100644 index 00000000..2e4df4a9 --- /dev/null +++ b/ui-react/src/lib/__tests__/spatial-boundaries.test.ts @@ -0,0 +1,119 @@ +import { describe, expect, it } from 'vitest'; + +import { + boundaryFeatureLabel, + boundaryGroups, + buildLayerOptions, + buildRegionOptions, + filterBoundaryOptions, + findBoundaryOption, + prettyRegionName, + regionFamilyOf, +} from '@/lib/spatial-boundaries'; + +const region = (id: string, name: string, category = 'Aquifers'): GeoJSON.Feature => ({ + type: 'Feature', + properties: { region_id: id, region_name: name, category_name: category }, + geometry: { type: 'Point', coordinates: [0, 0] }, +}); + +describe('boundaryFeatureLabel', () => { + it('labels a number-only layer from the configured prefix', () => { + expect(boundaryFeatureLabel({ GMAnum: 1 }, 'GMAnum', { label_prefix: 'GMA ' })).toBe('GMA 1'); + }); + + it('adds the name when the layer has one', () => { + expect( + boundaryFeatureLabel({ GMAnum: 8, GMAName: 'Gulf Coast' }, 'GMAnum', { + label_prefix: 'GMA ', + }), + ).toBe('GMA 8 — Gulf Coast'); + }); + + it('uses the configured label field and suffix for named layers', () => { + expect( + boundaryFeatureLabel({ Name: 'Anderson', Fips: 1 }, 'Name', { + label_field: 'Name', + label_suffix: ' County', + }), + ).toBe('Anderson County'); + }); +}); + +describe('regionFamilyOf', () => { + it('finds the family of a numbered name', () => { + expect(regionFamilyOf('TRINITY_53')?.label).toBe('Trinity'); + expect(regionFamilyOf('EDWARDS-TRINITY_3')?.label).toBe('Edwards-Trinity'); + }); + it('ignores names without a base or a number', () => { + expect(regionFamilyOf('Alwero')).toBeNull(); + expect(regionFamilyOf('1')).toBeNull(); + }); + it('prettifies underscore names only', () => { + expect(prettyRegionName('SEYMOUR_3')).toBe('Seymour 3'); + expect(prettyRegionName('GMA 12')).toBe('GMA 12'); + }); +}); + +describe('buildRegionOptions', () => { + const features = [ + region('zz1', 'TRINITY_2'), + region('zz2', 'TRINITY_10'), + region('zz3', 'TRINITY_1'), + region('aa9', 'Alwero', 'Watersheds'), + region('lone', 'CARRIZO_1'), + region('1790434732014', '1', 'Watersheds'), + ]; + const options = buildRegionOptions(features, 'region_id'); + + it('never puts an id in a label', () => { + for (const option of options) { + expect(option.label).not.toMatch(/zz\d|aa9|lone|1790434732014/); + } + }); + + it('collapses numbered parts into one family that selects every part', () => { + const family = options.find((o) => o.label === 'Trinity (3 parts)'); + expect(family?.values.sort()).toEqual(['zz1', 'zz2', 'zz3']); + expect(family?.parts?.map((p) => p.label)).toEqual(['Trinity 1', 'Trinity 2', 'Trinity 10']); + expect(options.filter((o) => o.label.startsWith('Trinity'))).toHaveLength(1); + }); + + it('leaves a lone numbered region as itself', () => { + expect(options.find((o) => o.label === 'Carrizo 1')?.parts).toBeUndefined(); + }); + + it('sorts by group then name, and lists the groups', () => { + expect(options.map((o) => o.label)).toEqual(['Carrizo 1', 'Trinity (3 parts)', '1', 'Alwero']); + expect(boundaryGroups(options)).toEqual(['Aquifers', 'Watersheds']); + }); + + it('filters by typing, keeping only matching parts of a family', () => { + const hits = filterBoundaryOptions(options, 'trinity 10'); + expect(hits).toHaveLength(1); + expect(hits[0]?.parts?.map((p) => p.label)).toEqual(['Trinity 10']); + expect(filterBoundaryOptions(options, 'alw').map((o) => o.label)).toEqual(['Alwero']); + expect(filterBoundaryOptions(options, 'nothing')).toEqual([]); + }); + + it('finds a saved single part or family by its value', () => { + expect(findBoundaryOption(options, 'zz2')?.label).toBe('Trinity 10'); + const family = options.find((o) => o.parts); + expect(findBoundaryOption(options, family?.value ?? '')?.label).toBe('Trinity (3 parts)'); + }); +}); + +describe('buildLayerOptions', () => { + it('labels and sorts GMA features', () => { + const features = [10, 2, 1].map( + (n): GeoJSON.Feature => ({ + type: 'Feature', + properties: { GMAnum: n }, + geometry: { type: 'Point', coordinates: [0, 0] }, + }), + ); + expect( + buildLayerOptions(features, 'GMAnum', { label_prefix: 'GMA ' }).map((o) => o.label), + ).toEqual(['GMA 1', 'GMA 2', 'GMA 10']); + }); +}); diff --git a/ui-react/src/lib/datasets/__tests__/upload-preview.test.ts b/ui-react/src/lib/datasets/__tests__/upload-preview.test.ts new file mode 100644 index 00000000..3eeb1965 --- /dev/null +++ b/ui-react/src/lib/datasets/__tests__/upload-preview.test.ts @@ -0,0 +1,122 @@ +import { describe, expect, it } from 'vitest'; +import type { GetModelTreeWithRegionsQuery, ModelConfigInfo } from '@/graphql/generated/modeling'; +import { rankSuggestionsByUse, suggestionUseText } from '@/lib/datasets/upload-preview'; + +const SV = 'https://w3id.org/okn/i/mint/'; + +function input(id: string, variable: string, format: string | null) { + return { + configuration_id: 'c', + input_id: id, + is_optional: false, + input: { + id, + label: id, + has_format: format, + presentations: [ + { + dataset_specification_id: id, + presentation_id: `${id}-p`, + presentation: { + id: `${id}-p`, + standard_variable: { id: SV + variable, label: variable }, + }, + }, + ], + }, + }; +} + +function config(id: string, inputs: ReturnType[], flow = false): ModelConfigInfo { + return { + id, + label: id, + flow: flow ? { configuration_id: id, thread_bindings_json: null } : null, + regions: [], + inputs, + outputs: [], + child_configurations: [], + } as unknown as ModelConfigInfo; +} + +const tree: GetModelTreeWithRegionsQuery = { + modelcatalog_software: [ + { + id: 's', + label: 's', + versions: [ + { + id: 'v', + label: 'v', + configurations: [ + config( + 'Dictionary flow', + [input('corpus', 'document_collection__text', 'directory')], + true, + ), + config( + 'Narrative flow', + [input('corpus2', 'document_collection__text', 'directory')], + true, + ), + config('HAND', [input('dem', 'land_surface__elevation', 'tif')]), + config('Narratives model', [input('nlp', 'corpus_nlp', null)]), + ], + }, + ], + }, + ], +}; + +// The semantic order the service returned for "RGV Transcripts — oral histories". +const hits = [ + { id: SV + 'corpus_nlp', label: 'corpus_nlp' }, + { + id: SV + 'document_collection_narrative_frame__description', + label: 'document_collection_narrative_frame__description', + }, + { id: SV + 'land_surface__elevation', label: 'land_surface__elevation' }, + { id: SV + 'document_collection__text', label: 'document_collection__text' }, +]; + +const labels = (ranked: { item: { label: string } }[]) => ranked.map((r) => r.item.label); + +describe('rankSuggestionsByUse', () => { + it('puts the variable a flow input accepts these files under first', () => { + const ranked = rankSuggestionsByUse(tree, hits, [{ name: 'transcript.json' }]); + expect(labels(ranked)).toEqual([ + 'document_collection__text', + 'corpus_nlp', + 'land_surface__elevation', + 'document_collection_narrative_frame__description', + ]); + expect(ranked.map((r) => r.fit)).toEqual([0, 1, 2, 3]); + }); + + it('keeps the semantic order among variables that fit equally', () => { + const ranked = rankSuggestionsByUse(tree, hits, []); + // No files yet: anything an input takes ranks above what nothing takes. + expect(labels(ranked)).toEqual([ + 'corpus_nlp', + 'land_surface__elevation', + 'document_collection__text', + 'document_collection_narrative_frame__description', + ]); + }); + + it('ranks nothing without the model tree but keeps every suggestion', () => { + expect(labels(rankSuggestionsByUse(undefined, hits, []))).toEqual(hits.map((h) => h.label)); + }); +}); + +describe('suggestionUseText', () => { + it('says who takes the variable and when the format is wrong', () => { + const ranked = rankSuggestionsByUse(tree, hits, [{ name: 'transcript.json' }]); + expect(ranked.map(suggestionUseText)).toEqual([ + 'Input to 2 flows', + 'Input to 1 model', + 'Input to 1 model, but not in this file format', + 'No registered flow or model takes this as an input', + ]); + }); +}); diff --git a/ui-react/src/lib/datasets/upload-preview.ts b/ui-react/src/lib/datasets/upload-preview.ts index ace22e0b..0ac23eee 100644 --- a/ui-react/src/lib/datasets/upload-preview.ts +++ b/ui-react/src/lib/datasets/upload-preview.ts @@ -76,6 +76,74 @@ export function consumersOfVariable( ); } +/** + * How useful a suggested variable is for these files, best first: + * 0 an input that declares a format takes these files, + * 1 an input takes the variable but declares no format (or no files are chosen yet), + * 2 inputs take the variable but none takes these files, + * 3 no registered flow or model takes it. + */ +export type SuggestionFit = 0 | 1 | 2 | 3; + +export interface RankedSuggestion { + item: T; + fit: SuggestionFit; + /** Flows and models with an input that takes the variable, whatever the format. */ + flows: number; + models: number; +} + +function suggestionFit(consumers: UploadConsumer[]): SuggestionFit { + if (consumers.length === 0) return 3; + if (consumers.some((c) => c.formatFits === true && (c.format ?? '').trim())) return 0; + if (consumers.some((c) => c.formatFits !== false)) return 1; + return 2; +} + +/** + * Order the semantic suggestions by whether a registered flow or model could use + * the files under that variable, keeping the semantic order within each group. + * Text similarity alone cannot tell "a corpus of transcripts" from "frames drawn + * from transcripts"; what the catalog's inputs accept can. + */ +export function rankSuggestionsByUse( + tree: GetModelTreeWithRegionsQuery | undefined, + suggestions: T[], + files: Array<{ name: string }>, +): RankedSuggestion[] { + return suggestions + .map((item, index) => { + const consumers = consumersOfVariable( + tree, + { id: item.id, label: item.label?.trim() || item.id }, + files, + ); + return { + index, + ranked: { + item, + fit: suggestionFit(consumers), + flows: consumers.filter((c) => c.kind === 'flow').length, + models: consumers.filter((c) => c.kind === 'model').length, + }, + }; + }) + .sort((a, b) => a.ranked.fit - b.ranked.fit || a.index - b.index) + .map(({ ranked }) => ranked); +} + +/** "Input to 6 flows and 1 model", or why nothing can use it. */ +export function suggestionUseText(ranked: RankedSuggestion): string { + const count = (n: number, word: string) => `${n} ${word}${n === 1 ? '' : 's'}`; + const parts = [ + ranked.flows ? count(ranked.flows, 'flow') : '', + ranked.models ? count(ranked.models, 'model') : '', + ].filter(Boolean); + if (parts.length === 0) return 'No registered flow or model takes this as an input'; + const users = parts.join(' and '); + return ranked.fit === 2 ? `Input to ${users}, but not in this file format` : `Input to ${users}`; +} + export interface CloseConsumer extends UploadConsumer { /** The close variable whose input this matched on. */ via: string; @@ -103,3 +171,36 @@ export function closeConsumers( } return found; } + +export interface GuidedSetupLinkInput { + datasetId: string; + variable: { label: string } | null; + exact: UploadConsumer[]; + close: CloseConsumer[]; + title: string; + startDate: string; + endDate: string; + /** Only set when the user picked a catalog region; a drawn box is derived from the dataset instead. */ + regionId?: string; +} + +/** + * The guided model setup URL for a just-uploaded dataset. The exact consumers + * travel as `modelIds` (recommended), the close matches as `closeModelIds`, and + * `modelNames` maps ids to readable names. Nothing here confirms anything: the + * wizard still asks the user to frame the question. + */ +export function guidedSetupLink(input: GuidedSetupLinkInput): string { + const params = new URLSearchParams({ datasetId: input.datasetId }); + if (input.exact.length) params.set('modelIds', input.exact.map((c) => c.id).join(',')); + if (input.close.length) params.set('closeModelIds', input.close.map((c) => c.id).join(',')); + const names: Record = {}; + for (const c of [...input.exact, ...input.close]) names[c.id] = c.name; + if (Object.keys(names).length) params.set('modelNames', JSON.stringify(names)); + if (input.variable) params.set('variable', input.variable.label); + if (input.title.trim()) params.set('goal', input.title.trim()); + if (input.startDate) params.set('startDate', input.startDate); + if (input.endDate) params.set('endDate', input.endDate); + if (input.regionId) params.set('regionId', input.regionId); + return `/modeling/problem-statements/start?${params.toString()}`; +} diff --git a/ui-react/src/lib/flow-authoring-api.ts b/ui-react/src/lib/flow-authoring-api.ts index cbd966a6..f0fe2ff9 100644 --- a/ui-react/src/lib/flow-authoring-api.ts +++ b/ui-react/src/lib/flow-authoring-api.ts @@ -11,6 +11,7 @@ */ import { getEnsembleManagerApiUrl, getModelCatalogApiUrl } from './config'; import type { FlowProblem, ProposedChain, ProposedSource, WireJson } from './flow-authoring'; +import type { SettingDetails } from './flow-settings'; export interface PieceSummary { inputs: number[]; @@ -25,7 +26,7 @@ export interface PieceSummary { string, { standard_variable_uri?: string; unit?: string; format?: string } >; - setting_details: Record; + setting_details: Record; } export interface FlowCheck { diff --git a/ui-react/src/lib/flow-authoring.ts b/ui-react/src/lib/flow-authoring.ts index 6619e0f9..272ffae9 100644 --- a/ui-react/src/lib/flow-authoring.ts +++ b/ui-react/src/lib/flow-authoring.ts @@ -15,6 +15,7 @@ */ import type { FlowPieceRow } from '@/graphql/queries/flows'; import { parseWires, parseSettings } from './flow-steps'; +import { VISIBILITIES, type SettingRule } from './flow-settings'; export interface WireJson { to: Record; @@ -70,6 +71,12 @@ export interface FlowDraft { steps: DraftStep[]; outputs: DraftOutput[]; bindings: DraftBindings; + /** + * What the author decided about each setting, by name: whether it is fixed, and + * whether a runner sees it, finds it under "Advanced settings", or does not see it. + * One setting name used by several steps is one setting of the flow. + */ + settings: Record; } export const MAX_STEPS = 24; @@ -85,6 +92,7 @@ export function emptyDraft(): FlowDraft { steps: [], outputs: [], bindings: { region: '', start_date: '', end_date: '', collections: [] }, + settings: {}, }; } @@ -114,6 +122,24 @@ function stepsJson(draft: FlowDraft) { })); } +/** + * The author's decisions as the catalog and the adapter take them: only a setting that + * has one, and only a setting some step has when `known` is given (a setting whose step + * was removed must not linger and fail the check). + */ +export function settingRulesJson(draft: FlowDraft, known?: ReadonlySet): Json { + const rules: Json = {}; + for (const [name, rule] of Object.entries(draft.settings)) { + if (known && !known.has(name)) continue; + const kept: SettingRule = { + ...(rule.visibility ? { visibility: rule.visibility } : {}), + ...(rule.fixed ? { fixed: true } : {}), + }; + if (Object.keys(kept).length > 0) rules[name] = kept; + } + return rules; +} + /** `editable` as the catalog takes it: a blank check or key is left out. */ function editableJson(editable: Json): Json { const { check, key, ...rest } = editable; @@ -136,8 +162,11 @@ function outputJson(output: DraftOutput, forSave: boolean): Json { return where; } -/** The body of the adapter's `POST /flows/check` and `POST /flows/propose`. */ -export function buildCheckBody(draft: FlowDraft) { +/** + * The body of the adapter's `POST /flows/check` and `POST /flows/propose`. `known` is + * the set of setting names the steps' pieces have, once the form has read them all. + */ +export function buildCheckBody(draft: FlowDraft, known?: ReadonlySet) { const bindings: Json = {}; for (const role of BINDING_ROLES) { const target = bindingJson(draft.bindings[role], false); @@ -155,11 +184,22 @@ export function buildCheckBody(draft: FlowDraft) { steps: stepsJson(draft), outputs: draft.outputs.map((output) => outputJson(output, false)), thread_bindings: bindings, + settings: settingRulesJson(draft, known), + // The settings the thread fills in: the check refuses one that is also fixed or hidden. + thread_settings: BINDING_ROLES.flatMap((role) => + draft.bindings[role].startsWith('setting:') + ? [draft.bindings[role].slice('setting:'.length)] + : [], + ), }; } -/** The body of `POST /flows` on the model catalog API. */ -export function buildSaveBody(draft: FlowDraft) { +/** + * The body of `POST /flows` on the model catalog API. The catalog writes each step's + * settings and the flow's `settings`, and copies the defaults, fixed values, choices and + * bounds onto the flow's parameter rows. + */ +export function buildSaveBody(draft: FlowDraft, known?: ReadonlySet) { const bindings: Json = {}; for (const role of BINDING_ROLES) { const target = bindingJson(draft.bindings[role], true); @@ -187,6 +227,7 @@ export function buildSaveBody(draft: FlowDraft) { steps: stepsJson(draft), outputs: draft.outputs.map((output) => outputJson(output, true)), thread_bindings: bindings, + settings: settingRulesJson(draft, known), }; } @@ -246,6 +287,13 @@ export function removeStep(draft: FlowDraft, index: number): FlowDraft { }; } +/** Record what the author decided about a setting. A rule with nothing in it is removed. */ +export function setSettingRule(draft: FlowDraft, name: string, rule: SettingRule): FlowDraft { + const { [name]: _dropped, ...others } = draft.settings; + const kept = Boolean(rule.visibility || rule.fixed); + return { ...draft, settings: kept ? { ...others, [name]: rule } : others }; +} + export function updateStep(draft: FlowDraft, index: number, change: Partial): FlowDraft { return { ...draft, @@ -606,6 +654,7 @@ export interface StoredFlowSource { flow: { thread_bindings_json?: unknown; outputs_json?: unknown; + settings_json?: unknown; steps: { position: number; etl_process_id?: string | null; @@ -718,6 +767,20 @@ export function draftFromStoredFlow(stored: StoredFlowSource): FlowDraft { ); } + const settings: Record = {}; + for (const [name, raw] of Object.entries( + isRecord(flow.settings_json) ? flow.settings_json : {}, + )) { + if (!isRecord(raw)) continue; + const visibility = (VISIBILITIES as readonly unknown[]).includes(raw.visibility) + ? (raw.visibility as SettingRule['visibility']) + : undefined; + settings[name] = { + ...(visibility ? { visibility } : {}), + ...(raw.fixed === true ? { fixed: true } : {}), + }; + } + return { software: { id: version?.software?.id ?? '', label: version?.software?.label ?? '' }, version: { id: version?.id ?? '', label: version?.version_id ?? '' }, @@ -729,5 +792,6 @@ export function draftFromStoredFlow(stored: StoredFlowSource): FlowDraft { steps, outputs, bindings, + settings, }; } diff --git a/ui-react/src/lib/flow-errors.ts b/ui-react/src/lib/flow-errors.ts index 79ba9346..de93f8c1 100644 --- a/ui-react/src/lib/flow-errors.ts +++ b/ui-react/src/lib/flow-errors.ts @@ -22,6 +22,7 @@ export const FLOW_ERROR_CODES = [ 'FLOW_ALREADY_RUNNING', 'FLOW_PLAN_NOT_YOURS', 'FLOW_ARGUMENT_FIXED', + 'FLOW_SETTING_REFUSED', 'STORED_SOURCE_REFUSED', 'FLOW_DOES_NOT_CHECK_OUT', 'FLOW_REGION_INVALID', @@ -305,6 +306,18 @@ export function flowErrorMessage(error: unknown): FlowErrorMessage | null { ], }; } + case 'FLOW_SETTING_REFUSED': { + const fixed = list(field(details, 'fixed')).map(text); + const hidden = list(field(details, 'hidden')).map(text); + return { + title: 'A setting is fixed or hidden by the flow', + lines: [ + "The flow's author fixed or hid these settings, so a run cannot change them. Remove your value and run again.", + ...fixed.map((name) => `${name} (fixed)`), + ...hidden.map((name) => `${name} (hidden)`), + ], + }; + } case 'STORED_SOURCE_REFUSED': { const problems = problemLines(list(field(details, 'problems'))); return { diff --git a/ui-react/src/lib/flow-settings.ts b/ui-react/src/lib/flow-settings.ts new file mode 100644 index 00000000..4d14fc12 --- /dev/null +++ b/ui-react/src/lib/flow-settings.ts @@ -0,0 +1,128 @@ +/** + * What a runner or a flow's author needs to know about a setting of a flow: the piece's + * description of it (a title, an explanation, a type, explained choices, bounds), the value + * a run starts from, and what the author decided (fixed, and whether it is shown, advanced + * or hidden). + * + * The description comes from the piece's JSON Schema through the adapter, and the author's + * decisions from the flow. Every key is optional: an older piece has only a type and a + * description, and a setting without them is described by its name. + */ +import { getModelCatalogApiUrl } from './config'; + +export const VISIBILITIES = ['shown', 'advanced', 'hidden'] as const; +export type Visibility = (typeof VISIBILITIES)[number]; + +export interface SettingChoice { + value: string | number | boolean; + title?: string; + description?: string; +} + +/** The piece's own description of one setting. */ +export interface SettingDetails { + type?: string; + default?: unknown; + description?: string; + title?: string; + choices?: SettingChoice[]; + minimum?: number; + maximum?: number; + /** The piece suggests keeping it out of the way. */ + advanced?: boolean; +} + +/** What the author decided about one setting of the flow. */ +export interface SettingRule { + visibility?: Visibility; + fixed?: boolean; +} + +/** One setting of a stored flow as a runner meets it. */ +export interface FlowSetting extends SettingDetails { + name: string; + /** The flow parameter row the thread binds a value to. */ + parameter_id: string; + visibility: Visibility; + fixed?: boolean; +} + +export interface FlowSettings { + settings: FlowSetting[]; + /** Settings the thread fills in itself (the dates). */ + thread_bound: string[]; +} + +export type SettingInputKind = 'choice' | 'number' | 'boolean' | 'text'; + +/** Which input suits a setting: a dropdown for choices, a number box, a tick box, or text. */ +export function settingInputKind(details: SettingDetails | undefined): SettingInputKind { + if (details?.choices?.length) return 'choice'; + if (details?.type === 'integer' || details?.type === 'number') return 'number'; + if (details?.type === 'boolean') return 'boolean'; + return 'text'; +} + +/** The name a person reads: the piece's title, else the name with its underscores spaced. */ +export function settingTitle(name: string, details?: Pick): string { + return details?.title?.trim() || name.replace(/_/g, ' '); +} + +export const choiceLabel = (choice: SettingChoice): string => choice.title ?? String(choice.value); + +/** The value a setting starts from, as the text an input shows. */ +export function settingText(value: unknown): string { + if (value === undefined || value === null) return ''; + return typeof value === 'string' ? value : JSON.stringify(value); +} + +/** Where a setting belongs in the form, from the author's choice or the piece's hint. */ +export function visibilityOf(rule: SettingRule | undefined, details?: SettingDetails): Visibility { + if (rule?.visibility && VISIBILITIES.includes(rule.visibility)) return rule.visibility; + return details?.advanced ? 'advanced' : 'shown'; +} + +const isRecord = (value: unknown): value is Record => + typeof value === 'object' && value !== null && !Array.isArray(value); + +function isSetting(value: unknown): value is FlowSetting { + return ( + isRecord(value) && + typeof value.name === 'string' && + typeof value.parameter_id === 'string' && + typeof value.visibility === 'string' && + (VISIBILITIES as readonly string[]).includes(value.visibility) + ); +} + +/** + * The adapter's answer for a flow, or null when it cannot be had (no token, no adapter, + * a catalog without the route, a malformed answer). The form then falls back to the flow's + * parameter rows, which carry the defaults, choices and bounds but not titles, explanations + * or the author's visibility. + */ +export async function fetchFlowSettings( + flowId: string, + token: string | null, + signal?: AbortSignal, +): Promise { + const base = getModelCatalogApiUrl(); + if (!base || !token) return null; + try { + const response = await fetch(`${base}/flows/${encodeURIComponent(flowId)}/settings`, { + headers: { Authorization: `Bearer ${token}` }, + signal, + }); + if (!response.ok) return null; + const body: unknown = await response.json(); + if (!isRecord(body) || !Array.isArray(body.settings)) return null; + return { + settings: body.settings.filter(isSetting), + thread_bound: Array.isArray(body.thread_bound) + ? body.thread_bound.filter((name): name is string => typeof name === 'string') + : [], + }; + } catch { + return null; + } +} diff --git a/ui-react/src/lib/spatial-boundaries.ts b/ui-react/src/lib/spatial-boundaries.ts new file mode 100644 index 00000000..90afd971 --- /dev/null +++ b/ui-react/src/lib/spatial-boundaries.ts @@ -0,0 +1,246 @@ +/** + * Boundary picker options for the Framing step's spatial map. + * + * A boundary option is what a person reads in the dropdown: a name, never an + * id. The id stays in `value`. Registered MINT regions that are numbered parts + * of one region (TRINITY_1 .. TRINITY_55) collapse into one family option whose + * `parts` the user can expand. + */ + +/** How a layer labels its features. Comes from the layer config, not the component. */ +export interface BoundaryLabelConfig { + /** Attribute holding the readable name. Defaults to the filter field. */ + label_field?: string; + /** Text before the label, e.g. "GMA " for a layer whose only attribute is a number. */ + label_prefix?: string; + /** Text after a label that is only the filter value, e.g. " County". */ + label_suffix?: string; +} + +export interface BoundaryOption { + /** The filter value saved with the selection. A family joins its parts' values with commas. */ + value: string; + /** Every filter value the option stands for. One for a single boundary. */ + values: string[]; + label: string; + /** Category heading for registered MINT regions. */ + group?: string; + /** Present on a family: the numbered parts, sorted. */ + parts?: BoundaryOption[]; +} + +/** Attributes some layers carry a name in, tried in order when the config names none. */ +const NAME_ATTRIBUTES = [ + 'GMAName', + 'GMA_Name', + 'GMA_NAME', + 'DistrictName', + 'Name', + 'name', + 'region_name', + 'label', +]; + +const collator = new Intl.Collator(undefined, { numeric: true, sensitivity: 'base' }); + +export function compareLabels(left: string, right: string): number { + return collator.compare(left, right); +} + +/** "TRINITY_53" becomes "Trinity 53". Names without an underscore are left as written. */ +export function prettyRegionName(name: string): string { + const trimmed = name.trim(); + if (!trimmed.includes('_')) return trimmed; + const spaced = trimmed.replace(/_+/g, ' '); + if (spaced !== spaced.toUpperCase()) return spaced; + return spaced.toLowerCase().replace(/(^|[\s-])([a-z])/g, (_m, lead: string, ch: string) => { + return `${lead}${ch.toUpperCase()}`; + }); +} + +/** + * The readable label for a feature. A named attribute wins; otherwise the + * filter value itself, wrapped in the layer's prefix and suffix. When the layer + * also has a name the label is "prefix value — name". + */ +export function boundaryFeatureLabel( + properties: Record | null | undefined, + filterField: string, + config: BoundaryLabelConfig = {}, +): string { + const props = properties ?? {}; + const raw = props[filterField]; + const value = raw == null ? '' : String(raw).trim(); + const prefix = config.label_prefix ?? ''; + const suffix = config.label_suffix ?? ''; + + const nameFields = config.label_field ? [config.label_field] : NAME_ATTRIBUTES; + let name = ''; + for (const field of nameFields) { + const candidate = props[field]; + if (candidate != null && String(candidate).trim() !== '') { + name = String(candidate).trim(); + break; + } + } + + if (name && name !== value) return `${prefix}${value} — ${name}`; + return `${prefix}${value}${suffix}`; +} + +/** + * The family a registered region belongs to, from its name: "TRINITY_53" and + * "Trinity 7" are both parts of the "trinity" family. A name with no trailing + * number, or only a number, is not part of a family. + */ +export function regionFamilyOf(name: string): { key: string; label: string } | null { + const match = /^(.*[^\s_-])[\s_-]+\d+$/.exec(name.trim()); + if (!match?.[1]) return null; + const pretty = /^(.*[^\s-])\s+\d+$/.exec(prettyRegionName(name)); + return { + key: match[1].toLowerCase().replace(/[\s_]+/g, ' '), + label: pretty?.[1] ?? prettyRegionName(match[1]), + }; +} + +interface RegionFeatureProps { + region_name?: unknown; + category_name?: unknown; + category_id?: unknown; +} + +/** + * Options for a registered-region layer. The name is the label and the + * category is the group. Numbered parts of one family (two or more in a + * category) collapse to one family option. + */ +export function buildRegionOptions( + features: GeoJSON.Feature[], + filterField: string, +): BoundaryOption[] { + const singles = new Map(); + for (const feature of features) { + const props = (feature.properties ?? {}) as Record & RegionFeatureProps; + const raw = props[filterField]; + if (raw == null || String(raw).trim() === '') continue; + const value = String(raw); + if (singles.has(value)) continue; + const rawName = props.region_name == null ? '' : String(props.region_name); + const family = regionFamilyOf(rawName); + const group = + props.category_name != null + ? String(props.category_name) + : props.category_id != null + ? String(props.category_id) + : undefined; + singles.set(value, { + value, + values: [value], + label: rawName ? prettyRegionName(rawName) : value, + group, + familyKey: family ? `${group ?? ''}|${family.key}` : undefined, + familyLabel: family?.label, + }); + } + + const byFamily = new Map>(); + for (const option of singles.values()) { + if (!option.familyKey) continue; + const members = byFamily.get(option.familyKey) ?? []; + members.push(option); + byFamily.set(option.familyKey, members); + } + + const result: BoundaryOption[] = []; + for (const option of singles.values()) { + const members = option.familyKey ? byFamily.get(option.familyKey) : undefined; + if (!members || members.length < 2) { + result.push(stripFamily(option)); + continue; + } + // Emit the family once, at its first member. + if (members[0] !== option) continue; + const parts = members.map(stripFamily).sort((a, b) => compareLabels(a.label, b.label)); + const values = parts.flatMap((part) => part.values); + result.push({ + value: values.join(','), + values, + label: `${option.familyLabel ?? option.label} (${parts.length} parts)`, + group: option.group, + parts, + }); + } + return sortBoundaryOptions(result); +} + +function stripFamily(option: BoundaryOption & { familyKey?: string; familyLabel?: string }) { + const { familyKey: _key, familyLabel: _label, ...rest } = option; + void _key; + void _label; + return rest; +} + +/** Options for a feature-service layer, labelled from the layer config. */ +export function buildLayerOptions( + features: GeoJSON.Feature[], + filterField: string, + config: BoundaryLabelConfig = {}, +): BoundaryOption[] { + const seen = new Set(); + const options: BoundaryOption[] = []; + for (const feature of features) { + const raw = feature.properties?.[filterField]; + if (raw == null || String(raw).trim() === '') continue; + const value = String(raw); + if (seen.has(value)) continue; + seen.add(value); + options.push({ + value, + values: [value], + label: boundaryFeatureLabel(feature.properties, filterField, config), + }); + } + return sortBoundaryOptions(options); +} + +/** Sort by group, then by name. Parts of a family sort the same way. */ +export function sortBoundaryOptions(options: BoundaryOption[]): BoundaryOption[] { + return [...options].sort( + (a, b) => compareLabels(a.group ?? '', b.group ?? '') || compareLabels(a.label, b.label), + ); +} + +/** The distinct groups, sorted. */ +export function boundaryGroups(options: BoundaryOption[]): string[] { + return [...new Set(options.flatMap((o) => (o.group ? [o.group] : [])))].sort(compareLabels); +} + +/** Every option including family parts, flat. */ +export function flattenBoundaryOptions(options: BoundaryOption[]): BoundaryOption[] { + return options.flatMap((option) => [option, ...(option.parts ?? [])]); +} + +/** + * Filter by what the user typed. A family stays when its own label or any part + * matches; when a part matched, only the matching parts are kept so the user + * sees where the match was. + */ +export function filterBoundaryOptions(options: BoundaryOption[], query: string): BoundaryOption[] { + const needle = query.trim().toLowerCase(); + if (!needle) return options; + const matches = (label: string) => label.toLowerCase().includes(needle); + return options.flatMap((option) => { + if (!option.parts) return matches(option.label) ? [option] : []; + if (matches(option.label)) return [option]; + const parts = option.parts.filter((part) => matches(part.label)); + return parts.length > 0 ? [{ ...option, parts }] : []; + }); +} + +/** The option for a saved filter value: a single boundary, a part, or a whole family. */ +export function findBoundaryOption( + options: BoundaryOption[], + value: string, +): BoundaryOption | undefined { + return flattenBoundaryOptions(options).find((option) => option.value === value); +} diff --git a/ui-react/src/lib/standard-variable-display.ts b/ui-react/src/lib/standard-variable-display.ts index 40ec48a5..de397196 100644 --- a/ui-react/src/lib/standard-variable-display.ts +++ b/ui-react/src/lib/standard-variable-display.ts @@ -1,3 +1,5 @@ +import { humanizeStandardVariable } from '@/lib/standard-variable-grammar'; + /** * How a standard variable reads in a list. * @@ -41,3 +43,42 @@ export function standardVariableDescription(sv: VariableLike): string | null { const text = (sv.description ?? '').trim(); return text && !text.startsWith(CATALOG_ID_PREFIX) ? text : null; } + +/** + * The title an option leads with: the label, humanized ("Document lexicon group + * — Term count" from `document_lexicon_group__term_count`). A row with no label + * borrows the id's tail when that is grammar-shaped, else reads as unnamed. + */ +export function standardVariableTitle(sv: VariableLike): string { + const source = isUnlabeledStandardVariable(sv) ? idTail(sv.id) : sv.label!.trim(); + if (isUnlabeledStandardVariable(sv) && !source.includes('__')) return standardVariableName(sv); + const { phenomenon, property } = humanizeStandardVariable(source); + if (phenomenon) return `${phenomenon} — ${property}`; + // Not grammar (no `__`): the label is the whole name, so only spell out its joiners. + const spaced = property.replace(/[~_]+/g, ' ').replace(/\s+/g, ' ').trim(); + return spaced.charAt(0).toUpperCase() + spaced.slice(1); +} + +/** A key that makes two rows naming the same variable collide; null for unlabeled rows. */ +export function standardVariableLabelKey(sv: VariableLike): string | null { + return isUnlabeledStandardVariable(sv) ? null : sv.label!.trim().toLowerCase(); +} + +/** Drop repeats by id, then (for labeled rows) by label, keeping the first seen. */ +export function dedupeStandardVariables( + rows: ReadonlyArray, + { byLabel = false }: { byLabel?: boolean } = {}, +): T[] { + const ids = new Set(); + const labels = new Set(); + const kept: T[] = []; + for (const row of rows) { + if (ids.has(row.id)) continue; + const key = byLabel ? standardVariableLabelKey(row) : null; + if (key !== null && labels.has(key)) continue; + ids.add(row.id); + if (key !== null) labels.add(key); + kept.push(row); + } + return kept; +} diff --git a/ui-react/src/pages/datasets/DatasetUploadPage.tsx b/ui-react/src/pages/datasets/DatasetUploadPage.tsx index 4c2a8bc7..944b56cd 100644 --- a/ui-react/src/pages/datasets/DatasetUploadPage.tsx +++ b/ui-react/src/pages/datasets/DatasetUploadPage.tsx @@ -35,7 +35,13 @@ import { } from '@/lib/ckan-upload'; import { useDebouncedValue } from '@/hooks/useDebouncedValue'; import { useSemanticSearch, useSimilarVariables } from '@/hooks/useSemanticSearch'; -import { closeConsumers, consumersOfVariable } from '@/lib/datasets/upload-preview'; +import { + closeConsumers, + consumersOfVariable, + guidedSetupLink, + rankSuggestionsByUse, + suggestionUseText, +} from '@/lib/datasets/upload-preview'; import { unionBoundingBox, type BoundingBox } from '@/lib/geo/bbox'; import { standardVariableName } from '@/lib/standard-variable-display'; import { useAuth } from '@/lib/auth/useAuth'; @@ -177,12 +183,21 @@ export function DatasetUploadPage() { // Standard variables suggested from what the user wrote. Never chosen for them. const aboutText = useDebouncedValue(`${title}\n${description}`.trim(), SUGGEST_DELAY_MS); const suggested = useSemanticSearch(aboutText, { target: 'svo', limit: 20, minLength: 4 }); + // Ranked by whether a registered flow or model takes these files under the + // variable, then by how close it is to the text. const suggestions = useMemo( - () => (suggested.results ?? []).filter((r) => r.label?.trim()), - [suggested.results], + () => + rankSuggestionsByUse( + tree, + (suggested.results ?? []).filter((r) => r.label?.trim()), + files, + ), + [tree, suggested.results, files], ); - const suggestionIds = useMemo(() => suggestions.map((r) => r.id), [suggestions]); - const shownSuggestions = suggestions.filter((r) => r.id !== variable?.id).slice(0, SUGGEST_SHOWN); + const suggestionIds = useMemo(() => suggestions.map((r) => r.item.id), [suggestions]); + const shownSuggestions = suggestions + .filter((r) => r.item.id !== variable?.id) + .slice(0, SUGGEST_SHOWN); const similar = useSimilarVariables(variable?.id, { limit: 5 }); const closeMatches = useMemo( @@ -304,7 +319,20 @@ export function DatasetUploadPage() { - - ))} + {shownSuggestions.map((ranked) => { + const s = ranked.item; + return ( +
  • + + {s.label} + {s.description && ( + + {s.description} + + )} + + {suggestionUseText(ranked)} + + + +
  • + ); + })}
    )} diff --git a/ui-react/src/pages/datasets/__tests__/DatasetUploadPage.test.tsx b/ui-react/src/pages/datasets/__tests__/DatasetUploadPage.test.tsx index efeb8926..aa391f6b 100644 --- a/ui-react/src/pages/datasets/__tests__/DatasetUploadPage.test.tsx +++ b/ui-react/src/pages/datasets/__tests__/DatasetUploadPage.test.tsx @@ -409,10 +409,9 @@ describe('DatasetUploadPage', () => { expect(sent).toHaveLength(1); expect(screen.getByText('dynamo-upload-x')).toBeInTheDocument(); expect(screen.getByTestId('upload-state')).toHaveTextContent('Private'); - expect(screen.getByRole('link', { name: 'Use in a thread' })).toHaveAttribute( - 'href', - '/modeling/problem-statements', - ); + const href = screen.getByRole('link', { name: 'Use in a thread' }).getAttribute('href')!; + expect(href.startsWith('/modeling/problem-statements/start?')).toBe(true); + expect(new URLSearchParams(href.split('?')[1]).get('datasetId')).toBe('dynamo-upload-x'); expect(screen.getByRole('link', { name: /dataset page/ })).toHaveAttribute( 'href', '/datasets/browse/dynamo-upload-x', @@ -493,6 +492,28 @@ describe('DatasetUploadPage', () => { expect(screen.getByTestId('who-can-use')).toHaveTextContent('Rain flow'); }); + it('"Use in a thread" opens guided setup with the dataset, exact models and close matches', async () => { + semantic = (u) => + u.includes('/similar?') + ? json({ target: 'svo', id: 'sv-1', results: [{ id: 'sv-2', label: 'rainfall_amount' }] }) + : json({ results: [] }); + await fill([csv('only.csv')]); + await screen.findByTestId('close-matches'); + await userEvent.click(screen.getByRole('button', { name: 'Upload' })); + await screen.findByTestId('upload-success'); + const href = screen.getByRole('link', { name: 'Use in a thread' }).getAttribute('href')!; + expect(href.startsWith('/modeling/problem-statements/start?')).toBe(true); + const params = new URLSearchParams(href.split('?')[1]); + expect(params.get('datasetId')).toBe('dynamo-upload-x'); + expect(params.get('modelIds')?.split(',')).toEqual(['flow-1', 'model-1']); + expect(params.get('closeModelIds')).toBe('flow-2'); + expect(params.get('variable')).toBe('precipitation_flux'); + expect(params.get('goal')).toBe('Rain 2020'); + expect(params.get('startDate')).toBe('2020-01-01'); + expect(params.get('endDate')).toBe('2020-12-31'); + expect(JSON.parse(params.get('modelNames')!)).toMatchObject({ 'flow-2': 'Wet flow' }); + }); + it('hides the suggestion sections quietly when the service is down', async () => { semantic = () => json({ error: 'down' }, 503); renderPage(); diff --git a/ui-react/src/pages/modeling/GuidedModelSetup.tsx b/ui-react/src/pages/modeling/GuidedModelSetup.tsx index df63e816..f74a7e2a 100644 --- a/ui-react/src/pages/modeling/GuidedModelSetup.tsx +++ b/ui-react/src/pages/modeling/GuidedModelSetup.tsx @@ -85,6 +85,16 @@ function initialIds(value: string | null, searchValue: string | null): string[] return [...new Set(values)]; } +function modelNamesParam(value: string | null): Record { + if (!value) return {}; + try { + const parsed: unknown = JSON.parse(value); + return parsed && typeof parsed === 'object' ? (parsed as Record) : {}; + } catch { + return {}; + } +} + function dateInputValue(value: Date | null | undefined): string { if (!value) return ''; const date = value instanceof Date ? value : new Date(value); @@ -152,11 +162,29 @@ export function GuidedModelSetup({ const [setThreadModels] = useSetThreadModelsMutation(); const [updateThread] = useUpdateThreadMutation(); - const [goal, setGoal] = useState(''); + // An upload hands over what it already knows. These only seed the fields; + // the user still edits them and confirms. + const uploadedVariable = searchParams.get('variable'); + const closeModelIds = useMemo( + () => initialIds(null, searchParams.get('closeModelIds')), + [searchParams], + ); + const modelNames = useMemo(() => modelNamesParam(searchParams.get('modelNames')), [searchParams]); + const recommendedForData = uploadedVariable + ? startingModelIds.filter((id) => !closeModelIds.includes(id)) + : []; + const contextModelIds = startingModelIds.filter( + (id) => !recommendedForData.includes(id) && !closeModelIds.includes(id), + ); + const nameOf = (id: string) => displayLabel(id, modelNames[id]); + + const [goal, setGoal] = useState(() => searchParams.get('goal') ?? ''); const [outcome, setOutcome] = useState(''); - const [regionId, setRegionId] = useState(''); - const [startDate, setStartDate] = useState('2000-01-01'); - const [endDate, setEndDate] = useState(new Date().toISOString().slice(0, 10)); + const [regionId, setRegionId] = useState(() => searchParams.get('regionId') ?? ''); + const [startDate, setStartDate] = useState(() => searchParams.get('startDate') || '2000-01-01'); + const [endDate, setEndDate] = useState( + () => searchParams.get('endDate') || new Date().toISOString().slice(0, 10), + ); const [selectedDatasetMetadata, setSelectedDatasetMetadata] = useState([]); const [destination, setDestination] = useState<'new' | 'existing'>('new'); const [existingProblemStatementId, setExistingProblemStatementId] = useState(''); @@ -444,11 +472,11 @@ export function GuidedModelSetup({ - {(modelId || datasetId) && ( + {(startingModelIds.length > 0 || startingDatasetIds.length > 0) && (
    Starting context
    - {startingModelIds.map((id) => ( + {contextModelIds.map((id) => ( + ) : ( + + )} + choose(option)} + > + + {option.label} + +
    + {isExpanded && + option.parts?.map((part) => ( +
    + choose(part)}> + + {part.label} + +
    + ))} + + ); + }; + + const ungrouped = visible.filter((option) => !option.group); + + return ( + { + setOpen(next); + if (!next) setQuery(''); + }} + > + + + + + + + + No matching boundaries. + {query.trim() === '' && ( + + choose(null)}> + + All boundaries + + + )} + {ungrouped.length > 0 && {ungrouped.map(renderOption)}} + {groups.map((group) => ( + + {visible.filter((option) => option.group === group).map(renderOption)} + + ))} + + + + + ); +} diff --git a/ui-react/src/pages/modeling/thread/MintParameters.tsx b/ui-react/src/pages/modeling/thread/MintParameters.tsx index 0776f463..015464c3 100644 --- a/ui-react/src/pages/modeling/thread/MintParameters.tsx +++ b/ui-react/src/pages/modeling/thread/MintParameters.tsx @@ -31,6 +31,15 @@ import { type ThreadAdapterPlan, } from '@/lib/adapter-execution'; import type { UnifiedParameterDefinition } from '@/lib/ensemble-manager'; +import { SettingInput } from '@/components/common/SettingInput'; +import { useFlowSettings } from '@/hooks/useFlowSettings'; +import { + settingText, + settingTitle, + type FlowSettings, + type SettingDetails, + type Visibility, +} from '@/lib/flow-settings'; import type { SpatialSelection } from './SpatialScopeMap'; // ─── Constants ───────────────────────────────────────────────────────────────── @@ -104,6 +113,51 @@ function spatialParameterOverride( } } +/** What a flow's setting is, for the form: how to describe it, and who sees it. */ +interface FlowSettingView { + details: SettingDetails; + visibility: Visibility; + fixed: boolean; + /** The value a run starts from, as text. Empty when nothing gives one. */ + defaultText: string; + /** The text of the fixed value, when the author fixed it. */ + fixedText: string; +} + +/** + * A flow setting as the form shows it. The adapter's description (titles, explained + * choices, the author's visibility and fixed flag) is used when it was read; without it + * the flow's parameter row still carries the type, default, choices and bounds, so a + * setting is never less than a text box. + */ +function flowSettingView(parameter: ModelParameter, settings: FlowSettings | undefined) { + const found = settings?.settings.find((setting) => setting.parameter_id === parameter.id); + const fromRow: SettingDetails = { + ...(parameter.type ? { type: parameter.type } : {}), + ...(parameter.description ? { description: parameter.description } : {}), + ...(parameter.default != null ? { default: parameter.default } : {}), + ...(parameter.accepted_values?.length + ? { choices: parameter.accepted_values.map((value) => ({ value })) } + : {}), + ...(parameter.min != null && Number.isFinite(Number(parameter.min)) + ? { minimum: Number(parameter.min) } + : {}), + ...(parameter.max != null && Number.isFinite(Number(parameter.max)) + ? { maximum: Number(parameter.max) } + : {}), + }; + const details: SettingDetails = found ? { ...fromRow, ...found } : fromRow; + const fixedText = parameter.value ?? (found?.fixed ? settingText(found.default) : ''); + const view: FlowSettingView = { + details, + visibility: found?.visibility ?? 'shown', + fixed: Boolean(parameter.value) || found?.fixed === true, + defaultText: settingText(details.default), + fixedText, + }; + return view; +} + // ─── Props ──────────────────────────────────────────────────────────────────── interface MintParametersProps { @@ -143,6 +197,7 @@ export function MintParameters({ }: MintParametersProps) { const modelIds = Object.keys(threadData.models ?? {}); const isConfigured = modelIds.length > 0; + const flowSettings = useFlowSettings(modelIds.filter((id) => threadData.models[id]?.isFlow)); // Shared with the wizard rail. Note it also requires an execution summary: a // model with no adjustable parameters would otherwise read as done before the @@ -163,11 +218,11 @@ export function MintParameters({ if (!param.accepted_values.includes(v)) { return `Accepted values: ${param.accepted_values.join(', ')}`; } - } else if (param.type === 'int') { + } else if (param.type === 'int' || param.type === 'integer') { const n = parseInt(v, 10); if (param.min != null && n < parseInt(param.min, 10)) return `Min is ${param.min}`; if (param.max != null && n > parseInt(param.max, 10)) return `Max is ${param.max}`; - } else if (param.type === 'float') { + } else if (param.type === 'float' || param.type === 'number') { const n = parseFloat(v); if (param.min != null && n < parseFloat(param.min)) return `Min is ${param.min}`; if (param.max != null && n > parseFloat(param.max)) return `Max is ${param.max}`; @@ -176,15 +231,60 @@ export function MintParameters({ return null; } + // ─ Which of a model's parameters are fixed, hidden, or the runner's to set ────── + // A flow's author may fix a setting (it shows as read-only) or hide it (it does not + // show). The adapter refuses a runner's value for either, so a value bound earlier for + // one that has since been fixed or hidden is dropped on save rather than sent. + function partitionParameters(mid: string, model: ThreadExecutionData['models'][string]) { + const rows = userParameters(model).sort((a, b) => { + if (a.position != null && b.position != null) return a.position - b.position; + return (a.name ?? '').localeCompare(b.name ?? ''); + }); + if (!model.isFlow) { + return { + fixed: rows.filter((p) => !!p.value), + adjustable: rows.filter((p) => !p.value), + withheld: [] as ModelParameter[], + }; + } + const viewOf = (p: ModelParameter) => flowSettingView(p, flowSettings[mid]); + const hidden = rows.filter((p) => viewOf(p).visibility === 'hidden'); + const fixed = rows.filter((p) => !hidden.includes(p) && viewOf(p).fixed); + return { + fixed, + adjustable: rows.filter((p) => !hidden.includes(p) && !fixed.includes(p)), + withheld: [...hidden, ...fixed], + }; + } + // ─ Parse values from the form for a parameter ───────────────────────────── - function getParamValues(modelId: string, param: ModelParameter, isFlow = false): string[] { + function getParamValues( + modelId: string, + param: ModelParameter, + isFlow = false, + flowDefault = '', + ): string[] { const form = formRefs.current[modelId]; if (!form) return []; - const el = form.elements.namedItem(param.id ?? '') as HTMLInputElement | null; + const el = form.elements.namedItem(param.id ?? '') as + | HTMLInputElement + | HTMLSelectElement + | null; + if (el instanceof HTMLSelectElement && el.multiple) { + // A model parameter with accepted values: every selected value is a run. + return Array.from(el.selectedOptions).map((option) => option.value); + } + if (isFlow) { + // A flow's setting is one value: a comma belongs to the value (a title, a + // list the flow parses itself), it does not start a sweep. A value left at the + // flow's default is not bound: the run then uses the default as the author's, + // which the runner accepts before it starts. + const checkbox = el instanceof HTMLInputElement && el.type === 'checkbox'; + const text = checkbox ? String(el.checked) : (el?.value ?? '').trim(); + if (checkbox && !el.checked && flowDefault === '') return []; + return text && text !== flowDefault ? [text] : []; + } const raw = el?.value ?? param.default ?? ''; - // A flow's setting is one value: a comma belongs to the value (a title, a - // list the flow parses itself), it does not start a sweep. - if (isFlow) return raw.trim() ? [raw.trim()] : []; return raw .split(/\s*,\s*/) .map((s) => s.trim()) @@ -255,8 +355,9 @@ export function MintParameters({ for (const mid of modelIds) { const model = threadData.models[mid]!; - const adjustable = userParameters(model).filter((p) => !p.value); + const { adjustable, withheld } = partitionParameters(mid, model); const bindings = newEnsembles[mid]?.bindings ?? {}; + for (const p of withheld) delete bindings[p.id ?? '']; const adapterPlans = threadData.adapter_plans?.[mid] ?? []; const adapterParameters = new Map(); for (const plan of adapterPlans) { @@ -312,7 +413,12 @@ export function MintParameters({ // If not in edit mode and already bound, keep existing if (!editMode && current && current.length > 0) continue; - const vals = getParamValues(mid, param, model.isFlow); + const vals = getParamValues( + mid, + param, + model.isFlow, + model.isFlow ? flowSettingView(param, flowSettings[mid]).defaultText : '', + ); const err = validateValues(param, vals); if (err) { newErrors[`${mid}::${param.id}`] = err; @@ -426,19 +532,16 @@ export function MintParameters({ const model = threadData.models[mid]!; const ensemble = threadData.model_ensembles[mid] ?? { id: '', bindings: {} }; - const fixedParams = userParameters(model) - .filter((p) => !!p.value) - .sort((a, b) => { - if (a.position != null && b.position != null) return a.position - b.position; - return (a.name ?? '').localeCompare(b.name ?? ''); - }); - - const adjustableParams = userParameters(model) - .filter((p) => !p.value) - .sort((a, b) => { - if (a.position != null && b.position != null) return a.position - b.position; - return (a.name ?? '').localeCompare(b.name ?? ''); - }); + const { fixed: fixedParams, adjustable: adjustableParams } = partitionParameters( + mid, + model, + ); + const flowView = (p: ModelParameter) => flowSettingView(p, flowSettings[mid]); + // The flow's author may keep rarely changed settings out of the way. + const advancedParams = model.isFlow + ? adjustableParams.filter((p) => flowView(p).visibility === 'advanced') + : []; + const shownParams = adjustableParams.filter((p) => !advancedParams.includes(p)); const adapterPlans = threadData.adapter_plans?.[mid] ?? []; const adapterParams = new Map< @@ -454,6 +557,118 @@ export function MintParameters({ } } + const renderRow = (p: ModelParameter) => { + const bindings = ensemble.bindings[p.id ?? ''] ?? []; + const displayValue = bindings.join(', '); + const isRegionGeoJson = p.default === '__region_geojson'; + const errKey = `${mid}::${p.id}`; + const view = model.isFlow ? flowView(p) : null; + // A model parameter with accepted values is picked from them. Picking + // several is several runs, as comma separated values are in the text box. + const accepted = + !model.isFlow && p.accepted_values?.length ? p.accepted_values : null; + const min = view ? view.details.minimum : p.min; + const max = view ? view.details.maximum : p.max; + // What is chosen already, or the default. A value the list does not hold + // (an old binding, say) stays visible and selected, flagged, so the + // validation names it instead of the save quietly losing it. + const selected = accepted + ? bindings.length > 0 + ? bindings + : (p.default ?? '').split(/\s*,\s*/).filter(Boolean) + : []; + const notAccepted = accepted + ? [...new Set(selected.filter((value) => !accepted.includes(value)))] + : []; + + return ( + + +
    + {view + ? settingTitle(p.name ?? '', view.details) + : (p.name ?? '').replace(/_/g, ' ')} +
    + {(view?.details.description ?? p.description) && ( +
    + {view?.details.description ?? p.description} +
    + )} + {(min != null || max != null) && ( +
    + {min != null && max != null + ? `Range: ${min} – ${max}.` + : min != null + ? `Min: ${min}.` + : `Max: ${max}.`} + {p.default && ` Default: ${p.default}`} +
    + )} + + + {!editMode ? ( + {displayValue || view?.defaultText || '—'} + ) : isRegionGeoJson ? ( + Region GeoJSON + ) : ( +
    + {accepted ? ( + <> + +
    + Hold Ctrl or Cmd to pick more than one: each value is a run. +
    + + ) : view ? ( + + ) : ( + + )} + {errors[errKey] && ( +
    + {errors[errKey]} +
    + )} +
    + )} + + + ); + }; + return (
  • @@ -490,14 +705,20 @@ export function MintParameters({
    - {(p.name ?? '').replace(/_/g, ' ')} + {model.isFlow + ? settingTitle(p.name ?? '', flowView(p).details) + : (p.name ?? '').replace(/_/g, ' ')}
    {p.description && (
    {p.description}
    )} - {p.default === '__region_geojson' ? 'Region GeoJSON' : p.value} + {p.default === '__region_geojson' + ? 'Region GeoJSON' + : model.isFlow + ? flowView(p).fixedText + : p.value} ))} @@ -534,64 +755,18 @@ export function MintParameters({ Values - - {adjustableParams.map((p) => { - const bindings = ensemble.bindings[p.id ?? ''] ?? []; - const displayValue = bindings.join(', '); - const isRegionGeoJson = p.default === '__region_geojson'; - const errKey = `${mid}::${p.id}`; - - return ( - - -
    - {(p.name ?? '').replace(/_/g, ' ')} -
    - {p.description && ( -
    {p.description}
    - )} - {(p.min != null || p.max != null) && ( -
    - {p.min != null && p.max != null - ? `Range: ${p.min} – ${p.max}.` - : p.min != null - ? `Min: ${p.min}.` - : `Max: ${p.max}.`} - {p.default && ` Default: ${p.default}`} -
    - )} - - - {!editMode ? ( - {displayValue || '—'} - ) : isRegionGeoJson ? ( - Region GeoJSON - ) : ( -
    - - {errors[errKey] && ( -
    - {errors[errKey]} -
    - )} -
    - )} - - - ); - })} - + {shownParams.map(renderRow)} + {advancedParams.length > 0 && ( +
    + + Advanced settings ({advancedParams.length}) + + + {advancedParams.map(renderRow)} +
    +
    + )}

  • ) : ( diff --git a/ui-react/src/pages/modeling/thread/SpatialScopeMap.tsx b/ui-react/src/pages/modeling/thread/SpatialScopeMap.tsx index c35cc070..70cf3b5d 100644 --- a/ui-react/src/pages/modeling/thread/SpatialScopeMap.tsx +++ b/ui-react/src/pages/modeling/thread/SpatialScopeMap.tsx @@ -8,8 +8,17 @@ import 'leaflet/dist/leaflet.css'; import { parseGeometry } from '@/pages/regions/regionUtils'; import { LIST_REGISTERED_BOUNDARY_REGIONS } from '@/graphql/queries/regions'; import { fetchSpatialLayers, spatialLayersAvailable } from '@/lib/spatial-layers'; +import { + boundaryGroups, + buildLayerOptions, + buildRegionOptions, + findBoundaryOption, + type BoundaryLabelConfig, + type BoundaryOption, +} from '@/lib/spatial-boundaries'; +import { BoundaryCombobox } from './BoundaryCombobox'; -interface SpatialLayerCatalogEntry { +interface SpatialLayerCatalogEntry extends BoundaryLabelConfig { id: string; label: string; uri: string; @@ -85,6 +94,9 @@ type RegisteredBoundaryRegionsData = { // pulling the much larger administrative level datasets into every map. const REGISTERED_BOUNDARY_CATEGORY_IDS = ['hydrology']; +/** Sentinel for the category select's "All categories" entry. */ +const ALL_CATEGORIES = '__all__'; + /** Cartographic preview of the spatial scope carried by the problem statement. */ export function SpatialScopeMap({ geometries = [], @@ -103,6 +115,7 @@ export function SpatialScopeMap({ const [selectedLayerId, setSelectedLayerId] = useState(''); const [layerFeatures, setLayerFeatures] = useState([]); const [selectedFeatureValue, setSelectedFeatureValue] = useState(''); + const [categoryChoice, setCategoryChoice] = useState(''); const [layerLoading, setLayerLoading] = useState(false); const [layerError, setLayerError] = useState(null); @@ -187,42 +200,49 @@ export function SpatialScopeMap({ const activeFilterField = filterField?.trim() || selectedLayer?.default_filter_field; - const featureOptions = useMemo(() => { + const isRegionLayer = selectedLayer?.source_type === 'mint-regions'; + const allOptions = useMemo(() => { if (!activeFilterField) return []; - const options = layerFeatures.flatMap((feature) => { - const rawValue = feature.properties?.[activeFilterField]; - if (rawValue == null || String(rawValue).trim() === '') return []; - const value = String(rawValue); - const name = - feature.properties?.GMAName ?? - feature.properties?.GMA_Name ?? - feature.properties?.GMA_NAME ?? - feature.properties?.Name ?? - feature.properties?.name ?? - feature.properties?.region_name ?? - feature.properties?.label ?? - ''; - return [ - { - value, - label: name ? `${value} — ${String(name)}` : value, - }, - ]; - }); - return options - .filter( - (option, index) => - options.findIndex((candidate) => candidate.value === option.value) === index, - ) - .sort((left, right) => left.label.localeCompare(right.label, undefined, { numeric: true })); - }, [activeFilterField, layerFeatures]); + return isRegionLayer + ? buildRegionOptions(layerFeatures, activeFilterField) + : buildLayerOptions(layerFeatures, activeFilterField, selectedLayer); + }, [activeFilterField, isRegionLayer, layerFeatures, selectedLayer]); + + // Registered regions span several categories. Show one at a time, starting + // with the saved selection's category, so the map is not a pile of overlaps. + const categories = useMemo(() => boundaryGroups(allOptions), [allOptions]); + const savedOption = findBoundaryOption( + allOptions, + selectedFeatureValue || filterValue?.trim() || '', + ); + const activeCategory = + categories.length > 1 && categoryChoice !== ALL_CATEGORIES + ? categories.includes(categoryChoice) + ? categoryChoice + : (savedOption?.group ?? categories[0] ?? '') + : ''; + const featureOptions = useMemo( + () => (activeCategory ? allOptions.filter((o) => o.group === activeCategory) : allOptions), + [activeCategory, allOptions], + ); + const visibleFeatures = useMemo( + () => + activeCategory + ? layerFeatures.filter((f) => f.properties?.category_name === activeCategory) + : layerFeatures, + [activeCategory, layerFeatures], + ); + const selectedOption = findBoundaryOption(allOptions, selectedFeatureValue); + const selectedValues = useMemo(() => new Set(selectedOption?.values ?? []), [selectedOption]); + const isSelectedFeature = (feature: GeoJSON.Feature) => + selectedValues.has(String(feature.properties?.[activeFilterField ?? ''])); useEffect(() => { const nextValue = filterValue?.trim() ?? ''; setSelectedFeatureValue( - nextValue && featureOptions.some((option) => option.value === nextValue) ? nextValue : '', + nextValue && findBoundaryOption(allOptions, nextValue) ? nextValue : '', ); - }, [featureOptions, filterValue]); + }, [allOptions, filterValue]); useEffect(() => { if (!selectedLayer) { @@ -249,6 +269,8 @@ export function SpatialScopeMap({ // (for example GMAnum=12) for ETL arguments. region_id: region.id.split('__').pop() ?? region.id, region_name: region.name, + category_id: region.category_id, + category_name: region.region_category?.name ?? region.category_id, }, geometry: parsed, } as GeoJSON.Feature, @@ -318,20 +340,20 @@ export function SpatialScopeMap({ () => ({ type: 'FeatureCollection', features: [ - ...layerFeatures.map((feature) => ({ + ...visibleFeatures.map((feature) => ({ ...feature, properties: { ...(feature.properties ?? {}), catalogLayer: true, selectedBoundary: - Boolean(selectedFeatureValue) && - String(feature.properties?.[activeFilterField ?? '']) === selectedFeatureValue, + selectedValues.size > 0 && + selectedValues.has(String(feature.properties?.[activeFilterField ?? ''])), }, })), ...selectedScopeFeatures, ], }), - [activeFilterField, layerFeatures, selectedFeatureValue, selectedScopeFeatures], + [activeFilterField, selectedValues, selectedScopeFeatures, visibleFeatures], ); const sourceIsCustom = Boolean( @@ -341,11 +363,8 @@ export function SpatialScopeMap({ ), ); const activeLayerLabel = sourceIsCustom ? 'Custom geometry source' : selectedLayer?.label; - const focusedFeatures = selectedFeatureValue - ? layerFeatures.filter( - (feature) => String(feature.properties?.[activeFilterField ?? '']) === selectedFeatureValue, - ) - : []; + const focusedFeatures = selectedFeatureValue ? visibleFeatures.filter(isSelectedFeature) : []; + const hasSelection = selectedValues.size > 0; const style = (feature?: GeoJSON.Feature): PathOptions => feature?.properties?.selectedBoundary @@ -363,12 +382,15 @@ export function SpatialScopeMap({ fillOpacity: 0, interactive: false, } - : { - color: '#1d4ed8', - weight: 1.5, - fillColor: '#3b82f6', - fillOpacity: 0.12, - }; + : hasSelection + ? // Everything but the selection is faint, so the choice stands out. + { color: '#94a3b8', weight: 1, fillColor: '#94a3b8', fillOpacity: 0.04 } + : { + color: '#1d4ed8', + weight: 1.5, + fillColor: '#3b82f6', + fillOpacity: 0.08, + }; return (
    @@ -386,6 +408,7 @@ export function SpatialScopeMap({ const nextId = event.target.value; setSelectedLayerId(nextId); setSelectedFeatureValue(''); + setCategoryChoice(''); const nextLayer = layers.find((layer) => layer.id === nextId); if (nextLayer) onLayerSelect?.(nextLayer); }} @@ -397,41 +420,50 @@ export function SpatialScopeMap({ ))} - {featureOptions.length > 0 && ( + {categories.length > 1 && ( )} + {allOptions.length > 0 && ( +
    + Boundary + { + setSelectedFeatureValue(option?.value ?? ''); + if (!option || !activeFilterField) return; + onFeatureSelect?.({ + filterField: activeFilterField, + filterValue: option.value, + label: option.label, + layer: selectedLayer, + geometry: boundaryGeometry(layerFeatures, activeFilterField, option), + }); + }} + /> +
    + )}
    )}
    @@ -457,7 +489,7 @@ export function SpatialScopeMap({ const option = featureValue == null ? undefined - : featureOptions.find((item) => item.value === String(featureValue)); + : findBoundaryOption(allOptions, String(featureValue)); const scopeName = option?.label ?? (featureValue == null ? activeLayerLabel : String(featureValue)); if (scopeName && (!isStoredScope || layerFeatures.length === 0)) { @@ -470,7 +502,7 @@ export function SpatialScopeMap({ ) { layer.on('click', () => { const value = String(feature.properties?.[activeFilterField]); - const option = featureOptions.find((item) => item.value === value); + const option = findBoundaryOption(allOptions, value); setSelectedFeatureValue(value); onFeatureSelect?.({ filterField: activeFilterField, @@ -497,7 +529,7 @@ export function SpatialScopeMap({
    {activeLayerLabel ?? 'Stored region geometry'} - {scopeId ? ` · ${label ?? scopeId}` : ''} + {scopeId && label ? ` · ${label}` : ''} {layerLoading ? 'Loading layer…' : (layerError ?? 'OpenStreetMap base map')}
    @@ -505,6 +537,20 @@ export function SpatialScopeMap({ ); } +/** The geometry a boundary option stands for: one feature's, or the union of a family's parts. */ +function boundaryGeometry( + features: GeoJSON.Feature[], + filterField: string, + option: BoundaryOption, +): GeoJSON.Geometry | undefined { + const values = new Set(option.values); + const geometries = features + .filter((feature) => values.has(String(feature.properties?.[filterField]))) + .flatMap((feature) => (feature.geometry ? [feature.geometry] : [])); + if (geometries.length <= 1) return geometries[0]; + return { type: 'GeometryCollection', geometries }; +} + function spatialLayerQueryUrl(uri: string): string | null { try { const url = new URL(uri); diff --git a/ui-react/src/pages/modeling/thread/__tests__/FlowSettingsDialog.test.tsx b/ui-react/src/pages/modeling/thread/__tests__/FlowSettingsDialog.test.tsx index 087d8269..17cbf772 100644 --- a/ui-react/src/pages/modeling/thread/__tests__/FlowSettingsDialog.test.tsx +++ b/ui-react/src/pages/modeling/thread/__tests__/FlowSettingsDialog.test.tsx @@ -20,6 +20,26 @@ describe('FlowSettingsDialog', () => { ); }); + it('lists every value the author chose, hidden and fixed ones and web addresses too', () => { + // The plan lists the author's values whatever the form shows, so the runner reads + // each one before accepting. Nothing here is filtered by visibility. + open({ + settings: { + defaults: { + taxonomy_uri: 'https://elsewhere.example/taxonomy.json', + chat_kind: 'openai_compatible', + max_files: 5, + }, + links: [], + }, + }); + const defaults = screen.getByTestId('flow-settings-defaults'); + expect(defaults).toHaveTextContent('taxonomy uri: https://elsewhere.example/taxonomy.json'); + expect(defaults).toHaveTextContent('chat kind: openai_compatible'); + expect(defaults).toHaveTextContent('max files: 5'); + expect(screen.getByRole('button', { name: 'Accept and run' })).toBeDisabled(); + }); + it('writes a linked setting with the step names, never as JSON', () => { open({ settings: { diff --git a/ui-react/src/pages/modeling/thread/__tests__/MintParameters.test.tsx b/ui-react/src/pages/modeling/thread/__tests__/MintParameters.test.tsx index 78d1a612..93d9db48 100644 --- a/ui-react/src/pages/modeling/thread/__tests__/MintParameters.test.tsx +++ b/ui-react/src/pages/modeling/thread/__tests__/MintParameters.test.tsx @@ -1,7 +1,7 @@ /** * Tests for MintParameters — parameter sweep configuration step. */ -import { describe, expect, it, vi } from 'vitest'; +import { afterEach, describe, expect, it, vi } from 'vitest'; import { screen, fireEvent, waitFor } from '@testing-library/react'; import type { ReactNode } from 'react'; import { renderWithProviders } from '@/test/utils/render'; @@ -205,14 +205,10 @@ describe('MintParameters', () => { expect( screen.getByRole('option', { name: 'TWDB Statewide GMA Boundaries' }), ).toBeInTheDocument(); - await waitFor(() => { - expect(screen.getByRole('option', { name: '8 — Gulf Coast' })).toBeInTheDocument(); - expect( - Array.from( - screen.getByLabelText('Spatial boundary feature').querySelectorAll('option'), - ).map((option) => option.textContent), - ).toEqual(['All boundaries', '2 — Central High Plains', '8 — Gulf Coast']); - }); + const picker = await screen.findByRole('combobox', { name: 'Spatial boundary feature' }); + fireEvent.click(picker); + expect(await screen.findByText('2 — Central High Plains')).toBeInTheDocument(); + expect(screen.getByText('8 — Gulf Coast')).toBeInTheDocument(); // The layers come from the Ensemble Manager; the browser does not call the adapter. expect(fetchMock).toHaveBeenCalledWith( 'http://em/v1/spatial/layers', @@ -326,7 +322,9 @@ describe('MintParameters', () => { fireEvent.change(screen.getByLabelText('Spatial boundary layer'), { target: { value: 'mint_registered_ground_water_availability_models' }, }); - expect(await screen.findByRole('option', { name: 'gams_12 — GMA 12' })).toBeInTheDocument(); + fireEvent.click(await screen.findByRole('combobox', { name: 'Spatial boundary feature' })); + expect(await screen.findByText('GMA 12')).toBeInTheDocument(); + expect(screen.queryByText(/gams_12/)).not.toBeInTheDocument(); vi.unstubAllGlobals(); Reflect.deleteProperty(window, '__MINT_CONFIG__'); @@ -604,3 +602,282 @@ describe('MintParameters', () => { }); }); }); + +describe('the settings of a flow', () => { + const MODE = { + name: 'match_mode', + parameter_id: 'p-mode', + visibility: 'shown', + title: 'How terms are counted', + description: 'The counting rule.', + type: 'string', + default: 'auto', + choices: [ + { value: 'auto', title: 'Automatic', description: 'Words and phrases.' }, + { value: 'longest', title: 'Longest match', description: 'No double counting.' }, + ], + }; + const FILES = { + name: 'max_files', + title: 'Most files', + parameter_id: 'p-files', + visibility: 'advanced', + type: 'integer', + minimum: 1, + maximum: 500, + default: 200, + }; + const SECRET = { + name: 'internal', + parameter_id: 'p-internal', + visibility: 'hidden', + default: 'x', + }; + const PINNED = { + name: 'grounding', + parameter_id: 'p-pinned', + visibility: 'shown', + fixed: true, + default: 'keyword', + }; + const row = (id: string, name: string, extra = {}) => ({ id, name, type: 'string', ...extra }); + + const flowData = (bindings: Record = {}): ThreadExecutionData => ({ + ...mockThreadDataUnconfigured, + models: { + 'model-1': { + ...mockThreadDataUnconfigured.models['model-1']!, + isFlow: true, + input_parameters: [ + row('p-mode', 'match_mode', { default: 'auto' }), + row('p-files', 'max_files', { type: 'integer', default: '200' }), + row('p-internal', 'internal', { default: 'x' }), + row('p-pinned', 'grounding', { default: 'keyword' }), + row('p-plain', 'title'), + ], + }, + }, + model_ensembles: { 'model-1': { id: 'ens-1', bindings } }, + }); + + const open = async ( + data = flowData(), + settings: unknown[] = [MODE, FILES, SECRET, PINNED], + ready = 'How terms are counted', + ) => { + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValue({ ok: true, json: async () => ({ settings, thread_bound: [] }) }), + ); + const onSave = vi.fn().mockResolvedValue(undefined); + renderWithProviders( + , + ); + // A flow whose step was saved opens read-only, as a finished step does. + fireEvent.click(await screen.findByTestId('edit-parameters-btn')); + // The form shows first; the settings answer follows and gives the titles. + await screen.findByText(ready); + return onSave; + }; + const saved = async (onSave: ReturnType) => { + fireEvent.click(screen.getByTestId('parameters-save-btn')); + await waitFor(() => expect(onSave).toHaveBeenCalled()); + return (onSave.mock.calls[0]![0] as Record }>)[ + 'model-1' + ]!.bindings; + }; + + afterEach(() => vi.unstubAllGlobals()); + + it('shows the title, the explanation and a dropdown filled with the default', async () => { + await open(); + expect(screen.getByText('The counting rule.')).toBeInTheDocument(); + const select = screen.getByTestId('param-input-p-mode') as HTMLSelectElement; + expect(select.tagName).toBe('SELECT'); + expect(select.value).toBe('auto'); + expect(screen.getByTestId('param-input-p-mode-explanation')).toHaveTextContent( + 'Words and phrases.', + ); + fireEvent.change(select, { target: { value: 'longest' } }); + expect(screen.getByTestId('param-input-p-mode-explanation')).toHaveTextContent( + 'No double counting.', + ); + }); + + it('keeps advanced settings in a collapsed section, with a number box and its bounds', async () => { + await open(); + const advanced = screen.getByTestId('advanced-settings-model-1') as HTMLDetailsElement; + expect(advanced.open).toBe(false); + expect(advanced).toHaveTextContent('Advanced settings'); + expect(advanced).toContainElement(screen.getByTestId('param-input-p-files')); + expect(screen.getByTestId('param-input-p-files')).toHaveAttribute('type', 'number'); + expect(screen.getByTestId('param-input-p-files')).toHaveAttribute('max', '500'); + expect(screen.getByTestId('param-input-p-mode').closest('details')).toBeNull(); + }); + + it('does not show a hidden setting, and lists a fixed one read-only', async () => { + await open(); + expect(screen.queryByTestId('param-input-p-internal')).not.toBeInTheDocument(); + expect(screen.queryByText('internal')).not.toBeInTheDocument(); + expect(screen.queryByTestId('param-input-p-pinned')).not.toBeInTheDocument(); + expect(screen.getByText(/expert modeler has selected/i)).toBeInTheDocument(); + expect(screen.getByText('grounding')).toBeInTheDocument(); + expect(screen.getByText('keyword')).toBeInTheDocument(); + }); + + it('binds only what the runner changed from the default, and a plain setting as typed', async () => { + const onSave = await open(); + fireEvent.change(screen.getByTestId('param-input-p-mode'), { target: { value: 'longest' } }); + fireEvent.change(screen.getByTestId('param-input-p-plain'), { target: { value: 'A, B' } }); + const bindings = await saved(onSave); + expect(bindings['p-mode']).toEqual(['longest']); + expect(bindings['p-plain']).toEqual(['A, B']); + // Left at the default: not a value the runner sent, so the flow's default is still + // listed for acceptance when the run starts. + expect(bindings['p-files']).toEqual([]); + }); + + it('drops a value bound earlier for a setting that is now hidden or fixed', async () => { + const onSave = await open( + flowData({ 'p-internal': ['y'], 'p-pinned': ['z'], 'p-mode': ['auto'] }), + ); + const bindings = await saved(onSave); + expect(bindings['p-internal']).toBeUndefined(); + expect(bindings['p-pinned']).toBeUndefined(); + }); + + it('refuses a number outside the limits the piece gave', async () => { + const data = flowData(); + data.models['model-1']!.input_parameters = [ + row('p-files', 'max_files', { type: 'integer', min: '1', max: '500', default: '200' }), + ]; + const onSave = await open(data, [FILES], 'Most files'); + fireEvent.change(screen.getByTestId('param-input-p-files'), { target: { value: '900' } }); + fireEvent.click(screen.getByTestId('parameters-save-btn')); + expect(await screen.findByTestId('param-error-p-files')).toHaveTextContent('Max is 500'); + expect(onSave).not.toHaveBeenCalled(); + }); + + it('still lists a fixed setting read-only from its row when the settings cannot be read', async () => { + vi.stubGlobal('fetch', vi.fn().mockResolvedValue({ ok: false, json: async () => ({}) })); + const data = flowData(); + data.models['model-1']!.input_parameters = [ + row('p-pinned', 'grounding', { default: 'keyword', value: 'keyword' }), + row('p-plain', 'title'), + ]; + renderWithProviders( + , + ); + fireEvent.click(screen.getByTestId('edit-parameters-btn')); + expect(screen.getByText(/expert modeler has selected/i)).toBeInTheDocument(); + expect(screen.getByText('keyword')).toBeInTheDocument(); + expect(screen.queryByTestId('param-input-p-pinned')).not.toBeInTheDocument(); + expect(screen.getByTestId('param-input-p-plain')).toBeInTheDocument(); + }); + + it('still shows a dropdown from the flows parameter rows when the settings cannot be read', async () => { + vi.stubGlobal('fetch', vi.fn().mockResolvedValue({ ok: false, json: async () => ({}) })); + const data = flowData(); + data.models['model-1']!.input_parameters = [ + row('p-mode', 'match_mode', { default: 'auto', accepted_values: ['auto', 'longest'] }), + ]; + renderWithProviders( + , + ); + fireEvent.click(screen.getByTestId('edit-parameters-btn')); + const select = screen.getByTestId('param-input-p-mode') as HTMLSelectElement; + expect(select.tagName).toBe('SELECT'); + expect(select.value).toBe('auto'); + expect(screen.getByText('match mode')).toBeInTheDocument(); + }); +}); + +describe('a model parameter with accepted values', () => { + const dataWith = (bindings: Record = {}): ThreadExecutionData => ({ + ...mockThreadDataUnconfigured, + models: { + 'model-1': { + ...mockThreadDataUnconfigured.models['model-1']!, + input_parameters: [ + { + id: 'param-s', + name: 'scheme', + type: 'string', + accepted_values: ['upstream', 'central', 'tvd'], + default: 'central', + }, + { id: 'param-t', name: 'timestep', type: 'int' }, + ], + }, + }, + model_ensembles: { 'model-1': { id: 'ens-1', bindings } }, + }); + const render_ = (bindings?: Record) => { + const onSave = vi.fn().mockResolvedValue(undefined); + renderWithProviders( + , + ); + return onSave; + }; + + it('is a multi-select with the default selected, and plain parameters stay text boxes', () => { + render_(); + const select = screen.getByTestId('param-input-param-s') as HTMLSelectElement; + expect(select.multiple).toBe(true); + expect(Array.from(select.selectedOptions).map((o) => o.value)).toEqual(['central']); + expect((screen.getByTestId('param-input-param-t') as HTMLInputElement).type).toBe('text'); + }); + + it('makes several selected values several runs', async () => { + const onSave = render_(); + const select = screen.getByTestId('param-input-param-s') as HTMLSelectElement; + for (const option of Array.from(select.options)) { + option.selected = option.value !== 'central'; + } + fireEvent.click(screen.getByTestId('parameters-save-btn')); + await waitFor(() => expect(onSave).toHaveBeenCalled()); + const bindings = ( + onSave.mock.calls[0]![0] as Record }> + )['model-1']!.bindings; + expect(bindings['param-s']).toEqual(['upstream', 'tvd']); + const summary = onSave.mock.calls[0]![1] as Record; + expect(summary['model-1']!.total_runs).toBe(2); + }); + + it('preselects the values already chosen instead of the default', () => { + render_({ 'param-s': ['tvd', 'upstream'] }); + const select = screen.getByTestId('param-input-param-s') as HTMLSelectElement; + expect(Array.from(select.selectedOptions).map((o) => o.value)).toEqual(['upstream', 'tvd']); + }); + + it('keeps a bound value the list does not hold, flagged, and says so on save', async () => { + // An old model saved comma separated values: one of them is not an accepted value. + const onSave = render_({ 'param-s': ['central', 'upwind'] }); + const select = screen.getByTestId('param-input-param-s') as HTMLSelectElement; + expect( + screen.getByRole('option', { name: 'upwind (not an accepted value)' }), + ).toBeInTheDocument(); + expect(Array.from(select.selectedOptions).map((o) => o.value)).toEqual(['central', 'upwind']); + fireEvent.click(screen.getByTestId('parameters-save-btn')); + expect(await screen.findByTestId('param-error-param-s')).toHaveTextContent('Accepted values'); + expect(onSave).not.toHaveBeenCalled(); + }); +}); diff --git a/ui-react/src/pages/modeling/thread/__tests__/SpatialScopeMap.test.tsx b/ui-react/src/pages/modeling/thread/__tests__/SpatialScopeMap.test.tsx new file mode 100644 index 00000000..4b33a008 --- /dev/null +++ b/ui-react/src/pages/modeling/thread/__tests__/SpatialScopeMap.test.tsx @@ -0,0 +1,156 @@ +import { fireEvent, screen, within } from '@testing-library/react'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +import { LIST_REGISTERED_BOUNDARY_REGIONS } from '@/graphql/queries/regions'; +import { renderWithProviders } from '@/test/utils/render'; +import { SpatialScopeMap } from '../SpatialScopeMap'; + +// jsdom has no SVG renderer for Leaflet vectors; the picker is what is under test. +vi.mock('react-leaflet', () => ({ + MapContainer: ({ children }: { children?: React.ReactNode }) =>
    {children}
    , + TileLayer: () => null, + GeoJSON: () => null, + useMap: () => ({ fitBounds: () => undefined }), +})); + +const square = (x: number) => ({ + type: 'Polygon', + coordinates: [ + [ + [x, 29], + [x + 1, 29], + [x + 1, 30], + [x, 30], + [x, 29], + ], + ], +}); + +const region = (id: string, name: string, categoryId: string, categoryName: string, x: number) => ({ + id: `mint__${id}`, + name, + category_id: categoryId, + region_category: { id: categoryId, name: categoryName }, + geometries: [{ geometry: square(x) }], +}); + +const REGIONS = [ + region('0B4wIEBZthkf9gclJk1F', 'SEYMOUR_3', 'hydrology', 'Hydrology', -100), + region('1ap0Inly6Lq3jYZgDDKy', 'TRINITY_53', 'hydrology', 'Hydrology', -99), + region('2ap0Inly6Lq3jYZgDDKy', 'TRINITY_2', 'hydrology', 'Hydrology', -98), + region('3wKEpR6sL8AtkDM8hXAo', 'Alwero', 'hydrology', 'Hydrology', -97), +]; + +function mount(props: Parameters[0] = {}) { + return renderWithProviders(, { + apolloMocks: [ + { + request: { + query: LIST_REGISTERED_BOUNDARY_REGIONS, + variables: { categoryIds: ['hydrology'] }, + }, + result: { data: { region: REGIONS } }, + }, + ], + }); +} + +async function openPicker() { + fireEvent.click(await screen.findByRole('combobox', { name: 'Spatial boundary feature' })); +} + +describe('SpatialScopeMap boundary picker', () => { + beforeEach(() => { + window.__MINT_CONFIG__ = { + HASURA_ENDPOINT: 'http://hasura/graphql', + AUTH_SERVER: '', + AUTH_CLIENT_ID: '', + AUTH_REALM: '', + AUTH_PROVIDER: 'keycloak', + ENSEMBLE_MANAGER_API: 'http://em/v1', + }; + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValue({ ok: true, json: async () => ({ layers: [] }) }), + ); + }); + + afterEach(() => { + vi.unstubAllGlobals(); + Reflect.deleteProperty(window, '__MINT_CONFIG__'); + }); + + it('shows region names sorted, with no ids, and one entry for a numbered family', async () => { + mount(); + await openPicker(); + const labels = within(await screen.findByRole('listbox')) + .getAllByRole('option') + .map((option) => option.textContent); + expect(labels).toEqual(['All boundaries', 'Alwero', 'Seymour 3', 'Trinity (2 parts)']); + expect(document.body.textContent).not.toMatch(/0B4wIEBZthkf9gclJk1F|1ap0Inly6Lq3jYZgDDKy/); + }); + + it('groups the options under their category', async () => { + mount(); + await openPicker(); + expect(await screen.findByText('Hydrology')).toBeInTheDocument(); + }); + + it('selects the whole family, and lets the user expand to one part', async () => { + const onFeatureSelect = vi.fn(); + mount({ onFeatureSelect }); + await openPicker(); + fireEvent.click(await screen.findByText('Trinity (2 parts)')); + expect(onFeatureSelect).toHaveBeenCalledTimes(1); + const whole = onFeatureSelect.mock.calls[0]?.[0]; + expect(whole.label).toBe('Trinity (2 parts)'); + expect(whole.filterField).toBe('region_id'); + expect(whole.filterValue.split(',').sort()).toEqual([ + '1ap0Inly6Lq3jYZgDDKy', + '2ap0Inly6Lq3jYZgDDKy', + ]); + expect(whole.geometry.type).toBe('GeometryCollection'); + expect(whole.geometry.geometries).toHaveLength(2); + + await openPicker(); + fireEvent.click(screen.getByRole('button', { name: 'Expand Trinity (2 parts)' })); + fireEvent.click(await screen.findByText('Trinity 53')); + expect(onFeatureSelect).toHaveBeenCalledTimes(2); + expect(onFeatureSelect.mock.calls[1]?.[0].filterValue).toBe('1ap0Inly6Lq3jYZgDDKy'); + expect(onFeatureSelect.mock.calls[1]?.[0].geometry.type).toBe('Polygon'); + }); + + it('filters as the user types', async () => { + mount(); + await openPicker(); + fireEvent.change(await screen.findByPlaceholderText('Search boundaries...'), { + target: { value: 'sey' }, + }); + const labels = within(screen.getByRole('listbox')) + .getAllByRole('option') + .map((option) => option.textContent); + expect(labels).toEqual(['Seymour 3']); + }); + + it('still shows a saved region by its readable name', async () => { + mount({ + scopeId: 'a-region-hash-id', + label: 'Texas Panhandle', + geometries: [square(-101)], + }); + const picker = await screen.findByRole('combobox', { name: 'Spatial boundary feature' }); + expect(picker).toHaveTextContent('Texas Panhandle'); + expect(document.body.textContent).not.toContain('a-region-hash-id'); + }); + + it('shows a saved boundary selection by name', async () => { + mount({ + filterField: 'region_id', + filterValue: '3wKEpR6sL8AtkDM8hXAo', + label: 'Alwero', + scopeId: '3wKEpR6sL8AtkDM8hXAo', + }); + const picker = await screen.findByRole('combobox', { name: 'Spatial boundary feature' }); + await vi.waitFor(() => expect(picker).toHaveTextContent('Alwero')); + }); +}); diff --git a/ui-react/src/pages/modeling/thread/wizard/FramingStep.tsx b/ui-react/src/pages/modeling/thread/wizard/FramingStep.tsx index 7a906519..2bc0791b 100644 --- a/ui-react/src/pages/modeling/thread/wizard/FramingStep.tsx +++ b/ui-react/src/pages/modeling/thread/wizard/FramingStep.tsx @@ -155,7 +155,7 @@ export function FramingStep({ geometries={(thread.region?.geometries ?? []) .map((geometry) => geometry.geometry) .filter(Boolean)} - label={spatialSelection?.feature?.label ?? thread.region?.name ?? regionId} + label={spatialSelection?.feature?.label ?? thread.region?.name} scopeId={spatialSelection?.feature?.filterValue ?? regionId} sourceUri={spatialSelection?.layer?.source_uri ?? spatialSelection?.layer?.uri} filterField={spatialSelection?.feature?.filterField} diff --git a/ui-react/src/pages/modeling/thread/wizard/ModelsStep.tsx b/ui-react/src/pages/modeling/thread/wizard/ModelsStep.tsx index 1b614b33..2b6c0a46 100644 --- a/ui-react/src/pages/modeling/thread/wizard/ModelsStep.tsx +++ b/ui-react/src/pages/modeling/thread/wizard/ModelsStep.tsx @@ -130,6 +130,31 @@ function rowFromConfig(cfg: ModelConfigInfo | ModelSetupInfo, parent?: ModelConf }; } +type OutcomeMatch = 'produces' | 'transforms'; + +/** The part of a standard-variable id or label that names it, lower-cased. */ +function outcomeName(value: string): string { + return slugFromUri(value.trim()).toLowerCase(); +} + +/** + * True when the row declares the outcome as one of its outputs. The id is + * compared first; the name (the URI slug or the label) bridges a flow whose + * output was registered under another namespace for the same variable. + */ +function rowProducesOutcome( + row: Pick, + outcome: string, + outcomeLabel?: string | null, +): boolean { + if (row.producesIds.includes(outcome)) return true; + const names = new Set([outcomeName(outcome), outcomeLabel ? outcomeName(outcomeLabel) : '']); + names.delete(''); + return [...row.producesIds, ...row.producesLabels].some( + (value) => value && names.has(outcomeName(value)), + ); +} + function flattenToRows( data: ReturnType['data'], ): ModelRow[] { @@ -153,10 +178,15 @@ function ModelCard({ row, checked, onToggle, + outcomeMatch, + outcomeLabel, }: { row: ModelRow; checked: boolean; onToggle: (id: string, checked: boolean) => void; + /** How the row reaches the chosen outcome, when one is chosen. */ + outcomeMatch?: OutcomeMatch | null; + outcomeLabel?: string | null; }) { // Asked only for a flow, and only once per flow per session. An unanswered or // failed check leaves the card pickable: plan and submit still refuse correctly. @@ -185,6 +215,19 @@ function ModelCard({ {row.name} {row.isFlow && }
    + {outcomeMatch && outcomeLabel && ( +
    + {outcomeMatch === 'produces' + ? `Produces ${outcomeLabel}` + : `Can be transformed into ${outcomeLabel}`} +
    + )} {row.takesCollection && (
    Takes a set of files as one input
    )} @@ -711,9 +754,11 @@ export function ModelsStep({ const indicatorRows = useMemo( () => indicator - ? allRows.filter((r) => r.producesIds.includes(indicator) || inferredModelIds.has(r.id)) + ? allRows.filter( + (r) => rowProducesOutcome(r, indicator, indicatorLabel) || inferredModelIds.has(r.id), + ) : allRows, - [allRows, indicator, inferredModelIds], + [allRows, indicator, indicatorLabel, inferredModelIds], ); const dataFeasibleRows = useMemo(() => { if (!hasRegionGeometry || availabilityLoading || availabilityError) return indicatorRows; @@ -756,23 +801,47 @@ export function ModelsStep({ ); }, [dataFilteredRows, searchText]); - const searchedRows = useMemo(() => { + const matchedRows = useMemo(() => { if (!debouncedSearchText.trim() || !semanticSearch.results) return localSearchedRows; const rankById = new Map(semanticSearch.results.map((result, index) => [result.id, index])); + const semanticRank = (row: ModelRow) => + Math.min( + Infinity, + ...row.searchIds + .map((id) => rankById.get(id)) + .filter((rank): rank is number => rank !== undefined), + ); + // The index may not carry a row (a flow's outputs, for one), so a row the + // plain text filter finds is kept and follows the semantic hits. + const found = new Set(localSearchedRows.map((row) => row.id)); return dataFilteredRows - .map((row) => ({ - row, - rank: Math.min( - ...row.searchIds - .map((id) => rankById.get(id)) - .filter((rank): rank is number => rank !== undefined), - ), - })) - .filter((item) => Number.isFinite(item.rank)) - .sort((a, b) => a.rank - b.rank) + .map((row) => ({ row, rank: semanticRank(row) })) + .filter((item) => Number.isFinite(item.rank) || found.has(item.row.id)) + .sort((a, b) => (a.rank === b.rank ? 0 : a.rank < b.rank ? -1 : 1)) .map((item) => item.row); }, [dataFilteredRows, debouncedSearchText, localSearchedRows, semanticSearch.results]); + // Rows that declare the outcome as an output come first, models and flows + // alike, whatever the search ranked them; then the rows reachable by a + // transform. The sort is stable, so each group keeps its own order. + const outcomeMatchById = useMemo(() => { + const matches = new Map(); + if (!indicator) return matches; + for (const row of allRows) { + if (rowProducesOutcome(row, indicator, indicatorLabel)) matches.set(row.id, 'produces'); + else if (inferredModelIds.has(row.id)) matches.set(row.id, 'transforms'); + } + return matches; + }, [allRows, indicator, indicatorLabel, inferredModelIds]); + const searchedRows = useMemo(() => { + if (outcomeMatchById.size === 0) return matchedRows; + const tier = (row: ModelRow) => { + const match = outcomeMatchById.get(row.id); + return match === 'produces' ? 0 : match === 'transforms' ? 1 : 2; + }; + return [...matchedRows].sort((a, b) => tier(a) - tier(b)); + }, [matchedRows, outcomeMatchById]); + const threadRegionId = thread.region_id ?? null; const { regionRows, otherRows } = useMemo(() => { if (!threadRegionId) return { regionRows: searchedRows, otherRows: [] as ModelRow[] }; @@ -793,10 +862,10 @@ export function ModelsStep({ return allRows.filter( (row) => selected.has(row.id) && - !row.producesIds.includes(indicator) && + !rowProducesOutcome(row, indicator, indicatorLabel) && !inferredModelIds.has(row.id), ); - }, [allRows, indicator, inferredModelIds, selectedIds]); + }, [allRows, indicator, indicatorLabel, inferredModelIds, selectedIds]); const toggleModel = useCallback((id: string, checked: boolean) => { setSelectedIds((prev) => { @@ -1023,6 +1092,8 @@ export function ModelsStep({ row={row} checked={selectedIds.has(row.id)} onToggle={toggleModel} + outcomeMatch={outcomeMatchById.get(row.id) ?? null} + outcomeLabel={indicatorLabel} /> )) )} diff --git a/ui-react/src/pages/modeling/thread/wizard/__tests__/ModelsStepOutcomeRanking.test.tsx b/ui-react/src/pages/modeling/thread/wizard/__tests__/ModelsStepOutcomeRanking.test.tsx new file mode 100644 index 00000000..eac71e6f --- /dev/null +++ b/ui-react/src/pages/modeling/thread/wizard/__tests__/ModelsStepOutcomeRanking.test.tsx @@ -0,0 +1,169 @@ +/** + * Choosing an outcome ranks the rows that produce it first, flows included, + * then the rows that can be transformed into it, and says why on each card. + */ +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import type { MockedResponse } from '@apollo/client/testing'; +import userEvent from '@testing-library/user-event'; +import { renderWithProviders, screen, waitFor } from '@/test/utils/render'; +import { GetModelTreeWithRegionsDocument, type Thread } from '@/graphql/generated/modeling'; +import { ModelOutcomeAdapterInferenceDocument, ModelsStep } from '../ModelsStep'; + +vi.mock('@/components/ui/use-toast', () => ({ useToast: () => ({ toast: vi.fn() }) })); +vi.mock('@/hooks/useFlowAvailability', () => ({ useFlowAvailability: () => null })); + +const semantic = vi.hoisted(() => ({ + results: null as Array<{ id: string }> | null, +})); +vi.mock('@/hooks/useSemanticSearch', () => ({ + useSemanticSearch: (query: string) => ({ + results: query.trim() ? semantic.results : null, + loading: false, + error: null, + }), +})); + +const REPORT = 'https://w3id.org/okn/i/mint/document_collection__sensemaking_report'; +const RAW = 'https://w3id.org/okn/i/mint/document_collection__raw_text'; + +function config(id: string, label: string, outId: string, outLabel: string, isFlow = false) { + return { + id, + label, + ...(isFlow ? { flow: { __typename: 'x', configuration_id: id } } : {}), + regions: [], + inputs: [], + outputs: [ + { + configuration_id: id, + output_id: `${id}-out`, + output: { + id: `${id}-out`, + label: outLabel, + has_format: null, + presentations: [ + { + dataset_specification_id: `${id}-out`, + presentation_id: `${id}-vp`, + presentation: { id: `${id}-vp`, standard_variable: { id: outId, label: outLabel } }, + }, + ], + }, + }, + ], + child_configurations: [], + }; +} + +const treeMock: MockedResponse = { + request: { query: GetModelTreeWithRegionsDocument }, + result: { + data: { + modelcatalog_software: [ + { + id: 'sw', + label: 'Sensemaking', + versions: [ + { + id: 'v', + label: '1', + configurations: [ + config('alpha', 'Alpha model', RAW, 'document_collection__raw_text'), + config('mid', 'Mid model', REPORT, 'document_collection__sensemaking_report'), + // Registered under another namespace: only the name matches. + config( + 'zeta', + 'Zeta flow', + 'https://example.org/vocab/document_collection__sensemaking_report', + 'document_collection__sensemaking_report', + true, + ), + ], + }, + ], + }, + ], + }, + }, +}; + +const adapterMock: MockedResponse = { + request: { query: ModelOutcomeAdapterInferenceDocument }, + result: { + data: { + adapterTransforms: [ + { + id: 't1', + name: 'raw-to-report', + description: null, + contracts: [ + { id: 'c1', role: 'input', standard_variable_uri: RAW, format: null, unit: null }, + { id: 'c2', role: 'output', standard_variable_uri: REPORT, format: null, unit: null }, + ], + }, + ], + }, + }, +}; + +const thread = { + __typename: 'thread', + id: 't1', + name: 'Report', + task_id: 'task1', + start_date: '2000-01-01', + end_date: '2026-01-01', + region_id: null, + driving_variable_id: null, + response_variable_id: REPORT, + response_variable: { id: REPORT, label: 'document_collection__sensemaking_report' }, + events: [], + permissions: [{ __typename: 'thread_permission', user_id: 'testuser', read: true, write: true }], + thread_models: [], +} as unknown as Thread; + +function renderStep() { + return renderWithProviders( + , + { apolloMocks: [treeMock, adapterMock] }, + ); +} + +const cardNames = () => + screen + .getAllByRole('checkbox') + .map((box) => box.getAttribute('aria-label')?.replace(/^Select /, '')); + +describe('ModelsStep outcome ranking', () => { + beforeEach(() => { + semantic.results = null; + }); + + it('puts the rows that produce the outcome first, flows included, then transforms', async () => { + renderStep(); + await screen.findByText('Zeta flow'); + // Tree order is alphabetical (Alpha, Mid, Zeta); the producers lead. + await waitFor(() => expect(cardNames()).toEqual(['Mid model', 'Zeta flow', 'Alpha model'])); + }); + + it('says why each row is listed', async () => { + renderStep(); + await screen.findByText('Zeta flow'); + await waitFor(() => expect(screen.getAllByTestId('outcome-reason')).toHaveLength(3)); + const reasons = screen.getAllByTestId('outcome-reason').map((el) => el.textContent); + expect(reasons).toEqual([ + 'Produces document_collection__sensemaking_report', + 'Produces document_collection__sensemaking_report', + 'Can be transformed into document_collection__sensemaking_report', + ]); + }); + + it('keeps exact matches ahead of semantic-only hits, and keeps a row the index lacks', async () => { + // The index ranks the transformed row first and does not carry the flow. + semantic.results = [{ id: 'alpha' }, { id: 'mid' }]; + renderStep(); + await screen.findByText('Zeta flow'); + await userEvent.type(screen.getByPlaceholderText(/filter models/i), 'e'); + await waitFor(() => expect(cardNames()).toEqual(['Mid model', 'Zeta flow', 'Alpha model'])); + }); +});