Repository navigation
feat(compare): add a Chart view of the summary rows - #218
Merged
Merged
Conversation
A backend module places a few named values, one per item, in SVG user units, and a template macro draws the marks and texts as given, with a table of every value for a screen reader. A missing value is a gap, a check is hollow and named, and nothing is ranked. Styles are a registry of drawing functions, so a new style is one function and one entry. An item's series gives it one colour in every chart, from eight --series-* tokens set for the light and the dark themes.
Compare gains a third view beside Side by side and Task by task. Each chart is a card. Its header holds a Style menu (Bars, Dots or Scatter), a Data menu of one metric, or X and Y menus for a scatter, each button naming its metric, and a Remove button. Add chart adds a card. The charts live in the URL, one chart= value each, and every link keeps them. The values are the side table's own cells, and a run's colour is its column's place, the same in every chart.
harenome
force-pushed
the
feat/compare-chart
branch
from
November 19, 2026 10:37
dfdd91f to
f9e1841
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #217
What
Compare gets a third view, Chart (
?view=chart), beside Side by side and Task by task. It has the same columns, cohort and controls row. The column rows (header, Workspace, Run) stay above the charts, so you change the runs as in the other views.Cards. Each chart is a card. Its header is one row of toolkit buttons, all
--tool-hhigh, with Remove at its right end:Every menu item is a link, and
NulliusTable.menuopens the menus, so the change adds no other script. A button never breaks its name over two lines. When a row is too narrow, whole buttons move to a second row, and Remove stays at the end of the first. At 1280 px, a Scatter card's Style, X and Y fit on one row, also with long metric names. A style change keeps the card's first metric. For Scatter, Y becomes the headline rate, or the cost when the rate is X. Add chart, in the actions slot beside Add a run, appends a Bars card of the headline rate.URL. Each card is one
chart=value, in order:chart=bars:adjusted_rate&chart=scatter:cost,adjusted_rate. The metric keys are stable names, not labels. Every link on the page keeps them, with the view and the cohort. Withoutchart=, the page shows three cards: Bars of the headline rate, Bars of the cost, and a Scatter of the cost against the rate. The headline rate is the adjusted rate, or the mean score for a benchmark that only scores.chart=alone shows one Add chart button. A card whose metric this comparison does not have keeps its controls and says "Not in this comparison."Styles.
Values. Each value is the side table's own cell: the number its difference uses, and the text it shows. A missing value has no mark, only its dash. A check is hollow and named "(check)". A check's $0.00 or 0 tokens is a real zero, as in the table, so it has a mark. Nothing is ranked. The tick labels use the row's unit: % for rates, $ for cost, k and M for tokens, seconds or minutes for time. A tick label that would touch the one before it is left out, but its gridline stays.
Colours. Each run has one colour in every chart: its column's place on the page picks
--series-1to--series-8, and the ninth column starts again at the first. The drawing data puts the index on each mark as a class (s1tos8), and the stylesheet maps each class to its token, so the template has no colour logic. A check stays hollow, with its outline in the run's colour. The baseline mark stays a dashed grey line. The key under a Scatter gives each run a small swatch beside its number and name, so a colour is never the only cue.The series palette. Eight new tokens,
--series-1to--series-8, instyle.css. They are muted blues, two teals, a brown and a plum, in the app's existing lightness range. None of them reads as an outcome colour for a pass (sage), a failure (red) or a warning (amber). The nearest pair is the dark brown and the warning amber, at ΔE 9.5, and every other pair is at 12.5 or more. The trace categories keep their own--cat-*colours, so a colour has one meaning on each page. The dark values serve the default dark theme and the charcoal, graphite and midnight palettes, and they pass on the surface of each.#215da5#3a75bf#179696#16a8a8#7d3f17#ad6a44#657dd1#748ee3#7e2e59#aa5d83#1587ba#3aa0d4#5e5197#7768be#2695ac#38a3baThe data-viz palette validator checks the colours against the card surface (
--surface-2). All checks pass in both modes:#edf0f4)#292b2f,#202024,#191f29,#18263c)ΔE is the OKLab distance ×100, with protanopia and deuteranopia simulated by Machado-Oliveira-Fernandes at full severity. Slots 6 to 8 repeat the blue and teal families, so a scatter of more than five runs also relies on the numbers and the key. Another view can use the palette with the class
s1tos8on a mark, orvar(--series-n)directly, for any set of items with a stable order.Chart module.
nullius/charts.pycomputes the geometry in SVG units.templates/_charts.htmldraws the marks and texts as it gets them, with no arithmetic. Each chart is an SVG withrole="img"and a title. Each mark has a tooltip, and a table for a screen reader gives every value. A registry,STYLES, gives each style its label, its drawing function and the names of the metrics it takes (("Metric",)or("X", "Y")). The card header reads it. A new style, for example Lines for pass@k after #146, is one function and one entry. The trace distributions (traces/charts.py) put durations in bins on a log axis. They share no scale with these charts, so they keep their own code.Docs: the results guide names the Chart view.
AGENTS.mdlists the module.Touches
backend (
nullius/charts.py, new) · routes (web/compare.py) · templates and static (_charts.htmlnew,compare/compare.html,base.html(ani-charticon),compare/columns.js,compare/comparison.css,style.css) · tests (test_categorical_charts.pynew,test_comparison_routes.py,field_contract.json) · docs (docs/results.md,AGENTS.md)Before and after
Seeded data: a showcase profile of Terminal-Bench 2.0 runs, seven compared (six agents and the oracle check). The view is new, so there is no before. The nine-run shots add the
nopcheck and a copy of the first run.Verified
In a throwaway rootless
python:3.12container with Node.js and uv, on a copy of the branch:python3 app/tests/parallel.py: 3004 tests in 154 modules, OK.pip install -r app/requirements.txt, thenpython3 app/tests/parallel.py app/ui_tests: 655 tests in 54 modules, OK.test_categorical_charts.pycovers the ticks and tick labels that would touch, a gap for a missing value, a zero and a negative value, a check, the baseline mark, a single run, a long name that wraps, very different magnitudes, a scatter of runs at one point, the numbers inside the plot, the registry, and each mark's series class, which cycles after eight.test_comparison_routes.pycovers the default cards, the URL shape, Add chart, Remove, a style change that keeps the metric, the Data, X and Y menus and the names on their buttons, no chart, a metric the cohort does not have, a check run, one run and an empty column, the column colours of eleven runs in every style, and the--series-*tokens in the dark and the light theme.Browser check, Chromium (Playwright 1.49.1), on the showcase data with seven runs, at 1280 px light and 390 px dark:
nophollow. With nine runs, the ninth has the first run's colour.mainand on this branch is the same, except the form tokens, the interface hash and the new icon in the sprite.Not checked: other browsers, a screen reader, touch.
Risk
?view=chartis new. Every other value ofviewstill opens Side by side.summary_rowsnow gives each row a key, a group, its number and its unit. The side table reads none of these, so its cells do not change.Related