perf(datagrid): draw data cells instead of building a view for each one (#2381)

CHANGELOG.md

@@ -7,13 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Changed
+
+- The data grid draws its cells instead of building a view for each one, so a result with hundreds of columns opens at once and holds a fraction of the memory. (#2381)
+
 ### Fixed
 
 - Flickering columns, blank columns, and an unpainted gap while scrolling a result with about 100 columns sideways. (#2381)
 - Find, arrow keys, and the inline editor unable to reach a column scrolled off the side of a wide result.
 - Return opening no editor on a row selected with the arrow keys.
 - Tab out of a row's last cell and Shift+Tab out of its first doing nothing.
 - Size All Columns to Fit leaving the far columns of a wide result unreachable.
+- A table with 500 columns pinning a core for 20 seconds and taking a gigabyte to open. (#2381)
 
 ## [0.67.1] - 2026-08-22
 

CLAUDE.md

@@ -208,7 +208,7 @@ To ship one: add the record type or field in CloudKit Console (or `xcrun cktool
 
 **The data grid header owns all of its own chrome, so nothing may ask AppKit to paint any of it**: `NSTableHeaderCell` and `NSTableHeaderView` both paint a fixed 28pt band that they centre vertically in whatever frame they are given, a 16pt column divider on `midY` and a 1pt rule at `midY + 13`. The data grid grows its header to 42pt for a column comment, so that band lands mid-cell: the rule crosses the comment's descenders and sits 8pt above the real bottom edge. `SortableHeaderChrome` is therefore the single owner of header geometry and colours, `SortableHeaderCell.draw(withFrame:in:)` never calls `super`, and `SortableHeaderView.draw(_:)` fills the background and rules the bottom edge itself. The trap is that the header view paints a second copy of that same band for `NSTableView.highlightedTableColumn`, driven by *state* rather than by a drawing call, so no cell override can reach it: setting it gives the sorted column a stray divider and a rule no other column has. TablePro already draws the sorted-column affordance itself (bold title, chevron, priority number, with `drawSortIndicator` overridden to nothing), so `highlightedTableColumn` is a redundant second channel and must stay unset. All sorted-column presentation goes through `SortableHeaderView.applySortState(_:schema:)`, which publishes the order natively through `tableView.sortDescriptors` (for accessibility; it paints nothing) and updates the cells. `SortableHeaderRenderingTests` rasterises the header and guards this. This shipped as a rule through the comment line and a stray divider on the sorted column (#2017).
 
-**The data grid's column window measures the viewport against the whole column run, so its rebase counts chrome only**: `ColumnWindowResolver` is pure and models every column from the first data column onward, and `DataGridColumnPool.applyColumnWindow` has to move the live viewport into that space before it can pick a range. Only chrome may be subtracted there. The leading spacer sits ahead of the first data column and holds exactly the width of the columns the window left out, which the resolver's model already carries, so counting it subtracts that width twice: the window walks left while the reader scrolls right, alternates between two ranges, and finally parks off screen, where the grid paints nothing until the table is reopened. That shipped as #2381, and a frame-by-frame capture of one scroll sweep across a 100-column table shows it plainly: the worst frame left 100% of the grid unpainted before the fix and 40pt, one column gutter, after it. Two measured AppKit facts hold the design up. A visible column occupies exactly `width + intercellSpacing.width` and a hidden one occupies nothing, spacing included, which is what lets the spacers keep the document width and therefore the scroll extent. And `rect(ofColumn:)` and `frameOfCell(atColumn:row:)` are both empty for a hidden column, so anything that reads a column's frame has to mount it first through `TableViewCoordinator.scrollColumnToVisible(tableColumnIndex:)`, which re-centres a bounded window instead of stretching the mounted range out to the target, because stretching mounts every column in between (848ms and 3,081 cell views for a match 90 columns away, 4.8s at 500). A test here measures through `rect(ofColumn:)` and never through the resolver, or it only proves the resolver agrees with itself.
+**The data grid's column window measures the viewport against the whole column run, so its rebase counts chrome only**: `NSTableColumn.isHidden` costs O(attached columns) per write, because AppKit walks every row view and re-sorts its subviews to rebuild the key view loop. Hiding the columns outside a viewport is therefore quadratic in the column count: measured at 233ms for 100 columns, 8.1s for 500 and 34s for 1000, with the relayout debris of those writes never reclaimed, which is where a 500-column table's 837MB went. No AppKit knob helps: `autorecalculatesKeyViewLoop = false`, `beginUpdates`/`endUpdates` and hiding before any row exists were all measured and none of them changed it. The grid therefore keeps every column attached and visible, and pays nothing for the ones off screen by not building a view for them: `tableView(_:viewFor:row:)` returns nil for every data column and `DataGridRowView` draws the cells the viewport touches with CoreText. Measured on a 500-column table: opening it went from 12.4s to 26ms and from 837MB to 3.9MB, and the whole table holds 26 views rather than 12,500. Never reintroduce a column window built on `isHidden`.
 
 **No fixed position in `tableColumns` names a data column**: the attached order is `[__rowNumber__, __leadingSpacer__, data columns, surplus pool slots, __trailingSpacer__]`, so `presentsColumn` is the question to ask, with `firstPresentedColumnIndex` and its neighbours beside it on `DataGridColumnPool`. `DataGridView.firstDataTableColumnIndex` was a hardcoded `1` that the leading spacer took over when windowing landed, and `isDataTableColumn` accepted the trailing spacer at the other end. The cell cursor was seeded onto a spacer whenever the selection moved without a click, so Down then Return did nothing on any table while the Edit menu item still validated as enabled, Tab out of a row's last cell and Shift+Tab out of its first were swallowed, and `scrollColumnToVisible` on a column the window had unmounted scrolled to the document origin instead of the column (#2381).
 

TablePro/Models/UI/ColumnIdentitySchema.swift

@@ -8,20 +8,8 @@ import AppKit
 struct ColumnIdentitySchema: Equatable {
     static let rowNumberIdentifier = NSUserInterfaceItemIdentifier("__rowNumber__")
 
-    /// Stand-ins for the columns the window leaves unmounted, holding their width so the document
-    /// keeps its full extent and the horizontal scroller still spans the whole result.
-    ///
-    /// Deliberately outside `dataColumnPrefix`: every loop that walks `tableView.tableColumns`
-    /// resolves a data index through `dataIndex(from:)` first, so a spacer is skipped by all of
-    /// them without any of those call sites learning about it.
-    static let leadingSpacerIdentifier = NSUserInterfaceItemIdentifier("__leadingSpacer__")
-    static let trailingSpacerIdentifier = NSUserInterfaceItemIdentifier("__trailingSpacer__")
     static let dataColumnPrefix = "dataColumn-"
 
-    static func isSpacer(_ identifier: NSUserInterfaceItemIdentifier) -> Bool {
-        identifier == leadingSpacerIdentifier || identifier == trailingSpacerIdentifier
-    }
-
     let identifiers: [NSUserInterfaceItemIdentifier]
     let columnNames: [String]
 

TablePro/Views/Results/CellOverlayBase.swift

@@ -52,13 +52,15 @@ class CellOverlayBase: NSObject {
         self.columnIndex = columnIndex
         tableView.addSubview(container)
         self.container = container
-        underlyingCell(in: tableView, row: row, column: column)?.applyOverlayActive(true)
+        setOverlayCell(CellPosition(row: row, column: columnIndex), in: tableView)
         selectionOverlay(in: tableView)?.needsDisplay = true
         installDismissObservers()
     }
 
-    private func underlyingCell(in tableView: NSTableView, row: Int, column: Int) -> DataGridCellView? {
-        tableView.view(atColumn: column, row: row, makeIfNecessary: false) as? DataGridCellView
+    /// The cell under the overlay draws no text of its own behind it. A drawn cell has no view to
+    /// carry that, so the coordinator holds it and repaints the cell either side of the change.
+    private func setOverlayCell(_ position: CellPosition?, in tableView: NSTableView) {
+        (tableView as? KeyHandlingTableView)?.coordinator?.overlayCell = position
     }
 
     private func selectionOverlay(in tableView: NSTableView) -> GridSelectionOverlay? {
@@ -73,7 +75,7 @@ class CellOverlayBase: NSObject {
         guard let activeContainer = container else { return }
         removeDismissObservers()
         if let hostTableView {
-            underlyingCell(in: hostTableView, row: row, column: column)?.applyOverlayActive(false)
+            setOverlayCell(nil, in: hostTableView)
             selectionOverlay(in: hostTableView)?.needsDisplay = true
         }
         activeContainer.removeFromSuperview()

TablePro/Views/Results/Cells/DataGridCellAccessoryGlyph.swift

@@ -0,0 +1,124 @@
+//
+//  DataGridCellAccessoryGlyph.swift
+//  TablePro
+//
+
+import AppKit
+
+/// The symbol a cell draws at its trailing edge, rasterised once per appearance.
+///
+/// Shared by every cell in the grid, so the cache is static: the glyph depends on the role and the
+/// appearance it is drawn into, never on which cell asked for it.
+@MainActor
+enum DataGridCellAccessoryGlyph {
+    enum Role: Hashable {
+        case foreignKeyNormal
+        case foreignKeyEmphasized
+        case chevronNormal
+        case chevronEmphasized
+        case chevronDisabled
+
+        init?(accessory: DataGridCellAccessory, isEmphasized: Bool, isDisabled: Bool) {
+            switch accessory {
+            case .none:
+                return nil
+            case .foreignKey:
+                self = isEmphasized ? .foreignKeyEmphasized : .foreignKeyNormal
+            case .chevron:
+                if isDisabled {
+                    self = .chevronDisabled
+                } else {
+                    self = isEmphasized ? .chevronEmphasized : .chevronNormal
+                }
+            }
+        }
+
+        var symbolName: String {
+            switch self {
+            case .foreignKeyNormal, .foreignKeyEmphasized:
+                return "arrow.forward"
+            case .chevronNormal, .chevronEmphasized, .chevronDisabled:
+                return "chevron.up.chevron.down"
+            }
+        }
+
+        /// The bare arrow spends its whole point size on the arrow itself, where the circled variant
+        /// spent most of it on the ring, so 14 here would draw an arrow half again as large as the
+        /// one it replaced. 12 keeps the ink at 11 x 9 in the 16 x 16 accessory rect, close to the
+        /// dropdown chevron's weight and to the 13pt cell text.
+        var pointSize: CGFloat {
+            switch self {
+            case .foreignKeyNormal, .foreignKeyEmphasized:
+                return 12
+            case .chevronNormal, .chevronEmphasized, .chevronDisabled:
+                return 10
+            }
+        }
+
+        var color: NSColor {
+            switch self {
+            case .foreignKeyNormal, .chevronNormal:
+                return .secondaryLabelColor
+            case .foreignKeyEmphasized, .chevronEmphasized:
+                return .alternateSelectedControlTextColor
+            case .chevronDisabled:
+                return .tertiaryLabelColor
+            }
+        }
+    }
+
+    struct Glyph {
+        let image: CGImage
+        let pointSize: NSSize
+    }
+
+    private struct Key: Hashable {
+        let role: Role
+        let appearance: NSAppearance.Name
+        let increasedContrast: Bool
+    }
+
+    private static var glyphs: [Key: Glyph] = [:]
+
+    /// Rasterizing resolves the dynamic symbol color, so a cached bitmap belongs to exactly one
+    /// appearance. Keying on the appearance is what keeps a dark window from being served the
+    /// light bitmap, and `NSAppearance.currentDrawing()` only reports the drawing appearance while
+    /// AppKit is inside `draw(_:)`, `updateLayer` or `layout`.
+    static func image(for role: Role) -> Glyph? {
+        let key = Key(
+            role: role,
+            appearance: NSAppearance.currentDrawing().name,
+            increasedContrast: NSWorkspace.shared.accessibilityDisplayShouldIncreaseContrast
+        )
+        if let cached = glyphs[key] { return cached }
+        guard let glyph = make(role) else { return nil }
+        glyphs[key] = glyph
+        return glyph
+    }
+
+    private static func make(_ role: Role) -> Glyph? {
+        let config = NSImage.SymbolConfiguration(pointSize: role.pointSize, weight: .regular)
+            .applying(.init(hierarchicalColor: role.color))
+        guard let image = NSImage(systemSymbolName: role.symbolName, accessibilityDescription: nil)?
+            .withSymbolConfiguration(config) else { return nil }
+        var rect = CGRect(origin: .zero, size: image.size)
+        guard let cgImage = image.cgImage(forProposedRect: &rect, context: nil, hints: nil) else { return nil }
+        return Glyph(image: cgImage, pointSize: image.size)
+    }
+
+    /// A symbol stretched to fill the accessory rect stops looking like a system symbol, so the
+    /// glyph draws at its own point size and the rect only ever clamps it. The clamp is one factor
+    /// across both axes, because clamping each axis on its own would distort the glyph exactly the
+    /// way filling the rect did. The origin rounds to whole points so a glyph narrower than its rect
+    /// by an odd number of points does not land on a half point and blur at 1x.
+    static func centeredRect(pointSize: NSSize, in rect: NSRect) -> NSRect {
+        let scale = min(1, rect.width / pointSize.width, rect.height / pointSize.height)
+        let size = NSSize(width: pointSize.width * scale, height: pointSize.height * scale)
+        return NSRect(
+            x: (rect.midX - size.width / 2).rounded(),
+            y: (rect.midY - size.height / 2).rounded(),
+            width: size.width,
+            height: size.height
+        )
+    }
+}

TablePro/Views/Results/Cells/DataGridCellAppearance.swift

@@ -0,0 +1,106 @@
+//
+//  DataGridCellAppearance.swift
+//  TablePro
+//
+
+import AppKit
+
+/// What one data cell looks like, resolved from its content and state.
+///
+/// Separated from the drawing so the decisions are testable without a view and without a graphics
+/// context: which font a NULL takes, when a modified cell keeps its tint, what colour text turns on
+/// a selected row. The renderer that consumes this makes no decisions of its own.
+@MainActor
+struct DataGridCellAppearance: Equatable {
+    let text: String
+    let font: NSFont
+    let textColor: NSColor
+    /// Painted behind the text, for a find match or a modified value.
+    let backgroundTint: NSColor?
+    let accessory: DataGridCellAccessory
+    /// Which symbol the accessory draws, resolved here because it follows the row's state rather
+    /// than anything the renderer can see.
+    let accessoryRole: DataGridCellAccessoryGlyph.Role?
+    /// The cell cursor on a selected row, which the selection fill would swallow a focus ring behind.
+    let drawsFocusBorder: Bool
+    /// The cell cursor on an unselected row.
+    let drawsFocusRing: Bool
+
+    static func resolve(
+        kind: DataGridCellKind,
+        content: DataGridCellContent,
+        state: DataGridCellState,
+        palette: DataGridCellPalette,
+        nullDisplayString: String,
+        onEmphasizedSelection: Bool,
+        hasOverlay: Bool
+    ) -> DataGridCellAppearance {
+        let deletedTextColor = state.visualState.isDeleted ? palette.deletedRowText : nil
+        let font: NSFont
+        let baseColor: NSColor
+
+        switch content.placeholder {
+        case .none:
+            font = palette.regularFont
+            baseColor = deletedTextColor ?? .labelColor
+        case .null, .empty:
+            font = palette.italicFont
+            baseColor = deletedTextColor ?? .secondaryLabelColor
+        case .defaultMarker:
+            font = palette.mediumFont
+            baseColor = deletedTextColor ?? .systemBlue
+        }
+
+        let findTint: NSColor? = state.isCurrentFindMatch ? palette.findMatchTint : nil
+        let modifiedTint: NSColor?
+        if state.visualState.isDeleted || state.visualState.isInserted {
+            modifiedTint = nil
+        } else if state.visualState.isModified(columnIndex: state.columnIndex) {
+            modifiedTint = palette.modifiedColumnTint
+        } else {
+            modifiedTint = nil
+        }
+
+        // A find match keeps its own highlight whatever else is true, and the text turns black
+        // against it. Otherwise a selected row's text takes the selection's own colour, and the
+        // modified tint stands down so the selection fill is not painted over.
+        let backgroundTint: NSColor?
+        let textColor: NSColor
+        if let findTint {
+            backgroundTint = findTint
+            textColor = .black
+        } else if onEmphasizedSelection {
+            backgroundTint = nil
+            textColor = .alternateSelectedControlTextColor
+        } else {
+            backgroundTint = modifiedTint
+            textColor = baseColor
+        }
+
+        let accessory = DataGridCellAccessory.visible(
+            for: kind,
+            isEditable: state.isEditable,
+            rawValue: content.rawValue
+        )
+        let isCursorVisible = state.isFocused && !hasOverlay
+        return DataGridCellAppearance(
+            text: DataGridCellContent.resolvedDisplayText(
+                content.displayText,
+                placeholder: content.placeholder,
+                isLargeDataset: state.isLargeDataset,
+                nullDisplayString: nullDisplayString
+            ),
+            font: font,
+            textColor: textColor,
+            backgroundTint: backgroundTint,
+            accessory: accessory,
+            accessoryRole: DataGridCellAccessoryGlyph.Role(
+                accessory: accessory,
+                isEmphasized: onEmphasizedSelection,
+                isDisabled: state.visualState.isDeleted
+            ),
+            drawsFocusBorder: isCursorVisible && onEmphasizedSelection,
+            drawsFocusRing: isCursorVisible && !onEmphasizedSelection
+        )
+    }
+}

TablePro/Views/Results/Cells/DataGridCellRegistry.swift

@@ -33,21 +33,6 @@ final class DataGridCellRegistry {
             }
     }
 
-    func dequeueCell(in tableView: NSTableView) -> DataGridCellView {
-        if let reused = tableView.makeView(
-            withIdentifier: DataGridCellView.reuseIdentifier,
-            owner: nil
-        ) as? DataGridCellView {
-            reused.nullDisplayString = nullDisplayString
-            return reused
-        }
-
-        let cell = DataGridCellView(frame: .zero)
-        cell.identifier = DataGridCellView.reuseIdentifier
-        cell.accessoryDelegate = accessoryDelegate
-        cell.nullDisplayString = nullDisplayString
-        return cell
-    }
 
     func makeRowNumberCell(
         in tableView: NSTableView,