Subgrid extraction

Overview

Subgrid extraction is a setting on dataset-output nodes (File Input, Operation, Filter, and the other types that emit a dataset). In Settings, turn on Subgrid extraction. The editor then shows a Subgrid tab that paints from that node’s reserved _source sheet.

The node’s output becomes one keyed workbook:

Unextracted sheets pass through
A sheet you painted is replaced by its named region keys (and {Name}_footer when that region has footer rows)
_source always holds the full native grid

Example: {sheet1, region1, region2, sheet3, _source}. Schema-from-node extracts pass-through sheets, regions, and footer sheets, and skips _source.

Do not add a new standalone Subgrid node. Existing Subgrid nodes on canvases stay as adapters: they still slice the upstream input, keep the old table or keyed shapes (no _source), and are hidden from Add node.

The standalone adapter still uses the same rectangle math below.

Picker

On File Input, the picker is the Source Data tab. On other dataset nodes it is the Subgrid tab. Region cards, Auto-create, Clear all, and the mini-map sit in the right-hand sidebar.

Click an unpainted source cell to add a one-cell region, or drag a cell span to add a larger region. Later gestures add more regions. Overlaps are allowed. Each painted region uses the same colour on its grid border, sidebar card, and mini-map box. Select a region on the grid, then rename or delete it from its card.
Each From and To entry has a crosshair button. Click the crosshair beside From or To, then click one cell to move only that corner. Escape cancels. The picker keeps at least one row and one column when you click across the opposite corner.
Set Header rows and Footer rows with the numeric controls in the selected region card. Those rows paint with a stronger fill along the top and bottom of the bound rectangle. Drag a selected region’s edges or corners to resize the bounds. Drag a column header’s right edge to change that column’s width. Zoom scales the table down so you can see more cells.
Half-open rectangle in row and column index space: [r0, r1) × [c0, c1)
Omitted end bounds extend through the last row or column of the current extent
Auto Select with AI (canvas node toolbar on the adapter) proposes a data-only rectangle for the active region without opening a chat thread
Auto width and Auto height are independent. Fixed size is a number (c1 - c0 columns, r1 - r0 rows). An auto axis grows at run time until the next empty row or column segment
Each region card holds name, From (top-left A1 cell), To (bottom-right A1 cell), independent From and To crosshair actions, Expand when adjacent data is available, Auto width, Auto height, fixed size, Header rows, Header delimiter, and Footer rows. Auto-create regions and Clear all regions are grid-wide.
The default-open Data map paints the whole source grid: every cell is an 8 by 4 pixel block, then that bitmap scales uniformly so its longest edge fits the sidebar. Coloured pixels show field type (number, date, text, boolean) with empty cells left blank. The key lists only the field types that appear in the source. A slate outline marks the occupied hull. Painted region boxes stay proportional to the grid. Click a box to select that region, or collapse the map from its heading
Large source tables load buffered row windows as you scroll. Overlapping rows stay visible while the next window loads and while a workflow run refreshes the source. The grid stays visible when a window starts on empty rows.
Native extraction always emits a keyed workbook because _source is always present. The legacy adapter still emits a flat table for one region with no footer.

Header rows

When a region has Header rows set, Subgrid promotes that many leading selected rows to field names. File Input must still show those labels as text (not Missing Data) so they can become field names.

Type the count in the selected region card. Auto-create regions can also propose a header count when it detects a header row. Clicking source cells only selects or creates regions; it does not change header or footer counts. Header rows use a stronger fill inside the painted rectangle.

0 keeps the upstream column keys
1 promotes the first selected row
2 or more flattens stacked rows. File Input writes an Excel merge into every cell of that merge, so a Buy group over Price becomes Buy.Price
Header delimiter (shown when the count is 2 or more) joins those stacked cells. The default is a dot
Those header rows are omitted from the data rows
A blank leaf cell keeps the upstream key
An empty parent cell is not filled from the left. A group that ends at P2 (Fees / Expenses) does not attach to Q3 All Income
Consecutive duplicate labels collapse (Ending Market Value on two rows stays one segment)
Explicit column names still override flattened labels

When a region has Footer rows set, Subgrid peels that many trailing selected rows off the body after headers. Footer rows keep the same column names as the body. They are not used as headers.

Type the count in the selected region card.

0 keeps trailing rows in the body
1 or more omits that many rows from the bottom of the remaining slice
Headers are taken first. Footer count cannot consume header rows
Footer rows paint with the same stronger fill as headers, along the bottom of the bound rectangle

A region named Holdings with footer rows emits two sheets: Holdings (body) and Holdings_footer (totals). A single region with footer rows uses the same keyed workbook shape as multiple regions, so the Data tab shows both sheets. Quick Add Schema from Subgrid adds an extract for output.Holdings_footer._sheetData.0 as well.

Multiple regions

On the picker, click an unpainted source cell to add a one-cell region, ready to expand or resize. Drag a cell span to add a larger region (for example A1 to C6). To move one corner, open the region card, click the crosshair beside From or To, then click its new cell. Overlapping regions are allowed. Region border colours match their sidebar cards and mini-map boxes without placing labels over source data. Hover a region card to strengthen its matching grid border and fade the other region overlays. Click a painted region or its sidebar card to select it; that card then shows its name, From and To cells, size, header, and footer controls. Rename the region in its card, then press Enter or leave the field to apply the change. The bullseye in the card header scrolls the picker to that region’s top-left cell. Delete is next to it. Drag the selected border’s edges or corners to resize the slice. Drag a column letter’s right edge to change width. Use the zoom slider to scale the table down and see more cells. The node still has a single output port. The default-open Data map paints every source cell as an 8 by 4 pixel block, then scales that bitmap uniformly to fit the sidebar. On File Input Source Data it paints from the loaded grid. On a Subgrid node it uses a bounded server raster. Coloured pixels show field type without sending extra source values to the browser. The key lists only the field types that appear in the source. A slate outline marks the occupied hull, and region boxes stay proportional above that layer. Click a mini-map box to select that region, or collapse the map from its heading.

One region, no footer: output is a table of the sliced rows
Two or more regions, or any region with footer rows: output is one row whose keys are the region names (and {Name}_footer when that region has footer rows). Each value is { _sheetName, _sheetData }, the same shape File Input uses for a multi-sheet workbook

Quick Add Schema from Subgrid creates a hub Schema on that output. When the output is keyed, it also adds one extract Schema per sheet, wired to output.{Name}._sheetData.0, so you can treat each slice (including {Name}_footer) as its own table.

Auto-create regions scans the source table on the workflow engine and adds a region for each 4-connected block of filled cells, including a single isolated cell. The browser receives only those origin cells; it does not download the full grid. Blocks that only touch on a diagonal stay separate. Existing painted regions are kept; a block that already sits inside a painted rectangle is skipped. Clear all regions removes every painted rectangle.

If a painted region has data in the neighbouring row or column along its edge (above, below, left, or right), an Expand control appears below From and To in that region’s card and the node reports a warning. Expand grows the rectangle until that edge is empty and clears the warning when no expandable edge remains. Auto width skips the right edge. Auto height skips the bottom edge, because the processor already grows those axes at run time.

Auto Select with AI

Auto Select with AI is a one-shot suggestion. It proposes a data-only regionRect and, when slice keys are empty, numeric-only, duplicate, or __EMPTY*, optional columnNames. It does not set headerRowCount.

This call is thread-free. It uses your signed-in session and the shared AI action rate limit. It does not apply the chat per-conversation spending cap. If Auto Select fails, the node keeps its previous selection.

Reload Convex after a backend change so the action you run matches the deployed code.

Auto width and Auto height

Each region can grow width and height independently.

Fixed width / height: the size is a number on the rectangle (c1 - c0 columns, r1 - r0 rows). You can type Width or Height on the Subgrid tab, or drag the matching edge.
Auto width: at run time the node keeps adding columns from c0 until the next column segment in the row window is empty, or the file edge.
Auto height: at run time the node keeps adding rows from r0 until the next row segment in the column window is empty, or the file edge.

You can mix them. A fixed-width, auto-height region keeps the chosen columns and grows down until that column span is blank. Stored end bounds stay on the region so turning a toggle off returns to the last manual size.

A blank row stops auto height even if more values sit further down. A blank column stops auto width even if more values sit further right.
Both auto flags together grow from a 1x1 origin, one axis at a time, so a footer below a blank row does not reopen width.

File Input – Load the workbook or table that Subgrid slices
Workflow orchestration – Canvas nodes, hashing, and server-side runs