[shibayan]

Summary

Closes #411.

OffscreenBuffer measured control characters as zero width (EastAsianWidth treats Cc/Cf as width 0), but wrote them to the console unchanged. The line count and cursor position it computed then disagreed with what the terminal showed:

Text	Terminal	Counted as	Effect
\n, \r in a message, item, or error	Line break / carriage return	Width 0, no new line	Redraw clears too few lines and leaves the old prompt behind
\t	Next tab stop	Width 0	Cursor ends up left of where it should be
ESC[31m	No width (color)	Width 4 ([31m)	Cursor ends up to the right
Cursor movement, clear screen, OSC	Moves the cursor or changes state	—	Corrupts the prompt
\b, \0, …	Moves left or prints nothing	Width 0	Items from external data are hard to tell apart

Typed keys were already safe, because FormBase rejects control characters. Messages, items (TextSelector), answers, default values (Tab), and exception messages can still carry them from external data.

Rules

OffscreenBuffer.Write, which every piece of prompt text goes through, now normalizes text with a new ConsoleText helper. Both the width calculation and the output use the normalized text:

• Line breaks (CRLF, LF, CR, NEL, LS, PS) become lines of the buffer, so multi-line messages and error messages render on several lines and are counted correctly. Cursor restoration sums the physical rows of the lines before the cursor, so a wrapped line before the input row doesn't misplace it.
• Tabs expand with spaces to the next multiple of 8 columns, counted from the start of the line. The column is accumulated incrementally, not remeasured.
• SGR sequences (ESC [ … m whose parameters are only digits, ; and :) are kept and take no width, so colored messages still work. A style ends with its line: each line is written separately and the console driver resets colors after every write, so the next line starts with the prompt's own colors.
• Other escape sequences are removed, because they can move the cursor or change terminal state:
  • other CSI sequences, including private ones such as ESC[>4;2m
  • OSC/DCS/SOS/PM/APC strings
  • two-character ESC sequences
• Other control characters are shown as Control Pictures (␀–␟, ␡); C1 controls are shown as U+FFFD.
• Text being edited (for example, a default value inserted with Tab) shows ESC as ␛ instead of applying sequences. The text before and after the cursor is written separately and must not split a sequence, and the cursor should move over every character.

No configuration was added. Only the display changes: Select/MultiSelect return the original items, and Input returns the text as entered. Plain text, which is almost all text, takes a fast path that returns the original string without allocating.

The docs (en/ja) describe the rules in a new "Control Characters" section under "Unicode Support".

Notes for reviewers

• SGR across lines: an earlier revision restored the effective SGR state at the start of each continuation line. Matching the terminal kept needing special cases: private parameters, default colors and resets against the driver's color, grouped resets, unknown codes, zero padding, and oversized parameters. We chose to let styles end with their line instead. Before this PR, a line break inside a message corrupted the redraw anyway.
• Line breaks are honored rather than flattened in every context, including items. Long items already wrap across rows today, so the same layout behavior applies.
• Grapheme-aware width (CJK, emoji ZWJ sequences, flags, combining marks) is unchanged. Existing EastAsianWidthTests and new cases cover it.

Testing

• ConsoleTextTests:
  • line-break variants
  • tab stops: after CJK text, emoji, and SGR, from a start column, after a line break, and 10,000 tabs
  • escape-sequence shapes: SGR, including the colon form, is kept with zero width; private CSI, other CSI, OSC including hyperlinks, two-char ESC, and unterminated and lone ESC are handled
  • SGR ending with its line
  • visible symbols for C0, DEL, and C1
  • unchanged printable text
• Form regressions:
  • multi-line message and multi-line error redraws
  • a wrapped line before the input row places the cursor on the input row
  • Select items with control characters still return the original value
  • escape sequences in edited input are shown literally
• Each form regression fails without its fix.
• dotnet build, dotnet test (1344 passed), and dotnet format --verify-no-changes.
• Native AOT verification (win-x64): 65/65 for net8.0 and net10.0.
• VitePress docs build.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

Private escape commands bypass filtering, wrapped multiline prompts misplace the cursor, and multiline messages lose SGR formatting.

3 open issues

• Filter private CSI parameters before preserving SGR state · New
• Preserve SGR formatting across generated line boundaries · New
• Restore cursor using physical rows for wrapped buffer lines · New

What changed in this PR

Normalizes prompt text before measurement and output to improve rendering consistency for external data, addressing #411.

Changes:

• Adds handling for line breaks, tabs, escape sequences, and visible control symbols.
• Integrates normalization into buffering and adds regression tests.
• Documents display rules in English and Japanese.

File	Description
tests/​Sharprompt.Tests/​Internal/​ConsoleTextTests.cs	Tests normalization and display widths.
tests/​Sharprompt.Tests/​Forms/​FormInteractionTests.cs	Tests multiline redraws and control-character rendering.
src/​Sharprompt/​Internal/​OffscreenBuffer.cs	Normalizes buffered text before measurement.
src/​Sharprompt/​Internal/​ConsoleText.cs	Implements normalization and escape-aware measurement.
docs/​ja/​advanced.md	Documents control-character behavior in Japanese.
docs/​en/​advanced.md	Documents control-character behavior in English.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Cursor-separated input can expose escape fragments, while tab measurement and SGR replay introduce quadratic processing or output growth.

1 open issue

• Compact SGR state instead of replaying full formatting history · New

2 previously missed issues

In code that hasn't changed since last review

Avoid quadratic tab normalization during prompt redraws

src/​Sharprompt/​Internal/​ConsoleText.cs:59

Each tab copies and remeasures the entire expanded line prefix, making normalization quadratic for tab-heavy external text. For example, 10,000 tabs produce 80,000 spaces but require scanning nearly 400 million characters. This work repeats on every prompt redraw and can stall interactive input. Track the display column incrementally and measure only newly appended text runs, preserving grapheme-aware widths and zero-width SGR handling.

Normalize complete input before splitting around the cursor

src/​Sharprompt/​Internal/​OffscreenBuffer.cs:67

OffscreenBufferExtensions.WriteInput writes the raw text before and after the cursor separately, so normalizing each write can split an escape sequence. With DefaultValue = "\x1b[31mred\x1b[0m", pressing Tab then LeftArrow changes the visible input from red to redm: the first write drops the incomplete reset, while the second prints its final m. Normalize the complete input and map the raw cursor position to its displayed position before splitting the output. Add a regression covering Tab followed by cursor movement within an escape sequence.

3 resolved since last review

• Filter private CSI parameters before preserving SGR state
• Restore cursor using physical rows for wrapped buffer lines
• Preserve SGR formatting across generated line boundaries

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Style continuation can restore incorrect formatting and generate quadratic output for distinct unsupported codes.

2 open issues

• Explicit default foreground reset is lost across lines · New
• SGR 54 reset is replayed after framing enable · New

1 previously missed issue

In code that hasn't changed since last review

Unknown SGR codes cause quadratic continuation output

src/​Sharprompt/​Internal/​ConsoleText.cs:324

The fallback retains every distinct unrecognized SGR code, and each newline replays all accumulated entries. For 1,024 lines containing codes 1000–2023 (ESC[1000mx\n, etc.), 9,216 input characters produce 2,634,240 normalized characters. The compact-state change therefore still permits quadratic allocation and terminal output for external multiline text. Track supported attributes explicitly and exclude unknown codes from continuation state; add a regression using distinct unknown codes.

1 resolved since last review

• Compact SGR state instead of replaying full formatting history

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Style continuation misparses zero-padded color modes and can cause quadratic output expansion.

1 open issue

• Zero-padded color modes lose or reset continuation colors · New

1 previously missed issue

In code that hasn't changed since last review

Oversized color tokens cause quadratic continuation expansion

src/​Sharprompt/​Internal/​ConsoleText.cs:373

The number of stored attributes is bounded, but each value can still contain an arbitrarily long raw token. For example, ESC[38:2:: followed by 2,048 zeroes, 255:0:0m, and 2,048 newlines expands 4,112 input characters into 4,229,136 normalized characters because the entire color token is replayed on every line. External text can therefore cause quadratic allocation and terminal output on each redraw, even with just one known attribute. Canonicalize supported parameters or bound the stored representation without truncating escape sequences, and add a regression for one oversized known attribute followed by many newlines.

2 resolved since last review

• SGR 54 reset is replayed after framing enable
• Explicit default foreground reset is lost across lines

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The new escape parser and cross-line style restoration need final human validation against real cross-platform terminals.

No open issues 🎉

1 resolved since last review

• Zero-padded color modes lose or reset continuation colors

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Retained SGR attributes need explicit resetting to prevent styles such as conceal from affecting subsequent input.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Fragments may leak SGR attributes across lines

src/​Sharprompt/​Internal/​OffscreenBuffer.cs:78

These fragments rely on DefaultConsoleDriver.Write calling Console.ResetColor() to end SGR styles. That restores colors, but does not guarantee resetting attributes such as underline or conceal. On terminals whose color reset leaves those attributes active, ESC[8mfirst\nsecond can therefore conceal the second line and subsequent input. Append an explicit full SGR reset to each fragment containing retained SGR sequences, and add a rendering-level regression that checks the reset is emitted before the next line.

_{Review effort: Balanced}