fix(detect): treat C0 control characters as valid UTF-8

[hiros0921]

Fixes #49.

Problem

isUTF8() accepts only TAB, LF, CR and printable ASCII, so a UTF-8 string that contains any other C0 control character is not detected as UTF-8.

const utf8 = Buffer.from('UTF8の文字列です', 'utf8');
const bs = Buffer.from([0x08]);

Encoding.detect(bs);                          // 'ASCII'
Encoding.detect(utf8);                        // 'UTF8'
Encoding.detect(Buffer.concat([utf8, bs]));   // 'UNICODE'  <- expected 'UTF8'

This is inconsistent with isASCII(), which accepts every byte in the ASCII range except ESC (0x1B). The same byte is reported as ASCII but not as UTF-8, even though every ASCII byte is valid UTF-8 by definition.

Measured on 2.2.0, appending one control character to the same UTF-8 string:

appended byte	detect()
0x08 BS	UNICODE
0x0B VT	UNICODE
0x0C FF	UNICODE
0x1B ESC	UNICODE
0x7F DEL	UNICODE
0x09 TAB, 0x0A LF, 0x0D CR	UTF8
0x00 NUL, 0x07 BEL	BINARY

Change

Accept the whole ASCII range except ESC:

if (b <= 0x7F && b !== 0x1B) {
  continue;
}

ESC is left out to match isASCII(), so that ISO-2022-JP data is not reported as UTF-8 by a direct isUTF8() call. Please let me know if you would rather have ESC accepted as well — detect() is unaffected either way, since isJIS runs before UTF8.

Why detect() does not get looser

Encoding.orders is UTF32 → UTF16 → BINARY → ASCII → JIS → UTF8 → EUCJP → SJIS → UNICODE. isBINARY (0x00-0x07, 0xFF) and isJIS are both tried before UTF8, so the inputs they already claim are unchanged. Only the previously misdetected inputs change:

input	before	after
UTF-8 + BS / VT / FF / DEL	UNICODE	UTF8
UTF-8 + NUL / BEL	BINARY	BINARY (unchanged)
Shift_JIS / EUC-JP / ISO-2022-JP / UTF-8 / ASCII	correct	unchanged

Tests

Added detect > UTF-8 with C0 control characters. The existing suite still passes (178 → 179).

I checked that the new test actually fails without the change:

without the fix:  1 failing   AssertionError: 'UNICODE' == 'UTF8'
with the fix:     179 passing

The test also pins the two behaviours that should not change: 0x00 still yields BINARY, and ISO-2022-JP data still yields JIS while detect(jis, 'UTF8') stays false.

Notes

I did not commit encoding.js / encoding.min.js, since #62 and #63 only touched src/ and tests/. Happy to rebuild them if you prefer them in the PR.

[polygonplanet]

@hiros0921
Thank you for the PR!
isUTF8() intentionally used a stricter range in the early days to reduce false positives when detecting other encodings. But that can now cause problems, and according to RFC 3629, the ASCII range (0x00 - 0x7F) is valid UTF-8. I will go ahead and merge this. Thank you!