SG-44961 Fix cp932 UnicodeDecodeError reading subprocess stderr on Japanese Windows - #367
Draft
stevelittlefish wants to merge 2 commits into
Draft
SG-44961 Fix cp932 UnicodeDecodeError reading subprocess stderr on Japanese Windows#367stevelittlefish wants to merge 2 commits into
stevelittlefish wants to merge 2 commits into
Conversation
…panese Windows _call_cmd_win32 read the stdout/stderr temp files without an explicit encoding, so open() used the OS locale codec (cp932 on Japanese Windows) and raised UnicodeDecodeError on non-ASCII subprocess output. Decode both files as utf-8 with errors="replace" so the reader never crashes. Also fix the win32 exception handler, which wrapped the traceback line list inside another list (stderr_lines = [traceback.format_exc().split()]), causing "".join(stderr_lines) in call_cmd to raise "TypeError: sequence item 0: expected str instance, list found". Match the Unix implementation, which assigns the list directly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #367 +/- ##
==========================================
+ Coverage 65.82% 66.09% +0.26%
==========================================
Files 24 24
Lines 1870 1870
==========================================
+ Hits 1231 1236 +5
+ Misses 639 634 -5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Covers both defects fixed in _call_cmd_win32: - Bug 1: a subprocess emitting bytes undecodable by the OS locale codec (0x81 0xff, illegal under cp932/cp1252/utf-8) must not crash the reader. - Bug 2: when the try block raises, the exception handler must leave stderr_lines as a flat list so call_cmd's "".join(...) does not raise TypeError. Tests are skipped on non-Windows since the code path is Windows-only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Jira: SG-44961
Problem
On Japanese Windows,
tk-framework-desktopservercrashes while reading a launched subprocess's stderr. Reported by SUBARU Corp. when opening a PSD from Flow Production Tracking with Photoshop 2026 (27.8) using a Japanese Photoshop UI. The same code is present unchanged onmaster, so this has effectively always been broken on non-utf-8 locales.There are two distinct defects in
command.py, both in_call_cmd_win32.Bug 1 (trigger): the stdout/stderr temp files are read with no explicit encoding, so
open()uses the OS locale codec, which is cp932 on Japanese Windows. Non-ASCII subprocess output (byte0x97) then raisesUnicodeDecodeError: 'cp932' codec can't decode byte 0x97.Bug 2 (turns it into a crash): the win32 exception handler wrapped the traceback line list inside another list (
stderr_lines = [traceback.format_exc().split()]), so"".join(stderr_lines)incall_cmdraisedTypeError: sequence item 0: expected str instance, list found. The Unix path assigns the list directly and is correct.Fix
Decode both temp files as utf-8 with
errors="replace"so the reader never crashes regardless of what the DCC emits, and assign the traceback list directly in the win32 exception handler to match the Unix implementation.Testing
Confirmed against the exact read/join logic: reading a stderr temp file containing byte
0x97as cp932 raisesUnicodeDecodeErrorinside thetry, and the malformed handler then produces theTypeErrorseen in the customer log. With the fix, the same input decodes with a replacement character and does not crash.QA reproduction recipe (Japanese system locale) is documented on the Jira ticket. English/Western Windows (cp1252) does not reproduce this because byte
0x97is valid in cp1252.