Skip to content

feat(text): Load extra string files next to the string table - #3460

Open
tintinhamans wants to merge 4 commits into
TheSuperHackers:mainfrom
tintinhamans:arctic/feat/layered-string-files
Open

tintinhamans wants to merge 4 commits into
TheSuperHackers:mainfrom
tintinhamans:arctic/feat/layered-string-files

Conversation

@tintinhamans

@tintinhamans tintinhamans commented Oct 10, 2026 •

Copy link
Copy Markdown

Merge after #3458. Closes #255

Right now the game reads one string file per language, so changing two labels means shipping a full generals.csf.

With this, every other .str and .csf in data\<Language>\ and data\ loads too, loose or in a big. A file only needs the labels it changes.

Precedence is the rule modders already know from bigs: whatever sorts first wins. Language folder before data\, files in name order, .str before .csf of the same name, and the game's own string file last so it only fills in what nothing else set. 000_mymod.str beats zz_other.str the same way 000_mymod.big beats zz_other.big.

Retail 1.04 has nothing extra to load. Steam 1.05 has Data\Patch.str in PatchData.big (the two Custom Mission labels) which the 1.05 exe reads and this code never did. That gets picked up now.

Two files, same label Result
Same path, loose and in a big Only the loose one loads
Same path in two bigs Only the first big by name loads
data\<Language>\ vs data\ Language folder wins
Same folder, two names First by name, .str beats .csf of the same name
Any extra file vs generals.csf The extra file wins

The first two rows are just how the file system already works. map.str stays a separate table that is only checked for labels the main one doesn't have. If parsing fails, the whole map string file is discarded rather than keeping a partial result.

Three commits: the parsers first push into a vector instead of a pre-sized array, then the parsers get bounds, then the layering. The vector drops the second pass over every str file that only counted END lines, the 500 entry slack, and the separate map.str parser, which was a copy of the str parser with a language filter. Malformed files get dropped instead of corrupting memory. For a csf that is oversized labels or text, a header count the file can't hold, a truncated file (including a partial record ID), or label and string counts that don't match the header. For a str it is a line, string or speech name longer than the buffer, a label with no quoted text, or a quote that never closes, which hangs startup on main. Explicit empty strings ("") remain valid. Every read is checked, so a bogus string count no longer spins. A Generals.str that fails to parse falls back to generals.csf.

Tested: INI CRC unchanged (FEAAE3F3 on main, with this, and with extra str files around). Same 11 labels out of LF, CRLF and mixed, loose and in a big, so #255's line ending concern doesn't apply. 22 scenarios in game, sheet below.

image

@coderabbitai

coderabbitai Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fe6845ad-4833-4979-92df-a15bb942f06b
📥 Commits

Reviewing files that changed from the base of the PR and between 325ee9304a88be7dcae745dacb52c54915d3f893 and 3fcdd098e8aa45512b17b89f04d3fdd646ba23b4.

📒 Files selected for processing: 1
  • Core/GameEngine/Source/GameClient/GameText.cpp

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

Walkthrough

The text manager merges .str and .csf records from language-specific and neutral directories, validates parsed input, and loads map strings through the shared parser. Archive directory traversal now follows the recursive-search option.

Changes

Text file loading

Layer / File(s) Summary
File selection and entry merging
Core/GameEngine/Source/GameClient/GameText.cpp
The manager collects and orders language-specific and neutral .str and .csf files. It merges entries in order, keeps the first definition for each label, and tries the configured .str file before falling back to the configured .csf file.
Vector parsing and input validation
Core/GameEngine/Source/GameClient/GameText.cpp
CSF and STR parsers return records in vectors and validate input reads and lengths. Quote and wave parsing report incomplete or oversized input. Map loading uses the shared STR parser with language filtering enabled.

Directory search behavior

Layer / File(s) Summary
Respect the recursive-search option
Core/GameEngine/Source/Common/System/ArchiveFile.cpp
Directory traversal skips child directories when recursive search is disabled.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature · Severity of issue fixed: Medium

Suggested reviewers: xezon

Merge Risk: 🔵 Low · up to 3fcdd

Extra strings can be lost if both primary string files fail. The remaining issue is narrow, but it warrants a fix or explicit acceptance before merging.

Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Issue #255 requires reliable parsing of plain .str files in .big archives. GameTextManager::collectStringFiles discovers .str files through TheFileSystem, and mergeStringFile routes them t…
Out of Scope Changes check Passed The archive file discovery, string parsing checks, vector-based parsing, and precedence handling support finding and safely loading string resources. No unrelated change is demonstrated.
Title check Passed The title clearly and concisely describes the main change: loading additional string files alongside the primary string table.
Description check Passed The description directly explains the additional .str and .csf loading behavior, precedence rules, parser validation, fallback behavior, and testing.
Full details: Linked Issues check

Explanation

Issue #255 requires reliable parsing of plain .str files in .big archives. GameTextManager::collectStringFiles discovers .str files through TheFileSystem, and mergeStringFile routes them to parseStringFile. The PR description reports matching results for LF, CRLF, and mixed line endings in loose files and .big archives. This supports the requirement.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 32e49e1c-a26d-4c7c-847c-55d9760cf9b9
📥 Commits

Reviewing files that changed from the base of the PR and between ab568af and ad6030d777a2481e46b3391196ae93290a264740.

📒 Files selected for processing (2)
  • Core/GameEngine/Source/Common/System/ArchiveFile.cpp
  • Core/GameEngine/Source/GameClient/GameText.cpp

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

Comment thread Core/GameEngine/Source/GameClient/GameText.cpp Outdated
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch 4 times, most recently from 7ea64ce to a93f4a7 Compare October 10, 2026 06:28

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

🟠 Major · Validate the CSF label count before allocation. · GameText.cpp:443

Core/GameEngine/Source/GameClient/GameText.cpp:443
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Validate the CSF label count before allocation.

getCSFInfo accepts a signed num_labels after checking only the header ID. mergeStringFile then passes that value to NEW StringInfo[capacity] before parseCSF can reject excess records. A negative or file-inconsistent count can request an invalid or excessive allocation and prevent text initialization.

Reject negative counts and counts larger than the remaining file bytes divided by the minimum label-record size.

🐛 Suggested fix
 			if ( header.id == CSF_ID )
 			{
-				textCount = header.num_labels;
+				const Int remainingBytes = file->size() - file->position();
+				const Int minLabelBytes = sizeof(Int) * 3;
+
+				if ( remainingBytes < 0 || header.num_labels < 0 ||
+					header.num_labels > remainingBytes / minLabelBytes )
+				{
+					break;
+				}
+
+				textCount = header.num_labels;
 
 				if ( header.version >= 2 )
 				{
 					language = (LanguageID) header.langid;

ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b595e7f8-c6f3-4828-a350-957039cab174
📥 Commits

Reviewing files that changed from the base of the PR and between 3f9d3b223c006b417f36b2578d2a2cc06866c578 and a93f4a701dc2205c49476af57d0e2cbe38b70f0f.

📒 Files selected for processing (1)
  • Core/GameEngine/Source/GameClient/GameText.cpp

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch 2 times, most recently from 01f10f4 to c79f7c4 Compare October 10, 2026 06:45
@tintinhamans
tintinhamans marked this pull request as ready for review October 10, 2026 06:54
Comment thread Core/GameEngine/Source/GameClient/GameText.cpp Outdated
@greptile-apps

greptile-apps Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

Loads extra .str and .csf files before the main string table.

  • Extra string files fill in and replace labels in the game table.
  • String tables are sized from parsed entries and checked as they load.
  • Archive file listings now follow the caller's subfolder choice.

No new actionable issues were found. The five earlier, unnumbered findings are addressed by the current read checks, buffer limits, count checks, and CSF fallback.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Language folder extras, sorted by name] --> B[Data folder extras, sorted by name]
  B --> C{Main STR loads?}
  C -- Yes --> E[Build text lookup]
  C -- No --> D[Load main CSF]
  D --> E
  F[Keep the first definition of each label] -.-> A
  F -.-> B
  F -.-> D
Loading

Reviews (8) · Last reviewed commit: "feat(text): Load extra string files next..." · Reviewed by Greptile

Comment thread Core/GameEngine/Source/GameClient/GameText.cpp Outdated
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch 2 times, most recently from 99ffeae to b754b90 Compare October 10, 2026 07:01
@tintinhamans
tintinhamans marked this pull request as draft October 10, 2026 07:03
Comment thread Core/GameEngine/Source/GameClient/GameText.cpp Outdated
Comment thread Core/GameEngine/Source/GameClient/GameText.cpp Outdated
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch from b754b90 to 01eb4bd Compare October 10, 2026 07:15
@tintinhamans tintinhamans changed the title feat(text): Load extra string files next to the string table, first file wins feat(text): Load extra string files next to the string table Oct 10, 2026
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch from 01eb4bd to bd77ebd Compare October 10, 2026 07:32
@tintinhamans
tintinhamans marked this pull request as ready for review October 10, 2026 07:36
Comment thread Core/GameEngine/Source/GameClient/GameText.cpp
@tintinhamans
tintinhamans marked this pull request as draft October 10, 2026 07:39

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: bb49ef62-8a47-4902-a46f-615802606014
📥 Commits

Reviewing files that changed from the base of the PR and between 01eb4bdf89e1ee6525cd9960426607fd64820146 and bd77ebda75252bd4c709d4d909399387473d7ab1.

📒 Files selected for processing (1)
  • Core/GameEngine/Source/GameClient/GameText.cpp

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 1 remain after this review.

Comment thread Core/GameEngine/Source/GameClient/GameText.cpp Outdated
Comment thread Core/GameEngine/Source/GameClient/GameText.cpp
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch 2 times, most recently from d03882c to a835031 Compare October 10, 2026 08:00

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8daf08b2-bd6d-478d-91d3-dd2b5548bd16
📥 Commits

Reviewing files that changed from the base of the PR and between bd77ebda75252bd4c709d4d909399387473d7ab1 and a835031c34c22081a2bbbca1c0fc02af7420b792.

📒 Files selected for processing (1)
  • Core/GameEngine/Source/GameClient/GameText.cpp

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 1 remain after this review.

Comment thread Core/GameEngine/Source/GameClient/GameText.cpp
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch from a835031 to 224c143 Compare October 10, 2026 08:13
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch from 224c143 to b7b7332 Compare October 10, 2026 08:19
@tintinhamans
tintinhamans marked this pull request as ready for review October 10, 2026 08:20
Comment thread Core/GameEngine/Source/GameClient/GameText.cpp Outdated
Comment thread Core/GameEngine/Source/GameClient/GameText.cpp Outdated
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch from b7b7332 to c32efb6 Compare October 12, 2026 20:32
@tintinhamans tintinhamans added Enhancement Is new feature or request Text Is text, string, localization related Gen Relates to Generals ZH Relates to Zero Hour labels Oct 12, 2026
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch from c32efb6 to 325ee93 Compare October 18, 2026 13:19

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 81013c44-3693-435c-b2d5-0e1a9cafa01f
📥 Commits

Reviewing files that changed from the base of the PR and between 224c143b40a86db74753171cc439ad0d2f8db4cd and 325ee9304a88be7dcae745dacb52c54915d3f893.

📒 Files selected for processing: 2
  • Core/GameEngine/Source/Common/System/ArchiveFile.cpp
  • Core/GameEngine/Source/GameClient/GameText.cpp

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 1 remain after this review.

Comment thread Core/GameEngine/Source/GameClient/GameText.cpp Outdated
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch from 325ee93 to 3fcdd09 Compare October 18, 2026 13:36
@tintinhamans
tintinhamans force-pushed the arctic/feat/layered-string-files branch from 3fcdd09 to 4e268ac Compare October 18, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Enhancement Is new feature or request Gen Relates to Generals Text Is text, string, localization related ZH Relates to Zero Hour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Plain string files cannot be read from within .big files

2 participants