Skip to content

Reimplement 88 Map::TileMapState functions at 100% - #366

Open
gynt wants to merge 34 commits into
mainfrom
reimpl/TileMapState-100
Open

gynt wants to merge 34 commits into
mainfrom
reimpl/TileMapState-100

Conversation

@gynt

@gynt gynt commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Reimplements 88 Map::TileMapState functions that match the original at 100%, taken from reimpl/TileMapState. The 17 TileMapState files already on main are unchanged.

  • Every function here measured 100% (raw, or normalized where only call targets differ) on reimpl/TileMapState, and again in a trial build of this branch with the data and function resolvers enabled.
  • The status lines of all 88 are set to 100.0% | Reimplemented.

Header changes

Each is in its own commit and limited to what these functions use:

  • Moat: tile is int (main's own UpgradeMapLogicToVersion_Unknown3 already reads it as *(uint*)&). x/y are short and fillProgress is char; both are tested signed in the original.
  • Rock::x/y: short, read with movsx.
  • Unit::field_0x83: a byte field rather than padding; triggerLoweredView clears it.
  • TileMapState:
    • signed int counters and flags: maxPitchDitchCount, temporaryTerrainTypeIndex, SEC_Section1052..1054, the forceUpdate* flags and buildingMin/MaxHeight
    • MacroLayer is short, read with movsx everywhere
    • mapping40x40 is char
    • field_0x5548d0 is a field rather than padding
  • validateWallPlacementAtTile takes its mapper command as short. The original compares it as a word, and that is what took the function from 81% to 100%.

These are generated headers, so the same changes need to go into Ghidra too.

LTCG

16 of these functions are added to cmake/compiler-flags-gl.txt, together with their already-finished callees, getBuildingSizeIndexMappingData among them.

  • The original keeps this in ecx across those calls. Only /GL with the callee compiled and enabled reproduces that; without it these functions measure 40-65%.
  • Only entries that measured better are listed.

Behaviour fixes

  • spawnEraserTileEffect no longer clears L_BUILDING; the original's mask is ~0x410b00.
  • placeDefensiveStructureTile gets its missing M_MAPPER_STAIR5 arm.
  • setupAllMapSections calls the empty 0x469F10 with UnitsState/LandscapeState in ecx, as the original does, through a __fastcall cast of DoNothing (as processGameTick does).

Note

updateBuildingPlacementRotationPreview, which is already on main, measures 46% in the trial build. It needs its /GL callee determineBuildingPlacementRotation, which is not finished yet and so is not part of this PR. This PR does not change that function.

Functions

Constructor_TileMapState, advanceMoatDigProgress, applyRockGraphicsToFootprint, changeMapOrientation, clearBuildingDisplayFlagsAndEntities, clearBuildingFootprintAndRemoveSiegeTower, clearBuildingFootprintAndResetUnits, clearBuildingFootprintWithEdgeRubble, clearBuildingFromTerrain, clearBuildingTilesAndTrees, clearDrawBridgeWater, clearFixedSizeTwoBuildingFootprint, clearInvalidMoatEntries, clearMoat, clearMoatDataAtTile, collectCliffEdgeTilesForClimbData, computeClimbRampRotation, computeTileAlongAxisOffset, computeWallCornerRenderRotation, countLogicPropertyInSurroundingTiles, countMoatsOwnedByEachPlayer, countUnfinishedMoatTilesForPlayer, countUnownedPitchDitches, createMoatData, destroyPitchDitch, destroyWallsOfPlayer, findNearestValidDigTileNearTarget, findTileInSameAreaAndNoTooHeightDifference, getBuildingHurtSFXID, getBuildingSizeForCommandBuildingType, getBuildingSizeIndexMappingData, getCastleBuildRangeForMapSize, getMaxWallHeightInBrushArea, getNonFarmFieldBuildingHealthAtTileOr1000, getOrientationThatIsWallTowerOrGatehouse, getPitchDitchIDForTile, getRubbleGraphicStageForDamageLevel, getTileForBrush, getTotalHeightAtTile, getWallFlagForOrientedDirection, hasHigherNeighborWithStairs, hasHigherPlainNeighborWithWallOrGatehouse, hasOnlyTowerNeighborsNoWalls, increaseHeightForTunnelSingleTile, increaseHeightForTunnelWithBrush, isCliffDropInDirection, isFootprintIndexOnEdge, isTileEnclosedByWalls, isTileEnclosedByWallsOrGates, isWallConnectionHeightValid, isWallCornerForCardinalDirection, isWallCornerForDiagonalDirection, markBuildingFootprintFlag, noop1, placeDefensiveStructureTile, planMoat, rebuildShowHiLayerFromHeights, removeMoat, resetAreaBasedOnLogicalLayer, resetHeightAndMapBorders, resetTileAndClearMoat, resetTileToDefaultState, returnMoatIDForPlayerIDs, returnOwnedMoatAtTile, rotateFearFactorBuildingVariations, setChangedLayerToThreeAndMapping0x40x40, setMapSize, setSignpostDistanceForCampaignMission, setupAllMapSections, setupBuildingSizeIndexMapping, setupLogicalMapBorders, setupTileMapSections, spawnEraserTileEffect, storeMinAndMaxHeightOfArea, swapMoatOwnership, swapPitchOwnership, swapWallOwnership, triggerLoweredView, unmarkBuildingFootprintFlag, updateAreaBasedOnSurrounding, updateGameRelatedValue, updateMacroLayer, updatePathLinkagesForBuilding, updatePitchDitches, updateShowHiLayerOrResetChangedLayer, upgradeMapFormatLogicLayer, validateBuildingPlacementAtTile, validateWallPlacementAtTile

🤖 Generated with Claude Code

https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC

gynt and others added 6 commits November 7, 2026 14:07
The original reads tile as a dword (main's own UpgradeMapLogicToVersion_Unknown3
already does *(uint*)&moats[i].tile), x and y with movsx, and tests
fillProgress signed (test al, al; jg / cmp dl, 4; jl).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
applyRockGraphicsToFootprint reads both with movsx.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
triggerLoweredView clears it next to heightDiv10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
maxPitchDitchCount, temporaryTerrainTypeIndex, SEC_Section1052..1054, the
forceUpdate flags and buildingMin/MaxHeight are compared signed in the
original; MacroLayer is read with movsx everywhere; mapping40x40 holds
char; the dword at 0x5548D0 is a field (setupAllMapSections clears it),
not padding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
The original compares it as a word; with the short type its
reimplementation reaches 100% (from 81%).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
From reimpl/TileMapState: every function that measures 100% there (raw or
normalized, i.e. only call targets differ), and again 100% in a trial
build of this branch with the data and function resolvers enabled.

- 88 sources in src/OpenSHC/Map/TileMapState/ (the 17 already on main are
  unchanged), with their status lines set to 100.0% Reimplemented
- cmake/compiler-flags-gl.txt: the 16 of them whose original keeps this
  in ecx across an already finished callee (getBuildingSizeIndexMappingData,
  setSignpostDistanceForCampaignMission, ...); /GL plus the enabled callee
  is what reproduces that, without it they measure 40-65%

Behaviour fixes carried over: spawnEraserTileEffect no longer clears
L_BUILDING (the original mask is ~0x410b00), setupAllMapSections calls the
empty 0x469F10 with UnitsState/LandscapeState in ecx as the original does,
placeDefensiveStructureTile has its missing M_MAPPER_STAIR5 arm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
@github-actions github-actions Bot added the declaration/type change This requires changes to the generated/exported types or definitions. label Nov 7, 2026
gynt and others added 3 commits November 7, 2026 17:55
… sources

Measured against a reccmp baseline of this branch before and after, in two
rounds: all 88 functions come out byte-identical in match, average 99.92,
every one still 100% raw or normalized. Nothing had to be reverted.

- rule 18: 70 hex literals to decimal, bit masks left in hex; 399/400 to
  MC_MAP_XY_LIMIT_INCLUSIVE/MC_MAP_XY_LIMIT, 80400 to MC_MAP_TILES_COUNT and
  398 to MC_MAP_XY_LIMIT - 2, matching what the sibling files already did
- rule 16: param_N named in 14 files. Four of those parameters are unused in
  the body, and since the function matches at 100% the original does not use
  them either, so they are called unused rather than given a guessed meaning
- rule 4: the three byte-sized layer fills in setupAllMapSections take
  sizeof of the layer. PillarGFXLayer's is a short fill, so its 80400 is an
  element count and stays MC_MAP_TILES_COUNT
- rule 1: the trailing return; of getBuildingSizeIndexMappingData and noop1
- rule 23: the ternary in computeWallCornerRenderRotation is another else-if
- rule 9: resetHeightAndMapBorders bounds its loop on a plain local, so its
  do-while became a for
- rule 21: the twelve remaining do-while loops carry a note instead. Their
  bound is this->constructionTileCount, which the getBuildingSizeIndexMappingData
  call inside the body writes, so a leading test would read the previous
  footprint's count. Those loops are required for correctness, not just for
  the match, and converting them would have been a behaviour bug
- block comments to //, which is what main's cpps use; duplicate using blocks
  dropped in three files; clear-masks spelled as the complement of the bits
  they clear, as the same lines already spell the bits they set; clang-format

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
… break

The twelve footprint loops test this->constructionTileCount, which the
getBuildingSizeIndexMappingData call in the body sets, so the test has to
stay at the end of the body. Writing that as while (true) with a trailing
break instead of do-while is codegen-neutral: all 88 functions measure
identical to the previous commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
Takes main's upstream Ghidra header sync, which already carries this
branch's TileMapState type fixes (and makes
forceUpdateLogicalAndMiscDisplayLayers int on top), so the headers are
main's. The /GL lists of both sides are kept. All 88 functions measure
identical to before the merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
@github-actions github-actions Bot removed the declaration/type change This requires changes to the generated/exported types or definitions. label Nov 7, 2026
gynt and others added 4 commits November 8, 2026 01:07
The six hand-unrolled copies were MSVC's own unrolling of a one-tile loop
body. Written as a plain loop it unrolls six-fold, as in the original, once
the decompiler's (char)playerID - 1U cast is dropped from the writes; with
the cast the body looks bigger to MSVC and it unrolls five-fold (0.905).
Still 100%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
Addresses the style-lint and cpp-linter reports on the PR:
- setupAllMapSections: the border graphic loop is a plain per-tile loop;
  MSVC unrolls it six-fold by itself.
- rebuildShowHiLayerFromHeights: a plain loop unrolls only six-fold where
  the original has eight, so the outer loop steps by 8 over an inner 0..7
  loop instead of eight hand-written copies.
- updatePathLinkagesForBuilding: trailing whitespace (clang-format).

All 88 functions measure identical to before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
Drop the comments that explained deviations, test BOOLEnum results
without != FALSE, remove casts to the destination's own type, name the
climb-ramp heights, write the setcc idioms in rotateFearFactorBuildingVariations
as if/else, make importHeight a BOOLEnum and let loops replace the
copies in countMoatsOwnedByEachPlayer and getTileForBrush.

All 88 functions measure unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaNCgNNvTDAKE2qbZC
`a = a op v` becomes `a op= v` (103 sites). Using-declarations that
TileMapState.hpp already provides (BOOLEnum, MappersEnum, BuildingType),
that other includes already make visible, or that were unused are removed.
Codegen-neutral: every function's /FA output is identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gynt and others added 4 commits November 11, 2026 13:03
…bounds

Removes 50 casts whose removal leaves every function's /FA output and the
compiler warnings unchanged; the four (int)(x & 1) casts in
placeDefensiveStructureTile stay, since dropping all of them together splits
a shared tail. Literal and MC_MAP_TILES_COUNT bounds that are the length of
the array being indexed become countof(), and MapConstants.hpp is no longer
included where nothing else used it. Codegen-neutral.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fillMemory_ByteValue takes a uchar, so the char literal only obscured a
plain zero. Codegen-neutral.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With main's headers, mapOrientation is a MapOrientationInt and the
forceUpdate* flags are BOOLEnum: compare against MO_NORTH/EAST/SOUTH/WEST
and assign TRUE/FALSE. PathConnectionLayer is now short, which makes the
(short) read casts redundant. reccmp unchanged (87/88 at 100%).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcehold sourcehold deleted a comment from github-actions Bot Nov 11, 2026
int treeID = this->OrganismLayer[tile];
if (treeID < countof(DAT_LandscapeState::instance.trees)) {
switch (DAT_LandscapeState::instance.trees[treeID].treeType) {
case 5:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use TreeType Enum

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 04a5f3e: the cases are OpenSHC::Map::Trees::TT_UNKNOWN_5 … TT_UNKNOWN_19. The function compiles to the same instructions.

case OpenSHC::Commands::M_MAPPER_GARDEN11:
case OpenSHC::Commands::M_MAPPER_GARDEN12:
case OpenSHC::Commands::M_MAPPER_OIL_SMELTER:
case 258:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use MapperEnum names

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 04a5f3e: M_MAPPER_UNKNOWN_248 … M_MAPPER_UNKNOWN_260, which main added (merged in 67247ca). No raw numbers are left in the switch. It compiles to the same instructions.

undefined4 TileMapState::getCastleBuildRangeForMapSize()
{
switch (this->mapSize) {
case 160:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use map size constants from MapConstants

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 04a5f3e: MC_MAP_SIZE_160, MC_MAP_SIZE_200, MC_MAP_SIZE_300 and MC_MAP_SIZE_400 (the last case was MC_MAP_XY_LIMIT). It compiles to the same instructions.

int TileMapState::getOrientationThatIsWallTowerOrGatehouse(uint x, uint y)
{
if (x > MC_MAP_XY_LIMIT_INCLUSIVE || y > MC_MAP_XY_LIMIT_INCLUSIVE) {
return 15;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return MapOrientation (see enum)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 31a645f: it returns MapOrientation in the header, the resolver and the definition, as Ghidra already has it. The "nothing found" result is MO_NONE. One caveat: the found orientation comes from SomeOrientationOrderArray = {0, 4, 2, 6, 1, 7, 3, 5}, so the diagonals 1/3/5/7 are returned too, and MapOrientation has no names for them. The function compiles to the same instructions.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up in 9882188: it now returns CardinalDirection instead. The result can be a diagonal (1/3/5/7), and MapOrientation has no names for those. CardinalDirection.hpp is main's generated header now (merged in 03523e2). Two caveats:

  • "Nothing found" is still 15, which CardinalDirection has no value for. It is written (CardinalDirection)MO_NONE; adding a CD_NONE = 15 to the Ghidra enum would let it read naturally.
  • Ghidra still types this function as returning MapOrientation, so the next export will put that back unless the signature is changed there too.

gynt and others added 4 commits November 12, 2026 16:56
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Exported by hand from Ghidra's /_HoldStrong/Map/Enums/CardinalDirection
(CD_NORTH = 0 ... CD_NORTHWEST = 7, CD_UPPER_LIMIT = 8), which had no
header yet. It names the second index of directionTranslationMatrix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As Ghidra already types it. The "no wall" result is MO_NONE (15). The other
results come from SomeOrientationOrderArray {0, 4, 2, 6, 1, 7, 3, 5}, so the
diagonals are returned as well. Codegen unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- countLogicPropertyInSurroundingTiles: CD_NORTH/EAST/SOUTH/WEST for the
  directionTranslationMatrix index
- resetTileAndClearMoat: TreeType values for the tree-state switch
- setupTileMapSections: changeMapOrientation(MO_NORTH)
- getCastleBuildRangeForMapSize: MC_MAP_SIZE_* cases
- getBuildingSizeForCommandBuildingType: M_MAPPER_UNKNOWN_248..260 from main

Each function compiles to the same instructions as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the declaration/type change This requires changes to the generated/exported types or definitions. label Nov 12, 2026
gynt and others added 3 commits November 12, 2026 17:35
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ntation)

The upstream Ghidra export types the parameter as MapOrientation, so the
definition takes one and processMapOrientationChange casts the pending
orientation it passes. Codegen unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion

The result is a neighbour direction taken from SomeOrientationOrderArray
{0, 4, 2, 6, 1, 7, 3, 5}, so the diagonals are returned too, which
MapOrientation has no names for. "Nothing found" stays 15 (MO_NONE);
CardinalDirection has no value for it. Codegen unchanged.

Ghidra still types the return as MapOrientation, so the next export will
revert this unless the function signature is changed there as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gynt and others added 2 commits November 14, 2026 00:28
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the declaration/type change This requires changes to the generated/exported types or definitions. label Nov 13, 2026
@gynt
gynt requested a review from TheRedDaemon November 13, 2026 23:53
int rockY = DAT_LandscapeState::instance.rocks[rockID].y;
int orientation = DAT_LandscapeState::instance.rocks[rockID].orientation + this->mapOrientation;
int index = 0;
if (orientation >= 8) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You created enums for this, if I remember right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, CardinalDirection has CD_UPPER_LIMIT = 8. The rock orientation is in eighths and mapOrientation adds 0/2/4/6, so the wrap is now if (orientation >= CD_UPPER_LIMIT) orientation -= CD_UPPER_LIMIT; (bae202f). Same instructions.

int size = DAT_LandscapeState::instance.rocks[rockID].size;
int surface = (orientation / 2) * size * size;

while (true) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be a do-while loop?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in bae202f: do { ... } while (index < this->constructionTileCount);. It compiles to the same instructions (the original is a do-while too; Ghidra renders the same loop in setMiscDisplayLayer as do ... while).

} else {
this->GfxLayer[tile]
= MACRO_CALL_MEMBER(OpenSHC::Map::LandscapeState_Func::getRandomRockImageOffset,
DAT_LandscapeState::ptr)((DAT_TileMapState::instance.RandomLayer[tile] & 3) + 1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the 3 marks the first two byte in this layer, but we have no info on them at the moment?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RandomLayer is not a bit field: setupTileMapSections fills every tile once with nextRandomNumber2(), and nothing in the game writes it afterwards (every other access is a read). & 3 picks one of four variants from that per-tile random number, and getRandomRockImageOffset(n) reads RandomRockImageGMImageOffsets[n], so (RandomLayer[tile] & 3) + 1 is one of the four single-tile rock images 1..4. renderMap uses the identical expression for rocks in the flat view, and the fire spread / archer-point code tests & 3 as a one-in-four chance. It stays & 3: the layer is a signed short, so % 4 would add the negative-remainder fix-up.

Comment on lines +27 to +31
this->screenSouthEastDirection = 3;
this->orientedRowStep = 1;
this->screenEastDirection = 2;
this->screenSouthDirection = 4;
this->screenSouthWestDirection = 5;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are some of these direction enum values, or are these numbers?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are CardinalDirection values. screenSouthEastDirection/screenEastDirection/screenSouthDirection/screenSouthWestDirection hold the map direction that points to screen south-east/east/south/south-west under the current orientation: the MO_NORTH set (SE 3, E 2, S 4, SW 5) turned by 2 per quarter turn. They are used as directionTranslationMatrix columns, and those columns are CardinalDirections: setupMovementDirectionTranslationMatrix fills column 2 with +1 (next tile in the row) and column 6 with -1, and clockwiseCardinalTranslationMatrix holds (0,-1), (1,-1), (1,0), (1,1), (0,1), (-1,1), (-1,0), (-1,-1) for columns 0..7. Done in 531e6e3: all four cases use CD_*. orientedRowStep stays 1/-1, because it is a step and not a direction. Same instructions.

buildingSizeTileIndex, widthOrHeight);
int tileUnk = DAT_ViewportRenderState::instance.translationMatrix[this->buildingY + buildingY].addXgetTile
+ this->buildingX + buildingX;
this->MiscDisplayLayer[tileUnk] &= ~0xc;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have an idea what these two bits mean?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. bae202f adds Map::LogicHelpers::MiscDisplay for the MiscDisplayLayer bits, and 0xC is MD_BUILDING_TILES = MD_BUILDING_MAIN_TILE (0x4) | MD_BUILDING_SECOND_TILEUnk (0x8). Both are set only by setMiscDisplayLayer: 0x4 on the tile it also stores in Building::miscDisplayLayerTile (or the single tile of a building smaller than 3x3), and that is the tile renderMap draws building-specific things on (the hut/farm image variant +1, the drawbridge overlay where (bits & 0xC) == 4). 0x8 goes on a second offset tile (on every tile for the granary and armory) and that tile is stored in Building+xPosition, which Ghidra marks fixme. Clearing both is what removing a building does here, in markBuildingFootprintFlag and in updateBuildingAreaTileGraphics. Same instructions.

{
int buildingSizeTileIndex = 0;

while (true) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do-while?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in bae202f: do { ... } while (buildingSizeTileIndex < this->constructionTileCount);. It compiles to the same instructions (the original is a do-while too; Ghidra renders the same loop in setMiscDisplayLayer as do ... while).

}
}
}
} else if (this->pitchDitches[id].state == 3) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pitch states yet unknown, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They can be pinned down from all the writers and readers:

  • 0: just placed (placePitchDitch). This update moves it to 1 on its next run.
  • 1: ready, not lit. Only a ditch in this state can be ignited: processEntityHitBuildingOrUnit sets 2 when a fire entity or burning arrow hits it, and the spread below does the same. updateGfxLayer hides a ditch in state < 2 from players who are not on its owner's team.
  • 2: burning. After 20 updates (field_0x12 is the timer) it sets fire to neighbouring ditches that are in state 1 and within 25 height, and moves to 3.
  • 3: burnt out. After 200 more updates the tile height is restored and the state becomes -1.
  • -1: remove. destroyPitchDitch is called on the next update.

There is no enum for them in Ghidra or the repo yet. A PitchDitchState (-1/0/1/2/3) and renaming field_0x12 to a timer would make this read naturally; it is left for the Ghidra side, since PitchDitch.state is a generated short field. ccb8650 also makes height an int, since it feeds abs(). Same instructions.

this->refreshCertainTileMap_old = this->refreshCertainTileMap;
this->refreshRelatedOne = 0;
for (int row = 0; row < countof(this->heightBasedScreenYOffset); row++) {
int target = ((uint)row >> 2) + 1;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be expressed more readable? That being said, I am not sure what is actually does. Divide through 4?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is a divide by 4: int const loweredOffset = row / 4 + 1; (bae202f). MSVC knows row is never negative, so the signed division compiles to the same shr edx, 2 as the (uint)row >> 2, and the function is unchanged (quick_gl and quick_diff both 1.0000). What it does: heightBasedScreenYOffset[h] is the screen Y offset at which a tile of height h is drawn (rebuildShowHiLayerFromHeights copies it into ShowHiLayer, renderMap uses it as the tile's Y offset). In the normal view it is h itself, and in the lowered (flattened, spacebar) view it is h / 4 + 1. This loop steps each entry towards one or the other a little each tick.

step = 0;
}

if (this->refreshCertainTileMap == 1) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these states? Or do they indicate numbers or indexes?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

States. refreshCertainTileMap is written only by triggerLoweredView (3 from the key/menu handlers that start the lowered view, 4 from the ones that end it, forced to 2 while flatViewToggleValue2 is set) and by the constructor (2). This function moves it back to 0 when the transition finishes. bae202f uses the LoweredViewTransition enum for it: LVT_NONE 0, LVT_SET_LOWERED 1 (jump to the lowered offsets), LVT_SET_NORMAL 2 (jump back), LVT_LOWER 3 and LVT_RAISE 4 (animate there). No caller passes 1, but this function handles it. Same instructions.

this->buildingPlacementFail = TRUE;
}
if ((this->LogicLayer[tile] & L_WALL_OR_GATEHOUSE) != 0 && this->BuildingLayer[tile] == 0) {
if ((this->WallOwnerLayer[tile] & 7) + 1 != playerID) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Owner part enum?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ccb8650: WO_OWNER_MASK from the new WallOwner enum (see destroyWallsOfPlayer). Same instructions.

@TheRedDaemon TheRedDaemon left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See all comments.

gynt and others added 5 commits November 15, 2026 22:43
- CardinalDirection for directionTranslationMatrix columns and direction
  parameters; MapOrientation steps for the neighbour rotation; CD_UPPER_LIMIT
  for the wrap and the queued-orientation sentinel
- if/else flag checks (L_WALL_OR_GATEHOUSE, L_STAIRS, L_CRENEL) instead of
  shift-and-mask
- new LoweredViewTransition (refreshCertainTileMap) and EditorRockType enums
- rotateFearFactorBuildingVariations: int timeGetTime() result, ERT_*/MO_*
- setupAllMapSections: 8 instead of '\b'; getTileForBrush: countof bound

Each function compiles to the same instructions as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaL4CQf53VmKTVHqhH
…rom review

- the construction-tile loops are do-while, testing constructionTileCount
  after the body as before (getBuildingSizeIndexMappingData sets it)
- new MiscDisplay enum for the MiscDisplayLayer bits, used for the literals
- storeMinAndMaxHeightOfArea: int height, casts dropped
- updateShowHiLayerOrResetChangedLayer: row / 4 + 1 and LVT_* values

Each function compiles to the same instructions as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaL4CQf53VmKTVHqhH
- new WallOwner enum: WO_OWNER_MASK (owner = playerID - 1 in the low 3 bits)
- getBuildingHurtSFXID: if/else on L_WALL_OR_GATEHOUSE instead of >> 7 & 2
- int locals where the original compares signed; redundant casts dropped
- computeTileAlongAxisOffset and getRubbleGraphicStageForDamageLevel:
  CardinalDirection / MapOrientation names

Each function compiles to the same instructions as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaL4CQf53VmKTVHqhH
- setSignpostDistanceForCampaignMission: MT_ECONOMIC
- updateMacroLayer: L2_NONE for the Logic2 mask argument
- advanceMoatDigProgress: L2_NONE

Each function compiles to the same instructions as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaL4CQf53VmKTVHqhH
- int x/y parameters where the original compares signed or callers pass
  sign-extended shorts: computeTileAlongAxisOffset, findNearestValidDigTile-
  NearTarget (range checks written as x < 0 || x > 399, same cmp/ja),
  getMaxWallHeightInBrushArea, getTileForBrush, increaseHeightForTunnel-
  SingleTile/WithBrush, placeDefensiveStructureTile
- int returns for getBuildingHurtSFXID, getCastleBuildRangeForMapSize,
  getRubbleGraphicStageForDamageLevel, getTotalHeightAtTile; BOOLEnum for
  isWallCornerForDiagonalDirection like its cardinal sibling
- int moatTileCount, lastTime, field_0x554a10; LoweredViewTransition for
  refreshCertainTileMap(_old) and triggerLoweredView
- getWallFlagForOrientedDirection: twoOrFour -> screenDirection
- CardinalDirection.hpp synced from main for CD_UNSPECIFIED (15), used for
  the "nothing found" result and the empty brush step

reccmp: every function equal or better; placeDefensiveStructureTile 90.2 -> 100%.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaL4CQf53VmKTVHqhH
@github-actions github-actions Bot added the declaration/type change This requires changes to the generated/exported types or definitions. label Nov 15, 2026
gynt and others added 2 commits November 16, 2026 07:35
New headers (generated format, to be added to Ghidra):
- PitchDitchState (+ PitchDitchStateShort): PDS_REMOVE -1, PDS_PLACED 0,
  PDS_READY 1, PDS_BURNING 2, PDS_BURNT_OUT 3; PitchDitch.state uses it and
  field_0x12 becomes burnTimer (20 ticks burning, 200 burnt out)
- HitSoundMaterial: HSM_NONE 0, HSM_WOOD 1, HSM_STONE 2, the return of
  getBuildingHurtSFXID (callers pick FX_ATTACK_WOOD / FX_ATTACK_STONE);
  its parameter is a tile, not a building id
- BuildingSizeIndexMapping { x, y, orientedPartIndex[4] } for the global
  that was int[14][169][6]; buildingRotationRelatedValue becomes
  buildingOrientedPartIndex
- MapOrientation gets MO_UPPER_LIMIT (8), the "no change queued" value of
  DAT_FutureMapOrientation

Field types and names:
- rockOrientation: MapOrientation, editorRockType: EditorRockType (the map
  editor big-rock buttons now use ERT_BIGROCK1..5 and MO_*)
- TerrainDefinedData.SomeOrientationOrderArray and
  ClimbLogicDefinedData.field_0x2c (-> neighbourSearchOrder): CardinalDirection[8]
- TerrainDefinedData.field_0x378c -> twoTileAxisOffsets,
  field_0x5549a8/ac -> int destinationNearTargetX/Y
- getRubbleGraphicStageForDamageLevel parameter -> fencePiece

reccmp: every function unchanged, MenuItemActionHandler_MapEditorLandscaping_
GeneralButtons 100%.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaL4CQf53VmKTVHqhH
Main's Ghidra sync carries the review's header proposals. Take main's
TileMapState headers and adapt the sources to its names: EditorRockType
now lives in Map/Rocks/Enums (the branch copy in Map/Enums is removed),
triggerLoweredView takes LoweredViewTransitionInt and getBuildingHurtSFXID
returns HitSoundMaterialInt.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmXCmaL4CQf53VmKTVHqhH
@github-actions github-actions Bot removed the declaration/type change This requires changes to the generated/exported types or definitions. label Nov 16, 2026
@gynt

gynt commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor Author

placeDefensiveStructureTile (0x005034A0): 100% before the main merge, 90.2% after it

Short version. The .cpp and the function's own signature are identical before and after the merge of origin/main (d146f8e). The result is decided by how many C++ declarations the translation unit contains, not by any type, and not by the order of the #includes. With identical compiler IL, MSVC's register allocator puts the x & 1 temporary in ecx in one case and in eax in the other. Only the ecx version lets the late tail-merge fold the M_MAPPER_CRENAL2 tail into the M_MAPPER_CRENAL one, which is the original's 145-instruction shape. The link is a hash in c2.dll (GrgImmedTupleHash) that buckets the address of a global by the global's TU-wide symbol number modulo 64; the details are in the c2.dll section below. There is no source-level fix. This also corrects my earlier reply on this function: the int x, int y signature did not cause the 90.2% -> 100% step. That step came from declarations added in the same commit.

reccmp across the branch

commit what it is placeDefensiveStructureTile
2a224ea before the review work 90.2%
233eacc types: header commit (int coordinates etc.) 100.0%
e4efe7a types: enums/struct commit 100.0%
d146f8e merge of origin/main 90.2%

The function's source is the same in all four. Every other function in the build list kept its score.

What changed in the headers at the merge

The function's own declaration and resolver are the same on both sides:

void placeDefensiveStructureTile(int playerID, int x, int y, MappersEnum param_4);
MACRO_FUNCTION_RESOLVER(void (TileMapState::*)(int, int, int, MappersEnum), ..., Address::SHC_3BB0A8C1_0x005034A0, ...)

src/OpenSHC/Map/TileMapState.hpp, pre-merge (e4efe7a):

#include "OpenSHC/Commands/MappersEnum.hpp"
#include "OpenSHC/IO/PackagedFileMagicNum.hpp"
#include "OpenSHC/Map/Buildings/BuildingFailReasonEnumInt.hpp"
#include "OpenSHC/Map/Buildings/BuildingType.hpp"
#include "OpenSHC/Map/Enums/CardinalDirection.hpp"
#include "OpenSHC/Map/Enums/EditorRockType.hpp"
#include "OpenSHC/Map/Enums/HitSoundMaterial.hpp"
#include "OpenSHC/Map/Enums/LoweredViewTransition.hpp"
#include "OpenSHC/Map/Enums/MapOrientation.hpp"
#include "OpenSHC/Map/Enums/MapOrientationInt.hpp"
#include "OpenSHC/Map/LogicHelpers/Logic1.hpp"
#include "OpenSHC/Map/LogicHelpers/Logic2.hpp"
#include "OpenSHC/Map/Moat.hpp"
#include "OpenSHC/Map/PitchDitch.hpp"
#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp"

post-merge (d146f8e):

#include "OpenSHC/Commands/MappersEnum.hpp"
#include "OpenSHC/IO/PackagedFileMagicNum.hpp"
#include "OpenSHC/Map/Buildings/BuildingFailReasonEnumInt.hpp"
#include "OpenSHC/Map/Buildings/BuildingType.hpp"
#include "OpenSHC/Map/Buildings/BuildingTypeShort.hpp"
#include "OpenSHC/Map/Enums/CardinalDirection.hpp"
#include "OpenSHC/Map/Enums/HitSoundMaterialInt.hpp"
#include "OpenSHC/Map/Enums/LoweredViewTransition.hpp"
#include "OpenSHC/Map/Enums/LoweredViewTransitionInt.hpp"
#include "OpenSHC/Map/Enums/MapOrientation.hpp"
#include "OpenSHC/Map/Enums/MapOrientationInt.hpp"
#include "OpenSHC/Map/LogicHelpers/Logic1.hpp"
#include "OpenSHC/Map/LogicHelpers/Logic2.hpp"
#include "OpenSHC/Map/Moat.hpp"
#include "OpenSHC/Map/PitchDitch.hpp"
#include "OpenSHC/Map/Rocks/Enums/EditorRockType.hpp"
#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp"

along with the using declarations for BuildingTypeShort, HitSoundMaterialInt, LoweredViewTransition, LoweredViewTransitionInt and Rocks::Enums::EditorRockType. The resolver header TileMapState.func.hpp changed with it.

src/OpenSHC/Map/Buildings/Building.hpp, pre-merge:

#include "OpenSHC/Game/Resources/ResourceTypeShort.hpp"
#include "OpenSHC/Map/Buildings/BuildingFlagSlot.hpp"
#include "OpenSHC/Map/Buildings/BuildingLogicalStateShort.hpp"
#include "OpenSHC/Map/Buildings/BuildingTypeShort.hpp"
#include "OpenSHC/Map/Buildings/Enums/DefensiveTowerEngineTypeByte.hpp"

post-merge (main's Ghidra sync; short field_0xc2 became BuildingTargetMarkerShort targetMarker):

#include "OpenSHC/Game/Resources/ResourceTypeShort.hpp"
#include "OpenSHC/Map/Buildings/BuildingFlagSlot.hpp"
#include "OpenSHC/Map/Buildings/BuildingLogicalStateShort.hpp"
#include "OpenSHC/Map/Buildings/BuildingTypeShort.hpp"
#include "OpenSHC/Map/Buildings/Enums/BuildingTargetMarkerShort.hpp"
#include "OpenSHC/Map/Buildings/Enums/DefensiveTowerEngineTypeByte.hpp"

Bisecting the headers

These were measured by compiling the one file (quick_diff.py, against the original's 145 instructions). The other headers were taken from the merged tree, and the resolver flags were identical in every run (the same 90 functions enabled).

TileMapState.hpp + .func.hpp Building.hpp result
pre-merge pre-merge 1.0000 (145 instructions)
pre-merge post-merge 0.9630 (152)
post-merge pre-merge 0.9630 (152)
post-merge post-merge 0.9630 (152)

Other headers main changed (ViewportRenderState.hpp, ViewportState.hpp, ViewportConstants.hpp, GameCore.hpp, Floater.hpp, MapConstants.hpp, PitchDitch.hpp, PitchDitchState.hpp, UnitState.hpp, MenuModalComposition.hpp, MenuTextInputState.hpp) make no difference: swapping all of them back to their pre-merge versions leaves the result unchanged. Either of the two changes above is enough on its own to produce the regression. Only with both headers at their pre-merge versions does the function compile to 1.0000.

Include order does not matter

The .cpp's own include list, as committed:

#include "../TileMapState.func.hpp"

#include "OpenSHC/Map/Buildings/BuildingsState.func.hpp"
#include "OpenSHC/Map/Navigation/PathFindingState.func.hpp"
#include "OpenSHC/UI/MinimapViewState.func.hpp"
#include "OpenSHC/Commands/MappersEnum.hpp"
#include "OpenSHC/Map/LogicHelpers/WallOwner.hpp"
#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp"

#include "OpenSHC/Globals/DAT_BuildingsState.hpp"
#include "OpenSHC/Globals/DAT_LandscapeState.hpp"
#include "OpenSHC/Globals/DAT_MinimapViewState.hpp"
#include "OpenSHC/Globals/DAT_PathFindingState.hpp"
#include "OpenSHC/Globals/DAT_ViewportRenderState.hpp"

Each order below was compiled with the pre-merge headers (TileMapState pair + Building.hpp from e4efe7a) and with the post-merge headers:

order pre-merge headers post-merge headers
as committed (above) 1.0000 0.9630
reversed 1.0000 0.9630
globals first 1.0000 0.9630
callee .func.hpps first 1.0000 0.9630
own .func.hpp last 1.0000 0.9630
enum headers first 1.0000 0.9630
#include "OpenSHC/Map/Buildings/Building.hpp" added first 1.0000 0.9630
#include "OpenSHC/Map/Buildings/BuildingTypeShort.hpp" added first 1.0000 0.9630
#include "OpenSHC/Rendering/ViewportRenderState.hpp" added first 1.0000 0.9630
#include "OpenSHC/Map/Buildings/Enums/BuildingTargetMarkerShort.hpp" added first 0.9630 0.9630
#include "OpenSHC/Map/Buildings/Enums/BuildingTargetMarkerShort.hpp" added after the last include 0.9630 0.9630
The permutations, literally

reversed:

#include "OpenSHC/Globals/DAT_ViewportRenderState.hpp"
#include "OpenSHC/Globals/DAT_PathFindingState.hpp"
#include "OpenSHC/Globals/DAT_MinimapViewState.hpp"
#include "OpenSHC/Globals/DAT_LandscapeState.hpp"
#include "OpenSHC/Globals/DAT_BuildingsState.hpp"
#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp"
#include "OpenSHC/Map/LogicHelpers/WallOwner.hpp"
#include "OpenSHC/Commands/MappersEnum.hpp"
#include "OpenSHC/UI/MinimapViewState.func.hpp"
#include "OpenSHC/Map/Navigation/PathFindingState.func.hpp"
#include "OpenSHC/Map/Buildings/BuildingsState.func.hpp"
#include "../TileMapState.func.hpp"

globals first:

#include "OpenSHC/Globals/DAT_BuildingsState.hpp"
#include "OpenSHC/Globals/DAT_LandscapeState.hpp"
#include "OpenSHC/Globals/DAT_MinimapViewState.hpp"
#include "OpenSHC/Globals/DAT_PathFindingState.hpp"
#include "OpenSHC/Globals/DAT_ViewportRenderState.hpp"
#include "../TileMapState.func.hpp"
#include "OpenSHC/Map/Buildings/BuildingsState.func.hpp"
#include "OpenSHC/Map/Navigation/PathFindingState.func.hpp"
#include "OpenSHC/UI/MinimapViewState.func.hpp"
#include "OpenSHC/Commands/MappersEnum.hpp"
#include "OpenSHC/Map/LogicHelpers/WallOwner.hpp"
#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp"

callee .func.hpps first:

#include "OpenSHC/Map/Buildings/BuildingsState.func.hpp"
#include "OpenSHC/Map/Navigation/PathFindingState.func.hpp"
#include "OpenSHC/UI/MinimapViewState.func.hpp"
#include "../TileMapState.func.hpp"
#include "OpenSHC/Commands/MappersEnum.hpp"
#include "OpenSHC/Map/LogicHelpers/WallOwner.hpp"
#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp"
#include "OpenSHC/Globals/DAT_BuildingsState.hpp"
#include "OpenSHC/Globals/DAT_LandscapeState.hpp"
#include "OpenSHC/Globals/DAT_MinimapViewState.hpp"
#include "OpenSHC/Globals/DAT_PathFindingState.hpp"
#include "OpenSHC/Globals/DAT_ViewportRenderState.hpp"

own .func.hpp last:

#include "OpenSHC/Map/Buildings/BuildingsState.func.hpp"
#include "OpenSHC/Map/Navigation/PathFindingState.func.hpp"
#include "OpenSHC/UI/MinimapViewState.func.hpp"
#include "OpenSHC/Commands/MappersEnum.hpp"
#include "OpenSHC/Map/LogicHelpers/WallOwner.hpp"
#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp"
#include "OpenSHC/Globals/DAT_BuildingsState.hpp"
#include "OpenSHC/Globals/DAT_LandscapeState.hpp"
#include "OpenSHC/Globals/DAT_MinimapViewState.hpp"
#include "OpenSHC/Globals/DAT_PathFindingState.hpp"
#include "OpenSHC/Globals/DAT_ViewportRenderState.hpp"
#include "../TileMapState.func.hpp"

enum headers first:

#include "OpenSHC/Commands/MappersEnum.hpp"
#include "OpenSHC/Map/LogicHelpers/WallOwner.hpp"
#include "OpenSHC/WindowsHelper/Enums/BOOLEnum.hpp"
#include "../TileMapState.func.hpp"
#include "OpenSHC/Map/Buildings/BuildingsState.func.hpp"
#include "OpenSHC/Map/Navigation/PathFindingState.func.hpp"
#include "OpenSHC/UI/MinimapViewState.func.hpp"
#include "OpenSHC/Globals/DAT_BuildingsState.hpp"
#include "OpenSHC/Globals/DAT_LandscapeState.hpp"
#include "OpenSHC/Globals/DAT_MinimapViewState.hpp"
#include "OpenSHC/Globals/DAT_PathFindingState.hpp"
#include "OpenSHC/Globals/DAT_ViewportRenderState.hpp"

So the order in which the declarations arrive is irrelevant; which declarations exist is what counts. BuildingTargetMarkerShort.hpp declares nothing this function uses (one 3-value enum, a short typedef and their static_assert_cpp98_objs), and adding it anywhere is enough to break the pre-merge state.

It is a count of declarations, not preprocessor text

Unrelated declarations were added after the last #include of the .cpp:

added pre-merge headers post-merge headers
nothing 1.0000 0.9630
1-3 × enum PdxDummyN { PDX_DUMMYN }; 1.0000 0.9630
4-6 enums 0.9630 0.9630
7-8 enums 0.9630 1.0000
1-6 × typedef short PdxShortN; 1.0000 0.9630
7-8 typedefs 0.9630 0.9630
1-256 #define PDX_MACRO_N N 1.0000 0.9630
one #define with 100-6400 tokens 1.0000 0.9630

Macros never move it, however many or however long. Real declarations do, in windows: the pre-merge state breaks at the 4th dummy enum or the 7th typedef, and the post-merge state comes back to 1.0000 with 7 or 8 dummy enums. So the merged code can be pushed back to 100% by declaring seven unused enums. That only shows the mechanism and would be no fix: the next header sync would move it again.

Which of the merge's declarations: the enums, not short vs ushort

Variations of the new enum/typedef headers, compiled like everything above (same resolver flags):

change result
merged headers, as they are 0.9630
typedef unsigned short BuildingTargetMarkerShort; instead of short 0.9630
typedef unsigned short PitchDitchStateShort; instead of short 0.9630
both typedefs unsigned short 0.9630
Building::targetMarker declared as plain short 0.9630
BuildingTargetMarker removed entirely (enum, typedef and the #include in Building.hpp) 0.9630
PitchDitchState enum dropped from the TU (PitchDitchStateShort.hpp without its #include "OpenSHC/Map/PitchDitchState.hpp") 0.9630
HitSoundMaterial enum dropped (HitSoundMaterialInt.hpp without its #include "OpenSHC/Map/Enums/HitSoundMaterial.hpp") 0.9630
any two of those three enums dropped 1.0000
all three dropped 1.0000
pre-merge headers, as they are 1.0000
plus #include "OpenSHC/Map/Buildings/Enums/BuildingTargetMarkerShort.hpp" (enum and typedef) 0.9630
same, with the typedef unsigned short 0.9630
plus only the BuildingTargetMarker enum 0.9630
plus only the BuildingTargetMarkerShort typedef 1.0000

The signedness of the typedefs and the field type change nothing. No single enum is at fault either; the next section explains why the count is what matters and why an enum weighs more than a typedef.

Where in the compiler it happens

Each phase was dumped from c2.dll (tools/compiler-internals/tuple_dump.py) under both header sets. The function's IL is byte-identical in initial, prelower, codegen, postlower and regalloc (the input to register allocation), temporary numbers included. The first difference is the output of register allocation (update):

 BLK 7  (M_MAPPER_CRENAL)
       or        dword ptr [edi + esi*4 + 1462624], 512
       add       byte ptr [esi + edi + 2751024], 98
-      mov       ecx, ecx        ; pre-merge: x & 1 lands in x's own register
-      and       ecx, 1
+      mov       eax, ecx        ; post-merge: x & 1 gets eax, x stays in ecx
+      and       eax, 1

and the same in BLK 14 (M_MAPPER_CRENAL2). In the pre-merge version the copy disappears, both crenel arms end in the same instruction sequence, and the late tail-merge folds the second one into the first (jmp L33). The post-merge version keeps the copy, the tail-merge does not fold the second arm, and its 7 instructions are emitted again. That is the whole 145 -> 152 difference, and it matches reccmp's diff against the original, which has the folded form.

The allocator's candidate table (reg_scores.py --invocation) shows why the two runs differ, although their IL is the same. The candidates are visited in a different order, and the scores the colourer accumulates differ as well:

candidate pre-merge key / weight post-merge key / weight
tile 451 / 112 385 / 76
this 383 / 110 311 / 74
mapper 50 / 32 46 / 32
x (crenel range) 26 / 4 26 / 4
x & 1 temporaries (ids 0x31, 0x43) 22 / 6 26 / 6
temporary id 0x1b — 155 / 40
candidates 28 29

In the post-merge run the x & 1 temporaries tie with x at 26. InsertInList (decoded in msvc-backend.md) breaks such a tie by symbol id, higher first, which puts the temporaries ahead of x. That matches what the allocator then does: the temporary does not get x's ecx. In the pre-merge run it scores 22, comes after x, and ends up in ecx with the copy coalesced away. The tie and the order are read from the compiler; that the order is what decides the register is my reading of the decoded rule, not something I traced instruction by instruction.

The visit order comes from the candidate walk at c2!0x10b11560. It iterates a bitset of candidate numbers with bsf and finds each candidate through a 1024-bucket table (and eax, 0x3ff; mov eax, [eax*4 + 0x10d15090], chained through +0x30). So the order follows the candidates' internal numbers. The next section shows where those numbers come from and how the declaration count reaches them.

The responsible code in c2.dll: GrgImmedTupleHash

Traced with a debugger probe on c2.dll (14.00.50727.762) in both header states, gated on this function's GlobalRegAlloc, with the names from c2.pdb. The IL is the same in both states, so I logged what the global register pass (Globregs, 0x10b563f0) does with it.

  1. CmdIdentifyImmedCandidates (0x10c26030) calls GrgRewriteImmedAsReg (0x10b56700) for every constant operand. Each distinct constant gets a register candidate ("globreg", numbered from _NextGlobregNum) and is filed in the 64-bucket _ImmedHashTable at 0x10d12170.

  2. GrgImmedTupleHash (0x10b5c050) picks the bucket. An integer constant (tuple kind 7) hashes by value, value & 0x3f. An address constant (kind 3) hashes by a field of the global's symbol record: [[[tuple+0x18]+8]]+0x28 & 0x3f. This function has three such address constants, the &...::instance of Instance<PathFindingState>, Instance<BuildingsState> and Instance<MinimapViewState>.

  3. That +0x28 field is a symbol number counted over the whole translation unit:

    global pre-merge +0x28 (bucket) post-merge +0x28 (bucket)
    Instance<PathFindingState>::instance 0x6eded (0x2d) 0x6edff (0x3f)
    Instance<BuildingsState>::instance 0x6edf9 (0x39) 0x6ee0b (0x0b)
    Instance<MinimapViewState>::instance 0x6ee04 (0x04) 0x6ee16 (0x16)

    All three are exactly 0x12 (18) higher after the merge. Measured on the pre-merge state, one enum PdxDummy { PDX_DUMMY }; adds 2, one typedef short adds 1, and 16 #defines add 0. That is why an enum weighs twice as much as a typedef, and why macros never mattered.

  4. CreateGlobregs (0x10b5abe0) first walks _ImmedHashTable bucket by bucket and frees every immediate candidate that turned out to be worthless (ColRemoveGlobregFromHashTable, 0x10b18790, called from 0x10b5ac39). The BuildingsState and MinimapViewState address candidates (globregs 0x25 and 0x26) are among the 29 freed. So their bucket decides where in the free order they go. That is the first point where the two states differ. Every constant, hash input and globreg number before it is identical:

    pre-merge  freed: 20 1d 19 11 e a 7 1f [26] 1c c 23 1a 1e 17 f 2 6 3 15 8 b 13 1b 21 [25] 24 5 18
    post-merge freed: 20 1d 19 11 e a 7 1f 1c c 23 1a [25] 1e 17 f [26] 2 6 3 15 8 b 13 1b 21 24 5 18
    
  5. Freed records go onto _FreeGlobs, and GrgNewGlobreg (0x10b5b7b0) reuses them last-in-first-out, keeping their old numbers (ColRemoveGlobregFromHashTable preserves +0x1c when it clears the record). The same CreateGlobregs call then creates the candidates for the real symbols (this, tile, x, the x & 1 temporaries, ...), so those get a permutation of the freed numbers. The first such allocations already differ: pre-merge 0x0b, 0x08, 0x15; post-merge 0x08, 0x15, 0x03.

  6. From there the candidate number drives everything that walks candidates. SpillObvious (0x10b11560) and the colouring walk iterate a bitset of numbers with bsf and look each up in _GlobregHash (0x10d15090, bucket number & 0x3ff). The splits, the spill-cost keys and the x / x & 1 tie in the table above follow from that, and so does eax vs ecx.

So the input that matters is the TU-wide symbol number of the three instance globals, modulo 64. Nothing about the declarations themselves matters, only how many symbols the front end numbered before this function.

It is periodic in 64

If number & 0x3f is the only path, the result must repeat every 64 symbol numbers. The tables below list the shift relative to the pre-merge headers (enum = 2, typedef = 1). The merged headers are shift 18.

Predicted first, then compiled:

state + dummies shift same as result
pre + 32 enums 64 0 1.0000
pre + 64 typedefs 64 0 1.0000
pre + 128 typedefs 128 0 1.0000
pre + 71 typedefs 71 7 0.9630
pre + 36 enums 72 8 0.9630
post + 23 enums 18 + 46 = 64 0 1.0000
post + 46 typedefs 64 0 1.0000
post + 39 enums 18 + 78 = 96 32 1.0000
post + 78 typedefs 96 32 1.0000

All nine came out as predicted. The full sweep of one period, pre + N typedefs for N = 0..63:

shift result
0-6 1.0000
7-30 0.9630
31-41 1.0000
42 0.9630
43-63 1.0000

40 of the 64 offsets give the original's shape. The merge landed on 18, in the middle of the bad window. Every boundary is a shift where one of the two freed address constants moves into a bucket that already holds another freed constant, which changes its place in the free order:

  • 7: BuildingsState wraps into bucket 0.
  • 31: MinimapViewState reaches 0x23.
  • 42: BuildingsState reaches 0x23.

Not every such landing is a boundary, though.

Only declarations before the function count. Four dummy enums after the last #include give 0.9630 (shift 8); the same four enums appended after the function body give 1.0000. That fits the earlier finding that include order is irrelevant: every #include comes before the body.

Macros

A #define consumes no symbol number, so macros never move this, but they don't fix it either. The merged headers are 18 numbers into a bad window. Converting enums to macros only helps if it happens to remove enough numbers to land in a good window again; removing any two of the three new enums does exactly that (table above). Any other change before this function in the TU moves the number again: a new enum, typedef, struct, member or inline function in any header the TU includes. Keeping the 100% would mean steering the TU's symbol count modulo 64 by hand, which I don't think the project should do.

The int signature was not the cause

void placeDefensiveStructureTile(int playerID, uint x, uint y, MappersEnum) (header, resolver and definition) against the int x, int y version:

pre-merge headers post-merge headers
uint x, uint y 1.0000 0.9630
int x, int y 1.0000 0.9630

The coordinate type is codegen-neutral in both states, which is what was measured before it went into the header commit. The 90.2% -> 100% step at 233eacc came from the declarations that commit added (the LoweredViewTransition include and the other new header content), the same way the merge's declarations took it back. My earlier reply credited it to the int change. That was wrong, and this report replaces it.

Conclusion

  • Nothing in the source or the generated types of placeDefensiveStructureTile is wrong, and the merge did not regress it in any real sense. Its last 7 instructions depend on the symbol number of three instance globals modulo 64: 40 of the 64 values give the original's shape, and the merge moved it from 0 to 18, which is in the bad window 7-30.
  • No ordering of #includes changes the outcome; only adding or removing declarations before the function does. Macros don't count, and neither does the signedness of the new typedefs. Padding the translation unit with dummy declarations would "fix" it until the next Ghidra sync, so I have not done that.
  • I'd record it as 90.2% with "c2 GrgImmedTupleHash: TU symbol count mod 64 (see PR comment)" instead of chasing it further. Six source rewrites of the crenel arms (casts, != 0, swapped tests, int command, statement order, the byte& height form from TileMapState-100-2) all stayed at 0.9630 or got worse.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants