Skip to content

Keep collection properties dirty-checked after reassignment; track iterator-based removals - #16282

Open
codeconsole wants to merge 3 commits into
apache:8.0.xfrom
codeconsole:fix/dirty-checking-collection-tracking-8.0.x
Open

Keep collection properties dirty-checked after reassignment; track iterator-based removals#16282
codeconsole wants to merge 3 commits into
apache:8.0.xfrom
codeconsole:fix/dirty-checking-collection-tracking-8.0.x

Conversation

@codeconsole

@codeconsole codeconsole commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Mongo Collection Property Dirty Tracking

On interception-based stores (MongoDB), two everyday mutation patterns silently escape dirty checking — save() reports success and persists nothing:

// 1. The defensive re-init — true for an EMPTY tracked list, because empty collections are falsy
if (!schedule.shares) {
    schedule.shares = []              // replaces the tracked wrapper with a plain ArrayList
}                                     // and [] == [] means the assignment isn't even flagged
schedule.shares.add(newShare)         // invisible: plain list, nothing marks the entity dirty
schedule.save(flush: true)            // writes nothing
// 2. Groovy's closure-based removal — removes via iterator().remove()
schedule.shares.removeAll { it.userId == userId }   // DirtyCheckingCollection doesn't override iterator()
schedule.save(flush: true)                          // writes nothing

Hibernate is unaffected — its flush-time snapshot comparison catches everything. Mongo relies exclusively on the DirtyChecking* wrappers, so anything that escapes them is lost. Hit in production: a schedule-sharing feature showed "shared" while the document kept shares: [].

The fix (interception only — no snapshots, no flush-time diffing)

1. Wrappers track every mutation path. iterator()/listIterator() now return dirty-marking iterators (covers removeAll(Closure), retainAll(Closure), removeIf), plus the missing direct overrides: retainAll(Collection), List.sort, List.replaceAll. Same approach as Hibernate's PersistentCollection.

2. Generated setters keep tracking across reassignment. Collection/List/Set/Map-typed properties assign through DirtyCheckingSupport.rewrap:

// generated setter, before:
void setShares(List shares) { markDirty("shares", shares); this.shares = shares }
// after:
void setShares(List shares) { markDirty("shares", shares); this.shares = (List) DirtyCheckingSupport.rewrap(this, "shares", this.shares, shares) }

rewrap wraps the new value only when the value being replaced was itself a tracked wrapper — otherwise it returns the raw value after one instanceof. Never-tracked properties (transient instances, Hibernate entities) behave exactly as before, and non-collection properties compile to identical bytecode.

3. Replacement wrappers are flagged isAssigned() (default method on DirtyCheckableCollection, so binary-compatible). PersistentEntityCodec then takes the full-rewrite path instead of per-element diffing — a replacement's layout need not match the stored array. Without the flag, a same-size replacement holding clean elements emitted no update at all.

Tests

Each escape is reproduced by a spec that fails without the fix:

  • DirtyCheckingCollectionSpec — 8 wrapper mutation paths that bypassed tracking
  • DirtyCheckCollectionReassignmentSpec — reassignment loses tracking (List/Set/Map); never-tracked values stay untouched
  • EmbeddedCollectionDirtyTrackingSpec — end-to-end against MongoDB, replicating the production shape (an auto-timestamped entity: the lastUpdated write during flush resets the explicit-save dirty marker, so persistence depends entirely on the wrappers)

…erator-based removals

Interception-based stores (MongoDB) rely exclusively on the
DirtyChecking* wrappers — there is no flush-time snapshot comparison —
so mutations that escape them are silently lost: save() sees a clean
entity and persists nothing.

Two real-world escapes:

1. Reassignment through a generated setter stored the raw value, so
   'entity.items = []' over a tracked list replaced the wrapper with a
   plain untracked ArrayList. The common defensive re-init
   'if (!entity.items) entity.items = []' triggers this on every load
   (an empty tracked collection is falsy in Groovy), and because
   [] == [] the equality-suppressed markDirty never flagged the
   assignment either. The in-place add() that followed was lost.

2. DirtyCheckingCollection never overrode iterator(), so every
   iterator-based removal — including Groovy's removeAll(Closure) and
   retainAll(Closure) and Java's removeIf — bypassed tracking, along
   with retainAll(Collection), List.sort and List.replaceAll.

The fix stays interception-only (no snapshots, no flush-time diffing):

- Wrappers override iterator()/listIterator() with dirty-marking
  iterators plus the missing direct mutators — the same approach as
  Hibernate's PersistentCollection.
- Generated setters for Collection/List/Set/Map-typed properties assign
  through DirtyCheckingSupport.rewrap, which wraps the incoming value
  ONLY when the value being replaced was itself a tracked wrapper. A
  never-tracked property (transient instance, or a store like Hibernate
  with its own dirty checking) stores the raw value as before, and
  non-collection properties compile to identical bytecode.
- A replacement wrapper is flagged isAssigned() so
  PersistentEntityCodec takes the full-rewrite path rather than
  per-element diffing — a replacement's layout need not match the
  stored array (a same-size replacement of clean elements previously
  emitted no update at all once wrapped).

Specs reproduce each escape before the fix: DirtyCheckingCollectionSpec
(wrapper mutation paths), DirtyCheckCollectionReassignmentSpec
(setter reassignment), and EmbeddedCollectionDirtyTrackingSpec
(end-to-end against MongoDB, replicating the production shape where an
auto-timestamped entity dropped an embedded-collection add).
@codecov

codecov Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.23810% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 54.7809%. Comparing base (c1b532a) to head (708aa2e).

Files with missing lines Patch % Lines
...ping/dirty/checking/DirtyCheckingCollection.groovy 88.2353% 2 Missing ⚠️
...re/mapping/dirty/checking/DirtyCheckingList.groovy 96.4286% 1 Missing ⚠️
...pping/dirty/checking/DirtyCheckingSortedSet.groovy 75.0000% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@                Coverage Diff                 @@
##                8.0.x     #16282        +/-   ##
==================================================
+ Coverage     54.7365%   54.7809%   +0.0444%     
- Complexity      20470      20507        +37     
==================================================
  Files            2103       2104         +1     
  Lines          101077     101152        +75     
  Branches        17928      17935         +7     
==================================================
+ Hits            55326      55412        +86     
+ Misses          37876      37862        -14     
- Partials         7875       7878         +3     
Files with missing lines Coverage Δ
...g/mongo/engine/codecs/PersistentEntityCodec.groovy 73.8872% <100.0000%> (+0.0777%) ⬆️
...ails/compiler/gorm/DirtyCheckingTransformer.groovy 78.9720% <100.0000%> (+0.8166%) ⬆️
...pping/dirty/checking/DirtyCheckableCollection.java 100.0000% <100.0000%> (ø)
...ore/mapping/dirty/checking/DirtyCheckingMap.groovy 50.0000% <100.0000%> (+10.0000%) ⬆️
...ore/mapping/dirty/checking/DirtyCheckingSet.groovy 100.0000% <100.0000%> (ø)
...mapping/dirty/checking/DirtyCheckingSupport.groovy 78.4314% <100.0000%> (+7.3787%) ⬆️
...re/mapping/dirty/checking/DirtyCheckingList.groovy 79.4872% <96.4286%> (+37.8205%) ⬆️
...pping/dirty/checking/DirtyCheckingSortedSet.groovy 83.3333% <75.0000%> (+83.3333%) ⬆️
...ping/dirty/checking/DirtyCheckingCollection.groovy 75.6757% <88.2353%> (+16.5848%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codeconsole codeconsole added this to the grails:8.0.0-RC1 milestone Sep 1, 2026
- SortedSet wrapper: construction, assigned flag, iterator-based removal
- ListIterator navigation methods (hasPrevious/previous/nextIndex/previousIndex)
- DirtyCheckingMap assigned flag on both constructors
- DirtyCheckableCollection.isAssigned() interface default (kept false for
  implementations that do not override it, e.g. PersistentCollection)
- rewrap: tracked-wrapper passthrough, bare-Collection fallback, and the
  defensive non-collection tail
@testlens-app

testlens-app Bot commented Sep 2, 2026

Copy link
Copy Markdown

✅ All tests passed ✅

🏷️ Commit: 708aa2e
▶️ Tests: 18960 executed
⚪️ Checks: 89/89 completed


Learn more about TestLens at testlens.app/docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant