Skip to content

fix(memory): enable workstation GC, LOH compaction, and nvidia-smi caching - #104

Merged
spelech merged 1 commit into
mainfrom
fix/memory-gc-and-telemetry-cache
Nov 1, 2026
Merged

spelech merged 1 commit into
mainfrom
fix/memory-gc-and-telemetry-cache

Conversation

@spelech

@spelech spelech commented Nov 1, 2026

Copy link
Copy Markdown
Owner

Summary

  • Disable Server GC (ServerGarbageCollection=false) and enable DATAS (GarbageCollectionAdaptationMode=1) in LocalLLMServerManager.csproj so freed heap segments are decommitted back to the OS on desktop machines.
  • Compact the Large Object Heap (GCLargeObjectHeapCompactionMode.CompactOnce) and trigger an aggressive blocking GC in MemoryLifecycleWatchdog when process working set exceeds 512 MB, managed heap exceeds 256 MB, or system RAM usage reaches 75%.
  • Trigger LOH compaction and aggressive GC in POST /api/memory/purge before reading post-purge memory telemetry.
  • Add a 3-second TTL cache in GpuTelemetryProvider (GetTelemetryAsync and GetGpuInfo) to avoid spawning
    vidia-smi thousands of times per day across concurrent polling, watchdog ticks, and health checks, while ensuring process handles are safely handled.
  • Add unit tests in HardwareEndpointsTests covering /api/memory/purge and GpuTelemetryProvider.GetTelemetryAsync TTL caching.

@spelech
spelech merged commit 0294247 into main Nov 1, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant