Skip to content

Implementing the Virtual File System as long with the OS File System - #175

Merged
mallory-scotton merged 47 commits into
mainfrom
ms/vfs
Nov 19, 2026
Merged

mallory-scotton merged 47 commits into
mainfrom
ms/vfs

Conversation

@mallory-scotton

@mallory-scotton mallory-scotton commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Type of Change

  • 🐞 Bug Fix (non-breaking change which fixes an issue)
  • ✨ New Feature (non-breaking change which adds functionality)
  • 💥 Breaking Change (fix or feature that would cause existing functionality to not work as expected)
  • 🧹 Refactor (code cleanup, architectural improvement, or progressive abstraction)
  • 📖 Documentation Update
  • 🧪 Tests / CI / Build System (CMake) Update

Description

Implements the two-tier OS filesystem and Virtual File System (VFS) in gp::core, migrates FileSink off std::FILE, and wires VFS lifecycle management into gp::Application.

Layer 0: OS FileSystem and FileHandle (gp::platform)

  • Added gp::platform::FileHandle (source/runtime/core/public/platforms/base/system/FileHandle.hpp) as an 8-byte non-polymorphic RAII wrapper around native OS file descriptors (HANDLE on Windows, int fd on POSIX), supporting sequential I/O (read, write, seek, tell, size, flush, truncate) and lock-free positional I/O (readAt, writeAt via Win32 OVERLAPPED offsets and POSIX pread/pwrite).
  • Added gp::platform::FileSystem across Windows, Unix, Linux, Apple, macOS, and generic backends (source/runtime/core/public/platforms/*/system/), covering file/directory queries, tree creation and removal, moves, copies, zero-allocation stack-buffered directory iteration (iterateDirectory, iterateDirectoryRecursive), bulk read/write helpers, and crash-safe atomic writes (writeAtomic using <path>.tmp.<seq>, CreateNew, flush(), and atomic rename).

Layer 1: Priority-Layered VFS (gp::vfs)

  • Added gp::vfs::VirtualFileSystem (source/runtime/core/public/vfs/VirtualFileSystem.hpp) backed by a zero-allocation .bss mount table (kMaxMountPoints = 32) guarded by SharedMutex.
  • Added IMountBackend with PhysicalMountBackend and MemoryMountBackend, priority overlay resolution (DeveloperOverride > ModOverride > Patch > DLC > Base), single-target ReadWrite routing, cross-mount streaming copyFile, sandbox escape rejection (VfsError::SandboxEscapeViolation), and deduplicated merged directory iteration.
  • Added enumerateMounts() and six AutoConsoleCommand diagnostics in VirtualFileSystem.cpp: vfs.Status, vfs.Mounts, vfs.Resolve, vfs.Exists, vfs.Stat, and vfs.Ls.

Application, Editor, and FileSink Integration

  • Replaced std::FILE and <cstdio> in gp::log::FileSink with gp::platform::FileHandle, creating missing parent directories on open and sharing read access (FileShareMode::Read) so logs can be read while open.
  • Wired VirtualFileSystem::initializeDefaultMounts() and VirtualFileSystem::shutdown() into gp::Application (ApplicationDesc, Application.hpp, Application.cpp) with -no-vfs and -vfs-app=<name> CLI overrides, preserving pre-mounted caller backends and resolving VFS URIs passed to -log-file.
  • Added a TODO(vfs) marker in EditorApplication::onInitialize() (source/launch/editor/private/Main.cpp) for future editor:// mounts.

Documentation and Tests

  • Drafted docs/adr/ADR-0060.md (Accepted), updated docs/adr/ADR-0044.md to Accepted, and registered ADR-0060 in docs/adr/INDEX.md.
  • Added unit tests in FileSystem.tests.cpp, VirtualFileSystem.tests.cpp, Logger.tests.cpp, and Application.tests.cpp.
  • Verified with clang-format --dry-run --Werror, a full windows-development build (0 warnings), gp-core-tests.exe (1,437 passed), and gp-application-tests.exe (13 passed).

Checklist

  • My code follows the gp-engine C++ Coding Standard.
  • I have compiled the engine successfully using the modern C++23 toolchain (GCC 13+, Clang 16+, or MSVC 19.38+) and CMake.
  • I have performed a self-review of my code and added comments, especially in complex areas like shader programming or math-heavy algorithms.
  • I have updated the relevant API reference or learning path documentation (if applicable).
  • My changes generate no new compiler warnings in the build log.
  • I have read and agree to the Code of Conduct and Contributing Guidelines.

GP Contributor License Agreement

I give to the Graphical Playground Team, LLC permission to license my contributions on any terms
they like. I am giving them this license in order to make it possible for them to accept my
contributions into their project.

As far as the law allows, my contributions come as is, without any warranty or condition, and I
will not be liable to anyone for any damages related to this software or this license, under any
kind of legal claim.

@mallory-scotton mallory-scotton self-assigned this Nov 19, 2026
@mallory-scotton mallory-scotton added the confirmed Issue has been verified and reproduced label Nov 19, 2026
@github-actions github-actions Bot added documentation Improvements or additions to documentation platform:android Specific to the Android platform platform:ios Specific to the iOS platform platform:visionos Specific to the Apple visionOS platform labels Nov 19, 2026
@github-actions github-actions Bot added the topic:editor Involves the editor UI or tooling label Nov 19, 2026
@mallory-scotton
mallory-scotton marked this pull request as ready for review November 19, 2026 15:46
@mallory-scotton
mallory-scotton requested review from a team as code owners November 19, 2026 15:46
@mallory-scotton
mallory-scotton merged commit e3f2e0e into main Nov 19, 2026
11 checks passed
@mallory-scotton
mallory-scotton deleted the ms/vfs branch November 19, 2026 15:46
@github-project-automation github-project-automation Bot moved this from Todo to Done in Engine Issues Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

confirmed Issue has been verified and reproduced documentation Improvements or additions to documentation enhancement Improvement to an existing feature high priority Requires immediate attention needs testing Requires testing before it can be closed needs work More work needed before it can be merged or closed performance Related to speed, memory usage, or efficiency platform:android Specific to the Android platform platform:ios Specific to the iOS platform platform:linuxbsd Specific to the Linux or BSD platform platform:macos Specific to the macOS platform platform:visionos Specific to the Apple visionOS platform platform:windows Specific to the Windows platform topic:core Involves core engine functionality topic:editor Involves the editor UI or tooling topic:platforms Involves platform-specific behavior or porting topic:tests Involves unit or integration tests

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant