Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
865dd8c
docs(adr): update status to accepted and adjust badge in ADR-0044
mallory-scotton Nov 19, 2026
9584552
add(adr): introduce ADR-0060 for Layer 0 OS FileHandle and I/O subsystem
mallory-scotton Nov 19, 2026
8db5d4f
add(adr): document ADR-0060 for zero-allocation OS FileHandle and I/O…
mallory-scotton Nov 19, 2026
456fe44
add(workspace): associate 'vfs' with 'dump' in file type mappings
mallory-scotton Nov 19, 2026
e4422e8
add(file-enums): define enums for file error codes, access modes, and…
mallory-scotton Nov 19, 2026
e480eae
add(file-handle): implement non-polymorphic, zero-allocation RAII fil…
mallory-scotton Nov 19, 2026
9c4d6c3
add(file-system): implement platform-specific file system interface
mallory-scotton Nov 19, 2026
e983f9b
add(file-types): implement file metadata structures and directory ent…
mallory-scotton Nov 19, 2026
0c32286
update(path-types): add IWYU pragma to StringView include
mallory-scotton Nov 19, 2026
9df1822
add(file-system): implement Win32 file and directory utilities
mallory-scotton Nov 19, 2026
12c850e
add(file-system): implement POSIX file and directory utilities for Unix
mallory-scotton Nov 19, 2026
d89d2b2
add(file-system): implement generic file system interface for cross-p…
mallory-scotton Nov 19, 2026
8b66bf8
add(file-system): implement Apple file system interface extending Unix
mallory-scotton Nov 19, 2026
733c116
add(file-system): implement Linux file system interface extending Unix
mallory-scotton Nov 19, 2026
24ed585
add(file-system): implement macOS file system interface extending Apple
mallory-scotton Nov 19, 2026
ca97c00
add(file-system): implement Apple, Linux, and macOS file system inter…
mallory-scotton Nov 19, 2026
a08c086
add(file-system): implement generic file system interface with basic …
mallory-scotton Nov 19, 2026
b11655a
add(file-system): implement Unix file system interface with core oper…
mallory-scotton Nov 19, 2026
e14ddf4
add(file-system): implement Windows file system interface with core o…
mallory-scotton Nov 19, 2026
e567433
add(vfs): implement abstract mount backend interface for virtual file…
mallory-scotton Nov 19, 2026
b3b1ec1
add(vfs): implement in-memory virtual filesystem backend
mallory-scotton Nov 19, 2026
8481253
add(vfs): implement physical mount backend for virtual file system
mallory-scotton Nov 19, 2026
cf96039
add(vfs): define enums for VFS error codes, mount priorities, and access
mallory-scotton Nov 19, 2026
7845f43
add(vfs): define enums and structures for virtual file system components
mallory-scotton Nov 19, 2026
02b471d
add(vfs): define structures and constants for virtual file system com…
mallory-scotton Nov 19, 2026
9a5bb64
add(vfs): implement virtual file system with mounting and URI handling
mallory-scotton Nov 19, 2026
bd45ee2
add(tests): implement comprehensive file system tests for core functi…
mallory-scotton Nov 19, 2026
9cb5edf
add(tests): implement unit tests for virtual file system functionality
mallory-scotton Nov 19, 2026
92fd655
update(vfs): mount editor-only content schemes and log virtual file s…
mallory-scotton Nov 19, 2026
1e6b551
add(vfs): implement virtual file system initialization and shutdown l…
mallory-scotton Nov 19, 2026
a684962
add(tests): implement virtual file system lifecycle and command line …
mallory-scotton Nov 19, 2026
17d0956
add(vfs): add vfsAppName and enableVirtualFileSystem to ApplicationDesc
mallory-scotton Nov 19, 2026
10ed5d8
fix(tests): remove unnecessary casts in VFS log URI resolution test
mallory-scotton Nov 19, 2026
687b8c7
update(logging): refactor FileSink to use platform-specific file hand…
mallory-scotton Nov 19, 2026
131a84e
fix(unix): remove unused parameter from FileSystem::open method
mallory-scotton Nov 19, 2026
7395b3b
fix(vfs): mark unused parameters in resolvePhysicalPath and openFileH…
mallory-scotton Nov 19, 2026
0859f39
fix(vfs): remove unused cast in PhysicalMountBackend constructor
mallory-scotton Nov 19, 2026
89b0fc6
add(vfs): implement mount enumeration and console commands for VFS
mallory-scotton Nov 19, 2026
c4c8551
update(logging): enhance FileSink to use FileHandle and improve docum…
mallory-scotton Nov 19, 2026
c85c868
fix(filehandle): remove unnecessary cast when closing file descriptor
mallory-scotton Nov 19, 2026
f814e11
fix(generic): remove unnecessary casts when setting read-only state
mallory-scotton Nov 19, 2026
5cf499b
add(vfs): implement toString functions for MountPriority and MountAccess
mallory-scotton Nov 19, 2026
26abf88
add(vfs): implement stateless callback signature for mount-table insp…
mallory-scotton Nov 19, 2026
099608e
add(vfs): implement enumerateMounts functions for active mount point …
mallory-scotton Nov 19, 2026
d007948
add(tests): implement file sink test for live read and append functio…
mallory-scotton Nov 19, 2026
544a954
fix(tests): remove unnecessary casts in file system test setup and te…
mallory-scotton Nov 19, 2026
d177f1d
add(tests): implement deterministic order and diagnostics for enumera…
mallory-scotton Nov 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/adr/ADR-0044.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
id: ADR-0044
title: Layered Virtual File System (VFS) Mount-Point and Logical URI Routing Architecture
date: 2026-11-01
status: proposed
status: accepted
authors:
- mallory-scotton
deciders:
Expand All @@ -21,7 +21,7 @@ deciders:
> `cache://`, `temp://`), isolating gameplay, shader, and asset streaming code from physical disk
> layouts and console sandbox partitions.

![Proposed][badge-proposed]
![Accepted][badge-accepted]

## Context

Expand Down Expand Up @@ -224,6 +224,8 @@ multiple physical directories into a single logical hierarchy.
`gp::application`, `gp::launch`).
* [ADR-0043](./ADR-0043.md): Zero-Allocation `.bss` Folder Discovery and Stack `PathBuffer` in
`gp::platform::Paths`.
* [ADR-0060](./ADR-0060.md): Zero-Allocation OS `FileHandle`, Positional `readAt`/`writeAt`, Atomic
Writes, and Stack Directory Iteration in `gp::platform::FileSystem`.

**Canonical references:**

Expand Down
335 changes: 335 additions & 0 deletions docs/adr/ADR-0060.md

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions docs/adr/INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,14 @@ Hardware Abstraction Layer (HAL).
> and mod override layers, to logical URI schemes (`engine://`, `game://`, `user://`,
> `appdata://`, `cache://`, `temp://`), isolating gameplay, shader, and asset streaming code from
> physical disk layouts and console sandbox partitions.
- [**ADR-0060**: Zero-Allocation OS `FileHandle`, Positional `readAt`/`writeAt`, Atomic Writes, and
Stack Directory Iteration in `gp::platform::FileSystem`](./ADR-0060.md)
> Establishes `gp::platform::FileSystem` and `gp::platform::FileHandle` in `gp::core` as the
> Layer 0 zero-heap-allocation OS file and directory I/O subsystem, pairing non-polymorphic RAII
> OS file descriptors with lock-free positional I/O (`pread`/`pwrite` and Win32 `OVERLAPPED`
> offsets), crash-safe atomic write-and-rename transactions (`writeAtomic`), and zero-allocation
> stack-buffered directory tree traversal (`iterateDirectory`) below Layer 1
> `gp::vfs::VirtualFileSystem` ([ADR-0044](./ADR-0044.md)).

### Concurrency & Event Routing

Expand Down
1 change: 1 addition & 0 deletions gp-engine.code-workspace
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@
"appleclang": "assembly",
"gcc": "assembly",
"intel": "assembly",
"vfs": "dump",
"platforms": "client",
"es": "i18n",
"fr": "i18n",
Expand Down
11 changes: 10 additions & 1 deletion source/launch/editor/private/Main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
#include "miscellaneous/CommandLine.hpp"
#include "profiling/Profiler.hpp"
#include "profiling/ProfilingControl.hpp"
#include "vfs/VirtualFileSystem.hpp"
#include <cstdio>

namespace gp
Expand Down Expand Up @@ -62,6 +63,14 @@ class EditorApplication : public Application
"EditorApplication::OnInitialize", GP_U32_COLOR_PURPLE, profiling::ProfilingControl::isZonesEnabled()
);

// TODO(vfs): Mount editor-only content schemes here (e.g., `editor://` pointing to the editor resources root,
// plus any active project or developer override layers) once the editor asset directory layout is
// defined. The six base ADR-0044 schemes (`engine://`, `game://`, `user://`, `appdata://`,
// `cache://`, `temp://`) are already mounted by `Application::initializeInternal()`.
GP_LOG_INFO(
LogEditor, "VirtualFileSystem Ready ({} active mounts).", gp::vfs::VirtualFileSystem::getMountCount()
);

const gp::console::IConsoleManager& consoleMgr = getConsoleManager();
GP_LOG_INFO(
LogEditor,
Expand Down Expand Up @@ -127,7 +136,7 @@ class EditorApplication : public Application
std::fputs(
"\n========================================================\n"
" [TEMP] Console Stdin Shell Active\n"
" Examples: help | cvar.list | cvar.dump | prof.Status | log.Stats | app.Stats | jobs.Stats\n"
" Examples: help | cvar.list | cvar.dump | prof.Status | log.Stats | app.Stats | jobs.Stats | vfs.Status | vfs.Mounts\n"
" Set CVar: t.TimeScale 0.5 | app.TargetFps=60 | log.GlobalLevel Warning\n"
" Commands: 'resume' (or Enter) returns to gp-editor | 'quit' / 'app.Quit' exits\n"
"========================================================\n",
Expand Down
93 changes: 92 additions & 1 deletion source/runtime/application/private/Application.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,16 @@
#include "input/GenericInputSystem.hpp"
#include "logging/LogFormatter.hpp"
#include "miscellaneous/CommandLine.hpp"
#include "platforms/base/system/PathTypes.hpp"
#include "profiling/Profiler.hpp"
#include "profiling/ProfilingControl.hpp"
#include "templates/Expected.hpp"
#include "templates/Optional.hpp"
#include "threading/roles/ThreadRole.hpp"
#include "threading/roles/ThreadRoleEnums.hpp"
#include "threading/roles/ThreadRoleTypes.hpp"
#include "vfs/VfsEnums.hpp"
#include "vfs/VirtualFileSystem.hpp"

namespace gp
{
Expand Down Expand Up @@ -196,6 +200,55 @@ void Application::applyCommandLineOverrides() noexcept
m_desc.fileSink.filePath = *customLogPath;
}
}

const Optional<bool> optVfs = CommandLine::tryGetBool("vfs");
if (optVfs.hasValue())
{
m_desc.enableVirtualFileSystem = *optVfs;
}

const Optional<StringView> optVfsApp = CommandLine::tryGetString("vfs-app");
if (optVfsApp.hasValue() && !optVfsApp->isEmpty())
{
m_desc.vfsAppName = *optVfsApp;
}
}

vfs::VfsError Application::initializeVirtualFileSystemInternal() noexcept
{
if (!m_desc.enableVirtualFileSystem)
{
return vfs::VfsError::None;
}

// Preserve pre-configured mounts if the host or test harness mounted schemes before Application::run().
if (vfs::VirtualFileSystem::getMountCount() > 0u)
{
m_ownsVfsInitialization = false;
return vfs::VfsError::None;
}

const gp::StringView effectiveAppName = !m_desc.vfsAppName.isEmpty() ? m_desc.vfsAppName : m_desc.backend.appName;
const gp::Expected<bool, vfs::VfsError> mountRes =
vfs::VirtualFileSystem::initializeDefaultMounts(effectiveAppName);
if (!mountRes.hasValue()) [[unlikely]]
{
vfs::VirtualFileSystem::shutdown();
m_ownsVfsInitialization = false;
return mountRes.error();
}

m_ownsVfsInitialization = true;
return vfs::VfsError::None;
}

void Application::shutdownVirtualFileSystemInternal() noexcept
{
if (m_ownsVfsInitialization)
{
vfs::VirtualFileSystem::shutdown();
m_ownsVfsInitialization = false;
}
}

bool Application::attachThreadRolesInternal() noexcept
Expand Down Expand Up @@ -311,7 +364,26 @@ void Application::initializeLoggingInternal() noexcept

if (m_desc.enableFileLogSink)
{
if (m_fileSink.open(m_desc.fileSink) == log::LogResult::Success)
log::FileSinkDesc resolvedFileSinkDesc = m_desc.fileSink;
gp::PathBuffer resolvedLogPath{};
bool canOpenFileSink = true;

if (!resolvedFileSinkDesc.filePath.isEmpty() && resolvedFileSinkDesc.filePath.contains("://"))
{
const gp::Expected<gp::PathBuffer, vfs::VfsError> resolved =
vfs::VirtualFileSystem::resolveWritablePhysicalPath(resolvedFileSinkDesc.filePath);
if (resolved.hasValue())
{
resolvedLogPath = resolved.value();
resolvedFileSinkDesc.filePath = resolvedLogPath.view();
}
else
{
canOpenFileSink = false;
}
}

if (canOpenFileSink && m_fileSink.open(resolvedFileSinkDesc) == log::LogResult::Success)
{
(void)logger.addSink(&m_fileSink);
}
Expand Down Expand Up @@ -633,12 +705,21 @@ bool Application::initializeInternal() noexcept
);

applyCommandLineOverrides();
const vfs::VfsError vfsInitError = initializeVirtualFileSystemInternal();
initializeLoggingInternal();
(void)console::IConsoleManager::get();

if (vfsInitError != vfs::VfsError::None) [[unlikely]]
{
GP_LOG_ERROR(
LogApplication, "VirtualFileSystem default mount initialization failed ({}).", vfs::toString(vfsInitError)
);
}

if (!attachThreadRolesInternal()) [[unlikely]]
{
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
return false;
}

Expand All @@ -663,6 +744,7 @@ bool Application::initializeInternal() noexcept
);
detachThreadRolesInternal();
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
return false;
}

Expand All @@ -676,6 +758,7 @@ bool Application::initializeInternal() noexcept
m_eventStream.shutdown();
detachThreadRolesInternal();
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
return false;
}
}
Expand All @@ -687,6 +770,7 @@ bool Application::initializeInternal() noexcept
m_eventStream.shutdown();
detachThreadRolesInternal();
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
return false;
}

Expand All @@ -705,6 +789,7 @@ bool Application::initializeInternal() noexcept
m_eventStream.shutdown();
detachThreadRolesInternal();
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
return false;
}
}
Expand Down Expand Up @@ -732,6 +817,7 @@ bool Application::initializeInternal() noexcept
m_eventStream.shutdown();
detachThreadRolesInternal();
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
return false;
}
}
Expand Down Expand Up @@ -766,6 +852,7 @@ bool Application::initializeInternal() noexcept
m_eventStream.shutdown();
detachThreadRolesInternal();
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
return false;
}

Expand All @@ -785,6 +872,7 @@ bool Application::initializeInternal() noexcept
m_eventStream.shutdown();
detachThreadRolesInternal();
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
return false;
}

Expand All @@ -805,6 +893,7 @@ bool Application::initializeInternal() noexcept
m_eventStream.shutdown();
detachThreadRolesInternal();
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
return false;
}

Expand All @@ -827,6 +916,7 @@ bool Application::initializeInternal() noexcept
m_eventStream.shutdown();
detachThreadRolesInternal();
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
return false;
}
}
Expand Down Expand Up @@ -885,6 +975,7 @@ void Application::shutdownInternal() noexcept
m_eventStream.shutdown();
detachThreadRolesInternal();
shutdownLoggingInternal();
shutdownVirtualFileSystemInternal();
m_isInitialized = false;
}

Expand Down
16 changes: 14 additions & 2 deletions source/runtime/application/public/Application.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
#include "threading/jobs/JobSystem.hpp"
#include "threading/roles/NamedThreadDispatcher.hpp"
#include "time/EngineClock.hpp"
#include "vfs/VfsForward.hpp"
#include "window/IWindow.hpp"
#include "window/WindowDesc.hpp" // IWYU pragma: keep

Expand Down Expand Up @@ -111,6 +112,7 @@ class GP_APPLICATION_API Application
gp::Atomic<bool> m_isRunning{ false };
bool m_isInitialized{ false };
bool m_isSuspended{ false };
bool m_ownsVfsInitialization{ false };
bool m_ownsLoggerInitialization{ false };
bool m_ownsGameThreadRole{ false };
bool m_ownsRenderThreadRole{ false };
Expand Down Expand Up @@ -473,10 +475,20 @@ class GP_APPLICATION_API Application
{}

private:
/// @brief Applies command-line concurrency and logging overrides (`-threads`, `-workers`, `-no-jobsystem`,
/// `-core-affinity`, `-no-log`, `-no-console-log`, `-log-file`) from `gp::CommandLine` to `m_desc`.
/// @brief Applies command-line concurrency, logging, and VFS overrides (`-threads`, `-workers`, `-no-jobsystem`,
/// `-core-affinity`, `-no-log`, `-no-console-log`, `-log-file`, `-no-vfs`, `-vfs-app`) from
/// `gp::CommandLine` to `m_desc`.
void applyCommandLineOverrides() noexcept;

/// @brief Mounts the six default ADR-0044 VFS schemes when `m_desc.enableVirtualFileSystem` is true and no mounts
/// are already registered by the caller.
/// @return `vfs::VfsError::None` on success (or if skipped), or the error returned by
/// `VirtualFileSystem::initializeDefaultMounts()`.
[[nodiscard]] vfs::VfsError initializeVirtualFileSystemInternal() noexcept;

/// @brief Unmounts all VFS mount points via `VirtualFileSystem::shutdown()` if this `Application` initialized them.
void shutdownVirtualFileSystemInternal() noexcept;

/// @brief Claims `NamedThread::Game` (and `Render` / `RHI` when `singleThreadedRenderAndRHI` is set) for the
/// calling thread.
/// @return True if all requested roles were attached or already held by the caller, false if another thread owns
Expand Down
12 changes: 12 additions & 0 deletions source/runtime/application/public/ApplicationDesc.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

#include "ApplicationTypes.hpp"
#include "backend/ApplicationBackendDesc.hpp"
#include "containers/views/StringView.hpp" // IWYU pragma: keep
#include "CoreMinimal.hpp"
#include "logging/LogDesc.hpp"
#include "threading/jobs/JobSystemDesc.hpp"
Expand Down Expand Up @@ -45,6 +46,12 @@ struct ApplicationDesc
/// @brief Configuration for the built-in buffered `FileSink` ([ADR-0054]).
log::FileSinkDesc fileSink{};

/// @brief Optional application subfolder name passed to `gp::vfs::VirtualFileSystem::initializeDefaultMounts()`
/// ([ADR-0044]).
/// @details When empty, `Application` falls back to `backend.appName` (and `VirtualFileSystem` falls back to the
/// executable stem if that is also empty). Can be overridden via `-vfs-app=<name>` on the command line.
gp::StringView vfsAppName{};

/// @brief Byte capacity of the lock-free SPSC event stream (automatically rounded up to a power of two).
gp::USize eventRingBufferBytes{ 16384u };

Expand Down Expand Up @@ -96,6 +103,11 @@ struct ApplicationDesc

/// @brief If true, registers the application's `TracySink` with `Logger::instance()`.
bool enableTracyLogSink{ true };

/// @brief If true, mounts the six default ADR-0044 VFS schemes (`engine://`, `game://`, `user://`, `appdata://`,
/// `cache://`, `temp://`) during startup and unmounts them on exit (can be overridden by `-no-vfs` on the
/// command line).
bool enableVirtualFileSystem{ true };
};

} // namespace gp
Loading
Loading