⚠️ Disclaimer: This repository serves as a technical showcase and architectural overview of a project developed during my internship at Aris888. Due to Non-Disclosure Agreements (NDA) and proprietary rights, the source code is not publicly available. This document demonstrates the system design, challenges solved, and technologies utilized.
In this project, I engineered a multisensory Virtual Reality experience combining immersive environment design, hardware-based scent diffusion, and Generative AI.
My primary focus was designing realistic 3D environments in Unity and implementing a proximity-based scent system using the AromaJoin SDK. This system triggers specific scents when users approach virtual flora. To further enhance immersion, I integrated a Python middleware layer to power a dynamic, context-aware NPC (Non-Player Character) using LLMs.
- Immersive Scene Design: Designed and optimized photorealistic Unity scenes from scratch.
- Hardware Integration (AromaJoin): Implemented the AromaJoin SDK to control scent diffusion hardware based on player position and distance to objects.
- Dynamic AI Interaction: Engineered a middleware solution to enable unscripted, voice-based conversations with NPCs.
| Component | Technology | Role |
|---|---|---|
| VR Engine | Unity (C#) | Environment Design, Physics, Raycasting |
| Haptic/Scent | AromaJoin SDK | Hardware Control & Scent Diffusion Logic |
| Middleware | Python (FastAPI) | REST API, LLM Orchestration |
| AI Core | LLM / GenAI | Dynamic Dialogue Generation |
The system operates on a Hybrid Architecture designed to balance immersion with intelligence. It is divided into two distinct execution loops:
- Local Execution (Zero Latency): Spatial interactions, such as distance calculations and scent triggering via the AromaJoin SDK, are processed locally within the Unity engine. This ensures instant feedback when the user approaches virtual flora, eliminating network lag for physical sensations.
- Cloud Execution (AI Processing): Complex logic, such as the NPC's conversational intelligence, is offloaded to a Python/FastAPI Middleware. This layer manages the state and communicates with the LLM to generate context-aware dialogue.
- Challenge: Triggering scents precisely when a user interacts with specific objects (e.g., smelling a flower) without overwhelming the user with constant diffusion.
- Solution: Integrated the AromaJoin SDK with Unity's physics engine. I implemented a distance-check algorithm (Raycasting & Colliders) that calculates the user's proximity to target objects (plants/flowers). The system dynamically adjusts the diffusion intensity based on distance and stops immediately when the user walks away.
- Challenge: Integrating an LLM for the NPC at the end of the project introduced potential latency, which can break immersion in VR.
- Solution: Implemented Asynchronous Programming in Python (
async/await) and C# (Coroutines). Designed a non-blocking "Thinking State" for the NPC, where it plays idle animations while waiting for the backend response, ensuring the frame rate remains stable.
- Challenge: Designing high-fidelity nature scenes while maintaining 90+ FPS for VR comfort.
- Solution: Utilized occlusion culling, baked lighting, and optimized 3D assets to ensure smooth performance despite the visual complexity.
|
|
|
|
(Above: Demonstration of the custom-designed VR environments where flora interactions trigger the AromaJoin hardware.)
Ege Karaca