> ## Documentation Index
> Fetch the complete documentation index at: https://hyperscape-ai.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Client Application

> React UI, 3D rendering, VRM avatars, and client architecture

# Client Application

The Hyperscape client is a React-based web application with Three.js WebGPU rendering. It provides a modern MMORPG experience with VRM avatars, responsive UI panels, and real-time multiplayer.

<Info>
  Client code lives in `packages/client/src/`. The rendering systems are in `packages/shared/src/systems/client/`.
</Info>

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────────┐
│                       React Application                         │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │ Auth Flow   │  │  Screens    │  │    Game Panels          │  │
│  │ PrivyAuth   │  │ - Login     │  │ - Inventory  - Skills   │  │
│  │ Wallet Auth │  │ - CharSel   │  │ - Equipment  - Bank     │  │
│  │             │  │ - GameClient│  │ - Combat     - Chat     │  │
│  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
├─────────────────────────────────────────────────────────────────┤
│                    Three.js WebGPU Renderer                     │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │ Scene Graph │  │  Cameras    │  │    Post-Processing      │  │
│  │ Terrain     │  │ Third Person│  │ - Bloom                 │  │
│  │ Entities    │  │ First Person│  │ - Tone Mapping          │  │
│  │ VRM Avatars │  │ RTS Mode    │  │ - Color Grading         │  │
│  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
├─────────────────────────────────────────────────────────────────┤
│                      Client World                               │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │  Network    │  │    Input    │  │     Systems             │  │
│  │  WebSocket  │  │  Keyboard   │  │ - Graphics  - Camera    │  │
│  │  Reconnect  │  │  Mouse      │  │ - Audio     - Loader    │  │
│  │             │  │  Touch      │  │ - Health    - XP Drops  │  │
│  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘
```

***

## Entry Point

The client entry point is `packages/client/src/index.tsx`:

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// Main entry point for the Hyperscape browser client
import { World, installThreeJSExtensions } from "@hyperscape/shared";
import React from "react";
import ReactDOM from "react-dom/client";

// Screens
import { LoginScreen } from "./screens/LoginScreen";
import { CharacterSelectScreen } from "./screens/CharacterSelectScreen";
import { GameClient } from "./screens/GameClient";

// Authentication (Privy + Wallet)
import { PrivyAuthProvider } from "./auth/PrivyAuthProvider";

// Embedded mode support for spectator views
import { EmbeddedGameClient } from "./components/EmbeddedGameClient";
import { isEmbeddedMode } from "./types/embeddedConfig";
```

***

## Authentication Flow

Authentication uses [Privy](https://www.privy.io/) for wallet-based login:

```
Login Screen → Privy Modal → Wallet Signature → Character Select → Game
```

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// auth/PrivyAuthProvider.tsx
export function PrivyAuthProvider({ children }: { children: React.ReactNode }) {
  return (
    <PrivyProvider
      appId={PRIVY_APP_ID}
      config={{
        loginMethods: ["wallet", "email", "google"],
        appearance: { theme: "dark" },
      }}
    >
      {children}
    </PrivyProvider>
  );
}
```

***

## Screens

### Login Screen

* Wallet connection via Privy
* Social login (email, Google)
* Session persistence

### Character Select Screen

* List existing characters
* Create new character
* Character preview with VRM avatar

### Game Client

* Main game loop
* World initialization
* UI overlay rendering

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// screens/GameClient.tsx
export function GameClient({ characterId }: { characterId: string }) {
  const worldRef = useRef<World | null>(null);
  
  useEffect(() => {
    const world = new World({
      isServer: false,
      systems: clientSystems,
    });
    
    world.connect({ characterId });
    worldRef.current = world;
    
    return () => world.destroy();
  }, [characterId]);
  
  return <CoreUI world={worldRef.current} />;
}
```

***

## UI Architecture

The client uses a clear separation between reusable UI primitives and game-specific code:

### UI Design System (`src/ui/`)

Pure, reusable UI components and systems:

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// From packages/client/src/ui/
components/               // Base UI primitives
  Window.tsx             // Draggable, resizable windows with anchor-based positioning
  TabBar.tsx             // Tab navigation with drag-to-combine
  DragOverlay.tsx        // Drag preview overlay
  Portal.tsx             // React portal for overlays
  MenuButton.tsx         // Menu buttons
  ItemSlot.tsx           // Item slot component
  // ... and more

core/                    // Core UI systems
  drag/                  // Drag-and-drop (@dnd-kit integration)
    DragContext.tsx      // Global drag state
    useDrag.ts           // Drag hook
    useDrop.ts           // Drop hook with race condition fixes
  edit/                  // Edit mode (L key to unlock)
  notifications/         // Notification system
  presets/               // Layout presets and cloud sync
  responsive/            // Breakpoint and mobile detection
  tabs/                  // Tab management and combining
  window/                // Window management with anchors

stores/                  # UI state management (Zustand)
  windowStore.ts         # Window positions with anchor-based positioning
  dragStore.ts           # Drag-and-drop state
  editStore.ts           # Edit mode state
  anchorUtils.ts         # Anchor positioning utilities
  // ... and more
```

**Anchor-Based Positioning:**

Windows use viewport anchors (Unity/Unreal-style) for responsive scaling:

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// From src/ui/stores/anchorUtils.ts
export type WindowAnchor =
  | "top-left" | "top-center" | "top-right"
  | "left-center" | "center" | "right-center"
  | "bottom-left" | "bottom-center" | "bottom-right";

// Key functions
getAnchorPosition(anchor, viewport)      // Get anchor coordinates
calculateOffsetFromAnchor(position, anchor, viewport)  // Calculate offset
repositionWindowForViewport(window, oldViewport, newViewport)  // Resize handler
```

### Game-Specific Code (`src/game/`)

All game logic and components:

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// From packages/client/src/game/
interface/               # Window management
  InterfaceManager.tsx   # Main UI orchestrator with drag-drop
  DefaultLayoutFactory.ts # Default window layouts with anchors
  useViewportResize.ts   # Responsive window positioning
  
panels/                  # Game panels
  InventoryPanel.tsx     # 28-slot inventory with drag & drop
  EquipmentPanel.tsx     # Equipment slots with stat display
  SkillsPanel.tsx        # All skills with XP progress bars
  CombatPanel.tsx        # Combat style selection with draggable styles
  BankPanel/             # Modular bank system
  TradePanel/            # Two-screen trade confirmation
  ActionBarPanel/        # Action bar with drag-drop support
  // ... and more

hud/                     # HUD elements
  StatusBars.tsx         # Health, prayer, run energy bars
  XPProgressOrb.tsx      # XP tracking orb with skill icons
  ActionProgressBar.tsx  # Skilling action progress
  Minimap.tsx            # Top-down map view
  MinimapCompass.tsx     # Compass overlay
  MinimapStaminaBar.tsx  # Stamina display
  level-up/              # Level-up notifications
  xp-orb/                # XP orb components

components/              # Game components
  chat/                  # Chat system (ChatBox, ChatInput, ChatMessage, ChatTabs)
  currency/              # Currency display and exchange
  dialog/                # Dialogue system
  equipment/             # Equipment components
  map/                   # World map
  quest/                 # Quest system
  settings/              # Settings panels
  skilltree/             # Skill tree
```

### Core UI (CoreUI.tsx)

The main UI wrapper that renders HUD elements and game panels:

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
export function CoreUI({ world }: { world: ClientWorld }) {
  const [ready, setReady] = useState(false);
  const [deathScreen, setDeathScreen] = useState(null);
  
  // Event handlers for loading, death, disconnect
  useEffect(() => {
    world.on(EventType.READY, () => setReady(true));
    world.on(EventType.PLAYER_DIED, handleDeath);
    world.on(EventType.DISCONNECTED, handleDisconnect);
  }, [world]);
  
  if (!ready) return <LoadingScreen />;
  
  return (
    <>
      <InterfaceManager world={world} />
      <StatusBars world={world} />
      <ActionProgressBar world={world} />
      <XPProgressOrb world={world} />
      {deathScreen && <DeathScreen {...deathScreen} />}
    </>
  );
}
```

***

## 3D Graphics System

### Renderer (WebGPU with WebGL Fallback)

The graphics system uses Three.js with WebGPU for high-performance rendering, with automatic WebGL fallback for environments that don't support WebGPU (e.g., WKWebView in Tauri, older browsers):

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// systems/client/ClientGraphics.ts
export class ClientGraphics extends SystemBase {
  private renderer: UniversalRenderer;
  private postProcessing: PostProcessingComposer;
  isWebGPU: boolean = true;
  
  async init() {
    // Create renderer (WebGPU preferred, WebGL fallback)
    this.renderer = await createRenderer({
      powerPreference: "high-performance",
      antialias: true,
    });
    
    this.isWebGPU = isWebGPURenderer(this.renderer);
    
    // Log backend capabilities
    if (this.isWebGPU) {
      logWebGPUInfo(this.renderer);
      const caps = getWebGPUCapabilities(this.renderer);
      console.log("[ClientGraphics] WebGPU features:", caps.features.length);
    } else {
      console.warn("[ClientGraphics] WebGPU unavailable (falling back to WebGL renderer)");
    }
    
    // Configure shadows
    // WebGPU: Cascaded Shadow Maps (CSM)
    // WebGL: Single directional light shadow
    configureShadowMaps(this.renderer, {
      cascades: 3,
      shadowMapSize: 2048,
    });
    
    // Post-processing (TSL-based, WebGPU only)
    // WebGL fallback disables post-processing
    this.usePostprocessing = (this.world.prefs?.postprocessing ?? true) && this.isWebGPU;
    
    if (this.usePostprocessing && isWebGPURenderer(this.renderer)) {
      this.postProcessing = createPostProcessing(this.renderer, {
        bloom: true,
        toneMapping: true,
        colorGrading: true,
      });
    }
  }
}
```

### Renderer Backend Detection

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// utils/rendering/RendererFactory.ts
export type RendererBackend = "webgpu" | "webgl";
export type UniversalRenderer = InstanceType<typeof THREE.WebGPURenderer>;

// Check if WebGPU is available
export async function isWebGPUAvailable(): Promise<boolean> {
  if (typeof navigator === "undefined") return false;
  
  const gpuApi = (navigator as NavigatorWithGpu).gpu;
  if (!gpuApi) return false;
  
  try {
    const adapter = await gpuApi.requestAdapter();
    return adapter !== null;
  } catch {
    return false;
  }
}

// Check if WebGL is available
export function isWebGLAvailable(): boolean {
  if (typeof document === "undefined") return false;
  
  const canvas = document.createElement("canvas");
  const gl = canvas.getContext("webgl2") || canvas.getContext("webgl");
  return gl !== null;
}

// Detect rendering capabilities
export async function detectRenderingCapabilities(): Promise<RenderingCapabilities> {
  const supportsWebGPU = await isWebGPUAvailable();
  if (supportsWebGPU) {
    return { supportsWebGPU: true, supportsWebGL: true, backend: "webgpu" };
  }
  
  const supportsWebGL = isWebGLAvailable();
  if (supportsWebGL) {
    return { supportsWebGPU: false, supportsWebGL: true, backend: "webgl" };
  }
  
  throw new Error(
    "Neither WebGPU nor WebGL is supported in this environment. " +
    "Please use a modern browser or a WebView with GPU acceleration enabled."
  );
}
```

### WebGL Fallback Features

When running on WebGL backend:

* **No TSL Post-Processing**: Bloom, tone mapping, and color grading are disabled
* **Simplified Shadows**: Single directional light instead of Cascaded Shadow Maps (CSM)
* **Auto Exposure**: Still works (tone mapping exposure is renderer-agnostic)
* **Settings Panel**: Displays "WebGL" instead of "WebGPU"

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// Environment.ts - WebGL shadow fallback
if (!useWebGPU) {
  this.csmShadowNode = null;
  this.csmNeedsAttach = false;
  this.needsFrustumUpdate = false;
  
  scene.add(this.sunLight);
  scene.add(this.sunLight.target);
  
  console.log(
    `[Environment] WebGL shadow map enabled (no CSM): mapSize=${csmConfig.shadowMapSize}, frustum=${baseFrustumSize * 2}`
  );
  return;
}
```

<Tip>
  The WebGL fallback uses `THREE.WebGPURenderer` with `forceWebGL: true` instead of switching to `THREE.WebGLRenderer`. This keeps the codebase unified while supporting both backends.
</Tip>

### Rendering Pipeline

1. **Pre-render**: Update matrices, frustum culling
2. **Shadow Pass**: Render cascaded shadow maps
3. **Main Pass**: Render scene with deferred lighting
4. **Post-Processing**: Bloom, tone mapping, effects (TSL-based)
5. **UI Overlay**: Render 2D React UI on top

### Model Loading & Transform Baking

The `ModelCache` system handles GLTF model loading with transform baking to prevent rendering issues:

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// From ModelCache.ts
private bakeTransformsToGeometry(scene: THREE.Object3D): void {
  // Ensure all matrices are up to date
  scene.updateMatrixWorld(true);

  // Apply transforms to each mesh's geometry
  scene.traverse((child) => {
    if (child instanceof THREE.Mesh && child.geometry) {
      // Clone geometry to avoid modifying shared geometry
      child.geometry = child.geometry.clone();

      // Apply world matrix to geometry (Three.js built-in method)
      child.geometry.applyMatrix4(child.matrixWorld);

      // Reset transform to identity
      child.position.set(0, 0, 0);
      child.rotation.set(0, 0, 0);
      child.scale.set(1, 1, 1);
      child.updateMatrix();
    }
  });
}
```

**Why Transform Baking?**

GLTF files can have transforms stored in various ways:

* Position/rotation/scale properties
* Baked into matrices
* Non-decomposable transforms (shear)

Baking all transforms into vertex positions guarantees correct rendering regardless of how the GLTF was exported from Blender or other 3D tools.

**Quaternion Normalization**:

Entity rotations use quaternions with all four components (x, y, z, w):

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// From Entity.ts
quaternion: config.rotation
  ? [
      config.rotation.x,
      config.rotation.y,
      config.rotation.z,
      config.rotation.w,  // Uses actual w value, not hardcoded 1
    ]
  : undefined,
```

This prevents "squished" or incorrectly rotated models that can occur when quaternion components are not properly normalized.

### Camera System

Supports multiple camera modes:

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// systems/client/ClientCameraSystem.ts
export class ClientCameraSystem extends SystemBase {
  private settings = {
    minDistance: 2.0,        // Min zoom
    maxDistance: 15.0,       // Max zoom
    minPolarAngle: Math.PI * 0.35,  // Pitch limits
    maxPolarAngle: Math.PI * 0.48,
    rotateSpeed: 0.9,        // RS3-like feel
    zoomSpeed: 1.2,
    shoulderOffsetMax: 0.15, // Over-the-shoulder offset
  };
}
```

| Mode             | Controls                                            |
| ---------------- | --------------------------------------------------- |
| **Third Person** | Right-drag to rotate, scroll to zoom, click-to-move |
| **First Person** | Pointer lock, WASD movement                         |
| **Top-down/RTS** | Pan, zoom, click-to-move                            |

***

## VRM Avatar System

Characters use VRM format avatars with humanoid bone mapping:

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// components/CharacterPreview.tsx
import { VRM, VRMLoaderPlugin, VRMUtils } from "@pixiv/three-vrm";
import { retargetAnimationToVRM } from "../utils/vrmAnimationRetarget";

export const CharacterPreview: React.FC<{ vrmUrl: string }> = ({ vrmUrl }) => {
  const vrmRef = useRef<VRM | null>(null);
  const mixerRef = useRef<THREE.AnimationMixer | null>(null);
  
  useEffect(() => {
    const loader = new GLTFLoader();
    loader.register((parser) => new VRMLoaderPlugin(parser));
    
    loader.load(vrmUrl, async (gltf) => {
      const vrm = gltf.userData.vrm as VRM;
      VRMUtils.rotateVRM0(vrm);  // Fix rotation for VRM 0.x
      
      // Retarget animations to VRM bones
      const mixer = new THREE.AnimationMixer(vrm.scene);
      const idleClip = await loadAnimation("idle.glb");
      const retargeted = retargetAnimationToVRM(idleClip, vrm);
      mixer.clipAction(retargeted).play();
      
      vrmRef.current = vrm;
      mixerRef.current = mixer;
    });
  }, [vrmUrl]);
};
```

### VRM Bone Mapping

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
type VRMHumanBoneName =
  | "hips" | "spine" | "chest" | "upperChest" | "neck" | "head"
  | "leftShoulder" | "leftUpperArm" | "leftLowerArm" | "leftHand"
  | "rightShoulder" | "rightUpperArm" | "rightLowerArm" | "rightHand"
  | "leftUpperLeg" | "leftLowerLeg" | "leftFoot" | "leftToes"
  | "rightUpperLeg" | "rightLowerLeg" | "rightFoot" | "rightToes";
```

***

## Client Systems

Located in `packages/shared/src/systems/client/`:

| System                     | Description                                |
| -------------------------- | ------------------------------------------ |
| `ClientGraphics.ts`        | WebGPU rendering, shadows, post-processing |
| `ClientCameraSystem.ts`    | Camera controls and collision              |
| `ClientNetwork.ts`         | WebSocket connection, reconnection         |
| `ClientInput.ts`           | Keyboard, mouse, touch handling            |
| `ClientAudio.ts`           | 3D positional audio, music                 |
| `ClientLoader.ts`          | Asset loading with progress                |
| `HealthBars.ts`            | Floating health bars over entities         |
| `Nametags.ts`              | Entity name labels                         |
| `DamageSplatSystem.ts`     | Floating damage numbers                    |
| `XPDropSystem.ts`          | XP gain notifications                      |
| `EquipmentVisualSystem.ts` | Equipment rendering on avatars             |
| `TileInterpolator.ts`      | Smooth tile-based movement                 |

***

## Embedded Mode

For stream overlays and spectator views:

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// URL params for embedded mode
?embedded=true
&mode=spectator|free
&agentId=AGENT_ID
&followEntity=ENTITY_ID
&quality=low|medium|high
&hiddenUI=inventory,skills,chat
```

```typescript theme={"theme":{"light":"github-light","dark":"css-variables"}}
// components/EmbeddedGameClient.tsx
export function EmbeddedGameClient() {
  const config = window.__HYPERSCAPE_CONFIG__;
  
  return (
    <World
      mode={config.mode}
      followEntity={config.followEntity}
      hiddenUI={config.hiddenUI}
    />
  );
}
```

***

## Related Documentation

* [Engine Architecture](/wiki/engine/overview)
* [Networking](/wiki/engine/networking)
* [Movement System](/wiki/game-systems/movement)