Controls

controls is a global singleton which provides an interface for VR, desktop and mobile inputs. These are the default controls and provide your standard third/first person controls for mobile/desktop and VR locomotion. This also stores controller state e.g. mouse interactions of VR hand/controller interactions and state.

Built-in player movement

The editor enables a complete local-player controller by default. Game scripts normally extend this controller instead of creating another player movement system.

On desktop, the built-in controller:

• uses WASD or the arrow keys for movement, Shift to run, Space to jump, and the mouse to rotate the camera;
• performs floor and wall collision detection;
• moves an internal camera target and copies that target to world.avatar.position every frame;
• places world.camera behind that target according to controls.zoom, controls.xShift, and controls.yShift;
• sets controls.moving, controls.falling, controls.floorHeight, and other movement state used by the avatar animation system.

Do not move world.avatar.position directly to implement normal local-player locomotion. The desktop controller will overwrite it on its next update. To teleport the local player and its camera target together, use:

world.avatar.teleport(new U.Vector3(0, 0, 5))

world.avatar.teleport() uses avatar root/feet coordinates, so y = 0 places the avatar on a floor at y = 0. The engine handles the private desktop controller eye-height offset internally.

To turn the equipped avatar and the desktop camera toward a world-space point together, use:

world.avatar.faceTowards(new U.Vector3(0, 0, 5))

The turn affects horizontal yaw and preserves the camera's current pitch.

world.avatar loads asynchronously and may not exist during a script's first frames. Guard access until it is available.

For a third-person shooter, retain the built-in movement and configure the camera rather than replacing the controller:

controls.zoomMin = 0
controls.zoomMax = 4
controls.setZoom(3, 0)
controls.lockZoom = true
controls.xShift = 0.15
controls.yShift = 0.1
controls.flyEnabled = false

While pointer lock is active, the player ray is cast through the center of the camera. controls.intersections contains hits against objects registered with world.addRaycastable, and controls.pressing represents the desktop primary mouse button. These are the normal starting points for camera-directed shooter aiming and firing.

Examples:

// set object's local postion to the left controller position (world space)
this.update = () => {
  this.position.copy(controls.left.position);
};

// Vibrate the controller when it clicks an object
this.click = (evt) => {
  // Pulse at max intensity, for 20ms
  // evt.hand is 'left' or 'right'
  controls[evt.hand].pulse(1.0, 20);
};

this.update = () => {
  // If using hand tracking
  if (controls.right.inputSource == 2) {
    this.position.copy(controls.right.hand["index-finger-tip"].position);
  }
};

// Set nested object's rotation/quaternion to controller quaternion
this.update = () => {
  // get inverse of parent quaternion
  const quaternion = new U.Quaternion();
  this.parent.getWorldQuaternion(quaternion);
  quaternion.invert();
  // copy it so that objects world rotation is 0, 0, 0
  this.quaternion.copy(quaternion);
  // then multiply / set from world controller rotation
  this.quaternion.multiply(controls.left.quaternion);
};

// Change walking speed on desktop, and locomotion speed in VR
controls.speed = 3;

// Set camera to first person
controls.setZoom(0, 0);

// Set camera to third person
controls.setZoom(2.5, 0);

controls.zoom : Number

Controls how far away the camera is from the player on mobile and desktop. Setting controls.zoom = 0 is first person and controls.zoom > 0.5 is third person, (3 is the default).

Prefer controls.setZoom(value, seconds) when changing zoom. It clamps the value to controls.zoomMin–controls.zoomMax and optionally transitions over the supplied duration. Mouse-wheel zoom uses the same limits. Set controls.lockZoom = true to prevent the mouse wheel from changing zoom during gameplay.

controls.desktop.cameraCollisionEnabled : Boolean

Defaults to true. The desktop/mobile camera sweeps a small sphere from its focus to the requested camera position, including shoulder and height offsets. It pulls inward immediately when blocked and eases outward when space clears. The sphere also encloses the camera's near-plane corners. controls.zoom keeps the requested distance, so collision can temporarily pull closer than zoomMin.

The check reuses player-collision bounds and triangle caches, respects object.collider = false, and is skipped in first person and XR. It also applies to fixed-angle cameras. Disable it with:

controls.desktop.cameraCollisionEnabled = false;

controls.zoomMin : Number

The minimum camera zoom accepted by controls.setZoom() and desktop mouse-wheel scrolling. The default is 0.

controls.zoomMax : Number

The maximum camera zoom accepted by controls.setZoom() and desktop mouse-wheel scrolling. The default is 4. Set the limits before calling setZoom():

controls.zoomMin = 1
controls.zoomMax = 8
controls.setZoom(5, 0)

controls.zoomMax should be greater than or equal to controls.zoomMin. If it is lower, the effective maximum is clamped to controls.zoomMin.

controls.yShift : Number

How far to move the camera above player eye height. This scales is multiplied by controls.zoom:

camera.position.y += controls.zoom * controls.yShift;

This is useful if you want to move the camera higher so the player's head doesn't overlap with the cursor. The default is 0.

controls.xShift : Number

Offset the avatar on the X axis, this can help make the cursor not overlap with the player's position

The value offsets the camera horizontally relative to the controller target. This is useful for an over-the-shoulder shooter camera.

controls.speed : Number

Multiplier for the built-in locomotion speed. The default is 1.

On desktop, the default grounded speeds are:

• walking: 2 metres per second;
• running with Left Shift: 5 metres per second.

Both speeds are multiplied by controls.speed. For example, controls.speed = 1.5 gives a walking speed of 3 metres per second and a running speed of 7.5 metres per second. Movement accelerates toward these speeds rather than reaching them immediately, and collisions can reduce the actual distance travelled.

controls.intersections : Array

An array of the interesected raycastable objects

controls.jumpEnabled : boolean

Flag to enable or disable jumping capability.

controls.flyEnabled : boolean

Flag to enable or disable flying capability.

controls.pressing : Boolean

Left click on desktop

controls.grabbing : Boolean

Right click on desktop

controls.floor : U.Object3D | null

The current object below the player, useful for sound effects, collisions.

controls.floorHeight : Number

The floors y position in world space.

controls.falling : boolean

Whether player controls are falling

controls.startPosition : Vector3

The initial camera position when the scene is loaded. This is used for resetting the player position on fall timeout. You can over-ride this. This can be useful for resetting to a point on a map when the player dies or falls for too long

controls.fallResetTime : Number : 2

The time before the player controls resets to the original startPostion when falling for x seconds. The default is 2 (seconds).

To disable this use: controls.fallResetTime = -1

Desktop

Explicit pointer lock from UI

For a button that deliberately enters mouse-look gameplay, request lock directly in its click handler, before awaiting asset loading:

controls.desktop.lock({ userGesture: true })

Call this from a native click, pointer, or keyboard event handler, before awaiting anything. The flag does not manufacture browser user activation. Viewport clicks and desktop 3D UI button releases are dispatched synchronously inside native pointer events so Safari receives the gesture too. On Chromium, a click during the Escape cooldown is remembered until that cooldown ends, provided transient user activation is still active. Blur and Stop cancel that queued click. Safari requests remain on the native event stack; after a browser denial there, click again with a fresh gesture. Ordinary lock() calls retain a short post-Escape guard. Explicit gesture requests otherwise go directly to the browser, which enforces its own recapture rules. cancelPendingLock(), input reset, Stop, and unlock cancel requests still in flight; a cancelled request that later succeeds is immediately unlocked. Both forms respect disabled controls and fixed-angle camera mode.

Camera modes

The default follow mode retains first/third-person mouse look and player movement. Use setCameraMode() to opt a desktop game into a camera with a fixed angle:

controls.desktop.setCameraMode('fixedAngle', {
  pitch: -Math.PI / 3, // radians, looking down
  yaw: 0,
  panSpeed: 24,       // world units per second
  edgeSize: 18,       // CSS pixels; 0 disables edge scrolling
  bounds: { minX: -100, maxX: 100, minZ: -100, maxZ: 100 },
})
controls.zoomMin = 12
controls.zoomMax = 30
controls.setZoom(30, 0)
controls.yShift = 0
world.camera.fov = 40
world.camera.updateProjectionMatrix()

Fixed angle mode disables pointer lock and mouse-look rotation. WASD still moves the local player through the built-in collision controller. Arrow keys, moving the pointer to the canvas edges, or dragging with the middle mouse button pan the camera independently of the player. The camera initially follows the player; panning detaches it. Press Home, or call controls.desktop.cameraRig.recenter(), to follow again. Space retains its existing jump behavior. Mouse-wheel zoom uses the configured zoom limits.

While moving, the avatar turns toward its actual horizontal travel direction, including strafing, reversing, and wall sliding. Its neck does not follow the overhead camera. An active world.avatar.faceTowards() request takes priority over movement facing and turns the avatar without rotating the camera. Camera focus is stored in controls.desktop.cameraRig.focus; panning never changes the player target. Bounds constrain the focus on the horizontal plane. pitch, yaw, panSpeed, edgeSize, and bounds can be changed through controls.desktop.cameraRig.configure(options). Panning pauses for UI, locked translation, or hidden tabs; blur clears held input. This mode is desktop only and does not take over mobile or XR controls.

For a temporary fixed-angle camera hold, call controls.desktop.cameraRig.setFocusOverride(position) with a world-space focus point. It copies the point and suspends following and panning until setFocusOverride(null) clears it. Call recenter() afterward to resume following the player. This changes camera focus only; avatar movement and respawn remain under their existing systems. The override also updates the camera while controls.desktop.enabled is false, allowing death cameras while locomotion is disabled.

Selection, unit orders, and button meanings remain the game's responsibility. Use controls.intersections for the object under the free cursor. For custom aiming, controls.desktop.getPointerRay(origin, direction) fills two supplied vectors using the same cursor coordinates as viewport interaction and returns whether a valid ray was available. It does not perform a raycast.

Call controls.desktop.setCameraMode('follow') to restore the previous camera orientation, pointer-lock setting, and avatar-facing setting. Stop Play also resets the mode. Games should restore any zoom/FOV settings they change in onStop. Independently of camera mode, controls.desktop.rotationEnabled = false disables mouse look without disabling movement or changing pointer lock.

The camera input regression checks can be run from u/ with:

node --test tests/desktop-controls.test.mjs

// Disable or enable using pointer lock. true by default
controls.desktop.pointerLockEnabled = true | false

controls.desktop.enabled = true | false

// Show or hide the shader cursor while pointer lock is active.
// This does not affect the operating-system pointer or XR hand/controller pointers.
controls.desktop.cursorEnabled = true | false

// Keep the local avatar facing the camera's horizontal direction.
// Camera pitch is ignored so the avatar remains upright. false by default.
controls.desktop.avatarFollowsCamera = true | false

// Actual world-space player displacement this update, after collision (metres).
// Reused by the controller; treat it as read-only. Cleared when movement is locked.
controls.desktop.worldMovement // U.Vector3

// Add an immediate world-space change in player velocity (metres/second).
controls.desktop.addImpulse(new U.Vector3(0, 8, 0))

// Add world-space acceleration for the next movement update
// (metres/second squared). Call each frame for sustained thrust.
controls.desktop.addAcceleration(new U.Vector3(0, 20, 0))

// Copy the current externally applied velocity into an existing vector.
controls.desktop.getExternalVelocity(result)

// Remove queued acceleration and externally applied velocity.
controls.desktop.clearExternalMotion()

controls.cursor // ShaderCursor; alias of controls.desktop.cursor (no longer a U.Mesh)

// You can create your own desktop specific controls, e.g.

controls.desktop = {
	update: (delta)=>{ /* your logic '*/ }
}

Shader cursor

The desktop center cursor is a small procedural quad drawn after the completed scene, UI overlays, and selection outlines. It uses one draw call and a reused 128-byte uniform buffer; it has no scene mesh, raycast registration, or world position. Size is in CSS pixels and remains constant across camera distance, FOV, display pixel ratio, and render resolution. It appears only while the viewport owns pointer lock and controls.desktop.cursorEnabled is true. Mobile touch and XR pointers are unchanged.

Use the built-in styles without writing a shader:

this.init = () => {
  controls.cursor.configure({
    shape: 'ring', // 'dot', 'ring', or 'cross'
    size: 24, // CSS pixels; includes transparent antialiasing margin
    color: [1, 1, 1, 1], // straight sRGB RGBA, each component 0–1
    outline: [0, 0, 0, 1],
    outlineWidth: 2, // CSS pixels
    hoverScale: 1.2, // multiplier while pointing at an interaction target
  });
};
this.onStop = () => controls.cursor.reset();

For custom shapes, provide WGSL defining fn cursor(p: vec2f) -> vec4f. The engine supplies the vertex stage, bindings, blend state, and final pass. p runs from -1 to +1 across the square, with +Y pointing up. Return straight sRGB RGBA; transparent pixels preserve the image behind the cursor. Keep the shape inside the square, leaving room for antialiasing. Use fwidth/smoothstep for smooth edges.

this.init = async () => {
  const cursor = controls.cursor;
  cursor.configure({ size: 28, color: [0.2, 0.9, 1, 1] });
  try {
    await cursor.setShader(/* wgsl */ `
fn cursor(p: vec2f) -> vec4f {
  let radius = 0.55 + settings.parameters[0].x * 0.15;
  let distance = abs(length(p) - radius) - 0.07;
  let aa = max(fwidth(distance), 0.001);
  let alpha = 1.0 - smoothstep(-aa, aa, distance);
  return vec4f(settings.color.rgb, settings.color.a * alpha);
}
`);
  } catch (error) {
    console.error('Cursor shader failed:', error);
  }
};
this.update = () => {
  // Update values without compiling a pipeline or allocating new buffers.
  controls.cursor.parameters[0] = 0.5;
};
this.onStop = () => controls.cursor.reset();

Available uniforms:

• settings.viewport: framebuffer width/height, current CSS size (including hover scale), time in seconds.
• settings.color / settings.outline: configured RGBA colors.
• settings.style: outline width, built-in shape index, framebuffer-to-CSS pixel ratio, reserved.
• settings.parameters: four vec4f values backed by cursor.parameters, a reusable Float32Array(16).

configure() changes uniforms; it does not compile shaders. setShader() compiles asynchronously and resolves to true when installed, or false if superseded by another shader, a reset, disposal, or renderer replacement. Compilation errors reject the promise and leave the previous cursor intact. Call it during setup or a style change, never every frame. reset() restores the default dot and clears custom parameters. Scene clearing recreates the cursor, and GPU device loss releases its buffer. Restore game customizations in init() and reset them in onStop.

Optional target-specific styles

Most games can keep the default cursor. Games that need different appearances can opt in with controls.cursor.resolveStyle. The engine passes the closest current desktop raycast hit, or null, once per viewport update. The callback runs synchronously; return a reusable handle from await cursor.createStyle(). Returning null or undefined keeps the configured base cursor and its normal hover feedback. cursor.target exposes the latest hit; treat it as read-only. Return a style created with { visible: false } to hide the cursor temporarily (for example, while dragging an object). Hidden styles skip the cursor draw pass.

let cursor, previousResolver, resolver;
this.init = async () => {
  cursor = controls.cursor;
  const normal = await cursor.createStyle({ hoverScale: 1 });
  const interactive = await cursor.createStyle({ shape: 'ring', size: 24 });
  const heart = await cursor.createStyle({
    size: 28,
    color: [1, 0.25, 0.6, 1],
    hoverScale: 1,
    code: /* wgsl */ `
fn cursor(p: vec2f) -> vec4f {
  let q = p * 1.5;
  let base = dot(q, q) - 1.0;
  let distance = base * base * base - q.x * q.x * q.y * q.y * q.y;
  let aa = max(fwidth(distance), 0.001);
  let alpha = 1.0 - smoothstep(-aa, aa, distance);
  return vec4f(settings.color.rgb, settings.color.a * alpha);
}
`,
  });
  previousResolver = cursor.resolveStyle;
  resolver = hit => {
    if (hit?.object.userData?.pet) return heart;
    if (!hit || hit.object.userData?.placementFloor) return normal;
    return interactive;
  };
  cursor.resolveStyle = resolver;
};
this.onStop = () => {
  if (cursor?.resolveStyle === resolver) cursor.resolveStyle = previousResolver;
};

The metadata in this example is game-defined; the engine does not assign pet or floor categories. Your resolver can use object identity, ancestors, game state, or your own registries instead. Avoid allocations, scene traversal, asset loading, and shader compilation inside the resolver.

Styles snapshot size, shape, colors, outline width, and hover scale at creation. They have their own mutable parameters array for animation. Supplying code precompiles WGSL during setup. Identical code shares a cached pipeline, even when colors or sizes differ. Switching styles selects the cached pipeline and updates the shared uniform buffer: one quad, one draw, one cursor pass. Unused styles never draw. Built-in styles compile lazily on first use; custom styles recompile asynchronously after device loss, using the base cursor until ready. Resolver errors fall back to the base cursor and are reported once per failing callback. reset() removes the resolver and clears cached styles and parameters. Existing handles can be reused but their pipelines must recompile.

Legacy scripts that manipulated controls.desktop.cursor.position, geometry, material, or scale must migrate to configure() and WGSL. cursorEnabled, setVisible(), and createCursor() remain available; controller updates own automatic visibility. This API styles the pointer indicator only and does not change aiming, raycasts, or input handling.

External motion is combined with ordinary player movement and passed through the built-in swept capsule collision. addAcceleration() accumulates calls until the next controller update and is then cleared automatically. Impulses persist as velocity; gravity affects vertical external motion, ground friction slows horizontal external motion, and blocked velocity is removed at collisions.

Mobile

controls.mobile.enable();

controls.mobile.disable();

controls.mobile = {
  update: (delta) => {
    /* your logic '*/
  },
};

XR

controls.locomotion : Locomotion

Handles locomotion within the VR environment.

controls.locomotion.enabled : boolean

Flag to enable or disable locomotion.

controls['left' or 'right'] : Hand

Left/right controller object, containing state and interaction data for that hand.

Hand : Object

Represents the right control interface, including position, rotation, and interaction states.

Hand joints:

'wrist'
'thumb-metacarpal'
'thumb-phalanx-proximal'
'thumb-phalanx-distal'
'index-finger-phalanx-proximal'
'index-finger-phalanx-intermediate'
'index-finger-phalanx-distal'
'middle-finger-phalanx-proximal'
'middle-finger-phalanx-intermediate'
'middle-finger-phalanx-distal'
'ring-finger-phalanx-proximal'
'ring-finger-phalanx-intermediate'
'ring-finger-phalanx-distal'
'pinky-finger-phalanx-proximal'
'pinky-finger-phalanx-intermediate'
'pinky-finger-phalanx-distal'

// Three.Group storing bone position, rotation
controls.left.hand['pinky-finger-phalanx-distal'] -> U.Group