Accessing Controller Inputs with Three.js and ECSY

Based on your code, I can see you're trying to implement a controller system using ECSY with Three.js. To access controller inputs like buttons and joysticks, you need to work with the XR controller objects and their associated events.

Accessing Controller Inputs

In Three.js, XR controllers provide access to input values through the gamepad property. Here's how you can modify your code to access these inputs:

1. Update your ControllerComponent

First, expand your ControllerComponent to store controller-specific data:

JAVASCRIPTCopied
import { Component, Types } from 'three/addons/libs/ecsy.module.js';

export class ControllerComponent extends Component { }

ControllerComponent.schema = {
    isLeft: { type: Types.Boolean, default: false }, 
    isRight: { type: Types.Boolean, default: false },
    controller: { type: Types.Ref },  // Store the XRController reference
    gamepad: { type: Types.Ref },     // Store the gamepad reference
    buttons: { type: Types.Array },   // Store button states
    axes: { type: Types.Array }       // Store joystick/axes values
};

2. Update your ControllerSystem

Modify your ControllerSystem to properly initialize and update controller data:

JAVASCRIPTCopied
import { System } from "three/examples/jsm/libs/ecsy.module.js";
import { ControllerComponent } from "../components/ControllerComponent.js";
import { Object3D } from "../components/Object3DComponent.js";
import * as THREE from "three";

export class ControllerSystem extends System {
    init(attributes) {
        this.scene = attributes.scene;
        this.controllers = attributes.controllers;
        this.raycaster = new THREE.Raycaster();
        this.tempMatrix = new THREE.Matrix4();
        
        // Set up controller entities if they don't exist yet
        if (this.controllers && this.controllers.length > 0) {
            this.setupControllers();
        }
    }
    
    setupControllers() {
        this.controllers.forEach((controller, index) => {
            // Create entity for each controller if it doesn't exist
            const entity = this.world.createEntity();
            const isLeft = index === 0; // Assuming first controller is left
            
            entity.addComponent(Object3D, { object: controller });
            entity.addComponent(ControllerComponent, {
                isLeft: isLeft,
                isRight: !isLeft,
                controller: controller
            });
            
            // Add event listeners for controller
            controller.addEventListener('connected', (event) => {
                const controllerEntity = this.getEntityByController(controller);
                if (controllerEntity) {
                    const data = event.data;
                    if (data.handedness === 'left') {
                        controllerEntity.getMutableComponent(ControllerComponent).isLeft = true;
                        controllerEntity.getMutableComponent(ControllerComponent).isRight = false;
                    } else if (data.handedness === 'right') {
                        controllerEntity.getMutableComponent(ControllerComponent).isLeft = false;
                        controllerEntity.getMutableComponent(ControllerComponent).isRight = true;
                    }
                }
            });
        });
    }
    
    getEntityByController(controller) {
        // Helper to find entity by controller reference
        let result = null;
        this.queries.controllers.results.forEach(entity => {
            const comp = entity.getComponent(ControllerComponent);
            if (comp.controller === controller) {
                result = entity;
            }
        });
        return result;
    }

    execute() {
        // Update controller input states
        this.queries.controllers.results.forEach(entity => {
            const controllerComp = entity.getMutableComponent(ControllerComponent);
            const object = entity.getComponent(Object3D).object;
            
            // Get gamepad data if available
            if (object.gamepad) {
                controllerComp.gamepad = object.gamepad;
                controllerComp.buttons = object.gamepad.buttons.map(b => b.pressed);
                controllerComp.axes = [...object.gamepad.axes];
                
                // Example of checking specific buttons
                const triggerPressed = controllerComp.buttons[0];
                const thumbstickPressed = controllerComp.buttons[3];
                
                // Example of accessing thumbstick/joystick values
                const thumbstickX = controllerComp.axes[2];
                const thumbstickY = controllerComp.axes[3];
                
                // You can now use these values for your interactions
                if (triggerPressed) {
                    // Handle trigger press
                    this.handleTriggerPress(entity);
                }
                
                if (Math.abs(thumbstickX) > 0.1 || Math.abs(thumbstickY) > 0.1) {
                    // Handle thumbstick movement
                    this.handleThumbstickMovement(entity, thumbstickX, thumbstickY);
                }
            }
            
            // Your raycasting code can go here
            // ...
        });
    }
    
    handleTriggerPress(entity) {
        // Implement trigger press behavior
        // For example, selecting objects
        const object = entity.getComponent(Object3D).object;
        
        // Update raycaster
        object.updateMatrixWorld();
        this.tempMatrix.identity().extractRotation(object.matrixWorld);
        this.raycaster.ray.origin.setFromMatrixPosition(object.matrixWorld);
        this.raycaster.ray.direction.set(0, 0, -1).applyMatrix4(this.tempMatrix);
        
        // Check for intersections
        const intersections = this.raycaster.intersectObjects(this.scene.children, true);
        if (intersections.length > 0) {
            // Handle intersection
            console.log("Selected object:", intersections[0].object);
        }
    }
    
    handleThumbstickMovement(entity, x, y) {
        // Implement thumbstick behavior
        // For example, movement or rotation
        console.log(`Thumbstick movement: x=${x}, y=${y}`);
    }
}

ControllerSystem.queries = {
    controllers: {
        components: [ControllerComponent, Object3D]
    }
};

3. Update your registration code

Make sure your controllers are properly passed to the system:

JAVASCRIPTCopied
// When setting up your XR session:
const controller1 = renderer.xr.getController(0);
const controller2 = renderer.xr.getController(1);
scene.add(controller1);
scene.add(controller2);

// Then when registering:
new useRegister({
    world: world,
    options: {
        requiredFeatures: ['draggable-default'],
        controllers: [controller1, controller2],
        template: { scene: scene }
        // other options...
    }
});

Button Mapping

For WebXR controllers, the button mapping typically follows this pattern:

• buttons[0]: Trigger (main selection button)
• buttons[1]: Squeeze/Grip button
• buttons[2]: Touchpad button (if available)
• buttons[3]: Thumbstick button (pressing the thumbstick)
• buttons[4] and above: Additional buttons (A, B, X, Y, etc.)

For axes:

• axes[0] and axes[1]: Touchpad X and Y (if available)
• axes[2] and axes[3]: Thumbstick X and Y

The exact mapping can vary between different VR devices, so you might need to adjust based on your target hardware.

Event-Based Approach

As an alternative to polling in the execute method, you can also use an event-based approach:

JAVASCRIPTCopied
// In your setupControllers method:
controller.addEventListener('selectstart', this.onSelectStart.bind(this));
controller.addEventListener('selectend', this.onSelectEnd.bind(this));
controller.addEventListener('squeezestart', this.onSqueezeStart.bind(this));
controller.addEventListener('squeezeend', this.onSqueezeEnd.bind(this));

This approach can be cleaner for button presses, while polling is better for continuous inputs like thumbstick position.

Remember that controller inputs are only available when running in an actual WebXR session, so make sure to test your code in a proper VR environment.