@web-engine-dev/ai-ml

Machine learning inference runtime for running trained neural network models in-browser. Provides a unified interface over ONNX Runtime Web (WASM + WebGPU backends) for gameplay AI applications like learned locomotion, style transfer, and behavior cloning.

Layer 7 · Gameplay

Features

• ONNX Runtime Web: Run ONNX models via WASM (CPU) or WebGPU (GPU) backends
• Unified Inference API: Load, warm up, and run models with typed tensor I/O
• Model Quantization: INT8 and FP16 quantized model support for smaller downloads
• Batch Inference: Process multiple inputs per call for agent populations
• Async Execution: Non-blocking inference keeps the game loop responsive
• Tensor Pooling: Reuse input/output tensor allocations to avoid GC pressure
• Model Caching: Cache compiled models in IndexedDB for instant reload
• Warm-Up: Pre-run inference on load to avoid first-frame latency spikes

Installation

npm install @web-engine-dev/ai-ml
# or
pnpm add @web-engine-dev/ai-ml

Quick Start

import { ModelRunner, InferenceBackend } from '@web-engine-dev/ai-ml';

// Load an ONNX model
const runner = new ModelRunner({
  backend: InferenceBackend.WEBGPU, // or WASM for CPU fallback
});

await runner.load('/models/locomotion.onnx');
await runner.warmUp(); // Pre-run to compile shaders

// Run inference
const input = runner.createTensor('float32', [1, 12], observationData);
const output = await runner.run({ observation: input });

// Use output for gameplay
const actions = output['actions'].data; // Float32Array

Use Cases

Application	Model Type	Input	Output
Learned Locomotion	RL policy	Joint angles, velocity	Torques, target angles
Behavior Cloning	Imitation	Game state observation	Action probabilities
Style Transfer	CNN	Texture image	Stylized texture
Content Generation	GAN/VAE	Latent vector	Generated content

Backend Selection

Backend	Speed	Compatibility	Best For
WebGPU	Fastest	Chrome 113+, Edge 113+	Large models, batch inference
WASM	Good	All modern browsers	Fallback, small models

Dependencies

• @web-engine-dev/assets — Model asset loading
• onnxruntime-web — ONNX Runtime Web inference engine