Splat Patching

Splat Patching is the layer of PAXAL that turns a captured scene into a game-world primitive. A renderer can paint a splat. A patched splat can also stop a wheel, register a dent, host a hinged door, and accept a composited prop. The capture stays the capture — the photoreal pixels are not touched. Patching is what the engine attaches around them so gameplay code has something to hold on to.

A 3D Gaussian splat is point-based, view-dependent appearance. Millions of fuzzy ellipsoids tinted by spherical harmonics. The renderer projects and blends them into pixels — beautifully — but there is no surface inside the file. No triangle a ray can hit. No vertex a constraint can bind to. Without patching, every drivable splat scene needs a parallel hand-modelled mesh, and the entire reason for capturing the world has just been undone.

Patching is the discipline of deriving the missing world from the capture itself — and keeping the two in sync as both evolve.

Patching is not a single technique; it is a family. PAXAL treats it as four distinct primitives that share a pipeline but solve different problems. The order matters — collision earns the rest.

The page is honest about what is shipped. Composition, deformation, and occlusion are real research lines with parked tickets — not announcements.

The shipped pipeline turns a Gaussian-splat capture into a watertight triangle mesh the physics layer can collide against. Three stages:

capture/room.compressed.ply
   │
   │  splat-transform … --filter-cluster --voxel-carve -K
   ▼
public/arena/room.collision.glb       (watertight triangle mesh)
   │
   │  loaded as trimesh collider at runtime
   ▼
physics:  walls-only trimesh  +  flat plane @ floorY = -3.8
spawn:    [ -2, -1, -1.4 ]

For the room arena specifically: the mesh is partitioned into a walls-only trimesh and a flat ground plane pinned at floorY = -3.8, with the vehicle spawning at [-2, -1, -1.4]. The vehicle drives smoothly; the walls stop you. The hybrid — concave trimesh for walls, infinite plane for floor — is portable to any captured interior with a single dominant ground level.

Once a splat scene has a collider, the rest of a game engine has somewhere to land. Vehicle physics runs against real-world capture, not against a hand-modelled stand-in. Doors get to hinge. Walls get to take damage. Props can be composited in at runtime and dropped onto a real floor. NPCs with their own colliders can stand inside the same space the camera was. The first proof point is the driving demo on the index route — a real scanned room, a vehicle that obeys it.

Patching is harder than it sounds because a Gaussian-splat scene has no vertex-and-triangle topology to attach constraints to. A hinge wants two rigid bodies and a pivot; a splat is a fuzzy ellipsoid with a colour and an opacity. The collision-mesh pipeline sidesteps the problem by extracting a parallel triangle mesh from the splat geometry and binding all physics to that. The splat renders; the mesh collides; the two are coupled by sharing a coordinate frame.

The remaining primitives can't cheat that cleanly. Deformation wants the splat parameters themselves to change when a wall is hit — opacity, scale, colour, per-splat — without ruining the appearance. Occlusion wants the engine to query a depth buffer the splat renderer wrote. Both are open research lines, not wishlist items, but neither is shipped today.

• 01Collision-mesh extraction. A splat-transform pass with cluster filtering and voxel carving emits a watertight .collision.glb alongside each compressed splat — the same step that ships the LOD ladder.
• 02Runtime trimesh collider. The GLB is loaded as a concave trimesh and registered with the physics world; the renderer never sees it. The room arena ships with a walls-only trimesh paired with an infinite ground plane.
• 03Vehicle physics on real captured geometry. Driving, braking, and wall contact all resolve against the extracted mesh, not a hand-modelled stand-in. The driving demo on / is the live proof point.

The next primitive is composition — parked-vehicle and NPC splats with their own colliders, dropped into a host scene at runtime. The engineering exists; the bottleneck is content. After that, deformation: per-splat dent maps that let geometry register impact without breaking the appearance. Then occlusion: an engine query against the depth buffer the splat renderer already writes, so the AI and the audio system can reason about line-of-sight. No dates; the order is the commitment.