>>289614201
Hmm, that is tricky. The camera object's local position is the same offset as your headset from your physical playspace origin. So your controller script is moving your playspace origin and dragging your headset along with it (in virtual space).

I think you could add an extra transform to your hierarchy like

PlayerObject
-- CancelPlayspaceXZ
---- Camera

then in LateUpdate(), you set CancelPlayspaceXZ's localPosition to the opposite of the Camera's localPosition in XZ, and move the top-level PlayerObject's position the same delta as you just adjusted. That way Camera/headset's position in PlayerObject's local space will always be (0, Y, 0), but the whole playerObject still follows your headset in XZ in the world.

You can also check when moving the playerObject's XZ according to the headset delta that if your collider is going to go inside an object, instead only move as far as possible to not collide. That means that when you move your head physically into that object, instead it'll look like the world will move to avoid you clipping, like how pavlov does it iirc. And for the Y position, same sort of thing to avoid going into the floor based on Camera.localPosition.y, so you actually move up and down slopes.