https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0152623
"However, previous measurements have shown that hand tremor causes a 0.2 to 0.5 mm trembling of surgical instruments (working length between 50 and 100 mm)"
So this means if your HMD tracking is at about 200-500 micrometers, aka .2 to .5 MM your accuracy is fine

AS EXPECTED SASUGA!
This means everything besides thecosmos cn easily be within par for a SURGEONS hands, now the only question is UPDATE RATE, don't light houses pulse 30 times a second effectively meaning you're tracking like a 30FPS mouse in a game?