Oculus inside-out tracking uses gyroscopes to track your hands, which is why it can track them for a while despite them going outside of the view of the cameras. 
The problem is when they're outside that view for an extended period or held stationary outside that view. 
It's not like your hands just fucking vanish when they move beyond your neck.