>>365562887
3.5GB is quite low for VRAM, that could be part of the cause. I had the same issue on my GTX 970.
You should turn down the supersampling in SteamVR and that will probably help a great deal. You may need to install OpenVR Advanced Settings to forcibly override it and lower it to 1x (or even lower).
Also with that little VRAM, you should hide models that are over 30 MB by default in your performance settings. Normally I hate blocking/hiding stuff, but in larger instances with low VRAM you will basically have to do this.