>>268894079
It's all shape keys actually. The more verts on your mesh, even if the blend shape only affects 1 vert, the bigger performance hit it causes. For the most performant avatar, you'll want your face on a separate mesh so that your total vert count on the shapekey-affected mesh is very small.