>>408189117
Turns out that unity saves textures in your material, even if the shader doesn't use it. And it seems like that it even saves it in the material if you replace it with a newer/different texture.
I was able to delete those textures in [spoiler]poiyomi with the button in pic related[/spoiler] and was able to shave off another 20 MB or so.
I'm at 60 MB now and I'm a pretty happy camper!