This is how I would answer your questions:
1) You should have no issue with vram on a 1080Ti with 11GB. I can run 80 on a 1070 with 8GB. Something else is bottlenecking you, even if it's showing up as a graphics card bottleneck. The EQ client is OOOOOOOLD, from a time when video cards had 4mb...