Invisibility potion does not work on Guardians

When I went to defeat an Ocean Monument I took water breathing and potion of invisibility 8 minutes.  I had no armor on and even had nothing in my hands and the guardians could still see and target me. I know if you have armor they can see you but had nothing so I died.