Dolphin's grace applied inconsistently

Dolphins follow you when you swim (like they are supposed to), but don't always give you dolphin's grace. They swim with you, then after 9-10 seconds of swimming you get the effect. Then after 5-6 seconds you lose the effect again, then 9-10 seconds later... you get the rest.