Rejoining a world or changing Dimension with Nausea resets the distortion speed

Steps to Reproduce:
1) Join a world and run /effect give @s nausea 99999 0
2) Rejoin the world or Travel to the nether
3) Notice that the Nausea effect starts again

This also works with Distortion Effects set to 0% as the green overlay slowly reappears after rejoining