Raids not moving or despawning

Whenever I go afk for a while, many raids spawn, and clump up if they fall into a body of water, but don't despawn and don't move, even if they see me. The crossbowers just shoot and don't try to get closer if I back up. If I unload and leave the chunks for a bit, they despawn