Entities still track Spectator Gamemode Players

When you attack an entity in survival then switch to spectator mode the entities will still track the player. One would expect them to 'untrack' you just as it does when you switch from survival to creative.

Steps to reproduce:
1) Attack an entity in survival
2) Change game mode to Spectator