Pufferfish find Spectators and players that are invisible.

When you go in to Spectator mode, the Pufferfish are still able to puff up because they detect a player nearby even when nobody else is near. This means that they detect players in any gamemode, which I feel should be fixed. They can also detect players while they are invisible.