Lag when looking at a group of players

When I join a server and I see a group or cluster of players, I immediately get lag no matter what I do. Even if I turn my graphics settings to the lowest and look at the players, I still lag. If I look away there is no lag.