Performance Drop After Entering Another Chunk

Whenever I enter another chunk, I get a huge performance drop. I went from around 250 fps down to 60 fps. It happens even if I enter the chunk repeatedly. It only happens when I cross the chunk border. Also, when flying in Creative, I can notice entering the new chunks and well, the TPS shows for itself.