Minecraft keeps lagging and freezing

when I join a realm it starts off pretty fine but then it starts to freeze and lag. In order for the game to unfreeze or stop lagging you have to press the Xbox button on the controller and it stops for maybe 6 seconds and then it does it again.