Huge Desync

When I play with my friend the game sometimes has MAJOR desync issues.

For example, when i host sometimes I can see him moving but I'm frozen on his screen and if I punch him nothing happens on his end for a few mins. I also cant teleport him and he cant see blocks that I place.