Massive amount of lag/delay when joining a friends realm

It is taking me 10 to 15 minutes to join a friends realm on minecraft for Xbox one (He only lives down the street from me). Once I join there is a massive delay/lag on all the things that I do. I cannot open chests or furnaces, and I cannot pick up the blocks that I mine. I have no noise for opening doors or gates and mobs do not attack me when I go near them , instead I appear to be taking the damage from the mobs several minutes later. I have checked my internet speeds , and it is all working at the usual high speed. I am not experiencing this problem on the other games I play such as call of duty. I have had this problem for the last 2 days now.