Adding friend to server causes the server to kick both of us off and sign me out

Join your world and add a friend. After a few seconds your friend will be kicked, as well as your screen freezing and you getting kicked soon after. When you try and add them back quickly, you'll find the message on the side saying you aren't signed into your Xbox live account anymore.