Can't invite friends, can't join friends and can't see their activity

My friend and I can't invite each other to our games, simply put there is no invite button. We have each other added as friends on the Xbox app and through the Minecraft app itself.
Also when I try to join my friends world through the main menu it shows up as "Your friends are not playing Minecraft right now" even though we are clearly playing and appearing online.