I can't play with Xbox One users... (Only on realms or servers)

I've tried to do everything I can do... but for some reason I can't play with two Xbox users (friends on xbox).

I don't really know what to say... It just that the world will never load and the message 'unable to connect to world' will appear. Maybe it could be my connection... because I tried in other networks and I can play with them... But I don't know... why? My connection is fine... So why I can't play with them?

If it helps... here's the name of the users

-Dark branding

-GruelingSand552

If I get to find another person from Xbox and stills happens... then I guess is all xbox users.