XBox login issue when playing on servers

Quite often when playing a server I get the disconnected message and when I try to rejoin it tells me to log into my xbox account, which it already is, and won't go away until I restart my game.