Controller Doesn't Work Until You Restart World

I'm using the Xbox One Controller w/ 3.5 mm audio jack via Micro USB and the game (but not the OS) does not respond to it if I connect in-game when the focus is on another window. I have to restart the world for it to start functioning.