OP privileges not working

I have been able to play just fine, then all of a sudden I was unable to change people's game mode. I have mostly just been playing with my girlfriend on my xbox, but my brother joined me and I didnt have that ability anymore.