Whenever I have armor on, I can't take it off or change it.

Whenever I put armor on, I can't take it off or swap it with other armor. This happens on creative and survival worlds as well as in games on servers. It will let me select the armor, but nothing happens after.