Inventory close bug

When I press my inventory button (TAB for me), it opens the inventory perfectly fine, but when I press it again to close, it doesn't close, so I close it with escape.  This is in creative and survival.