Inventory won't close when bound to a mouse button

I found this bug when I was trying to do a mouse-only challenge where I rebind the most essential keys to the mouse and then only use said mouse when playing, and not my keyboard. I opened my inventory with middle mouse button to craft my crafting table, but it wouldn't close again.

I used esc to leave and go check my key binds, then went back to verify that it wouldn't close. I then tried resetting it to E, which worked perfectly, then left click, which also wouldn't close, then a random keyboard key, in this case P, which also worked perfectly.

Steps to reproduce:

1. Open key binds
2. Set open/close inventory to a mouse button
3. Enter world
4. Press newly bound mouse button
5. Pressing the button again will not leave the inventory