Binding "Open/Close Inventory" to a special Mouse click (e.g. 4 or 5) won't Close inventory, only Open

I know a similar bug was resolved around the Tab key (MC-145691), but this is happening to me when I try to bind Mouse Button 4 or Mouse Button 5 to the action of "Open/Close Inventory" (that usually defaults to E). Clicking such mouse buttons will Open the Inventory, but doing so again will not Close it. It also does not Close other UI elements such as Villager trading windows. I've tested it against different keybinds and they do work as intended, so my guess is this is related to mouse-clicking events. I also had the same issue when testing the action to be keybound to Right-click of the mouse.