Side Mouse buttons don't work on menus and inventory

When in a world and not using an inventory or menu, the side mouse buttons work perfectly fine. Yet whenever I open a menu or inventory (wether to rebind a key or to close a chest) they don't work at all, completely not registering