When using side mouse buttons for keybinds, can't quickly move items.

When using normal keybinds, if you hover over an item in a chest or in your inventory with your mouse you can press a button in your hotbar and the item will move there. If you're using the front mouse button or back mouse button, that doesn't work. It gets really frustrating after a while.