Hotbar incorrectly selecting items

For awhile now, i have a bug affecting the selection of items through both scroll wheel and the numbers. Whenever i use a tool for a period of time holding down right click and try to swap to a different tool or block or food or whatever. It rubber bands back to the first item i had and sort of "sticks" to the slot. 

I've tested this on a single player world and a "spigot" server from my friend. The only way i've found to influence this is through my frame rate. When i tanked it through limiting my frame rate to 30 fps plus sharing screens over discord it seems to happen way less often. And this isn't unique to 1.14 either, i tried going back to 1.13 and it still happens there. albeit less frequently