Odd Inventory Glitch

Note: This glitch only happens on servers, whether it is modded or not, I've seen it happen on Vanilla servers too.

A lot of the time, this weird inventory glitch happens. Say I had a diamond pickaxe and a dirt block in my inventory. I press 'E' to go into my inventory and want to move my dirt block to my hotbar, and it would turn invisible. If I click it again, of course, it is fixed. But it is rather strange and I have been experiencing it for quite a time now. Probably since 1.8.