Inventory Crash on Server

It's hard to explain, but if you follow the steps you will see what I mean.

1. Check your ping to a server, that's how much time you have
2. Go on that server
3. You have to press `e` twice within your ping, see step 1
4. Minecraft crashes

There is also a delay when opening your inventory on servers after updating to 1.16