swing speed from pickaxe kept after breaking

What I expected to happen: My swing speed to be the normal fist speed when the pickaxe was broken and left an empty slot.

What happened: The swing speed from the pickaxe was kept even though the slot was empty, since the pickaxe was broken.

How to reproduce:
1. Break an pickaxe with unbreaking on.
2. punch something with the slot the pickaxe used to be in.

Note: Unbreaking can sometimes break the pickaxe before it actually breaks, which is another bug that could cause this one to occur.