Problems with switching between hotbar items during elytra flight

Multiple times now, I have swapped between rockets and other items mid-elytra flight, and the selected item has quicky gone back to what i had selected before. Sometimes it takes 3 to 4 attempts to switch items. This happened on a Realm.