Arrow of harming II sticks in offhand, can't take out

I'm playing in Creative and turned into Survival in the nether. Then I put a harming II arrow in my offhand slot to try and shoot a ghast but when I try to take it out, it goes back there. Then I tried to turn back into Creative, still doesn't get out. I have to rejoin the world and then I can take it out of my offhand slot.