Switching between survival and spectator causes you to fall through blocks

I was figuring out the new gamemode switching mechanics, and started to hit F3 + n quickly. even time I would switch back to survival It would appear I was being set a fraction of a block to low, so I would barely clip into the next block underneath of me.