Spectators are affected by vines and ladders

Whilst in spectator mode, when holding shift (or CTRL in my case) to descend, it becomes glitchy when directly above, neither letting you on the ladder nor letting you descend normally. The screenshot isn't a good way to show it, but here is how to reproduce it. Dig down a few blocks and place a ladder in the hole. Switch to Spectator and try to go down the ladder.