Mining in survival carries over to adventure

Whenever you start mining a block in survival, switch over to adventure, and try to mine the block again, it appears like you're mining it (only client-sided). When you try to mine any other block in the world or use a different slot in your hotbar, it does not work and behaves like adventure mode should. Works on singleplayer and multiplayer.