Casting fishing rod while looking straight down freezes server

The bug

When you cast the fishing rod while looking straight down the server freezes. This can be seen for example by punching mobs, trying to retract the fishing rod, ... or in case of a multiplayer server the server will be shutdown by watchdog.