You can see behind yourself on a horse using a chat glitch.

If you are riding a horse, it seems with the kind of "F8" sensitivity, if you quickly look down or up and open the chat, the view will continue to go in that direction allowing you to see behind you with the world upside down.