I can't feed my horses

When I right click to feed my horse, it mounts me onto the horse and I can't feed it. If I hold shift to keep myself from mounting on the horse it opens the GUI. There doesn't seem to be a way to feed my horses.