Cannot push a button when sitting in a minecart.

I was unable to to push a button while sitting in a minecart. This button is just outside the cart - see screenshot. This behaviour is erratic - sometimes, after jumping out and in the cart, I am able push it again.