Redstone block powers sticky piston wierdly

If you power a sticky piston with a redstone block on the bottom of the piston then put a redstone block on the extended end of the piston, remove the first redstone block, the piston is stays extended