Iron golem aggro after changing from easy to peaceful

Attack an iron golem while on easy then change the difficulty to peaceful and the iron golems still attack. Save and quit, then load the world again and they are passive again on peaceful and easy. I haven't tested the other difficulty settings.