Endermen attack constantly when you're in the nether

Whenever I go into the blue warped forest I look down but no matter what I do I still get attacked by endermen I kill one four of them come after me I kill those more show up until I die... Please fix if there's already an issue like mine sorry