Pillager w/ banner not attacking me (aka player)

Hello, I was playing in my minecraft world and I noticed that pillagers with banners seem to be passive towards me (when near they don't attack. Even if I attack them. They only patrol) I am not sure if it also applies with npcs (wandering traders, villagers, etc.) I need to do testing. Thanks, -Slushiiez