Pillagers don't attack the Allays

The Pillagers hold the Allays captive in their outposts and forest mansions. This denotes their hostile attitude towards the Allays. However, if they are released, the Pillagers do not attack them