After I raided pillager outpost/watchtower, then moved my house into pillager outpost/watchtower, then few hours later, pillager spawns again, and again

After I raided pillager outpost, and set as my new home, I didn't saw pillagers coming into it, after a few hours of playing, I saw pillagers spawn into it, I built a fence to protect my house, then pillagers spawn again, fix it, it was my house, but pillagers didn't attack my wolves, and my horse, please fix it, so I can stay it for long, I want pillagers to be gone

I put a torch in pillager outpost inside and outside, and also the fence, it spawns also

I always wanted to stay at Pillager Outpost, no more pillagers spawning into Pillager watchtower