Illager patrol spawn rate seems to be too high

The problem is that whenever I leave my house and go out to one of the open fields in my world, there are groups of Illagers everywhere, they are very difficult to defeat and it makes doing anything a pain.