Phantom spawn rate and glitching in walls

To replicate I'd say just make a box with a roof and 3 walls and leave the front open with a height of 3 blocks just so you can walk in and out. First off I've been going to sleep but when it turns to night and I go out wandering it seems the phantoms just keeping spawning. First it'll be 2 or 3, then a minute later 5 or 6 then after that there's like 10 to 15 of them constantly barraging me, I can't hardly do anything at night because of it and I've gone to bed. Also they are seeking to get stuck in walls and I wasn't able to kill it until the sun destroyed it.