Allay has trouble recognising noteblock when it doesn't make a sound every few minutes

I have to hit a noteblock for an allay to bring items there in the first place (which may be the way it's supposed to be), but it will stop bringing them there after a very short period of time (under 3 minutes) and instead bring them to me again.