After the player enters Nether, player's Allay gets rid of the leash tied to the fence and no longer follows the player

How to make a broken Allay

1. tied Allay who has a item from the player with a leash to a fence
2. enter the Nether Portal a few steps away from the Allay. Allay won't follow the player enter the Nether.(at this time, Allay is broken)
3. go back to the overworld
4. the Allay fly aimlessly, wait a seconds then the leash will be pulled off to the limit and break, or they have already done when you spend your time in the Nether

How broken Allay act

broken Allay don't fly back to the player when they're too far away from the player, they just fly aimlessly.
They still pick up items what they supposed to take, fly to your (or note block) and throw the items.

If they are leashed, the leash wouldn't affect their movement at all. they just fly around, pick up items and throw them to player (or note block).

When the leash is pulled off to the limit, the leash breaks.

Fix broken Allay

Take their item away and punch them.