Item alignment against anvils is directional

If you try to align a Item against a Anvil it only works from two of the four sides.

From my testing the item is fast after the alignment from only two of the four directions.

This should be consistent so it should always be fast, or is there any reason why it shouldn't?