Birch trees planted next to blocks with fences on them don't mature

Birch trees planted next to blocks with fences on them stay as saplings forever and won't mature.

Seems like if it isn't going to mature in that spot it should prevent you planting it in the first place.