Skins swapping

While my friend and I play, our skins always end up swapping on us. Ill become his skin, yet still see him as his, and he'll become my skin, and see mine as mine. Its annoying, and it sucks to not be your own skin as you play. This has been happening since 1.13 skin system update