Inventory lost when opening 1.20.4 world in 1.20.5 and returning to 1.20.4

What I expected to happen was...:
I expected to find my inventory as it was when I opened a world in 1.20.4 which has been previously opened in 1.20.5

What actually happened was...:
I opened the world in 1.20.5 and the inventory was there, but when I opened it again in 1.20.4 I got an empty inventory as well as empty chests and furnaces.

Steps to Reproduce:
1. Create a world in 1.20.4
2. Get items in your inventory and leave the world
3. Rejoin using 1.20.5 and exit, then rejoin with 1.20.4

Tip: affected version is 1.20.6 since I couldn't get 1.20.4 and 1.20.5 in affected versions, someone help me