Setting worldspawn doesn't respect Y value / players always spawn at top-most block

I'm making a custom map in which the player, once dead, respawns in an area made for it. (The first picture shows the outside).

So I used the command ''/setworldspawn 64 104 177 0.0'' to make the player spawn in the spot I wanted to (second picture).

But after killing myself and testing it, the game respawns me in a platform above, which shouldn't be the case because I marked the Y value to be 103 and the block there it's a valid spawning point. (Third picture).

I have also tried reducing the Spawn Radius to 0 and to 1, but it behaved the same.