Can't use ^ with ~

I can use ~ with an actual coordinate but I can't use ^ with either. I have to use ^ ^ ^ or none at all

Example: "tp @s ~1 0 ^5" won't work but something like "~1 0 ~5" or "^1 ^ ^5" will