Minecraft not saving port number when using direct connect

To join a server I used direct connect and put in the IP address and port, but this particular server was not using port 25565 it used 25589
so i had to type 00.00.000.000:25589 to connect, however, when I quit game, then tried to join, I joined a wrong server because the :25589 had been removed, leaving only 00.00.000.000
numbers are censored to ensure quality and privacy of the server owner
Note: I also tried using the "X" (close) button and quit game, same result, Did this 5 times with mods and without mods, same result.
below I will upload some images of the results.