High render distance memory leak

When exiting a world with a high render distance it doesn't decrease the memory usage.

Steps to Reproduce:

1. Open Minecraft
2. Open Task Manager in the performance > memory tab on a separate monitor.
3. In Minecraft, lower your render distance to 16.
4. Create or enter a world.
5. Increase your render distance to 50+
6. Minecraft will use multiple gigs of memory
7. Exit the world

Observed Results:

• Exiting the world only lowers the memory usage by 2GBs
• Closing and restarting Minecraft rests on memory usage.

Expected Results:

• Exiting the world lowers the memory to when you first started the game.