Resource pack fallback to reduced level of detail, even with the required hardware

The Bug:

Loading a high resolution texture pack with raytracing on a PC with sufficient RAM and VRAM still gives the error:

Resource Pack Fallback

Low memory. Textures will have a reduced level of detail.

Steps to reproduce:

1) Load up either the latest stable or beta of the game. I've reproduced the same error on both.

2) Find a high resolution resource pack (in my case, I'm using Hardtop VanillaccurateRTX 512x) and install it by double clicking

3) Load the pack, turn on Raytracing and load up a world or realm.

4) You should see the error pop up in the top screen.

 

This shouldn't happen as I'd expect my PC to be able to run a 512x pack. (Important Specs: Ryzen 9 5900X, 32GB 3600 MHz C16, RTX 3090) and looking at task manager, I don't see the VRAM usage go past 4 - 5 GB and the RAM usage only hits 10GB before going back down.

 

I can also confirm that the error does affect textures as the glass in this pack is bugged when using 512x but using the 128x texture lets the glass go back to normal textures as you don't get the error with that resolution (I've attached the bugged glass below)

 

EDIT: Attached the menu, error message and VRAM/RAM usage as pictures