texturepacks with res higher than 256x render the game unplayable

When using a texturepack with a resolution higher than 256x (tried with 512x), the game start to render only one to two frames per seconds.

I tried differents texturepack resolutions (on the same world with the same camera position) and here are the results:
default texturepack: ~230fps
64x: ~200fps
128x: ~190fps
256x: ~150fps
512x: ~1fps

This problem seems to affect all current 1.14.x versions (1.14 to 1.14.4-pre5), I saw no significant fps drop when using the same texture pack on pre-1.14 versions.

When using a 512x texturepack, the VRAM of my graphic card is only 2/3 used, same for my RAM.

I'm using the "vanillaBDCraft" texturepack from bdcraft.net.