Water caves and aquifers are not connected to dry caves.
Henrik Kniberg's video showed how water caves and aquifers connect to dry caves (https://www.youtube.com/watch?v=u0l_h8-LIaw&t=6s). As a result: to get into the cave you need to dig the blocks.