The solution would involve having a custom render file for your project, accessing a Pd array from C++ through the Heavy API, then load the audio file from C++ into the array.
Examples of C++ code to read audio files into arrays are in
04-Audio/sample-loader. If your files are many and/or very large (i.e.: more than about 300MB of RAM in total), then you may have to only preload the beginning of the file into the arrays and read the rest of the file from disk upon request(see e.g.: