RNBO audio buffers on Bela Gem Multi

Antrees

Hello there,

I’m a new owner of a Bela Gem Multi, my aim is to use it for a sound installation requiring multichannel audio played on headphones. For this, I wanted to use RNBO and then export the C++ code to run it on Bela. My actual project is to have a realtime selection of a lot of audio files (more than 500). There are predominantly under 30 seconds and goes to around 5 minutes for the longer ones. Right now, it does not work when I build the exported C++ code with the IDE. In the console, I can see that it’s loading files but it stops when it exceeds 59 files (mp3 320 CBR, around 32Mo in total).

So I tried with less files to see if it would work at the some point. It does work when I load around 40-50 mp3 files (around 26.5Mo total) or 40-50 WAVE files (48-24, around 190Mo), when I go over that it doesn’t. Although, I’m quite versed running Max and RNBO patches on my computer, it’s fairly new to me to run programs on a embedded platform like Bela. I suppose I need to be particularly aware of the technical limitations (computer power and memory usage), but I thought I was not exceeding Bela limits for this particular project.

In RNBO, I use a very simple patch (screenshot below), to select the files read by a groove~ which buffer is changed by a multibuffer~. I declare all my audio files with buffer~ objects, they seem to be recognized in the console of the IDE. It seems it’s the working method right now in RNBO to use buffers, even though it’s really painful to name individual files like this.

So my question is, what could be the reason for not supporting this amount of audio files ? And what could I do to make it work ? It seems the RNBO part is working, but it might not be the best way to do that when using Bela.

Thanks a lot for your help !

giuliomoro

There's 512 MB of RAM on the board. Files are probably stored in RAM uncompressed, most likely 32-bit floats, so 190MB of 24-bit WAV files will be 253 MB in RAM. Add in everything else going on and you are getting pretty close to filling up the RAM. While you may be able to do surgical work to free up a few more MB, the fundamental issue is that when you have a lot of samples, they should not be all loaded in RAM at the same time, rather you should be streaming them from disk when necessary. If you need the files to start playing back with no further wait when you trigger them, you'll normally load the first few samples in RAM, so they are immediately accessible on play, and while you play them you load the rest of the file from disk.

Does RNBO support that natively in any way? Alternatively, we have a AudioFilePlayer class that does just that, so in principle you can do it with a bit of work on the C++ side, some messaging from RNBO to the C+++ wrapper and some virtual audio channels into RNBO.

Antrees

Thank you for your answer Giulio ! I had the impression I was not doing this properly. You confirm what I read on the C++ examples on the Gem, and other ressources in that language. I'm clearly not proficient in C++ unfortunately, even if I can more or less understand little bits sometimes. I've succeeded in running the example sample-streamer — and I can see sample-reader too — on my Bela but that's it. Making the necessary "plumbing" between Bela and RNBO is way out of my league and the time I have to work on this. I should have been more cautious before choosing to work with the Gem board and RNBO.

For RNBO, I need to ask on C74 forum if it's possible or not. My impression reading the docs, is that it's not the case.

giuliomoro

For RNBO, I need to ask on C74 forum if it's possible or not. My impression reading the docs, is that it's not the case.

See what you get back from that, then we can look into the best way of doing it.

Antrees

Hello Giulio, here is the answer from Alex Norman working at C74 (post here) :

Unfortunately RNBO doesn't have a good way to do what you're trying to do here internally. All specified buffers are loaded into RAM. Maybe some day we could have a way to stream from disk, but we don't have that now (it would be a major architectural change).
It looks like the Bela integration isn't setup to change dataref mappings at runtime (only load), but support for that would likely be beneficial for others as well. Even with runtime loading support though you'd have to figure out a way to instrument changing the mappings.

I'm all ears for your suggestions about how we could make it work, thanks a lot !

I think it could be beneficial for people working on sound installations like this one. Here, I see Bela Gem Multi as a way to create an interactive and generative multichannel sample-based installation. Lowering the barrier of entry by using RNBO is really convenient for people like me who already work with Max for a long time, and who doesn't have the ressources to learn another language correctly. I'm an electronic composer and sound artist, I'm sufficientily knowledgedable — all things considered — for coding my own tools, but don't always have a lot of extra time to go deeper. This specific project I'm working on is just the beginning, I have already ideas about how visitors could interact with it in the future. For now, it is a "proof of concept" to see if the fundamentals works.

giuliomoro

In render.cpp, here we could inject more "dummy" audio inputs that are actually coming from streaming files from disk.

Turning this

	for(unsigned int c = 0; c < nInChannels; ++c)
	{
		if(c < context->audioInChannels)
			inputs[c] = (float*)(context->audioIn + c * nFrames);
		else
			inputs[c] = (float*)(context->analogIn + (c - nAnalogParameters - context->audioInChannels) * nFrames);
	}

into

	for(unsigned int c = 0; c < nInChannels; ++c)
	{
		if(c < context->audioInChannels)
			inputs[c] = (float*)(context->audioIn + c * nFrames);
		else if(c - nAnalogParameters - context->audioInChannels < context->analogIn)
			inputs[c] = (float*)(context->analogIn + (c - nAnalogParameters - context->audioInChannels) * nFrames);
		else
			inputs[c] = //read file here and copy it as an audio input
	}

then you'd have additional audio inputs containing the data from the file.

The question is how you select the files. It looks like it is possible to receive messages from rnbo in C++. How soon after sending the message do you need to start playing the file?

Antrees

Thank you Giulio ! I select the files by a number generated by an "urn" object in RNBO. As soon as the number is chosen the file needs to start playing. Is it what you wanted to know ?

For you to better understand my case, I've exported the same RNBO patch with only 5 short buffers using this way of choosing (that you can see in the screenshot on my first post). You can find and download the rnbo_source.cpp file along all the other files — media included — of a RNBO C++ export in this shared folder on my server. You can find the maxpat and rnbopat files, to try this on your machine too.

giuliomoro

What latency can you accept between selecting the number and starting the playback?

If there's a bit of room, let's say 50 to 100 ms is acceptable, then it's much easier to code than if it has to be immediate.

Antrees

Oh yeah, there's room, because there is no need to synchronize anything. The only important thing is to avoid any perceived gaps when we go from one playing file to the next. But if it's perceived, I can avoid silence by mixing another sound coming from another player that will fill the gap.

giuliomoro

how many files will you be playing at once?

Antrees

I will play three files maximum.