• Audio

  • udp audio over ethernet

great getting to play with the bela for the last few days. i can seee a particular space for it in my system but i need to comprehend what is possible.

i would like to route audio into and out of the bela from my raspberry pi based network. i use the udp~ send/recieve to send audio from an instance of pd with no dac...only netowrk out. i then funnel all the audio, over udp, to an instance that is only a dac, which takes almost no cpu power.i found that this allows me to treat the network of pi's, of which there are now 3, as 12 cores rather than 3 computers.

what would be very iinteresting would be to send the udp stream to the bela where it would come out of the line and speaker outputs. in addition, id like to capture my adc signal using bela and instantly send it over ethernet to any of my networked synthesis engines that would need it.

so basically, any help that could be provided in the pursuit of
1. accepting a stereo udp~ audio stream from pd-extended
2. routing it to the line and speaker outputs
3. capturing audio using the line inputs and sending it over ethernet udp to the connected pi network running pd-extended, for use as vocal input.

any pointers in this direction are greatly appreciated.

cheers,

onyx

    onyxashanti found that this allows me to treat the network of pi's, of which there are now 3, as 12 cores rather than 3 computers.

    That's an interesting setup. How do you deal with clock synchronization? Are you using one single "master" device for I/O and all the others only to process audio, sending and receiving it over the network?
    If they are all outputting audio, don't they "drift" away at some point?

    onyxashanti 1. accepting a stereo udp~ audio stream from pd-extended

    Is the source for udp~ available anywhere? It turns out that netsend/netreceive/udpsend/udpreceive, do the networking I/O in the audio thread, which is completely crazy. If [udp~] believes similiarly, then that would require to be rewritten, otherwise you could just compile it for Bela.

    Alternatively: on Bela I already rewrote [netsend] and [netreceive] so that they do the networking bits in a separate thread, so they are safe to use, so what about using this vanilla replacement for udp~?

    alt text

          thanks for the reply :-)
          my system has no master clock. i am the master clock. everything is udp'd between instances and works really well. there is no drift because everything is realtime gestural. like this

          all syncronization is a mental pattern and gestural interaction.

          sorry for the typo. it is udpsend~ and udprecieve~.

          does netsend/netreceive work at signal rate?

            onyxashanti does netsend/netreceive work at signal rate?

            The above abstraction makes it work at signal rate, by sending lists of blocks of audio.

              dope! i will play with that and see what comes of it and post back when i get to another stage. thank you. im looking very forward to playing with the platform.

              4 months later

              ok, so happy autumn :-)

              for the last few months, ive been doing refinement on the bodyware for interfacing with the software system on the raspberry pi 3 and just used the audio hat ive been using, but i kept the bela somewhere i could see it and designed some 3d printed stuff for it to be integrated into my system and now i have the mind-space to dedicate some time towards its integration.

              i can access the IDE from chromium on the pi3 over ehternet using my own static ip and i am sending audio from pd-l2ork to pure data then pd to bela using a variation of the patch you shared with me above.

              on the pi3 i am runnning this patch

              alt text

              and this is what i have on the bela side

              alt text

              the issue is that although it does get there:

              -the audio is really harsh, spitty noise
              -i get an array of error messages:

              "underrun detected: x blocks dropped" (x being anywhere from 1-3 blocks)
              "x mode switches detected on the audio thread" (x being a number that rises into the millions)

              i was getting a number of other error messages before i updated the bela and reloaded the patches. at first, there was no audio coming thru at all.

              this happening over ethernet and not usb. i changed the bela's static ip to 192.168.1.116 so it works within the range of ip's i already use. everything works, but it just doesnt sound good.

              the cpu on the bela is around 50% while this is all happening, with the ide open but falls to around 12% when i close the browser.

              the current idea is to do my synthesis in pd-l2ork then use the [udpsend~ ] object to send it to the pd instance that runs the patch above to connect to the bela over ethernet.

              im still going through the help files and forum posts but any insight you might have on how to solve this audio distortion issue, are greatly appreciated.

              could it be possible to use the above patch from within pd-l2ork to talk to the bela or is there something special about the pd-vanilla [netsend] object?

              thank you

              First off, could I ask you to update the Bela code to the latest version? I just updated libpd, so maybe that will help.

              Second: can you increase the blocksize on Bela to be the same as what is sent to [s blocksize] ?

              Third: I think that [udpsend~] will send binary data, so it should be matched by a [netreceive -b ] at the receiving end (my mistake above)

              Fourth: make sure the sample rate on the two devices is the same.

              Fifth: there is some overhead associated with sending/receiving UDP packets. If you could join the two channels you are sending into a single packet, that would make the networking more efficient, although you'd have to do slightly more work to split them out to two separate arrays. Mind you: you shall keep the overall size of the packet < the MTU (maximum transmission unit) of the network transport layer. This normally is 1500 bytes for ethernet. In binary mode, each sample will be 4 bytes, so you should be able to send up to 1500/4 = 375 samples per packet (including both channels).

              Sixth: I think you should implement some buffering in your array. Currently, you are expecting that you will receive exactly blocksize samples over UDP for each blocksize samples running on Bela, and that these will come in exactly once for each audio callback. What if one packet is slightly late? It will not be there on time and you will get an underrun (not detected by the IDE, but ultimately you will be reading the same block twice from the array). What if one packet arrives slightly early? [netsend] will receive two packets in the same audio block and the second one will override the first one in the array.

              Seventh: even if the sample rate on the two devices is the same, they will drift over time (because physics). Therefore, sooner or later you will have a dropout at the receiving end (if the sender is slower than the receiver), or you will have an increased latency (if the sender is faster than the receiver), until you run out of space in your buffer and you get a click that takes you back "in sync" till the next drift.

              Eighth: there may be a number of other pitfalls I cannot think about just now. It's not an easy task.

              funny story...

              thank you for your reply. i updated the distro and made the changes you suggested which changed the tone of the distroted audio but didnt cure it, so i played around with some of the settings on the pi, which is running a realtime kernel. i found that i could run my patches with a cheap usb audio interface, but hadnt really considered it. note that ive been trying to get to a point of using less moving parts, whether hardware or software. i was surprised because my synthesis system cant run on standard raspbian using a a usb soundcard. so i decided to try getting rid of the usb soundcard and just using the onboard sound, AND IT WORKED!! ive never been able to use onboard sound before! albeit at 22.5khz and 20ms latency, but it works...its sounds interestingly shitty rather than unuseably shitty.

              so i think im going to investigate the bela more slowly and play with it while i vector my sonocyb system toward maxing out the potential of the raspberry pi to be useable on its own with nothing else other than what it comes with (analog audio out, bluetooth, wifi and pi-fm radio tx)

              thank you for your help. i look forward to coninuing to explore the bela at a pace where i can take time to understand its nuances.

              cheers,

              onyx

              Great!
              Yes, USB audio tends to be fairly bad on the Pi. You should go for an I2S soundcard (basically a dedicated Hat). In this paper we used the IQaudIO Pi-DAC+ (and also some USB dongle for comparison). This has no inputs, only 2 outputs.

              thats the card i have now. it is beautiful, but as this project becomes an open source project, i dont want it to be a neccessity. if i can make the onboard sound usable, it is all up from there as the rpi has a rep as the worst sound on any computer. i feel like wireless at this end of the process can be very interesting now. ethernet is still gonna be in there though, once the 3rd computer gets added...maybe...