• Audio

  • Realtime audio drivers in Rust

I'll upload the current state of things on Github as soon as I can. What I should do first is complete the writeup that explains my approach and provides guidelines. Although there are not many sound-producing gadgets available yet (as I'm writing these lines, only a phaser!), the general design is ready for comments, and things should move much faster now.

The Sample library was my inspiration: it was a mind-expanding revelation for me that iterator-like objects will give you an API that's both efficient and easy to use, even when you change sample rate. But I think that ordinary Rust Iterators, as used in Sample, are not suited for audio, so I designed my own Signal trait. In addition to the .next() method there's a need for an .init() that takes account of the parameters known only at runtime, like sample rate. And Sample's choice of RefCell<Rc<T>> (if my memory serves me) when an output needs to feed many inputs is quite inefficient. So my other important design choice was to use &Cell<T> for sharing signals. But this makes my framework impossible to use when audio streams have a 'static lifetime, which is the standard for 90% of audio ffis in Rust, so I spent the last three weeks trying to find a workaround. I finally got to hear a sound on my MacBook by tweaking mitchmindtree's coreaudio-rs port.

It's great to hear that you can now write Supercollider UGens in Rust, and I like your idea of a Rusty approach to multi-threading, which should be quite efficient.

So I'll keep in touch, that's for sure.

    padenot I see you're based in Paris. Will you be at tomorrow's Rust meetup? I intend to go.

      12 days later

      alcofribas Sorry I missed a few days -- the sample crate relatively recently scrapped the reliance on Iterator in favor of fn next(&mut self) -> S directly. It'd be great to see if we can meet on that point now.

      And yes, Bus is not at all safe for realtime synthesis. It not only allocates but also has the potential to block. I made something to replace it using the bounded_spsc_queue crate, which is very efficient and can be used between threads. (The Arc is only a performance penalty each time clone is called, so we can safely create the queue in a lower-priority task.) This code isn't officially public, but is part of a larger library that adds some higher-level abstractions using sample as a dependency.

      I'd love to just look through the code if you're willing to share (even off GitHub). I'm about to introduce the Bela into an undergraduate course and would love to at least demo Rust. (There are a few comp science students in the class.)

        a year later

        A quick update on this. We wrote a couple patches the other day with Andrew after he contacted me because I'd had just gotten into the swing of things and basically wrote more or less the same binding crate (= rust package) he did, asking me whether I'd like to collaborate.

        We ended up finishing his bela-rs crate, which exposes a safe interface for all features of the Bela API (setup, render, teardown, auxillary tasks, all the various ports, cape things, etc.), and I think it's ready for people to use.

        Everything is on github:

        • bela-sys, which is an unsafe (in the rust meaning, i.e., it can segfault and things) crate exists in two, compatible, flavours:
          • My bela-sys: the binding header is vendored, has a setup script, potentially easier to setup.
          • Andrew's bela-sys: the binding header is generated on the fly during build, better when using a pre-release Bela image
        • bela-rs, exposes a safe (can't segfault or do anything nasty to memory, etc.) Rust API for all of the Bela native API, and a few utility functions.

        There are a few examples in each crate (cargo run --example hello for example). Cross-compilation is preferred, but not mandatory, Rust being rather slow to compile (especially on ARM it seems), but fast to run, this is the fastest setup, I find. Simply copying the executable to the board and running it there works, Rust produces a monolithic executable, finding the right .so on the board.

        Performance is, of course, as good as C++, but you have access to a massive amount of rust packages already written (a bunch of state of the art lock free stuff in particular). I've been running moderately complex interactive applications written in it without any issue, be it performance, compatibility, crashes or anything.

        A few examples:
        - https://github.com/padenot/monome-bela-seq, a sequencer that outputs eurorack triggers using the analog outputs, controlled by a Monome grid (depending on the modules you use, there might be a need for a few bits of electronics to bring the voltage up from the 0V-5V the board can output)
        - https://github.com/padenot/monome-rs, a crate to use a Monome grid (no code change between desktop and bela), used in monome-bela-seq but available on its own
        - https://github.com/padenot/mlr-rs a port of the classic mlr in rust (I think I've only pushed the desktop code, will push the bela code soon hopefully), a classic live sample-cutting application, also using monome-rs

        Let us know in the github issues or here if you have any question or face any issues or anything !

        2 years later

        Hello-

        Curious if this is still active? Or if there is any chance of official Rust support for Bela?

          Nothing breaking has changed in the Bela API since then (it has only been expanded a bit ), so I guess it should just work.

          djensenius Or if there is any chance of official Rust support for Bela?

          There are no hard and fast rules as to how a language becomes "officially supported" by Bela, but we'd need at least a few users who use it routinely, otherwise there is the risk of:
          a) the support breaking, because of API changes, old documentation, etc
          b) spending development time on something that benefits no one

          None of us on the core team of Bela is a Rust developer, which means we wouldn't be in the position of maintaining it and adding features to it without a significant community contribution and/or interest . A possible pathway towards getting "out of the box" support for Rust on Bela, would be for someone (e.g.: you, and/or @padenot / @andrewcsmith if they have kept using it beyond their great contributions above) should make sure the steps above still work, then come up with at least a minimal set of examples to cover the basics (think something like the "Fundamentals" examples we currently provide for C++, or the examples for Supercollider and Csound). In the process, hopefully the functionalities will receive further test and possibly additions.

          I am happy to help every step of the way with integration, but I'd rather not to have to write anything on the Rust side (because I don't know how to!).

          As a side note, a "success story": Supercollider support was added with community contribution where I took part of the integration, and I still maintain it today. We are a few revisions behind the official repo, but now looking at merge upstream in a future not too far away. In the process, I have - to this day - written more lines of Supercollider C++ source code than Supercollider programs.

            afaik all of the above still works. I'm not working on this at the moment because of unrelated real-life reasons, but hopefully I'll be able to start again in a few months.

            Most of the rust bindings at automatically generated from the header files, and then we wrap it into an idiomatic Rust API, to get the nice safety property of rust and a bit better ergonomics. It's not particularly high maintenance if it doesn't work, I can have a look and fix it. If a pull request comes to fix something, I'll look at it quickly.

            10 months later

            As a big fan of Rust🦀, I am also interested in using Rust rather than C++ to program my new Bela (arrived yesterday).

            Sadly, neither bela-sys nor bela-rs are compatible with a) current Rust and b) current Bela (the settings struct changed for example). I was able to work around most of the compile issues with some effort:

            https://github.com/l0calh05t/bela-rs/tree/bela0.3.8b_rust1.52.0
            https://github.com/l0calh05t/bela-sys/tree/bela0.3.8b_rust1.52.0

            The build process still isn't nice (need to manually scp a bunch of files from the Bela to a cryptic <HOME>/.cargo/git/checkouts/bela-sys-<HASH>/<SHORT-HASH> folder) and I haven't gotten around to the "actually running it on Bela"-part, but at the very least: the examples compile and link!

            @l0calh05t thanks for the update. I always hope that I'll find the time to dig into Rust at some point! The changes in the BelaInitSettings struct have been made so that it maintains binary compatibility with the older versions. Not sure if that helps with your effort, because the only related change I seem to see is that here https://github.com/l0calh05t/bela-rs/commit/a32aacdae50f712d9aee17eafe7fd3091a0f2bf9 you commented out all the lines that were actually setting the settings ...

              giuliomoro yes, I removed them as the fields didn't appear to exist anymore in the new BelaInitSettings struct. I assume

              int unused1;
char unused2[MAX_UNUSED2_LENGTH];

              is where they used to be? Since @andrewcsmith 's bela-sys generates the Rust structs directly from the header, binary compatibility wasn't helpful. It may work with @padenot 's bela-sys since that one doesn't auto-generate the Rust code, in that case I could add them back in (but everything would have to be binary compatible and it could silently break in the future).

                  In any case, the bela-sys hello seems to run (although I haven't dared plug anything in yet see EDIT below)

                  root@bela:~# ./hello
Bela_initAudio()
Starting in high-performance mode
Starting with period size 16 ;analog enabled
DAC level 0.000000 dB; ADC level 0.000000 dB; headphone level -6.000000 dB
Detected hardware: CtagBeastBela
Hardware specified by user:
Hardware specified in belaconfig:
Hardware to be used: CtagBeastBela
Bela_getHwConfig()
fifoFactor: 1
core audioFrames: 16
PRU memory mapped to ARM:
digital: 0xb6316000 0xb6316400
audio: 0xb6317000 0xb6317200 0xb6317400 0xb6317500
analog: 0xb6307000 0xb6307080 0xb6307100 0xb6307180
analog offset: 0xffff2000 0xffff2080 0xffff2100 0xffff2180
Bela_startAudio
startAudioInilne
Using embedded PRU code
_________________Audio Thread!
^Caudio thread ended
Stopping audio...
Bela_cleanupAudio()

                  On the other hand, the bela-rs examples only output Setting up, except the sample example which stops at an assertion (probably because it didn't expect to find a Bela BEAST combo)

                  root@bela:~# ./sample
Setting up
thread '<unnamed>' panicked at 'assertion failed: `(left == right)`
  left: `16`,
 right: `2`', examples\sample.rs:26:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 9
Aborted

                  EDIT:
                  Sound seems to work. The difference regarding the output of the bela-rs hello comes down to not doing bela_sys::Bela_setVerboseLevel(1);

                    l0calh05t yes, I removed them as the fields didn't appear to exist anymore in the new BelaInitSettings struct.

                    I see!

                    enableBelaCapeButtonMonitoring has become stopButtonPin, codecI2CAddress, receivePort, transmitPort, serverName have been removed because they had been unused for ages. Some of those became unused, some others have been replaced with something that has been added in the mean time (namely at least audioThreadDone, not sure if anything else).

                    l0calh05t (but everything would have to be binary compatible and it could silently break in the future)

                    I am aiming to keep binary compatibility going ahead.

                    l0calh05t the bela-sys hello seems to run

                    great!

                    l0calh05t except the sample example which stops at an assertion (probably because it didn't expect to find a Bela BEAST combo)

                    that's it by the looks of things!

                              giuliomoro enableBelaCapeButtonMonitoring has become stopButtonPin

                              Okay, but that substitution implies different semantics, right? bela-rs considered it a boolean and stopButtonPin can be -1 or some pin number. Is there any way to check which ones are valid? Otherwise calling the method "safe" would be a lie (but checking of setting invariants is already quite incomplete). In any case

                              /// How many audio input channels [ignored]
int numAudioInChannels;
/// How many audio out channels [ignored]
int numAudioOutChannels;

                              seem to be obsolete? audioThreadDone, board and projectName appear to be new. Supporting audioThreadDone would be the most complex of the additions.

                                  giuliomoro that's it by the looks of things!

                                  Yes, but indirectly it is due to a limitation of the dasp (formerly sample crate) which appears to only support fixed-size (at compile time) frames (if I'm reading the documentation correctly).

                                    Urghh... You could run board_detect which returns the board name and infer the channel count from that at compile time ... Hackish ...

                                      giuliomoro That wouldn't work for cross-compilation. dasp is a separate crate anyway, so that can be ignored for now

                                        l0calh05t Okay, but that substitution implies different semantics, right?

                                        that's correct, but as far as I knew no one was actually setting it to anything else than default, so this was sort of a compromise: keep ABI compatibility for everything else while changing the meaning for this (to my knowledge) unused flag. Yeah, pretty poor, but ultimately AFAIK the only thing that relied on dynamically linking to Bela is Supercollider, which I also maintain and which didn't set that specific flag.

                                        l0calh05t Supporting audioThreadDone would be the most complex of the additions.

                                        that's an optional function callback, you are unlikely to need it for anything. It was required for Supercollider, as it didn't have a way of being notified and gracefully shutdown if the audio thread ended because the button was pressed.

                                        l0calh05t s there any way to check which ones are valid?

                                        you mean which values are valid for stopButtonPin? Any value is "valid". -1 or any number >= 128 are considered "off", values between 0 and 127 are considered "on" and the value is the pin number that will get monitored.

                                        l0calh05t /// How many audio input channels [ignored]
                                        int numAudioInChannels;
                                        /// How many audio out channels [ignored]
                                        int numAudioOutChannels;
                                        seem to be obsolete?

                                        You seem right. I may have left them there in case i actually wanted to use them ... it could be helpful to specify manually how many of the available I/O one wants to actually use ... currently you can do it only in a limited way by setting e.g: --board=Bela (or .board = BelaHw_CtagFace) to only use 4in/8out when a CTAG Beast is also connected... But as usual there are other higher-priority things going on :-(

                                                  giuliomoro the only thing that relied on dynamically linking to Bela is Supercollider, which I also maintain and which didn't set that specific flag.

                                                  Rust links shared libraries by default AFAIK, but it should be possible to change this. The allocation functions for the settings struct aren't used yet though, but doing that shouldn't be to complicated either.

                                                  giuliomoro you mean which values are valid for stopButtonPin? Any value is "valid". -1 or any number >= 128 are considered "off", values between 0 and 127 are considered "on" and the value is the pin number that will get monitored.

                                                  Ok, but can any value in [0, 128) actually be used? What if something else is assigned to the pin as well?

                                                  giuliomoro currently you can do it only in a limited way by setting e.g: --board=Bela (or .board = BelaHw_CtagFace) to only use 4in/8out when a CTAG Beast is also connected... But as usual there are other higher-priority things going on

                                                  What happens if you set a board that isn't present / available (i.e., Bela when only a Beast/Face is used or BelaMini on a Bela)?

                                                          l0calh05t What happens if you set a board that isn't present / available (i.e., Bela when only a Beast/Face is used or BelaMini on a Bela)?

                                                          it will fail and not start. See https://github.com/BelaPlatform/Bela/blob/master/core/RTAudio.cpp#L452-L479

                                                          l0calh05t Ok, but can any value in [0, 128) actually be used? What if something else is assigned to the pin as well?

                                                          It will let you do that ... if you end up picking one of the pins that the Bela Digital channels are on and you set it to an output, this will affect the readings, but if it's used as an input, it will work. There are no checks to avoid any such clashes, we just make sure it doesn't crash. It's up to the user to select a pin that makes sense. Again, most likely this is just for those who want to spin their own board or something like that, not really for day-to-day usage ...