• Audio

  • Realtime audio drivers in Rust

I'm interested in using the Bela with the programming language Rust to implement some realtime audio processing in a relatively small system. Currently, the problem is that most of my code relies on portaudio, but that's negotiable. I think it won't take too much to write some sort of an FFI interface, but having a handle on getting a callback through a C API (as C++ is not possible with Rust FFI) will be essential.

Rust allows for low-level memory control, pointers, all the fun you'd expect from C, so I would just like to be able to write a safe API wrapper that can be used by other Rust programmers as well.

Edit: I am able to cross-compile to the armv7, and even install the Rust compiler itself on the Bela. Realtime scheduling is the hiccup. So assume my experience level is "pretty good" with things like interfacing between C and Rust at a low level, and "pretty low" regarding microcontrollers. I'm obviously more interested in wrapping the Bela API than in writing my own very, very low-level xenomai wrapper.

Thanks. I wasn't able to find this myself. It looks like he took a slightly different approach than I was planning. Instead of compiling new binaries using Rust calling an external C library, he's declaring the Rust methods as C functions and calling it from the usual C++ core routines.

This viral post blink an LED with Rust and Beaglebone Black was very well-received in the Rust community, so I'm thinking the Rust/Bela thing could be an interesting merge. Rust & Bela have some serious power.

Do you have any example of Bela Core being compiled as an external lib, statically linked into another project and called via the C ABI?

13 days later

I want to say that I'm very much interested in a Rust port for Bela, using whatever means that will do the job. I'm (slowly) writing a DSP library in Rust that's meant to run efficiently on a Bela, but I don't know C++ and I have no experience with the Rust ffi yet.

The last time I looked I had the impression that you could implement a basic instrument (just the render loop) using only C, and that would be easy to achieve in the current state of Rust. Was I right about this? It would already be great to have this, you can already do a lot in Bela without having to worry about threads.

    alcofribas Do you have some code online for your DSP library? I'm working on something similar, but (more importantly) I'm a fairly frequent contributor to the sample crate and am planning for these bindings to easily interact with that crate. I know Rust pretty well at this point, and have done FFI as well as created a SuperCollider UGen using Rust.

    My end goal, though, is to be able to use Rust's move semantics (Send/Sync) to enforce any kind of contracts assumed by the Bela task creation process. I'd love to build up channel abstractions to communicate between the realtime and non-realtime OS processes, doing complex analyses offline (in a lower-priority task) while being able to ensure that the more difficult thread-safety issues are taken care of by the Rust compiler. Hit me up if you're interested and I'll let you know when I start that project on Github.

      My patches are quite outdated, I'll try to rebase them on current master if there is interest.

      The rust branch on my fork also has setup scripts (that pull a binary, install it, check if everything works fine, etc.), and automatic binding generation. It's not finished, and it's generating a rust binding file that is too big and contains unnecessary things. However, it worked very well.

      Now that a C ABI is exposed, the little C function I was using won't be necessary, and things will be cleaner. bindgen is also much better.

        I'll upload the current state of things on Github as soon as I can. What I should do first is complete the writeup that explains my approach and provides guidelines. Although there are not many sound-producing gadgets available yet (as I'm writing these lines, only a phaser!), the general design is ready for comments, and things should move much faster now.

        The Sample library was my inspiration: it was a mind-expanding revelation for me that iterator-like objects will give you an API that's both efficient and easy to use, even when you change sample rate. But I think that ordinary Rust Iterators, as used in Sample, are not suited for audio, so I designed my own Signal trait. In addition to the .next() method there's a need for an .init() that takes account of the parameters known only at runtime, like sample rate. And Sample's choice of RefCell<Rc<T>> (if my memory serves me) when an output needs to feed many inputs is quite inefficient. So my other important design choice was to use &Cell<T> for sharing signals. But this makes my framework impossible to use when audio streams have a 'static lifetime, which is the standard for 90% of audio ffis in Rust, so I spent the last three weeks trying to find a workaround. I finally got to hear a sound on my MacBook by tweaking mitchmindtree's coreaudio-rs port.

        It's great to hear that you can now write Supercollider UGens in Rust, and I like your idea of a Rusty approach to multi-threading, which should be quite efficient.

        So I'll keep in touch, that's for sure.

          padenot I see you're based in Paris. Will you be at tomorrow's Rust meetup? I intend to go.

            12 days later

            alcofribas Sorry I missed a few days -- the sample crate relatively recently scrapped the reliance on Iterator in favor of fn next(&mut self) -> S directly. It'd be great to see if we can meet on that point now.

            And yes, Bus is not at all safe for realtime synthesis. It not only allocates but also has the potential to block. I made something to replace it using the bounded_spsc_queue crate, which is very efficient and can be used between threads. (The Arc is only a performance penalty each time clone is called, so we can safely create the queue in a lower-priority task.) This code isn't officially public, but is part of a larger library that adds some higher-level abstractions using sample as a dependency.

            I'd love to just look through the code if you're willing to share (even off GitHub). I'm about to introduce the Bela into an undergraduate course and would love to at least demo Rust. (There are a few comp science students in the class.)

              a year later

              A quick update on this. We wrote a couple patches the other day with Andrew after he contacted me because I'd had just gotten into the swing of things and basically wrote more or less the same binding crate (= rust package) he did, asking me whether I'd like to collaborate.

              We ended up finishing his bela-rs crate, which exposes a safe interface for all features of the Bela API (setup, render, teardown, auxillary tasks, all the various ports, cape things, etc.), and I think it's ready for people to use.

              Everything is on github:

              • bela-sys, which is an unsafe (in the rust meaning, i.e., it can segfault and things) crate exists in two, compatible, flavours:
                • My bela-sys: the binding header is vendored, has a setup script, potentially easier to setup.
                • Andrew's bela-sys: the binding header is generated on the fly during build, better when using a pre-release Bela image
              • bela-rs, exposes a safe (can't segfault or do anything nasty to memory, etc.) Rust API for all of the Bela native API, and a few utility functions.

              There are a few examples in each crate (cargo run --example hello for example). Cross-compilation is preferred, but not mandatory, Rust being rather slow to compile (especially on ARM it seems), but fast to run, this is the fastest setup, I find. Simply copying the executable to the board and running it there works, Rust produces a monolithic executable, finding the right .so on the board.

              Performance is, of course, as good as C++, but you have access to a massive amount of rust packages already written (a bunch of state of the art lock free stuff in particular). I've been running moderately complex interactive applications written in it without any issue, be it performance, compatibility, crashes or anything.

              A few examples:
              - https://github.com/padenot/monome-bela-seq, a sequencer that outputs eurorack triggers using the analog outputs, controlled by a Monome grid (depending on the modules you use, there might be a need for a few bits of electronics to bring the voltage up from the 0V-5V the board can output)
              - https://github.com/padenot/monome-rs, a crate to use a Monome grid (no code change between desktop and bela), used in monome-bela-seq but available on its own
              - https://github.com/padenot/mlr-rs a port of the classic mlr in rust (I think I've only pushed the desktop code, will push the bela code soon hopefully), a classic live sample-cutting application, also using monome-rs

              Let us know in the github issues or here if you have any question or face any issues or anything !

              2 years later

              Hello-

              Curious if this is still active? Or if there is any chance of official Rust support for Bela?

                Nothing breaking has changed in the Bela API since then (it has only been expanded a bit ), so I guess it should just work.

                djensenius Or if there is any chance of official Rust support for Bela?

                There are no hard and fast rules as to how a language becomes "officially supported" by Bela, but we'd need at least a few users who use it routinely, otherwise there is the risk of:
                a) the support breaking, because of API changes, old documentation, etc
                b) spending development time on something that benefits no one

                None of us on the core team of Bela is a Rust developer, which means we wouldn't be in the position of maintaining it and adding features to it without a significant community contribution and/or interest . A possible pathway towards getting "out of the box" support for Rust on Bela, would be for someone (e.g.: you, and/or @padenot / @andrewcsmith if they have kept using it beyond their great contributions above) should make sure the steps above still work, then come up with at least a minimal set of examples to cover the basics (think something like the "Fundamentals" examples we currently provide for C++, or the examples for Supercollider and Csound). In the process, hopefully the functionalities will receive further test and possibly additions.

                I am happy to help every step of the way with integration, but I'd rather not to have to write anything on the Rust side (because I don't know how to!).

                As a side note, a "success story": Supercollider support was added with community contribution where I took part of the integration, and I still maintain it today. We are a few revisions behind the official repo, but now looking at merge upstream in a future not too far away. In the process, I have - to this day - written more lines of Supercollider C++ source code than Supercollider programs.

                  afaik all of the above still works. I'm not working on this at the moment because of unrelated real-life reasons, but hopefully I'll be able to start again in a few months.

                  Most of the rust bindings at automatically generated from the header files, and then we wrap it into an idiomatic Rust API, to get the nice safety property of rust and a bit better ergonomics. It's not particularly high maintenance if it doesn't work, I can have a look and fix it. If a pull request comes to fix something, I'll look at it quickly.

                  10 months later

                  As a big fan of Rust🦀, I am also interested in using Rust rather than C++ to program my new Bela (arrived yesterday).

                  Sadly, neither bela-sys nor bela-rs are compatible with a) current Rust and b) current Bela (the settings struct changed for example). I was able to work around most of the compile issues with some effort:

                  https://github.com/l0calh05t/bela-rs/tree/bela0.3.8b_rust1.52.0
                  https://github.com/l0calh05t/bela-sys/tree/bela0.3.8b_rust1.52.0

                  The build process still isn't nice (need to manually scp a bunch of files from the Bela to a cryptic <HOME>/.cargo/git/checkouts/bela-sys-<HASH>/<SHORT-HASH> folder) and I haven't gotten around to the "actually running it on Bela"-part, but at the very least: the examples compile and link!

                  @l0calh05t thanks for the update. I always hope that I'll find the time to dig into Rust at some point! The changes in the BelaInitSettings struct have been made so that it maintains binary compatibility with the older versions. Not sure if that helps with your effort, because the only related change I seem to see is that here https://github.com/l0calh05t/bela-rs/commit/a32aacdae50f712d9aee17eafe7fd3091a0f2bf9 you commented out all the lines that were actually setting the settings ...

                    giuliomoro yes, I removed them as the fields didn't appear to exist anymore in the new BelaInitSettings struct. I assume

                    int unused1;
char unused2[MAX_UNUSED2_LENGTH];

                    is where they used to be? Since @andrewcsmith 's bela-sys generates the Rust structs directly from the header, binary compatibility wasn't helpful. It may work with @padenot 's bela-sys since that one doesn't auto-generate the Rust code, in that case I could add them back in (but everything would have to be binary compatible and it could silently break in the future).

                        In any case, the bela-sys hello seems to run (although I haven't dared plug anything in yet see EDIT below)

                        root@bela:~# ./hello
Bela_initAudio()
Starting in high-performance mode
Starting with period size 16 ;analog enabled
DAC level 0.000000 dB; ADC level 0.000000 dB; headphone level -6.000000 dB
Detected hardware: CtagBeastBela
Hardware specified by user:
Hardware specified in belaconfig:
Hardware to be used: CtagBeastBela
Bela_getHwConfig()
fifoFactor: 1
core audioFrames: 16
PRU memory mapped to ARM:
digital: 0xb6316000 0xb6316400
audio: 0xb6317000 0xb6317200 0xb6317400 0xb6317500
analog: 0xb6307000 0xb6307080 0xb6307100 0xb6307180
analog offset: 0xffff2000 0xffff2080 0xffff2100 0xffff2180
Bela_startAudio
startAudioInilne
Using embedded PRU code
_________________Audio Thread!
^Caudio thread ended
Stopping audio...
Bela_cleanupAudio()

                        On the other hand, the bela-rs examples only output Setting up, except the sample example which stops at an assertion (probably because it didn't expect to find a Bela BEAST combo)

                        root@bela:~# ./sample
Setting up
thread '<unnamed>' panicked at 'assertion failed: `(left == right)`
  left: `16`,
 right: `2`', examples\sample.rs:26:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 9
Aborted

                        EDIT:
                        Sound seems to work. The difference regarding the output of the bela-rs hello comes down to not doing bela_sys::Bela_setVerboseLevel(1);

                          l0calh05t yes, I removed them as the fields didn't appear to exist anymore in the new BelaInitSettings struct.

                          I see!

                          enableBelaCapeButtonMonitoring has become stopButtonPin, codecI2CAddress, receivePort, transmitPort, serverName have been removed because they had been unused for ages. Some of those became unused, some others have been replaced with something that has been added in the mean time (namely at least audioThreadDone, not sure if anything else).

                          l0calh05t (but everything would have to be binary compatible and it could silently break in the future)

                          I am aiming to keep binary compatibility going ahead.

                          l0calh05t the bela-sys hello seems to run

                          great!

                          l0calh05t except the sample example which stops at an assertion (probably because it didn't expect to find a Bela BEAST combo)

                          that's it by the looks of things!

                                    giuliomoro enableBelaCapeButtonMonitoring has become stopButtonPin

                                    Okay, but that substitution implies different semantics, right? bela-rs considered it a boolean and stopButtonPin can be -1 or some pin number. Is there any way to check which ones are valid? Otherwise calling the method "safe" would be a lie (but checking of setting invariants is already quite incomplete). In any case

                                    /// How many audio input channels [ignored]
int numAudioInChannels;
/// How many audio out channels [ignored]
int numAudioOutChannels;

                                    seem to be obsolete? audioThreadDone, board and projectName appear to be new. Supporting audioThreadDone would be the most complex of the additions.