that builds and runs fine for me. Do you know what version of the core code you have? Also the image number would be of interest (grep v0 /etc/motd).

I checked that earlier memory allocation again and it looks good, can you try commenting out lines 343 to 352 and see if that changes anything?

Also, can you try compiling simply default_libpd_render.cpp in your project folder without the lines from main.cpp?

Interesting. What flags did you use for the compiler/linker? Just so we're on the same page. From what I understand, g++ on Bela is just an alias for arm-linux-gnueabihf-g++, correct? As in, it will link against those libraries without having to call that linker/compiler explicitly. I recently updated the core, around when we started this project, so it is recent. However, I will post that info in a little while.

I will try all of that and get back to you with the info shortly. Although, if I do not include the lines from main.cpp the linker complains of an undefined reference to main(), since I am compiling this outside of the IDE, I assume.

Hmm ok, I had just built it as a Bela program. Why don't you send me the full command line you use to build the file?

g++ should be an alias for arm-linux-gnueabihf-g++ , yes. We use clang++ for all of the Bela stuff, but g++ should work equally fine. To see all the flags used by Bela for compilation and linking, just add AT= to your command line when building a project. For instance, after modifying some files in the project myProject, run this (equivalent to adding AT= in the "Make parameters" field in the IDE and hitting the "build" button in the IDE)

make -C ~/Bela PROJECT=myProject AT=

Thanks, that AT= addition was helpful just to see how it was building internally. Here's the output for that when pointed towards a test PD project without the custom render and main files:

/usr/bin/clang++ -Llib/ -pthread -o "/root/Bela/projects/test/test" build/core/FormatConvert.o build/core/OscillatorBank_routines.o build/core/math_runfast.o build/core/Gpio.o build/core/I2c_Codec.o build/core/PulseIn.o build/core/scope_ws.o build/core/RTAudio.o build/core/UdpClient.o build/core/WriteFile.o build/core/RTAudioCommandLine.o build/core/OSCClient.o build/core/WriteFile_c.o build/core/AuxTaskRT.o build/core/board_detect.o build/core/AuxTaskNonRT.o build/core/Midi.o build/core/AuxiliaryTasks.o build/core/I2c_TouchKey.o build/core/Midi_c.o build/core/Scope.o build/core/PruBinary.o build/core/PRU.o build/core/UdpServer.o build/core/OSCServer.o build/core/GPIOcontrol.o build/core/Spi_Codec.o build/core/JSONValue.o build/core/DigitalChannelManager.o build/core/JSON.o ./build/core/default_main.o ./build/core/default_libpd_render.o -Wl,--no-as-needed -L/usr/xenomai/lib -lcobalt -lmodechk -lpthread -lrt -lprussdrv -lstdc++ -Wl,--no-as-needed -L/usr/xenomai/lib -lcobalt -lmodechk -lpthread -lrt -lasound -lseasocks -lNE10 -lmathneon -lsndfile -lpd -lpthread

So, I've tried using that combination, building as follows:

Invoking: GCC C++ Compiler
g++ -std=c++14 -I/usr/local/include/libpd/ -I/home/juniper/Downloads/liblo-0.29 -I/home/juniper/Downloads/boost_1_69_0 -I/root/Bela/include -pthread -O3 -g3 -Wall -c -fmessage-length=0 -MMD -MP -MF"Main.d" -MT"Main.o" -o "Main.o" "../Main.cpp"

Invoking: GCC C++ Linker
g++ -o "bela_custom_main" ./Main.o -L/root/Bela/lib/ -pthread -lbelaextra -lbela -llo -Wl,--no-as-needed -L/usr/xenomai/lib -lcobalt -lmodechk -lpthread -lrt -lprussdrv -lstdc++ -Wl,--no-as-needed -L/usr/xenomai/lib -lcobalt -lmodechk -lpthread -lrt -lasound -lseasocks -lNE10 -lmathneon -lsndfile -lpd -lpthread

And this compiles and links successfully, but I get the same error. Same thing if I switch out g++ for clang++

grep v0 /etc/motd
Bela image, v0.3.6b, 23 October 2018

Not sure where to look for the core code version

    Right the problem is that in the compilation step you don't have the command-line -D options that Bela uses.

    If I run

    make -C ~/Bela PROJECT=test-we run AT=

    I get

    clang++ -I/root/Bela/projects/test-we -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -DNDEBUG -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/test-we/build/Main.d" -o "/root/Bela/projects/test-we/build/Main.o" "/root/Bela/projects/test-we/Main.cpp"

    Some of those defines are needed for the header files to work properly. include/Midi.h, for instance, requires XENOMAI_SKIN_native or XENOMAI_SKIN_posix to be defined. This will change at some point when we drop support for the native skin, but for now it's important to have them. Also, I think you should make sure you use the same include paths (including the xenomai ones).

    Ah! Thanks so much. That did it. Of course, I was running AT= on a project without any custom source files, so the compiler was never invoked at all :facepalm:

    FYI, we have decided to use OSC for all the communication between the Touchkeys process and Pd. This simplifies things, and the actual MIDI on/off we require is just encapsulated in an OSC message automatically anyway.

    As you suggested, I have edited the way the Mappings work to avoid unnecessary reallocation during a state change. My solution was to add a member MRPMapping* mrpMapping_; to the PianoKey class, which is pre-allocated when the class is instantiated. The mapping is simply reset() and engage()d when active, and disengage()d when idle. Seems to work well so far. That was certainly a good suggestion!

    The next stage is seeing how it runs on the Bela, as is (likely extremely slow, if at all) and then seeing what we can strip out. My gut would be to look at the Scheduler classes first, and try to eliminate them. I think there is a chance that all actions (except for timeouts, obviously) could simply be performed immediately, without too much ill effect. We may not need timeouts at all, really, since we are not receiving from an OSC or MIDI device (other than sysex messages from the external audio routing hardware). If we do, then we could simply delegate only the timeout actions to the scheduler, and perform all other actions (mappings, mainly) immediately. It's a theory that is worth testing, at least.

    Let me know if you have any more insights in to the matter, you've been more than helpful so far.

    Edit: For reference/posterity, here's the final compile command:
    g++ -std=c++14 -I/usr/local/include/libpd/ -I/home/juniper/Downloads/liblo-0.29 -I/home/juniper/Downloads/boost_1_69_0 -I/root/Bela/include -I/root/Bela/build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -DNDEBUG -O3 -g3 -Wall -c -fmessage-length=0 -MMD -MP -MF"Main.d" -MT"Main.o" -o "Main.o" "../Main.cpp"

    One thing I would like to try, and I will let you know how this works out:

    If communicating through USB (UART) we are not utilizing Bela's real-time capabilities for anything but the audio thread, which is doing the lifting for our Pd patch. Hence, I think it is worth trying to communicate to the Bela via ethernet from a dedicated board (likely a Raspberry Pi), which will be running the Touchkeys code and generating OSC messages.

    My thinking is this, although that would be adding another layer of latency, it may be acceptable and more importantly, it would ensure that the audio processing blocks always have enough time to complete each cycle, maintaining their real-time promise, and other threads don't have to compete for the CPU time that is left over. In essence, this would constitute a mult-core machine (in abstract, at least).

    The RPi 3 B+ has a 1.4GHz A8 quad-core CPU, with 1 GB of RAM, so the increased performance may actually beat out any latency introduced from the ethernet interface. I will post updates once we get some testing done on this.

    Did you hit the CPU limit on Bela already?
    Do you have a sense of how much CPU is needed by the audio engine and by the MRP software on Bela
    Adding a computer to the game seems more hassle than is needed, if it's not strictly necessary.

    I haven't been able to test the CPU load yet. That is more of a contingency plan than anything else. We are also trying to incorporate other features (wifi enabled parameter adjustment and presets, machine learning features), which might end up pushing us over the line. Ideally, yes we wouldn't go that route.

    alt text

    Something is not right here. I have everything working on my personal (Ubuntu Linux) machine to the point where I can run my adapted Touchkeys code on my machine and send OSC messages to Bela which is running out Pd patch, and everything works as expected and is quite responsive.

    However, once I attempt to run Touchkeys on the Bela at the same time, it hangs while attempting to open the Touchkey device. gdb does not seem to play well with the Bela core, so I cannot see the exact line where it is hanging (blocking perhaps?) but through deduction it is this line: device_ = open(inputDevicePath, O_RDWR | O_NOCTTY | O_NDELAY); Is there any reason this would not work on Bela? That is should I use the same flags as you did here: https://github.com/giuliomoro/serial-piano-scanner/blob/master/SerialInterface.cpp. Namely _handle = open(portname, O_RDWR | O_NOCTTY | O_SYNC).

    I am attempting to run all threads as vanilla pthreads. Is it possible my pthread calls are being wrapped (and turned into realtime threads) because of the flags being passed to the compiler? Or does that happen just during linking? Do you think there something else that I need to change about the LDFLAGS I posted above?

    For now, I will attempt building the Touchkeys code alone, with the only the normal CFLAGS and LDFLAGS to rule those out as the culprits, and to allow me to use gdb if needed.

    Thanks again.

      wa3573 Invoking: GCC C++ Linker
      g++ -o "bela_custom_main" ./Main.o -L/root/Bela/lib/ -pthread -lbelaextra -lbela -llo -Wl,--no-as-needed -L/usr/xenomai/lib -lcobalt -lmodechk -lpthread -lrt -lprussdrv -lstdc++ -Wl,--no-as-needed -L/usr/xenomai/lib -lcobalt -lmodechk -lpthread -lrt -lasound -lseasocks -lNE10 -lmathneon -lsndfile -lpd -lpthread

      Nothing here is automatically wrapping pthread calls.

      wa3573 That is should I use the same flags as you did here: https://github.com/giuliomoro/serial-piano-scanner/blob/master/SerialInterface.cpp. Namely _handle = open(portname, O_RDWR | O_NOCTTY | O_SYNC).

      That was used to open an actual UART port. I have not tried opening a USB serial instead.

      Can you simply cat /dev/tty... from the terminal for the port you want to use? Does it work?

          so in this case portname is actually "/dev/serial/by-id/usb-APM_TouchKeys_B6A358563433-if00". On both my machine and Bela, cat /dev/serial/by-id/usb-APM_TouchKeys_B6A358563433-if00 echoes terminal input until it receives a SIGINT, as expected. I'm not sure if /dev/tty... ports would work for out case because we will (likely) be using a USB hub, as we need to connect to two USB interfaces (the key sensors and the MRP routing hardware). Not sure how the tty... ports are assigned in that case.

          So, after compiling and running the Touchkeys app alone on Bela, the same issue persists. I can confirm open() is not functioning as expected.

          Thread 1 "serial-piano-sc" hit Breakpoint 1, 0xb6fbfbea in open () from /lib/arm-linux-gnueabihf/libpthread.so.0
          (gdb) step
          Single stepping until exit from function open,
          which has no line number information.
          Starting the TouchKeys on /dev/serial/by-id/usb-APM_TouchKeys_8D73428C5751-if00 ... failed: Failed to open
          [Thread 0xb2879450 (LWP 1048) exited]

          What is the return value of open() ? Send me the exact code you are using, and I can try with the piano scanner I have here.

          I'll get you the return value shortly. If you're interested, the full working project is available in the repo: https://github.com/wa3573/Drexel_MRP_Key_Scanner/tree/master/serial-piano-scanner

          It's very stable and working well for us on my machine. The relevant part for this discussion would be in TouchkeyDevice.cpp specifically:

          int device_; // File descriptor

          ...

          bool TouchkeyDevice::openDevice(const char * inputDevicePath)
          {
          // If the device is already open, close it
          if (isOpen())
          closeDevice();
          device_ = open(inputDevicePath, O_RDWR | O_NOCTTY | O_NDELAY);
          if (device_ < 0) {
          // fprintf(stderr, "%s", explain_open(inputDevicePath, O_RDWR | O_NOCTTY | O_NDELAY, 0));
          return false;
          }
          return true;
          }

          Edit:

          Ah, you know, I have been getting lucky in some regard. I just realized an error in my adapter class which replaces JUCE's CriticalSection and ScopedLock with functional adaptations using the pthread_ library. I made the error of having ScopedLock inherit from CriticalSection, so at it's constructor implicitly called CriticalSection and effectively made a new mutex, which is not the intended functionality. (ScopedLock sl = ScopedLock(&criticalSection_); is called to lock, and when that object goes out of scope, it's destructor is called, which exits the enclosed criticalSection). Anyway, that is remedied now, and the repo has been updated.

          Not sure if this was having an effect on my problem above, but it is possible. I will retest tomorrow.

          I just got a Raspberry Pi to experiment with, and interestingly, while the Pi is able to open the device file and obtain a file descriptor, it cannot successfully detect the presence of the Touchkeys device. More testing ahead, but this leads me to wonder if there is a fundamental difference in the way these "files" are being accessed by their ARM Linux implementations.

          Here's the solution (for the RPi at least, but both are embedded Debian distros so I suspect it would work on Bela as well)

          First: learn the Baud rate of the device:

          stty -F /dev/serial/by-id/usb-APM_TouchKeys_8D73428C5751-if00

          output:

          speed 9600;

          Then, git rid of that pesky O_NDELAY ~= O_NONBLOCK flag. This may work fine in most cases, but we're chewing up CPU cycles just to get an EAGAIN when there's no data. Add the O_SYNC flag which ensures any data written to the port is flushed before the write() call returns, seemingly more reliably than the existing flush calls written in.

          In place of blocking, set up a pure timed read using the termios attributes, by setting VMIN to 0 and VTIME to 5 for a 0.5 second timeout. Then setup the termios attributes for non-canonical mode with the baud rate attained above.

          So finally, using a couple helper functions (second answer here https://stackoverflow.com/questions/6947413/how-to-open-read-and-write-from-serial-port-in-c) we have the new openDevice() logic:

          ...

          if (isOpen())
	closeDevice();

device_ = open (inputDevicePath, O_RDWR | O_NOCTTY | O_SYNC);

if (device_ < 0) {
	return false;
}

set_interface_attribs(device_, B9600);
set_mincount(device_, 0);                // set to timed read

return true;

          Seems to work pretty consistently, so far.

          Ok, so it seems the only problem was with the speed of the port? I am actually surprised it was B9600 ... I am wondering if that's just a dummy number when dealing with USB serial?

          All the rest seems to be just optimizations (blocking upon read seems indeed a better option in general). Maybe O_SYNC doesn't help much in terms of CPU performance; I think the integrity of the data is not at risk either way. I guess this only helps when using multi-threaded programs?

          Yeah that seemed very slow to me as well. Very possible it's a dummy value. I'll experiment with different speeds. Not sure if that was the root of the problem, threading likely played a role. O_SYNC is possibly unnecessary, I will also try without that flag.

          Apart from it being purely speed related, my best guess would be that using a timed read had some impact.

          2 months later

          Here's an update:

          First of, I've got to thank you for all your help, this wouldn't have been possible without it.

          We've stuck with the Touchkeys software running on a RPi, and then transmitting OSC to Bela which is running a Pd patch to do the DSP, it works well, and the Bela doesn't seem to have the CPU resources to run both DSP and the Touchkeys software at the same time. Here's a video demo from last term:

          Pardon the wildly out-of-tune piano. We're working on enhancements and usability improvements now, and I am building an enclosure to solidify the system's cohesion.
          alt text

          alt text

          My next question for you:
          do you know what specific connector is used for the 3-pin audio connectors on the audio expander caplet? I am making a custom panel output with stereo 3.5mm jacks, and would like to purchase connectors, rather than cut and splice the connectors which we have already.

          I've also addressed a couple of bugs in the Touchkeys code which you all might be interested in.