Help with underruns using SDT for Puredata on Bela?

Many thanks for the response, @giuliomoro :

giuliomoro sometimes it's easier to set those options from the command-line when calling make, without need to patch the file:

Thanks! I thought that wouldn't work, because the Makefile didn't specify PD_INCLUDE to begin with, but since only CFLAGS uses that variable anyways, this invocation now makes sense to me!

giuliomoro - the proper way: use [declare] to pre-load the library.

Great - thanks for this, for some reason I thought [declare] wasn't vanilla PD - I just tried it out, and for the install procedure described above, [declare -lib SDT] works fine on the Bela.

giuliomoro Nope, that's not a problem by itself.

Good - thanks for confirming that, nice to get it out of the way.

giuliomoro So you get a low average CPU usage, but for each of the spikes you get a dropout. This is often mitigated by increasing Bela's blocksize.

Thanks for this - so knowing this, and along with information from https://forum.bela.io/d/715-block-size-with-puredata-on-the-bela, I did the following:

First, I added [block~ 128] to a subpatch in the patch, and so the entire patch now looks like this (also, here is TestSDT/_main.pd):

I tried having different values of [block~ N] vs. different blocksizes from Bela's Project Settings - and it seems, the [block~ N] didn't influence underruns much, but Project Settings did.

Then I also recompiled libpd ( as per https://forum.bela.io/d/101-compiling-puredata-externals/47 ), setting DEFDACBLKSIZE first to 64, then experimenting again with [block~ N]/Project Settings, then recompiling libpd with DEFDACBLKSIZE of 128, then experimenting again with [block~ N]/Project Settings - and it seems the biggest influence, regardless, was the Project settings

So in the end I went back to DEFDACBLKSIZE of 16, and made the following script, runblock.sh, which should be placed and run from the same folder where _main.pd resides:

#!/usr/bin/env bash
# NB: you need to be inside of a PD project folder on the Bela to run this
set -x
BLKSZA="$1"
BLKSZB="$2"
sed -i 's/block~ \([0-9]*\)/block~ '$BLKSZA'/' _main.pd
grep block _main.pd
# don't use build_project.sh --clean here, it deletes the newly compiled executable!
# just manually rm instead
rm -v $(find . -type f -executable)
~/Bela/scripts/build_project.sh --force -n $(readlink -f .)/
# ts program: `apt install moreutils` on a Bela with internet
./TestSDT -p $BLKSZB 2>&1 | LC_ALL=C ts '[%Y-%m-%d %H:%M:%S]'

So, basically first argument of the script is the "logical" block size as set by [block~] which is directly changed inside the PD file, and the second argument is the "hardware" block size set by -p command line argument when running the final executable. Here are a couple of runs:

root@bela:~/Bela/projects/TestSDT# bash runblock.sh 512 512
+ BLKSZA=512
+ BLKSZB=512
+ sed -i 's/block~ \([0-9]*\)/block~ 512/' _main.pd
+ grep block _main.pd
#X obj 104 65 block~ 512;
+ ./TestSDT -p 512
+ LC_ALL=C
+ ts '[%Y-%m-%d %H:%M:%S]'
[2018-11-23 15:29:20] Underrun detected: 1 blocks dropped
[2018-11-23 15:29:22] Underrun detected: 1 blocks dropped
[2018-11-23 15:29:31] Underrun detected: 1 blocks dropped
[2018-11-23 15:29:41] Underrun detected: 1 blocks dropped
[2018-11-23 15:29:53] Underrun detected: 1 blocks dropped
[2018-11-23 15:29:54] Underrun detected: 1 blocks dropped
[2018-11-23 15:29:58] Underrun detected: 1 blocks dropped

root@bela:~/Bela/projects/TestSDT# bash runblock.sh 128 512
+ BLKSZA=128
+ BLKSZB=512
+ sed -i 's/block~ \([0-9]*\)/block~ 128/' _main.pd
+ grep block _main.pd
#X obj 104 65 block~ 128;
+ LC_ALL=C
+ ts '[%Y-%m-%d %H:%M:%S]'
+ ./TestSDT -p 512
[2018-11-23 15:30:21] Underrun detected: 1 blocks dropped
[2018-11-23 15:30:32] Underrun detected: 1 blocks dropped
[2018-11-23 15:30:42] Underrun detected: 1 blocks dropped
[2018-11-23 15:30:54] Underrun detected: 1 blocks dropped
[2018-11-23 15:30:55] Underrun detected: 1 blocks dropped
[2018-11-23 15:30:59] Underrun detected: 1 blocks dropped

root@bela:~/Bela/projects/TestSDT# bash runblock.sh 16 512
+ BLKSZA=16
+ BLKSZB=512
+ sed -i 's/block~ \([0-9]*\)/block~ 16/' _main.pd
+ grep block _main.pd
#X obj 104 65 block~ 16;
+ LC_ALL=C
+ ts '[%Y-%m-%d %H:%M:%S]'
+ ./TestSDT -p 512
[2018-11-23 15:32:13] Underrun detected: 1 blocks dropped
[2018-11-23 15:32:15] Underrun detected: 1 blocks dropped
[2018-11-23 15:32:24] Underrun detected: 1 blocks dropped
[2018-11-23 15:32:34] Underrun detected: 1 blocks dropped

So - if I increase the hardware block size, via -p to 512, the underruns decrease significantly in frequency, but they still occur - sometimes at a second apart, sometimes at 10 second apart - but I noticed, almost always in sync with the [metro]ed [bang] from the patch; changing the logical block size here does not seem to have much effect.

Changing the hardware block size, via -p to 1024, decreases the frequency of underruns even further - however, this causes a high-pitched noise to appear on the output, so I can't really use it. Curiously, this appearance of high-pitched noise for -p 1024 (vs -p 512) appeared all the same for the different DEFDACBLKSIZE libpd builds.

giuliomoro The usual suspect when you get low average CPU usage but repeated underruns is that the CPU load for the audio callback is not constant. Some Pd vanilla objects (e.g.: [fft~], [sigmund~], [fiddle~], possibly [rms~]) have such a property: most of the time they do nothing, they just copy input samples into a buffer. Every so often, when the buffer is full, they run some expensive computation (e.g.: FFT) in the audio thread, which takes longer than the time available to process one block of data. So you get a low average CPU usage, but for each of the spikes you get a dropout.

giuliomoro Now, by a quick inspection of the code you linked, I don't see such a behaviour in that specific object, but I may be wrong. Also, maybe there is somewhere else in your patch that causes that?

Yeah, that could be it. When I do grep '\~' TestSDT/_main.pd, I can see besides the vanilla DSP objects *~, dac~, block~, the only other DSP objects in use are impact~ and breaking~. The breaking~ we saw earlier, and https://github.com/SkAT-VG/SDT/blob/master/src/Pd/impact~.c has this DSP function:

  t_impact *x = (t_impact *)(w[1]);
  t_float *in0 = (t_float *)(w[2]);
  t_float *in1 = (t_float *)(w[3]);
  t_float *in2 = (t_float *)(w[4]);
  t_float *in3 = (t_float *)(w[5]);
  t_float *in4 = (t_float *)(w[6]);
  t_float *in5 = (t_float *)(w[7]);
  int n = (int)w[8];
  double tmpOuts[2 * SDT_MAX_PICKUPS];
  int i, k;
  
  for (k = 0; k < n; k++) {
    SDTInteractor_dsp(x->impact, *in0++, *in1++, *in2++, *in3++, *in4++, *in5++, tmpOuts);
    for (i = 0; i < x->nOuts; i++) {
      x->outBuffers[i][k] = (t_float)tmpOuts[i];
    }
  }
return w + 9;

... which looks short == fast, but then SDTInteractor_dsp is in https://github.com/SkAT-VG/SDT/blob/master/src/SDT/SDTInteractors.c, and it looks a lot more complicated.

Now, even though breaking~ goes into impatch~ mostly through a DSP line, the fact that I see the underruns in sync with the triggering bang, tells me that possibly, the triggering bang "introduces energy into the system", thus causing some of the if'd portions of SDTInteractor_dsp, possibly like:

  if (v0 && x->obj0) {
    p = x->obj1 ? SDTResonator_getPosition(x->obj1, x->contact1) : 0.0;
    SDTResonator_setPosition(x->obj0, x->contact0, p);
    SDTResonator_setVelocity(x->obj0, x->contact0, v0);
}

... might now run (as opposed to not running for the most of the time, when the system is not "bang"ed); and since SDTInteractor_dsp is in itself called from a for loop from impact_perform, this might cause additional burden to the CPU - enough for an underrun to happen occasionally. But if that is the case, then I don't see an obvious way to solve it, - apart from, well, faster math (tried in optimizations alredy done), and faster CPU. Does this makes sense?

Is there anything else I could possibly try?

Thanks again, @giuliomoro :

giuliomoro that's NOT how you are supposed to use [block~]: it only affects the subpatch it is contained in (and all its subpatches), so that one is currently doing nothing.

I see, thanks - so I guess, I should move all processing into a subpatch, which also contains [block~], and then add two [outlet~]s, so as to connect to [dac~] in the main patch (so the [block~] and [dac~] don't conflict, as per help file)?

giuliomoro uops, forgot to mention that the maximum block size at which inputs and outputs all work fine is 128.

Got it - this example I need just for outputs, so its possible that's why I didn't notice anything wrong by going to 512.

giuliomoro It seems that something is being triggered by the metro that causes this underrun. However, none of those functions looks particularly expensive, but there would be need to look more in detail into them.

Yeah, I had a similar hunch about the functions, too, but wasn't really sure... Guess more debugging is needed here - if I get a bit further will this, will post here...