Hi! I'm attempting to load a trained model as-per this library -> https://github.com/f0uriest/keras2c

So far I can clone the Github to my Bela in a fresh project (MYPROJECT), compile the test code according to the instructions (https://f0uriest.github.io/keras2c/usage.html) using the example files provided https://github.com/f0uriest/keras2c/tree/master/example) and run that program successfully on my command line on the Bela board.

Where I fail is setting up my render.cpp to do the same thing, and I'm convinced it's due to linking issues (or maybe some c vs. c++ thing?). When I load a model in render.cpp (similar to https://github.com/f0uriest/keras2c/blob/master/example/example_test_suite.c)

Which returns this error:

example.c:(.text+0x840): undefined reference to k2c_dense(k2c_tensor*, k2c_tensor const*, k2c_tensor const*, k2c_tensor const*, void (*)(float*, unsigned int), float*)'
example.c:(.text+0x882): undefined reference tok2c_dense(k2c_tensor, k2c_tensor const, k2c_tensor const, k2c_tensor const, void ()(float, unsigned int), float)'
example.c:(.text+0x8fc): undefined reference to k2c_lstm(k2c_tensor*, k2c_tensor const*, float*, k2c_tensor const*, k2c_tensor const*, k2c_tensor const*, float*, int, int, void (*)(float*, unsigned int), void (*)(float*, unsigned int))'
example.c:(.text+0x930): undefined reference tok2c_dense(k2c_tensor, k2c_tensor const, k2c_tensor const, k2c_tensor const, void ()(float, unsigned int), float)'
collect2: error: ld returned 1 exit status

I've tried to fix the linking by using this command (inspired by a fix in the references + the instructions from keras2c repo):

make -C ~/Bela PROJECT=MYPROJECT COMPILER=gcc CFLAGS="-I/root/Bela/projects/MYPROJECT/ -std=c99" LDFLAGS="-L/root/Bela/projects/MYPROJECT/include" LDLIBS="-lkeras2c -lm /root/Bela/projects/MYPROJECT/example.c"

Notably, if I exclude the example.c reference, i get an issue with having undefined references to things defined in that file:

/root/Bela/projects/MYPROJECT/render.cpp:674: undefined reference to example_initialize()'
/root/Bela/projects/MYPROJECT/render.cpp:676: undefined reference toexample(k2c_tensor, k2c_tensor)'
/root/Bela/projects/MYPROJECT/render.cpp:677: undefined reference to example(k2c_tensor*, k2c_tensor*)'
/root/Bela/projects/MYPROJECT/render.cpp:678: undefined reference toexample(k2c_tensor, k2c_tensor)'
/root/Bela/projects/MYPROJECT/render.cpp:679: undefined reference to example(k2c_tensor*, k2c_tensor*)'
/root/Bela/projects/MYPROJECT/render.cpp:680: undefined reference toexample(k2c_tensor, k2c_tensor)

Which to me indicates some shortcoming in my approach where the compiler isn't getting the right links. I also explored using this approach: https://github.com/gosha20777/keras2cpp but the cmake version & c++17 version requirements weren't compatible and it seems like a longer journey than keras2c.

And finally, I've now noticed that if i just run rename my render.cpp to render.c all these problems go away, but then I can't seem to use things like AudioFile:

In file ./libraries/AudioFile/AudioFile.h: 'string' file not found column: 10, line: 2

Referenced posts:
https://forum.bela.io/d/629-problem-linking/2 --> really useful for command line interactions w/ bela. i tried the "first approach" without success.
https://forum.bela.io/d/2223-compiling-with-external-libraries/2 -> more insight to the make commands
https://forum.bela.io/d/2786-using-tflite-within-a-bela-project/3 -> tflite setup information

    dakqyn it's due to linking issues (or maybe some c vs. c++ thing?

    It could be that: when including a C-only file into a C++ file, you need to enclose it in an extern "C" statement, e.g.:

    extern "C" {
    #include "myfile.h"
    }

    That was exactly it, worked like a charm!

    Sadly trying to predict inside the render block seems to crash my program - it processes a sample in setup just fine

    void render(BelaContext *context, void *userData)
    {
        for(unsigned int n = 0; n < context->audioFrames; n++) {
            if(++gReadPointer > gSampleBuffer.size())
                gReadPointer = 0;
    	float out = gSampleBuffer[gReadPointer];
    	// iterate window
    	for (int i = 0; i < 7; i++) {
    	    model_in[i] = model_in[i+1];
    	}
            model_in[7] = out;
    	model(&model_in_tensor, &model_out_tensor);
    	float pred = model_out[0];
            audioWrite(context, n, 0, pred);
            audioWrite(context, n, 1, pred);
        }
    }

    This code is using a model that takes in an array of 8 floats, returns 1 so I'm trying to process the audio as a sliding window and then run the inference method:

    #include <Bela.h>
    #include <vector>
    #include <iostream>
    #include <libraries/AudioFile/AudioFile.h>
    #include "include/k2c_include.h" 
    #include "models/model.h" 
    
    std::string gFilename = "guitar.wav";
    std::vector<float> gSampleBuffer;
    int gReadPointer = 0;
    
    // ml model
    float model_in[8] = {0, 0, 0, 0, 0, 0, 0, 0,}; 
    float model_out[1] = {0};
    k2c_tensor model_in_tensor = {model_in,2,8,{8,1,1,1,1}};
    k2c_tensor model_out_tensor = {model_out,1,1,{1,1,1,1,1}}; 
    
    bool setup(BelaContext *context, void *userData)
    {
    	ml(&model_in_tensor, &model_out_tensor);
    	std::cout << model_out[0] << std::endl;
    	
    	// load buffer
    	gSampleBuffer = AudioFileUtilities::loadMono(gFilename);
    	return true;
    }
    
    void render(BelaContext *context, void *userData)
    {
        for(unsigned int n = 0; n < context->audioFrames; n++) {
            if(++gReadPointer > gSampleBuffer.size())
                gReadPointer = 0;
    
    		float out = gSampleBuffer[gReadPointer];
    		
    		// iterate sliding window
    		for (int i = 0; i < 7; i++) {
    		    model_in[i] = model_in[i+1];
    		}
    		model_in[7] = out;
    
    		model(&model_in_tensor, &model_out_tensor);
    		
    		float pred = model_out[0];
    
    		audioWrite(context, n, 0, pred);
    		audioWrite(context, n, 1, pred);
        }
    }

    Note: I wrapped the header files themselves with:

    #ifdef __cplusplus
    extern "C" {  
    #endif 

    The error is that it starts to run and then the audio crackles and then it shuts down:

    Running project ...
    -0.0113054
    Bela has disconnected. Any changes you make will not be saved. Check your USB connection and reboot Bela.

      dakqyn Bela has disconnected. Any changes you make will not be saved. Check your USB connection and reboot Bela

      Right, that's not a build error and not an error per-se. It means that you are using a lot of CPU when the program starts. You should see the on-board LED blink at regular intervals and you should be able to stop the program and gain control of the IDE again by briefly pressing the button on the Bela cape and wait a few seconds.

      You could put the following in the "User command line arguments": --board Batch --codec-mode p=95,s=1,i=10000 . That will make the project run without processing live audio, running 10000 iterations of the audio callback and it will give you an estimate CPU usage, so you can use that as a reference of how much you need to scale back the processing if at all.

        6 days later

        Thanks Giulio, that worked great, I'm definitely using too much CPU.

        The library that compiles my model outputs:

        Average time over 10 tests: 4.469000e-04 s

        which my back of the envelope math indicated would be fast enough to be called repeatedly in the render function (256 block size & 22050khz). I initialize/terminate the model in setup/cleanup which I expect allocates memory. Still getting CPU time limit exceeded, but will keep investigating!

        how often are you running your model in the render() function?

        giuliomoro You could put the following in the "User command line arguments": --board Batch --codec-mode p=95,s=1,i=10000 .

        what do you get when running this?

        a year later

        I seem to have a similar issue: I'm trying to link two old C-files and I get the same error message, also after adding the extern "C" {...} These are my files: render.cpp, arm.c, drive.cpp, hci.c, arm.h, drive.h, hci.h
        The error messages appear at compile time and say that one of the C-files hci.c has undefined references to some functions. Those functions are defined in the CPP-file drive.cpp. I have the #include "drive.h" at the top of the file hci.c I have previously used these files in an Arduino setup but then I had resaved them as CPP. I guess I could try that, but it is nice to leave them exactly as they were written, in 1996(!) Thanks!

          m3rten Those functions are defined in the CPP-file drive.cpp.

          In drive.cpp are you doing:

          extern "C" {
          #include "drive.h"
          };

          ?

          That solved the problem! Thanks a lot!