Problem compiling simpleFX_Analog.dsp

DDigital-Larry · May 29, 2019

https://github.com/BelaPlatform/Bela/wiki/Compiling-Faust-code-for-Bela

It looks like I got distcc working!

However, the linker step fails. I suspect something amiss with the Faust http library.

gary@audio-workstation:~/faust/examples/bela$ faust2bela -tobela -gui simpleFX_Analog.dsp
Send to bela
/home/gary/faust/examples/bela/simpleFX_Analog
Start communication with bela
/home/gary/faust/examples/bela/simpleFX_Analog
simpleFX_Analog
Checking the board is up and running at root@192.168.7.2...done
Stop running process...
Run bela now
uploadBuildRun START
Copying new source files to BeagleBone...Using rsync...
   render.cpp
Automatically detected PROJECT_TYPE: cpp 
Running on __stretch__ with Xenomai __3__
systemctl stop bela_startup || true
PID=`grep bela-audio  /proc/xenomai/sched/stat | cut -d " " -f 5 | sed s/\s//g`; if [ -z $PID ]; then [ true = true ] || echo "No process to kill"; else [  true = true  ] || echo "Killing old Bela process $PID"; kill -2 $PID; sleep 0.2; kill -9 $PID 2> /dev/null; fi; screen -X -S Bela quit > /dev/null; exit 0;
killall scsynth 2>/dev/null& killall sclang 2>/dev/null& true
echo 'Building render.cpp...'
Building render.cpp...
/usr/bin/clang++  -I/root/Bela/projects/simpleFX_Analog -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/simpleFX_Analog/build/render.d" -o "/root/Bela/projects/simpleFX_Analog/build/render.o" "/root/Bela/projects/simpleFX_Analog/render.cpp" 
echo ' ...done'
 ...done
echo ' '
 
echo 'Linking...'
Linking...
/usr/bin/clang++  -Llib/  -pthread -o "/root/Bela/projects/simpleFX_Analog/simpleFX_Analog" build/core/FormatConvert.o build/core/OscillatorBank_routines.o build/core/math_runfast.o build/core/Gpio.o build/core/I2c_Codec.o build/core/PulseIn.o build/core/scope_ws.o build/core/RTAudio.o build/core/UdpClient.o build/core/WriteFile.o build/core/RTAudioCommandLine.o build/core/OSCClient.o build/core/WriteFile_c.o build/core/AuxTaskRT.o build/core/board_detect.o build/core/AuxTaskNonRT.o build/core/Midi.o build/core/AuxiliaryTasks.o build/core/I2c_TouchKey.o build/core/Midi_c.o build/core/Scope.o build/core/PruBinary.o build/core/PRU.o build/core/UdpServer.o build/core/OSCServer.o build/core/GPIOcontrol.o build/core/Spi_Codec.o build/core/JSONValue.o build/core/DigitalChannelManager.o build/core/JSON.o ./build/core/default_main.o    /root/Bela/projects/simpleFX_Analog/build/render.o -lHTTPDFaust -Wl,--no-as-needed -L/usr/xenomai/lib -lcobalt -lmodechk -lpthread -lrt -lprussdrv -lstdc++  -Wl,--no-as-needed -L/usr/xenomai/lib -lcobalt -lmodechk -lpthread -lrt -lasound -lseasocks -lNE10 -lmathneon  -lsndfile 
//usr/local/lib/libHTTPDFaust.a(HTTPDSetup.cpp.o): In function `httpdfaust::HTTPDSetup::~HTTPDSetup()':
/tmp/distcc/distccd_430b6057.ii:(.text+0x34): undefined reference to `MHD_stop_daemon'
//usr/local/lib/libHTTPDFaust.a(HTTPDSetup.cpp.o): In function `httpdfaust::HTTPDSetup::stop()':
/tmp/distcc/distccd_430b6057.ii:(.text+0xa4): undefined reference to `MHD_stop_daemon'
//usr/local/lib/libHTTPDFaust.a(HTTPDSetup.cpp.o): In function `httpdfaust::HTTPDSetup::~HTTPDSetup()':
/tmp/distcc/distccd_430b6057.ii:(.text+0x118): undefined reference to `MHD_stop_daemon'
//usr/local/lib/libHTTPDFaust.a(HTTPDServer.cpp.o): In function `httpdfaust::HTTPDServer::~HTTPDServer()':
/tmp/distcc/distccd_502b6055.ii:(.text+0x48): undefined reference to `MHD_stop_daemon'
//usr/local/lib/libHTTPDFaust.a(HTTPDServer.cpp.o): In function `httpdfaust::HTTPDServer::~HTTPDServer()':
/tmp/distcc/distccd_502b6055.ii:(.text+0x8c): undefined reference to `MHD_stop_daemon'
//usr/local/lib/libHTTPDFaust.a(HTTPDServer.cpp.o): In function `httpdfaust::HTTPDServer::start(int)':
/tmp/distcc/distccd_502b6055.ii:(.text+0xd8): undefined reference to `MHD_start_daemon'
//usr/local/lib/libHTTPDFaust.a(HTTPDServer.cpp.o): In function `httpdfaust::HTTPDServer::send(MHD_Connection*, char const*, char const*, int)':
/tmp/distcc/distccd_502b6055.ii:(.text+0x130): undefined reference to `MHD_create_response_from_buffer'
/tmp/distcc/distccd_502b6055.ii:(.text+0x15c): undefined reference to `MHD_add_response_header'
/tmp/distcc/distccd_502b6055.ii:(.text+0x174): undefined reference to `MHD_add_response_header'
/tmp/distcc/distccd_502b6055.ii:(.text+0x184): undefined reference to `MHD_queue_response'
/tmp/distcc/distccd_502b6055.ii:(.text+0x190): undefined reference to `MHD_destroy_response'
//usr/local/lib/libHTTPDFaust.a(HTTPDServer.cpp.o): In function `httpdfaust::HTTPDServer::page(MHD_Connection*, char const*)':
/tmp/distcc/distccd_502b6055.ii:(.text+0x3e8): undefined reference to `MHD_create_response_from_fd'
/tmp/distcc/distccd_502b6055.ii:(.text+0x414): undefined reference to `MHD_add_response_header'
/tmp/distcc/distccd_502b6055.ii:(.text+0x42c): undefined reference to `MHD_add_response_header'
/tmp/distcc/distccd_502b6055.ii:(.text+0x43c): undefined reference to `MHD_queue_response'
/tmp/distcc/distccd_502b6055.ii:(.text+0x448): undefined reference to `MHD_destroy_response'
//usr/local/lib/libHTTPDFaust.a(HTTPDServer.cpp.o): In function `httpdfaust::HTTPDServer::answer(MHD_Connection*, char const*, char const*, char const*, char const*, unsigned int*, void**)':
/tmp/distcc/distccd_502b6055.ii:(.text+0xc30): undefined reference to `MHD_get_connection_values'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:479: recipe for target '/root/Bela/projects/simpleFX_Analog/simpleFX_Analog' failed
make: *** [/root/Bela/projects/simpleFX_Analog/simpleFX_Analog] Error 1
Connection to 192.168.7.2 closed.
gary@audio-workstation:~/faust/examples/bela$

#================================
The ZIP file contains only one file:

gary@audio-workstation:~/faust/examples/bela$ unzip libHTTPDFaust.zip 
Archive:  libHTTPDFaust.zip
  inflating: libHTTPDFaust.a         
gary@audio-workstation:~/faust/examples/bela$

This does not agree with the directions given. They hint that there should be some .so* files also:

libHTTPDFaust.so*

The libHTTPDFaust.a file is in the /usr/local/lib folder on Bela.

root@bela:~/Bela/resources/network# ls -l /usr/local/lib
total 6476
drwxr-xr-x 3 root root     4096 Oct 15  2018 SuperCollider
-rw-r--r-- 1 root root   657816 May 26 20:37 libHTTPDFaust.a
-rwxr-xr-x 1 root root    95796 Oct 15  2018 libfdt-1.4.4.so
-rw-r--r-- 1 root root   129020 Oct 15  2018 libfdt.a
lrwxrwxrwx 1 root root       11 Oct 15  2018 libfdt.so -> libfdt.so.1
lrwxrwxrwx 1 root root       15 Oct 15  2018 libfdt.so.1 -> libfdt-1.4.4.so
lrwxrwxrwx 1 root root       10 Oct  9  2018 libpd.so -> libpd.so.1
-rwxr-xr-x 1 root root   599912 Oct  9  2018 libpd.so.1
-rw-r--r-- 1 root staff   11550 Oct 15  2018 libprussdrv.a
-rw-r--r-- 1 root staff   14696 Oct 15  2018 libprussdrv.so
-rw-r--r-- 1 root staff   28866 Oct 15  2018 libprussdrvd.a
-rw-r--r-- 1 root staff   29800 Oct 15  2018 libprussdrvd.so
-rw-r--r-- 1 root staff 3777630 Oct 15  2018 libseasocks.a
lrwxrwxrwx 1 root staff      20 Oct 15  2018 libseasocks.so -> libseasocks.so.1.3.2
-rw-r--r-- 1 root staff 1249404 Oct 15  2018 libseasocks.so.1.3.2
drwxrwsr-x 4 root staff    4096 Sep 27  2018 python2.7
drwxrwsr-x 3 root staff    4096 Oct 15  2018 python3.5
root@bela:~/Bela/resources/network#

#==================
I tried again without the -gui option and the program appears to be running on Bela.

root@bela:/usr/local/lib# ps ax | grep -i simple
15634 pts/2    Ss+    0:00 make --no-print-directory QUIET=true -C /root/Bela PROJECT=simpleFX_Analog CL= LDLIBS= CPPFLAGS= AT= run
15670 pts/2    S+     0:00 /bin/sh -c sync& cd /root/Bela/projects/simpleFX_Analog && /root/Bela/projects/simpleFX_Analog/simpleFX_Analog 
15672 pts/2    SLl+   0:00 /root/Bela/projects/simpleFX_Analog/simpleFX_Analog
15684 pts/0    S+     0:00 grep --color=auto -i simple
root@bela:/usr/local/lib#

#==================================
Sorry I don't have any audio plugged in at the moment to actually try it out 8⁽⁽

I am highly motivated to help solve this issue as using something like Faust is the only way I am going write effects code. I can't operate at the C++ level. Not that I don't understand C++ but it is so far below the conceptual level that Faust allows it's not worth spending time on it.

==============

ok I see many references to MHD_stop_daemon for example, what is that I wonder, well I found this, next question is how do I integrate it with this project?

https://www.gnu.org/software/libmicrohttpd/manual/libmicrohttpd.html

=========

installing libmicrohttpd directly on Bela.

wget https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-latest.tar.gz
tar xvzf libmicrohttpd-latest.tar.gz 
cd libmicrohttpd-0.9.63/
./configure 
make
sudo make install

ls -l /usr/local/lib | grep http

root@bela:~/Downloads/libmicrohttpd-0.9.63# ls -l /usr/local/lib | grep http
-rw-r--r-- 1 root root   639692 May 29 05:21 libmicrohttpd.a
-rwxr-xr-x 1 root root      997 May 29 05:21 libmicrohttpd.la
lrwxrwxrwx 1 root root       24 May 29 05:21 libmicrohttpd.so -> libmicrohttpd.so.12.50.0
lrwxrwxrwx 1 root root       24 May 29 05:21 libmicrohttpd.so.12 -> libmicrohttpd.so.12.50.0
-rwxr-xr-x 1 root root   389772 May 29 05:21 libmicrohttpd.so.12.50.0
root@bela:~/Downloads/libmicrohttpd-0.9.63#

Build still fails with lots of this kind of thing:

/tmp/distcc/distccd_502b6055.ii:(.text+0x3e8): undefined reference to `MHD_create_response_from_fd'
/tmp/distcc/distccd_502b6055.ii:(.text+0x414): undefined reference to `MHD_add_response_header'
/tmp/distcc/distccd_502b6055.ii:(.text+0x42c): undefined reference to `MHD_add_response_header'
/tmp/distcc/distccd_502b6055.ii:(.text+0x43c): undefined reference to `MHD_queue_response'
/tmp/distcc/distccd_502b6055.ii:(.text+0x448): undefined reference to `MHD_destroy_response'

Now I just need to try to figure out how to link that library in...

DDigital-Larry · May 29, 2019

Next question, when I ran the faust2bela script while watching ~/distcc.log, nothing is appended to the log file (this is on the host of course). The file is actually compiled and running on the Bela though. What do I need to do to enable the distcc to run when I run faust2bela -tobela also from the host?

giuliomoro · May 29, 2019

PLEASE format your posts properly.
Enclose blocks of code or terminal logs with ``` , .e.g:
```
/tmp/distcc/distccd_502b6055.ii.text+0x3e8): undefined reference to MHD_create_response_from_fd' /tmp/distcc/distccd_502b6055.ii:(.text+0x414): undefined reference toMHD_add_response_header'
/tmp/distcc/distccd_502b6055.ii.text+0x42c): undefined reference to MHD_add_response_header'
```

so that it gets visualized as

/tmp/distcc/distccd_502b6055.ii:(.text+0x3e8): undefined reference to `MHD_create_response_from_fd'
/tmp/distcc/distccd_502b6055.ii:(.text+0x414): undefined reference to `MHD_add_response_header'
/tmp/distcc/distccd_502b6055.ii:(.text+0x42c): undefined reference to `MHD_add_response_header'

I have been editing your posts to be able to read them.

Digital-Larry This does not agree with the directions given. They hint that there should be some .so* files also:

libHTTPDFaust.so*

some files are missing there, I will fix that shortly.

Digital-Larry ok I see many references to MHD_stop_daemon for example, what is that I wonder, well I found this, next question is how do I integrate it with this project?

https://www.gnu.org/software/libmicrohttpd/manual/libmicrohttpd.html

The instructions from here actually say:

Install libmicrohttpd-dev on the board:
connect the board to the internet and run
apt-get install libmicrohttpd-dev

you may have skipped that?

Digital-Larry ext question, when I ran the faust2bela script while watching ~/distcc.log, nothing is appended to the log file (this is on the host of course). The file is actually compiled and running on the Bela though. What do I need to do to enable the distcc to run when I run faust2bela -tobela also from the host?

You need to enable it to build all Bela projects. It is described in the distcc-on-bela documentation:

Using distcc by default on Bela
Make the following edit to the Bela Makefile:

 endif

 ifeq ($(COMPILER), clang)
-  CLANG_PATH?=/usr/bin/clang
+ CLANG_PATH=distcc-clang
   CC=$(CLANG_PATH)
   CXX=$(CLANG_PATH)++
   DEFAULT_CPPFLAGS += -DNDEBUG # Maybe we should add back in -no-integrated-as?

have you done these steps yet? It doesn't look like it, because your linker line starts with /usr/bin/clang++ instead of distcc-clang+

giuliomoro · May 29, 2019

giuliomoro some files are missing there, I will fix that shortly.

now fixed.

DDigital-Larry · May 29, 2019

giuliomoro Thank you!!!! Sorry about the bad formatting, it wasn't obvious what I should be doing, but I suppose I should be aware this is a markdown environment. My building of the httplib locally was probably unnecessary as I had previously used the apt-get step. Using apt-get gives you 0.9.51 while building from source gives you a version that is a couple of years more recent, not that I think it matters.

I did not edit the Bela makefile.

I'll try this later today. I'm sick with the flu and have a pounding headache. 8⁽ But I trust that if this succeeds, I will be singing in the sunshine. 8^) with pitch shifted delays

DDigital-Larry · May 29, 2019

giuliomoro THANK YOU!!!!! The updated library files made a bunch of difference. However I had to copy them to /usr/lib as /usr/local/lib did not work. OMG I can see the GUI! I'm even using remote access software so I'm viewing the Bela attached to my Linux box downstairs on USB while I'm upstairs using Windows. Tonight I will try out some audio tests!

giuliomoro · May 29, 2019

Digital-Larry However I had to copy them to /usr/lib as /usr/local/lib did not work.

hmmm not sure why that would be the case. Possibly it just needed running ldconfig ... Or possibly, the linker would find the file libHTTPDFaust.a you previously had in /usr/local/lib and looked no further? Not sure.

DDigital-Larry · May 30, 2019

giuliomoro let me know if you would like me to test any theories you may have

giuliomoro · May 30, 2019

not sure really. However I have an explanation as to why it worked with the shared library (.so) and not with the static (.a) one:

a static library is simply an archive of object files, while a shared library - among other things - has been linked with all the libraries it depends upon. So when using the static library, in order for that to work, you need to explicitly pass to the linker the library(ies) that your your static library depends upon (in this case: -lmicrohttpd), so that the external symbols in the objects files that make up the .a file can be resolved. When using the shared library, instead, these symbols have been already resolved when it was linked together in a .so file, so there is no need to explicitly state it again.

DDigital-Larry · Jun 1, 2019

Still having trouble at this step.

Using distcc by default on Bela
Make the following edit to the Bela Makefile:

By "the Bela Makefile" I presume that you mean the file /root/Bela/Makefile. Sorry, it wasn't obvious to me.

I found the lines and made the change.

echo 'Building render.cpp...'
Building render.cpp...
distcc-clang++  -I/root/Bela/projects/delay01 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/delay01/build/render.d" -o "/root/Bela/projects/delay01/build/render.o" "/root/Bela/projects/delay01/render.cpp" 
distcc[2855] ERROR: nonblocking connect to 192.168.7.1:3632 failed: Connection refused
distcc[2855] (dcc_build_somewhere) Warning: failed to distribute and fallbacks are disabled
Makefile:434: recipe for target '/root/Bela/projects/delay01/build/render.o' failed
make: *** [/root/Bela/projects/delay01/build/render.o] Error 103
Connection to 192.168.7.2 closed.
gary@audio-workstation:~/faust/examples/bela$

DDigital-Larry · Jun 1, 2019

OK the distccd wasn't running. Now that I started that, I get:

distccd[14751] (dcc_standalone_server) not detaching
distccd[14751] (dcc_new_pgrp) already a process group leader
distccd[14751] (dcc_log_daemon_started) preforking daemon started (3.3.2 x86_64-pc-linux-gnu, built May 26 2019 14:11:45)
distccd[14751] (dcc_create_kids) up to 1 children
distccd[14751] (dcc_create_kids) up to 2 children
distccd[14751] (dcc_create_kids) up to 3 children
distccd[14751] (dcc_create_kids) up to 4 children
distccd[14751] (dcc_create_kids) up to 5 children
distccd[14751] (dcc_create_kids) up to 6 children
distccd[14752] (dcc_check_client) connection from 192.168.7.2:59746
distccd[14752] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
distccd[14752] (dcc_r_token_int) got DIST00000001
distccd[14752] (dcc_r_token_int) got ARGC00000013
distccd[14752] (dcc_r_argv) reading 19 arguments from job submission
distccd[14752] (dcc_r_token_int) got ARGV0000000f
distccd[14752] (dcc_r_token_string) got 'clang++-3.9-arm'
distccd[14752] (dcc_r_argv) argv[0] = "clang++-3.9-arm"
distccd[14752] (dcc_r_token_int) got ARGV0000000e
distccd[14752] (dcc_r_token_string) got '-march=armv7-a'
distccd[14752] (dcc_r_argv) argv[1] = "-march=armv7-a"
distccd[14752] (dcc_r_token_int) got ARGV0000000a
distccd[14752] (dcc_r_token_string) got '-mfpu=vfp3'
distccd[14752] (dcc_r_argv) argv[2] = "-mfpu=vfp3"
distccd[14752] (dcc_r_token_int) got ARGV0000001c
distccd[14752] (dcc_r_token_string) got '-fasynchronous-unwind-tables'
distccd[14752] (dcc_r_argv) argv[3] = "-fasynchronous-unwind-tables"
distccd[14752] (dcc_r_token_int) got ARGV00000003
distccd[14752] (dcc_r_token_string) got '-O3'
distccd[14752] (dcc_r_argv) argv[4] = "-O3"
distccd[14752] (dcc_r_token_int) got ARGV0000000e
distccd[14752] (dcc_r_token_string) got '-march=armv7-a'
distccd[14752] (dcc_r_argv) argv[5] = "-march=armv7-a"
distccd[14752] (dcc_r_token_int) got ARGV00000010
distccd[14752] (dcc_r_token_string) got '-mtune=cortex-a8'
distccd[14752] (dcc_r_argv) argv[6] = "-mtune=cortex-a8"
distccd[14752] (dcc_r_token_int) got ARGV00000010
distccd[14752] (dcc_r_token_string) got '-mfloat-abi=hard'
distccd[14752] (dcc_r_argv) argv[7] = "-mfloat-abi=hard"
distccd[14752] (dcc_r_token_int) got ARGV0000000a
distccd[14752] (dcc_r_token_string) got '-mfpu=neon'
distccd[14752] (dcc_r_argv) argv[8] = "-mfpu=neon"
distccd[14752] (dcc_r_token_int) got ARGV00000010
distccd[14752] (dcc_r_token_string) got '-ftree-vectorize'
distccd[14752] (dcc_r_argv) argv[9] = "-ftree-vectorize"
distccd[14752] (dcc_r_token_int) got ARGV0000000b
distccd[14752] (dcc_r_token_string) got '-ffast-math'
distccd[14752] (dcc_r_argv) argv[10] = "-ffast-math"
distccd[14752] (dcc_r_token_int) got ARGV0000000a
distccd[14752] (dcc_r_token_string) got '-std=c++11'
distccd[14752] (dcc_r_argv) argv[11] = "-std=c++11"
distccd[14752] (dcc_r_token_int) got ARGV0000000c
distccd[14752] (dcc_r_token_string) got '-Wno-varargs'
distccd[14752] (dcc_r_argv) argv[12] = "-Wno-varargs"
distccd[14752] (dcc_r_token_int) got ARGV00000005
distccd[14752] (dcc_r_token_string) got '-Wall'
distccd[14752] (dcc_r_argv) argv[13] = "-Wall"
distccd[14752] (dcc_r_token_int) got ARGV00000002
distccd[14752] (dcc_r_token_string) got '-c'
distccd[14752] (dcc_r_argv) argv[14] = "-c"
distccd[14752] (dcc_r_token_int) got ARGV00000012
distccd[14752] (dcc_r_token_string) got '-fmessage-length=0'
distccd[14752] (dcc_r_argv) argv[15] = "-fmessage-length=0"
distccd[14752] (dcc_r_token_int) got ARGV00000002
distccd[14752] (dcc_r_token_string) got '-o'
distccd[14752] (dcc_r_argv) argv[16] = "-o"
distccd[14752] (dcc_r_token_int) got ARGV0000002a
distccd[14752] (dcc_r_token_string) got '/root/Bela/projects/delay01/build/render.o'
distccd[14752] (dcc_r_argv) argv[17] = "/root/Bela/projects/delay01/build/render.o"
distccd[14752] (dcc_r_token_int) got ARGV00000026
distccd[14752] (dcc_r_token_string) got '/root/Bela/projects/delay01/render.cpp'
distccd[14752] (dcc_r_argv) argv[18] = "/root/Bela/projects/delay01/render.cpp"
distccd[14752] (dcc_r_argv) got arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/delay01/build/render.o /root/Bela/projects/delay01/render.cpp
distccd[14752] (dcc_scan_args) scanning arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/delay01/build/render.o /root/Bela/projects/delay01/render.cpp
distccd[14752] (dcc_scan_args) found object/output file "/root/Bela/projects/delay01/build/render.o"
distccd[14752] (dcc_scan_args) found input file "/root/Bela/projects/delay01/render.cpp"
distccd[14752] compile from render.cpp to render.o
distccd[14752] (dcc_run_job) output file /root/Bela/projects/delay01/build/render.o
distccd[14752] (dcc_input_tmpnam) input file /root/Bela/projects/delay01/render.cpp
distccd[14752] (dcc_r_token_int) got DOTI0018d9d3
distccd[14752] (dcc_r_file) received 1628627 bytes to file /tmp/distccd_1db50d83.ii
distccd[14752] (dcc_r_file_timed) 1628627 bytes received in 0.146217s, rate 10877kB/s
distccd[14752] (dcc_set_input) changed input from "/root/Bela/projects/delay01/render.cpp" to "/tmp/distccd_1db50d83.ii"
distccd[14752] (dcc_set_input) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/delay01/build/render.o /tmp/distccd_1db50d83.ii
distccd[14752] (dcc_set_output) changed output from "/root/Bela/projects/delay01/build/render.o" to "/tmp/distccd_1dc50d83.o"
distccd[14752] (dcc_set_output) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_1dc50d83.o /tmp/distccd_1db50d83.ii
distccd[14752] (dcc_check_compiler_masq) /usr/local/bin/clang++-3.9-arm is not a symlink
distccd[14752] (dcc_spawn_child) forking to execute: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_1dc50d83.o /tmp/distccd_1db50d83.ii
distccd[14752] (dcc_spawn_child) child started as pid14971
distccd[14971] (dcc_new_pgrp) entered process group
distccd[14971] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[14752] (dcc_collect_child) cc child 14971 terminated with status 0x100
distccd[14752] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[14752] (dcc_x_token_int) send DONE00000001
distccd[14752] (dcc_x_token_int) send STAT00000100
distccd[14752] (dcc_x_file) send 248 byte file /tmp/distcc_161f0d83.stderr with token SERR and compression 69
distccd[14752] (dcc_x_token_int) send SERR000000f8
distccd[14752] (dcc_x_file) send 0 byte file /tmp/distcc_16d70d83.stdout with token SOUT and compression 69
distccd[14752] (dcc_x_token_int) send SOUT00000000
distccd[14752] (dcc_x_token_int) send DOTO00000000
distccd[14752] clang++-3.9-arm /root/Bela/projects/delay01/render.cpp on localhost failed
distccd[14752] job complete
distccd[14752] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[14752] (dcc_job_summary) client: 192.168.7.2:59746 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:988ms clang++-3.9-arm /root/Bela/projects/delay01/render.cpp

in the distcc log.

On host console I get:

echo 'Building render.cpp...'
Building render.cpp...
distcc-clang++  -I/root/Bela/projects/delay01 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/delay01/build/render.d" -o "/root/Bela/projects/delay01/build/render.o" "/root/Bela/projects/delay01/render.cpp" 
distcc[3031] ERROR: compile /root/Bela/projects/delay01/render.cpp on 192.168.7.1 failed
distcc[3031] (dcc_build_somewhere) Warning: remote compilation of '/root/Bela/projects/delay01/render.cpp' failed, retrying locally
distcc[3031] (dcc_build_somewhere) Warning: failed to distribute and fallbacks are disabled
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:434: recipe for target '/root/Bela/projects/delay01/build/render.o' failed
make: *** [/root/Bela/projects/delay01/build/render.o] Error 1
Connection to 192.168.7.2 closed.
gary@audio-workstation:~/faust/examples/bela$

DDigital-Larry · Jun 1, 2019

Think you were talking about this before, but not sure I understood.

ary@audio-workstation:~/faust/examples/bela$ cat /usr/local/bin/*arm
#!/bin/bash
/usr/bin/clang -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $@
#!/bin/bash
/usr/bin/clang++ -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $0
gary@audio-workstation:~/faust/examples/bela$

These don't currently point to clang-3.9. They just point to clang which is actually version 6. I think you were saying that it has to be version 3.9?

However, this step works:

distcc-clang -c ~/Bela/resources/network/udp-client.c

DDigital-Larry · Jun 1, 2019

gary@audio-workstation:~$ ld -V
GNU ld (GNU Binutils for Ubuntu) 2.30
  Supported emulations:
   elf_x86_64
   elf32_x86_64
   elf_i386
   elf_iamcu
   i386linux
   elf_l1om
   elf_k1om
   i386pep
   i386pe
gary@audio-workstation:~$

giuliomoro · Jun 1, 2019

Digital-Larry By "the Bela Makefile" I presume that you mean the file /root/Bela/Makefile.

yes

Digital-Larry On host console I get:

I guess that's technically the "client" console (i.e.: what you see there is while you are logged in on Bela).

So it seems that the compiler is failing, and my guess is that it is because it is being passed -march=armv7-a but no -target armv7l-unknown-linux-gnueabihf, and this - I guess - is because you are not calling the wrapper but you are calling the vanilla compiler directly (through the symlink you created).
The point of the wrapper is exactly to automatically add -target armv7l-unknown-linux-gnueabihf to all calls.

I think what is happening is what I mentioned in the other thread:

giuliomoro But if you do put some symlinks there, they should be pointing NOT to the regular clang, but to the clang-3.9-arm and clang++-3.9-arm files you created in /usr/local/bin/, and they will eventually call clang with the appropriate option -target armv7l-unknown-linux-gnueabihf, to tell it to build code for ARM.

Digital-Larry gary@audio-workstation:/usr/local/lib$ ls -l distcc/
total 0
lrwxrwxrwx 1 root root 14 May 28 17:45 clang-3.9-arm -> /usr/bin/clang
lrwxrwxrwx 1 root root 16 May 28 17:45 clang++-3.9-arm -> /usr/bin/clang++

Is this working? I find it surprising, but I guess it may depend on how you clang has been built. This way you are simply using the regular clang on your computer, but it should need a -target armv7l-unknown-linux-gnueabihf option in order to know it should build for arm (at least, it does with mine, which I installed with macports on Mac).

basically, remove your current symlinks from /usr/local/lib/distcc/ and make some new ones that point NOT to the vanilla clang/clang++, but to the wrapper clang-3.9-arm/clang++-3.9-arm you created:

sudo rm /usr/local/lib/distcc/*
sudo ln -s /usr/local/bin/clang-3.9-arm/ /usr/local/lib/distcc/
sudo ln -s /usr/local/bin/clang++-3.9-arm/ /usr/local/lib/distcc/

Digital-Larry However, this step works:

distcc-clang -c ~/Bela/resources/network/udp-client.c

I see now how that can easily give false positives: it just checks that the server can compile, but does not check the nature of the resulted file. If you were to do file a.out, it would probably say it's an x86 file and not an ARM one.
So the test should probably be amended to :

distcc-clang -c ~/Bela/resources/network/udp-client.c -march=armv7-a

This should then fail similarly to the how the Bela project is failing in your current configuration, and should be fixed with the updated symlinks.

DDigital-Larry · Jun 1, 2019

giuliomoro thanks for your continued help... will check all this!

DDigital-Larry · Jun 2, 2019

OK let's see where we are now.

root@bela:~/Bela# grep distcc Makefile 
  CLANG_PATH=distcc-clang
root@bela:~/Bela#

reference command line build on bela works (without specifying architecture):

root@bela:~/Bela/resources/network# distcc-clang -v -c udp-client.c
clang version 3.9.1-9 (tags/RELEASE_391/rc2)
Target: armv7l-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6
Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6
Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6.3.0
Selected GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
Candidate multilib: .;@m32
Selected multilib: .;@m32
 "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name udp-client.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -fuse-init-array -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include -internal-externc-isystem /usr/include/arm-linux-gnueabihf -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /root/Bela/resources/network -ferror-limit 19 -fmessage-length 80 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o - -x c udp-client.c
clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target armv7l-unknown-linux-gnueabihf
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include
 /usr/include/arm-linux-gnueabihf
 /usr/include
End of search list.
clang version 3.9.1-19ubuntu1 (tags/RELEASE_391/rc2)
Target: armv7l-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/bin
 "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name distccd_ba396596.i -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -coverage-file /tmp/distccd_ba666596.o -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 0 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/distccd_ba666596.o -x cpp-output /tmp/distccd_ba396596.i
clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target x86_64-pc-linux-gnu
#include "..." search starts here:
End of search list.
root@bela:~/Bela/resources/network#

reference build works with specifying architecture:

root@bela:~/Bela/resources/network# distcc-clang -c -v ~/Bela/resources/network/udp-client.c -march=armv7-a
clang version 3.9.1-9 (tags/RELEASE_391/rc2)
Target: armv7l-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6
Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6
Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6.3.0
Selected GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
Candidate multilib: .;@m32
Selected multilib: .;@m32
 "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name udp-client.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -fuse-init-array -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include -internal-externc-isystem /usr/include/arm-linux-gnueabihf -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /root/Bela/resources/network -ferror-limit 19 -fmessage-length 80 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o - -x c /root/Bela/resources/network/udp-client.c
clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target armv7l-unknown-linux-gnueabihf
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include
 /usr/include/arm-linux-gnueabihf
 /usr/include
End of search list.
clang version 3.9.1-19ubuntu1 (tags/RELEASE_391/rc2)
Target: armv7l-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/bin
 "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name distccd_ff4c663b.i -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -coverage-file /tmp/distccd_fcec663b.o -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 0 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/distccd_fcec663b.o -x cpp-output /tmp/distccd_ff4c663b.i
clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target x86_64-pc-linux-gnu
#include "..." search starts here:
End of search list.
root@bela:~/Bela/resources/network#

Here's the distcclog from the Ubuntu machine:

distccd[14752] (dcc_check_client) connection from 192.168.7.2:59770
distccd[14752] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
distccd[14752] (dcc_r_token_int) got DIST00000001
distccd[14752] (dcc_r_token_int) got ARGC00000006
distccd[14752] (dcc_r_argv) reading 6 arguments from job submission
distccd[14752] (dcc_r_token_int) got ARGV0000000d
distccd[14752] (dcc_r_token_string) got 'clang-3.9-arm'
distccd[14752] (dcc_r_argv) argv[0] = "clang-3.9-arm"
distccd[14752] (dcc_r_token_int) got ARGV00000002
distccd[14752] (dcc_r_token_string) got '-v'
distccd[14752] (dcc_r_argv) argv[1] = "-v"
distccd[14752] (dcc_r_token_int) got ARGV00000002
distccd[14752] (dcc_r_token_string) got '-c'
distccd[14752] (dcc_r_argv) argv[2] = "-c"
distccd[14752] (dcc_r_token_int) got ARGV0000000c
distccd[14752] (dcc_r_token_string) got 'udp-client.c'
distccd[14752] (dcc_r_argv) argv[3] = "udp-client.c"
distccd[14752] (dcc_r_token_int) got ARGV00000002
distccd[14752] (dcc_r_token_string) got '-o'
distccd[14752] (dcc_r_argv) argv[4] = "-o"
distccd[14752] (dcc_r_token_int) got ARGV0000000c
distccd[14752] (dcc_r_token_string) got 'udp-client.o'
distccd[14752] (dcc_r_argv) argv[5] = "udp-client.o"
distccd[14752] (dcc_r_argv) got arguments: clang-3.9-arm -v -c udp-client.c -o udp-client.o
distccd[14752] (dcc_scan_args) scanning arguments: clang-3.9-arm -v -c udp-client.c -o udp-client.o
distccd[14752] (dcc_scan_args) found input file "udp-client.c"
distccd[14752] (dcc_scan_args) found object/output file "udp-client.o"
distccd[14752] compile from udp-client.c to udp-client.o
distccd[14752] (dcc_run_job) output file udp-client.o
distccd[14752] (dcc_input_tmpnam) input file udp-client.c
distccd[14752] (dcc_r_token_int) got DOTI00016dac
distccd[14752] (dcc_r_file) received 93612 bytes to file /tmp/distccd_ba396596.i
distccd[14752] (dcc_r_file_timed) 93612 bytes received in 0.011009s, rate 8304kB/s
distccd[14752] (dcc_set_input) changed input from "udp-client.c" to "/tmp/distccd_ba396596.i"
distccd[14752] (dcc_set_input) command after: clang-3.9-arm -v -c /tmp/distccd_ba396596.i -o udp-client.o
distccd[14752] (dcc_set_output) changed output from "udp-client.o" to "/tmp/distccd_ba666596.o"
distccd[14752] (dcc_set_output) command after: clang-3.9-arm -v -c /tmp/distccd_ba396596.i -o /tmp/distccd_ba666596.o
distccd[14752] (dcc_check_compiler_masq) /usr/local/bin/clang-3.9-arm is not a symlink
distccd[14752] (dcc_spawn_child) forking to execute: clang-3.9-arm -v -c /tmp/distccd_ba396596.i -o /tmp/distccd_ba666596.o
distccd[14752] (dcc_spawn_child) child started as pid32524
distccd[32524] (dcc_new_pgrp) entered process group
distccd[32524] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[14752] (dcc_collect_child) cc child 32524 terminated with status 0
distccd[14752] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[14752] (dcc_x_token_int) send DONE00000001
distccd[14752] (dcc_x_token_int) send STAT00000000
distccd[14752] (dcc_x_file) send 1002 byte file /tmp/distcc_b8066596.stderr with token SERR and compression 69
distccd[14752] (dcc_x_token_int) send SERR000003ea
distccd[14752] (dcc_x_file) send 0 byte file /tmp/distcc_b8246596.stdout with token SOUT and compression 69
distccd[14752] (dcc_x_token_int) send SOUT00000000
distccd[14752] (dcc_x_file) send 2192 byte file /tmp/distccd_ba666596.o with token DOTO and compression 69
distccd[14752] (dcc_x_token_int) send DOTO00000890
distccd[14752] clang-3.9-arm udp-client.c on localhost completed ok
distccd[14752] job complete
distccd[14752] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[14752] (dcc_job_summary) client: 192.168.7.2:59770 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:318ms clang-3.9-arm udp-client.c
distccd[14753] (dcc_check_client) connection from 192.168.7.2:59772
distccd[14753] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
distccd[14753] (dcc_r_token_int) got DIST00000001
distccd[14753] (dcc_r_token_int) got ARGC00000006
distccd[14753] (dcc_r_argv) reading 6 arguments from job submission
distccd[14753] (dcc_r_token_int) got ARGV0000000d
distccd[14753] (dcc_r_token_string) got 'clang-3.9-arm'
distccd[14753] (dcc_r_argv) argv[0] = "clang-3.9-arm"
distccd[14753] (dcc_r_token_int) got ARGV00000002
distccd[14753] (dcc_r_token_string) got '-c'
distccd[14753] (dcc_r_argv) argv[1] = "-c"
distccd[14753] (dcc_r_token_int) got ARGV00000029
distccd[14753] (dcc_r_token_string) got '/root/Bela/resources/network/udp-client.c'
distccd[14753] (dcc_r_argv) argv[2] = "/root/Bela/resources/network/udp-client.c"
distccd[14753] (dcc_r_token_int) got ARGV0000000e
distccd[14753] (dcc_r_token_string) got '-march=armv7-a'
distccd[14753] (dcc_r_argv) argv[3] = "-march=armv7-a"
distccd[14753] (dcc_r_token_int) got ARGV00000002
distccd[14753] (dcc_r_token_string) got '-o'
distccd[14753] (dcc_r_argv) argv[4] = "-o"
distccd[14753] (dcc_r_token_int) got ARGV0000000c
distccd[14753] (dcc_r_token_string) got 'udp-client.o'
distccd[14753] (dcc_r_argv) argv[5] = "udp-client.o"
distccd[14753] (dcc_r_argv) got arguments: clang-3.9-arm -c /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
distccd[14753] (dcc_scan_args) scanning arguments: clang-3.9-arm -c /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
distccd[14753] (dcc_scan_args) found input file "/root/Bela/resources/network/udp-client.c"
distccd[14753] (dcc_scan_args) found object/output file "udp-client.o"
distccd[14753] compile from udp-client.c to udp-client.o
distccd[14753] (dcc_run_job) output file udp-client.o
distccd[14753] (dcc_input_tmpnam) input file /root/Bela/resources/network/udp-client.c
distccd[14753] (dcc_r_token_int) got DOTI00016eeb
distccd[14753] (dcc_r_file) received 93931 bytes to file /tmp/distccd_20816628.i
distccd[14753] (dcc_r_file_timed) 93931 bytes received in 0.012232s, rate 7499kB/s
distccd[14753] (dcc_set_input) changed input from "/root/Bela/resources/network/udp-client.c" to "/tmp/distccd_20816628.i"
distccd[14753] (dcc_set_input) command after: clang-3.9-arm -c /tmp/distccd_20816628.i -march=armv7-a -o udp-client.o
distccd[14753] (dcc_set_output) changed output from "udp-client.o" to "/tmp/distccd_20dc6628.o"
distccd[14753] (dcc_set_output) command after: clang-3.9-arm -c /tmp/distccd_20816628.i -march=armv7-a -o /tmp/distccd_20dc6628.o
distccd[14753] (dcc_check_compiler_masq) /usr/local/bin/clang-3.9-arm is not a symlink
distccd[14753] (dcc_spawn_child) forking to execute: clang-3.9-arm -c /tmp/distccd_20816628.i -march=armv7-a -o /tmp/distccd_20dc6628.o
distccd[14753] (dcc_spawn_child) child started as pid32547
distccd[32547] (dcc_new_pgrp) entered process group
distccd[32547] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[14753] (dcc_collect_child) cc child 32547 terminated with status 0
distccd[14753] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[14753] (dcc_x_token_int) send DONE00000001
distccd[14753] (dcc_x_token_int) send STAT00000000
distccd[14753] (dcc_x_file) send 0 byte file /tmp/distcc_27c16628.stderr with token SERR and compression 69
distccd[14753] (dcc_x_token_int) send SERR00000000
distccd[14753] (dcc_x_file) send 0 byte file /tmp/distcc_27996628.stdout with token SOUT and compression 69
distccd[14753] (dcc_x_token_int) send SOUT00000000
distccd[14753] (dcc_x_file) send 2192 byte file /tmp/distccd_20dc6628.o with token DOTO and compression 69
distccd[14753] (dcc_x_token_int) send DOTO00000890
distccd[14753] clang-3.9-arm /root/Bela/resources/network/udp-client.c on localhost completed ok
distccd[14753] job complete
distccd[14753] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[14753] (dcc_job_summary) client: 192.168.7.2:59772 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:286ms clang-3.9-arm /root/Bela/resources/network/udp-client.c
distccd[14754] (dcc_check_client) connection from 192.168.7.2:59774
distccd[14754] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
distccd[14754] (dcc_r_token_int) got DIST00000001
distccd[14754] (dcc_r_token_int) got ARGC00000007
distccd[14754] (dcc_r_argv) reading 7 arguments from job submission
distccd[14754] (dcc_r_token_int) got ARGV0000000d
distccd[14754] (dcc_r_token_string) got 'clang-3.9-arm'
distccd[14754] (dcc_r_argv) argv[0] = "clang-3.9-arm"
distccd[14754] (dcc_r_token_int) got ARGV00000002
distccd[14754] (dcc_r_token_string) got '-c'
distccd[14754] (dcc_r_argv) argv[1] = "-c"
distccd[14754] (dcc_r_token_int) got ARGV00000002
distccd[14754] (dcc_r_token_string) got '-v'
distccd[14754] (dcc_r_argv) argv[2] = "-v"
distccd[14754] (dcc_r_token_int) got ARGV00000029
distccd[14754] (dcc_r_token_string) got '/root/Bela/resources/network/udp-client.c'
distccd[14754] (dcc_r_argv) argv[3] = "/root/Bela/resources/network/udp-client.c"
distccd[14754] (dcc_r_token_int) got ARGV0000000e
distccd[14754] (dcc_r_token_string) got '-march=armv7-a'
distccd[14754] (dcc_r_argv) argv[4] = "-march=armv7-a"
distccd[14754] (dcc_r_token_int) got ARGV00000002
distccd[14754] (dcc_r_token_string) got '-o'
distccd[14754] (dcc_r_argv) argv[5] = "-o"
distccd[14754] (dcc_r_token_int) got ARGV0000000c
distccd[14754] (dcc_r_token_string) got 'udp-client.o'
distccd[14754] (dcc_r_argv) argv[6] = "udp-client.o"
distccd[14754] (dcc_r_argv) got arguments: clang-3.9-arm -c -v /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
distccd[14754] (dcc_scan_args) scanning arguments: clang-3.9-arm -c -v /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
distccd[14754] (dcc_scan_args) found input file "/root/Bela/resources/network/udp-client.c"
distccd[14754] (dcc_scan_args) found object/output file "udp-client.o"
distccd[14754] compile from udp-client.c to udp-client.o
distccd[14754] (dcc_run_job) output file udp-client.o
distccd[14754] (dcc_input_tmpnam) input file /root/Bela/resources/network/udp-client.c
distccd[14754] (dcc_r_token_int) got DOTI00016eeb
distccd[14754] (dcc_r_file) received 93931 bytes to file /tmp/distccd_ff4c663b.i
distccd[14754] (dcc_r_file_timed) 93931 bytes received in 0.011458s, rate 8006kB/s
distccd[14754] (dcc_set_input) changed input from "/root/Bela/resources/network/udp-client.c" to "/tmp/distccd_ff4c663b.i"
distccd[14754] (dcc_set_input) command after: clang-3.9-arm -c -v /tmp/distccd_ff4c663b.i -march=armv7-a -o udp-client.o
distccd[14754] (dcc_set_output) changed output from "udp-client.o" to "/tmp/distccd_fcec663b.o"
distccd[14754] (dcc_set_output) command after: clang-3.9-arm -c -v /tmp/distccd_ff4c663b.i -march=armv7-a -o /tmp/distccd_fcec663b.o
distccd[14754] (dcc_check_compiler_masq) /usr/local/bin/clang-3.9-arm is not a symlink
distccd[14754] (dcc_spawn_child) forking to execute: clang-3.9-arm -c -v /tmp/distccd_ff4c663b.i -march=armv7-a -o /tmp/distccd_fcec663b.o
distccd[14754] (dcc_spawn_child) child started as pid32551
distccd[32551] (dcc_new_pgrp) entered process group
distccd[32551] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[14754] (dcc_collect_child) cc child 32551 terminated with status 0
distccd[14754] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[14754] (dcc_x_token_int) send DONE00000001
distccd[14754] (dcc_x_token_int) send STAT00000000
distccd[14754] (dcc_x_file) send 1002 byte file /tmp/distcc_f203663b.stderr with token SERR and compression 69
distccd[14754] (dcc_x_token_int) send SERR000003ea
distccd[14754] (dcc_x_file) send 0 byte file /tmp/distcc_f22c663b.stdout with token SOUT and compression 69
distccd[14754] (dcc_x_token_int) send SOUT00000000
distccd[14754] (dcc_x_file) send 2192 byte file /tmp/distccd_fcec663b.o with token DOTO and compression 69
distccd[14754] (dcc_x_token_int) send DOTO00000890
distccd[14754] clang-3.9-arm /root/Bela/resources/network/udp-client.c on localhost completed ok
distccd[14754] job complete
distccd[14754] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[14754] (dcc_job_summary) client: 192.168.7.2:59774 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:320ms clang-3.9-arm /root/Bela/resources/network/udp-client.c

DDigital-Larry · Jun 2, 2019

Now trying "faust2bela" from the Ubuntu machine. /usr/Bela/Makefile still references distcc.

gary@audio-workstation:~/faust/examples/bela$ faust2bela -tobela -gui FXChaine2.dsp 
Send to bela
/home/gary/faust/examples/bela/FXChaine2
Start communication with bela
/home/gary/faust/examples/bela/FXChaine2
FXChaine2
Checking the board is up and running at root@192.168.7.2...done
Stop running process...
Run bela now
uploadBuildRun START
Copying new source files to BeagleBone...Using rsync...
Automatically detected PROJECT_TYPE: cpp 
Running on __stretch__ with Xenomai __3__
systemctl stop bela_startup || true
PID=`grep bela-audio  /proc/xenomai/sched/stat | cut -d " " -f 5 | sed s/\s//g`; if [ -z $PID ]; then [ true = true ] || echo "No process to kill"; else [  true = true  ] || echo "Killing old Bela process $PID"; kill -2 $PID; sleep 0.2; kill -9 $PID 2> /dev/null; fi; screen -X -S Bela quit > /dev/null; exit 0;
killall scsynth 2>/dev/null& killall sclang 2>/dev/null& true
echo 'Building render.cpp...'
Building render.cpp...
distcc-clang++  -I/root/Bela/projects/FXChaine2 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/FXChaine2/build/render.d" -o "/root/Bela/projects/FXChaine2/build/render.o" "/root/Bela/projects/FXChaine2/render.cpp" 
distcc[5035] ERROR: compile /root/Bela/projects/FXChaine2/render.cpp on 192.168.7.1 failed
distcc[5035] (dcc_build_somewhere) Warning: remote compilation of '/root/Bela/projects/FXChaine2/render.cpp' failed, retrying locally
distcc[5035] (dcc_build_somewhere) Warning: failed to distribute and fallbacks are disabled
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:434: recipe for target '/root/Bela/projects/FXChaine2/build/render.o' failed
make: *** [/root/Bela/projects/FXChaine2/build/render.o] Error 1
Connection to 192.168.7.2 closed.
gary@audio-workstation:~/faust/examples/bela$

This still seems to be the failure:

/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe

This link takes us to some hints on that error message, which seem to culminate in "install the toolchain". The question is in the context of Android NDK though so I don't know if it applies here.

https://stackoverflow.com/questions/52646860/ld-unrecognised-emulation-mode-armelf-linux-eabi-when-cross-compiling-with-a

Here's the distcc log from this step:

distccd[14755] (dcc_check_client) connection from 192.168.7.2:59776
distccd[14755] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
distccd[14755] (dcc_r_token_int) got DIST00000001
distccd[14755] (dcc_r_token_int) got ARGC00000013
distccd[14755] (dcc_r_argv) reading 19 arguments from job submission
distccd[14755] (dcc_r_token_int) got ARGV0000000f
distccd[14755] (dcc_r_token_string) got 'clang++-3.9-arm'
distccd[14755] (dcc_r_argv) argv[0] = "clang++-3.9-arm"
distccd[14755] (dcc_r_token_int) got ARGV0000000e
distccd[14755] (dcc_r_token_string) got '-march=armv7-a'
distccd[14755] (dcc_r_argv) argv[1] = "-march=armv7-a"
distccd[14755] (dcc_r_token_int) got ARGV0000000a
distccd[14755] (dcc_r_token_string) got '-mfpu=vfp3'
distccd[14755] (dcc_r_argv) argv[2] = "-mfpu=vfp3"
distccd[14755] (dcc_r_token_int) got ARGV0000001c
distccd[14755] (dcc_r_token_string) got '-fasynchronous-unwind-tables'
distccd[14755] (dcc_r_argv) argv[3] = "-fasynchronous-unwind-tables"
distccd[14755] (dcc_r_token_int) got ARGV00000003
distccd[14755] (dcc_r_token_string) got '-O3'
distccd[14755] (dcc_r_argv) argv[4] = "-O3"
distccd[14755] (dcc_r_token_int) got ARGV0000000e
distccd[14755] (dcc_r_token_string) got '-march=armv7-a'
distccd[14755] (dcc_r_argv) argv[5] = "-march=armv7-a"
distccd[14755] (dcc_r_token_int) got ARGV00000010
distccd[14755] (dcc_r_token_string) got '-mtune=cortex-a8'
distccd[14755] (dcc_r_argv) argv[6] = "-mtune=cortex-a8"
distccd[14755] (dcc_r_token_int) got ARGV00000010
distccd[14755] (dcc_r_token_string) got '-mfloat-abi=hard'
distccd[14755] (dcc_r_argv) argv[7] = "-mfloat-abi=hard"
distccd[14755] (dcc_r_token_int) got ARGV0000000a
distccd[14755] (dcc_r_token_string) got '-mfpu=neon'
distccd[14755] (dcc_r_argv) argv[8] = "-mfpu=neon"
distccd[14755] (dcc_r_token_int) got ARGV00000010
distccd[14755] (dcc_r_token_string) got '-ftree-vectorize'
distccd[14755] (dcc_r_argv) argv[9] = "-ftree-vectorize"
distccd[14755] (dcc_r_token_int) got ARGV0000000b
distccd[14755] (dcc_r_token_string) got '-ffast-math'
distccd[14755] (dcc_r_argv) argv[10] = "-ffast-math"
distccd[14755] (dcc_r_token_int) got ARGV0000000a
distccd[14755] (dcc_r_token_string) got '-std=c++11'
distccd[14755] (dcc_r_argv) argv[11] = "-std=c++11"
distccd[14755] (dcc_r_token_int) got ARGV0000000c
distccd[14755] (dcc_r_token_string) got '-Wno-varargs'
distccd[14755] (dcc_r_argv) argv[12] = "-Wno-varargs"
distccd[14755] (dcc_r_token_int) got ARGV00000005
distccd[14755] (dcc_r_token_string) got '-Wall'
distccd[14755] (dcc_r_argv) argv[13] = "-Wall"
distccd[14755] (dcc_r_token_int) got ARGV00000002
distccd[14755] (dcc_r_token_string) got '-c'
distccd[14755] (dcc_r_argv) argv[14] = "-c"
distccd[14755] (dcc_r_token_int) got ARGV00000012
distccd[14755] (dcc_r_token_string) got '-fmessage-length=0'
distccd[14755] (dcc_r_argv) argv[15] = "-fmessage-length=0"
distccd[14755] (dcc_r_token_int) got ARGV00000002
distccd[14755] (dcc_r_token_string) got '-o'
distccd[14755] (dcc_r_argv) argv[16] = "-o"
distccd[14755] (dcc_r_token_int) got ARGV0000002c
distccd[14755] (dcc_r_token_string) got '/root/Bela/projects/FXChaine2/build/render.o'
distccd[14755] (dcc_r_argv) argv[17] = "/root/Bela/projects/FXChaine2/build/render.o"
distccd[14755] (dcc_r_token_int) got ARGV00000028
distccd[14755] (dcc_r_token_string) got '/root/Bela/projects/FXChaine2/render.cpp'
distccd[14755] (dcc_r_argv) argv[18] = "/root/Bela/projects/FXChaine2/render.cpp"
distccd[14755] (dcc_r_argv) got arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/FXChaine2/build/render.o /root/Bela/projects/FXChaine2/render.cpp
distccd[14755] (dcc_scan_args) scanning arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/FXChaine2/build/render.o /root/Bela/projects/FXChaine2/render.cpp
distccd[14755] (dcc_scan_args) found object/output file "/root/Bela/projects/FXChaine2/build/render.o"
distccd[14755] (dcc_scan_args) found input file "/root/Bela/projects/FXChaine2/render.cpp"
distccd[14755] compile from render.cpp to render.o
distccd[14755] (dcc_run_job) output file /root/Bela/projects/FXChaine2/build/render.o
distccd[14755] (dcc_input_tmpnam) input file /root/Bela/projects/FXChaine2/render.cpp
distccd[14755] (dcc_r_token_int) got DOTI0019610a
distccd[14755] (dcc_r_file) received 1663242 bytes to file /tmp/distccd_261c6794.ii
distccd[14755] (dcc_r_file_timed) 1663242 bytes received in 0.150477s, rate 10794kB/s
distccd[14755] (dcc_set_input) changed input from "/root/Bela/projects/FXChaine2/render.cpp" to "/tmp/distccd_261c6794.ii"
distccd[14755] (dcc_set_input) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/FXChaine2/build/render.o /tmp/distccd_261c6794.ii
distccd[14755] (dcc_set_output) changed output from "/root/Bela/projects/FXChaine2/build/render.o" to "/tmp/distccd_27a96794.o"
distccd[14755] (dcc_set_output) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_27a96794.o /tmp/distccd_261c6794.ii
distccd[14755] (dcc_check_compiler_masq) /usr/local/bin/clang++-3.9-arm is not a symlink
distccd[14755] (dcc_spawn_child) forking to execute: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_27a96794.o /tmp/distccd_261c6794.ii
distccd[14755] (dcc_spawn_child) child started as pid32659
distccd[32659] (dcc_new_pgrp) entered process group
distccd[32659] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[14755] (dcc_collect_child) cc child 32659 terminated with status 0x100
distccd[14755] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[14755] (dcc_x_token_int) send DONE00000001
distccd[14755] (dcc_x_token_int) send STAT00000100
distccd[14755] (dcc_x_file) send 248 byte file /tmp/distcc_58bf6794.stderr with token SERR and compression 69
distccd[14755] (dcc_x_token_int) send SERR000000f8
distccd[14755] (dcc_x_file) send 0 byte file /tmp/distcc_5b7e6794.stdout with token SOUT and compression 69
distccd[14755] (dcc_x_token_int) send SOUT00000000
distccd[14755] (dcc_x_token_int) send DOTO00000000
distccd[14755] clang++-3.9-arm /root/Bela/projects/FXChaine2/render.cpp on localhost failed
distccd[14755] job complete
distccd[14755] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[14755] (dcc_job_summary) client: 192.168.7.2:59776 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:1109ms clang++-3.9-arm /root/Bela/projects/FXChaine2/render.cpp

giuliomoro · Jun 2, 2019

This is very confusing to me. The error message you get seems to be about the linker, ld, and it seems to come from the host, because of the list of supported emulations. The weird thing about this is that distcc should never attempt to run the linker on the host (only the compiler), and also that the line that fails really has nothing to do with the linker.

So this is the line that is run on the host and fails:

Digital-Larry distccd[14755] (dcc_spawn_child) forking to execute: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_27a96794.o /tmp/distccd_261c6794.ii

As you see, it tries to compile the pre-processed file /tmp/distccd_261c6794.ii into the object file /tmp/distccd_27a96794.o. The pre-processed file is generated by distcc on the client (Bela), where it runs cpp (the C-preprocessor) on the file to bring in all the includes and apply all the defines and resolve the macros, generating a stand-alone C++ file that has no dependency on any external files.

Let's troubleshoot this with incremental steps. Create a simple C++ file with no includes on the host:

Create a simple C++ file with no includes on the HOST: create a file called test.ii:

int func()
{
    return 0;
}

and then run the same command as above (except for the file paths at the end) :

clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.ii

if this is successful, make sure the generated file test.o is indeed an ARM binary:

$ file test.o
test.o: ELF 32-bit LSB relocatable, ARM, version 1 (SYSV), not stripped

if the above worked fine, then copy the test.ii file to the board and run ON THE BOARD:

distcc-clang++ -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.ii

and if this is successful, check that the generated file is indeed an ARM binary:

$ file test.o
test.o: ELF 32-bit LSB relocatable, ARM, version 1 (SYSV), not stripped

if this is still good, then rename test.ii to test.cpp and run the same line again, this time with test.cpp at the end:

distcc-clang++ -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.cpp

if this is still good, still on the board, do cd /root/Bela/ and there try to re-run the full line that was failing while building your project:

distcc-clang++  -I/root/Bela/projects/FXChaine2 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/FXChaine2/build/render.d" -o "/root/Bela/projects/FXChaine2/build/render.o" "/root/Bela/projects/FXChaine2/render.cpp"

does this successfully generated the file /root/Bela/projects/FXChaine2/build/render.o? And is it an ARM binary?