Problem compiling simpleFX_Analog.dsp

DDigital-Larry · May 29, 2019

Next question, when I ran the faust2bela script while watching ~/distcc.log, nothing is appended to the log file (this is on the host of course). The file is actually compiled and running on the Bela though. What do I need to do to enable the distcc to run when I run faust2bela -tobela also from the host?

giuliomoro · May 29, 2019

PLEASE format your posts properly.
Enclose blocks of code or terminal logs with ``` , .e.g:
```
/tmp/distcc/distccd_502b6055.ii.text+0x3e8): undefined reference to MHD_create_response_from_fd' /tmp/distcc/distccd_502b6055.ii:(.text+0x414): undefined reference toMHD_add_response_header'
/tmp/distcc/distccd_502b6055.ii.text+0x42c): undefined reference to MHD_add_response_header'
```

so that it gets visualized as

/tmp/distcc/distccd_502b6055.ii:(.text+0x3e8): undefined reference to `MHD_create_response_from_fd'
/tmp/distcc/distccd_502b6055.ii:(.text+0x414): undefined reference to `MHD_add_response_header'
/tmp/distcc/distccd_502b6055.ii:(.text+0x42c): undefined reference to `MHD_add_response_header'

I have been editing your posts to be able to read them.

Digital-Larry This does not agree with the directions given. They hint that there should be some .so* files also:

libHTTPDFaust.so*

some files are missing there, I will fix that shortly.

Digital-Larry ok I see many references to MHD_stop_daemon for example, what is that I wonder, well I found this, next question is how do I integrate it with this project?

https://www.gnu.org/software/libmicrohttpd/manual/libmicrohttpd.html

The instructions from here actually say:

Install libmicrohttpd-dev on the board:
connect the board to the internet and run
apt-get install libmicrohttpd-dev

you may have skipped that?

Digital-Larry ext question, when I ran the faust2bela script while watching ~/distcc.log, nothing is appended to the log file (this is on the host of course). The file is actually compiled and running on the Bela though. What do I need to do to enable the distcc to run when I run faust2bela -tobela also from the host?

You need to enable it to build all Bela projects. It is described in the distcc-on-bela documentation:

Using distcc by default on Bela
Make the following edit to the Bela Makefile:

 endif

 ifeq ($(COMPILER), clang)
-  CLANG_PATH?=/usr/bin/clang
+ CLANG_PATH=distcc-clang
   CC=$(CLANG_PATH)
   CXX=$(CLANG_PATH)++
   DEFAULT_CPPFLAGS += -DNDEBUG # Maybe we should add back in -no-integrated-as?

have you done these steps yet? It doesn't look like it, because your linker line starts with /usr/bin/clang++ instead of distcc-clang+

giuliomoro · May 29, 2019

giuliomoro some files are missing there, I will fix that shortly.

now fixed.

DDigital-Larry · May 29, 2019

giuliomoro Thank you!!!! Sorry about the bad formatting, it wasn't obvious what I should be doing, but I suppose I should be aware this is a markdown environment. My building of the httplib locally was probably unnecessary as I had previously used the apt-get step. Using apt-get gives you 0.9.51 while building from source gives you a version that is a couple of years more recent, not that I think it matters.

I did not edit the Bela makefile.

I'll try this later today. I'm sick with the flu and have a pounding headache. 8⁽ But I trust that if this succeeds, I will be singing in the sunshine. 8^) with pitch shifted delays

DDigital-Larry · May 29, 2019

giuliomoro THANK YOU!!!!! The updated library files made a bunch of difference. However I had to copy them to /usr/lib as /usr/local/lib did not work. OMG I can see the GUI! I'm even using remote access software so I'm viewing the Bela attached to my Linux box downstairs on USB while I'm upstairs using Windows. Tonight I will try out some audio tests!

giuliomoro · May 29, 2019

Digital-Larry However I had to copy them to /usr/lib as /usr/local/lib did not work.

hmmm not sure why that would be the case. Possibly it just needed running ldconfig ... Or possibly, the linker would find the file libHTTPDFaust.a you previously had in /usr/local/lib and looked no further? Not sure.

DDigital-Larry · May 30, 2019

giuliomoro let me know if you would like me to test any theories you may have

giuliomoro · May 30, 2019

not sure really. However I have an explanation as to why it worked with the shared library (.so) and not with the static (.a) one:

a static library is simply an archive of object files, while a shared library - among other things - has been linked with all the libraries it depends upon. So when using the static library, in order for that to work, you need to explicitly pass to the linker the library(ies) that your your static library depends upon (in this case: -lmicrohttpd), so that the external symbols in the objects files that make up the .a file can be resolved. When using the shared library, instead, these symbols have been already resolved when it was linked together in a .so file, so there is no need to explicitly state it again.

DDigital-Larry · Jun 1, 2019

Still having trouble at this step.

Using distcc by default on Bela
Make the following edit to the Bela Makefile:

By "the Bela Makefile" I presume that you mean the file /root/Bela/Makefile. Sorry, it wasn't obvious to me.

I found the lines and made the change.

echo 'Building render.cpp...'
Building render.cpp...
distcc-clang++  -I/root/Bela/projects/delay01 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/delay01/build/render.d" -o "/root/Bela/projects/delay01/build/render.o" "/root/Bela/projects/delay01/render.cpp" 
distcc[2855] ERROR: nonblocking connect to 192.168.7.1:3632 failed: Connection refused
distcc[2855] (dcc_build_somewhere) Warning: failed to distribute and fallbacks are disabled
Makefile:434: recipe for target '/root/Bela/projects/delay01/build/render.o' failed
make: *** [/root/Bela/projects/delay01/build/render.o] Error 103
Connection to 192.168.7.2 closed.
gary@audio-workstation:~/faust/examples/bela$

DDigital-Larry · Jun 1, 2019

OK the distccd wasn't running. Now that I started that, I get:

distccd[14751] (dcc_standalone_server) not detaching
distccd[14751] (dcc_new_pgrp) already a process group leader
distccd[14751] (dcc_log_daemon_started) preforking daemon started (3.3.2 x86_64-pc-linux-gnu, built May 26 2019 14:11:45)
distccd[14751] (dcc_create_kids) up to 1 children
distccd[14751] (dcc_create_kids) up to 2 children
distccd[14751] (dcc_create_kids) up to 3 children
distccd[14751] (dcc_create_kids) up to 4 children
distccd[14751] (dcc_create_kids) up to 5 children
distccd[14751] (dcc_create_kids) up to 6 children
distccd[14752] (dcc_check_client) connection from 192.168.7.2:59746
distccd[14752] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
distccd[14752] (dcc_r_token_int) got DIST00000001
distccd[14752] (dcc_r_token_int) got ARGC00000013
distccd[14752] (dcc_r_argv) reading 19 arguments from job submission
distccd[14752] (dcc_r_token_int) got ARGV0000000f
distccd[14752] (dcc_r_token_string) got 'clang++-3.9-arm'
distccd[14752] (dcc_r_argv) argv[0] = "clang++-3.9-arm"
distccd[14752] (dcc_r_token_int) got ARGV0000000e
distccd[14752] (dcc_r_token_string) got '-march=armv7-a'
distccd[14752] (dcc_r_argv) argv[1] = "-march=armv7-a"
distccd[14752] (dcc_r_token_int) got ARGV0000000a
distccd[14752] (dcc_r_token_string) got '-mfpu=vfp3'
distccd[14752] (dcc_r_argv) argv[2] = "-mfpu=vfp3"
distccd[14752] (dcc_r_token_int) got ARGV0000001c
distccd[14752] (dcc_r_token_string) got '-fasynchronous-unwind-tables'
distccd[14752] (dcc_r_argv) argv[3] = "-fasynchronous-unwind-tables"
distccd[14752] (dcc_r_token_int) got ARGV00000003
distccd[14752] (dcc_r_token_string) got '-O3'
distccd[14752] (dcc_r_argv) argv[4] = "-O3"
distccd[14752] (dcc_r_token_int) got ARGV0000000e
distccd[14752] (dcc_r_token_string) got '-march=armv7-a'
distccd[14752] (dcc_r_argv) argv[5] = "-march=armv7-a"
distccd[14752] (dcc_r_token_int) got ARGV00000010
distccd[14752] (dcc_r_token_string) got '-mtune=cortex-a8'
distccd[14752] (dcc_r_argv) argv[6] = "-mtune=cortex-a8"
distccd[14752] (dcc_r_token_int) got ARGV00000010
distccd[14752] (dcc_r_token_string) got '-mfloat-abi=hard'
distccd[14752] (dcc_r_argv) argv[7] = "-mfloat-abi=hard"
distccd[14752] (dcc_r_token_int) got ARGV0000000a
distccd[14752] (dcc_r_token_string) got '-mfpu=neon'
distccd[14752] (dcc_r_argv) argv[8] = "-mfpu=neon"
distccd[14752] (dcc_r_token_int) got ARGV00000010
distccd[14752] (dcc_r_token_string) got '-ftree-vectorize'
distccd[14752] (dcc_r_argv) argv[9] = "-ftree-vectorize"
distccd[14752] (dcc_r_token_int) got ARGV0000000b
distccd[14752] (dcc_r_token_string) got '-ffast-math'
distccd[14752] (dcc_r_argv) argv[10] = "-ffast-math"
distccd[14752] (dcc_r_token_int) got ARGV0000000a
distccd[14752] (dcc_r_token_string) got '-std=c++11'
distccd[14752] (dcc_r_argv) argv[11] = "-std=c++11"
distccd[14752] (dcc_r_token_int) got ARGV0000000c
distccd[14752] (dcc_r_token_string) got '-Wno-varargs'
distccd[14752] (dcc_r_argv) argv[12] = "-Wno-varargs"
distccd[14752] (dcc_r_token_int) got ARGV00000005
distccd[14752] (dcc_r_token_string) got '-Wall'
distccd[14752] (dcc_r_argv) argv[13] = "-Wall"
distccd[14752] (dcc_r_token_int) got ARGV00000002
distccd[14752] (dcc_r_token_string) got '-c'
distccd[14752] (dcc_r_argv) argv[14] = "-c"
distccd[14752] (dcc_r_token_int) got ARGV00000012
distccd[14752] (dcc_r_token_string) got '-fmessage-length=0'
distccd[14752] (dcc_r_argv) argv[15] = "-fmessage-length=0"
distccd[14752] (dcc_r_token_int) got ARGV00000002
distccd[14752] (dcc_r_token_string) got '-o'
distccd[14752] (dcc_r_argv) argv[16] = "-o"
distccd[14752] (dcc_r_token_int) got ARGV0000002a
distccd[14752] (dcc_r_token_string) got '/root/Bela/projects/delay01/build/render.o'
distccd[14752] (dcc_r_argv) argv[17] = "/root/Bela/projects/delay01/build/render.o"
distccd[14752] (dcc_r_token_int) got ARGV00000026
distccd[14752] (dcc_r_token_string) got '/root/Bela/projects/delay01/render.cpp'
distccd[14752] (dcc_r_argv) argv[18] = "/root/Bela/projects/delay01/render.cpp"
distccd[14752] (dcc_r_argv) got arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/delay01/build/render.o /root/Bela/projects/delay01/render.cpp
distccd[14752] (dcc_scan_args) scanning arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/delay01/build/render.o /root/Bela/projects/delay01/render.cpp
distccd[14752] (dcc_scan_args) found object/output file "/root/Bela/projects/delay01/build/render.o"
distccd[14752] (dcc_scan_args) found input file "/root/Bela/projects/delay01/render.cpp"
distccd[14752] compile from render.cpp to render.o
distccd[14752] (dcc_run_job) output file /root/Bela/projects/delay01/build/render.o
distccd[14752] (dcc_input_tmpnam) input file /root/Bela/projects/delay01/render.cpp
distccd[14752] (dcc_r_token_int) got DOTI0018d9d3
distccd[14752] (dcc_r_file) received 1628627 bytes to file /tmp/distccd_1db50d83.ii
distccd[14752] (dcc_r_file_timed) 1628627 bytes received in 0.146217s, rate 10877kB/s
distccd[14752] (dcc_set_input) changed input from "/root/Bela/projects/delay01/render.cpp" to "/tmp/distccd_1db50d83.ii"
distccd[14752] (dcc_set_input) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/delay01/build/render.o /tmp/distccd_1db50d83.ii
distccd[14752] (dcc_set_output) changed output from "/root/Bela/projects/delay01/build/render.o" to "/tmp/distccd_1dc50d83.o"
distccd[14752] (dcc_set_output) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_1dc50d83.o /tmp/distccd_1db50d83.ii
distccd[14752] (dcc_check_compiler_masq) /usr/local/bin/clang++-3.9-arm is not a symlink
distccd[14752] (dcc_spawn_child) forking to execute: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_1dc50d83.o /tmp/distccd_1db50d83.ii
distccd[14752] (dcc_spawn_child) child started as pid14971
distccd[14971] (dcc_new_pgrp) entered process group
distccd[14971] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[14752] (dcc_collect_child) cc child 14971 terminated with status 0x100
distccd[14752] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[14752] (dcc_x_token_int) send DONE00000001
distccd[14752] (dcc_x_token_int) send STAT00000100
distccd[14752] (dcc_x_file) send 248 byte file /tmp/distcc_161f0d83.stderr with token SERR and compression 69
distccd[14752] (dcc_x_token_int) send SERR000000f8
distccd[14752] (dcc_x_file) send 0 byte file /tmp/distcc_16d70d83.stdout with token SOUT and compression 69
distccd[14752] (dcc_x_token_int) send SOUT00000000
distccd[14752] (dcc_x_token_int) send DOTO00000000
distccd[14752] clang++-3.9-arm /root/Bela/projects/delay01/render.cpp on localhost failed
distccd[14752] job complete
distccd[14752] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[14752] (dcc_job_summary) client: 192.168.7.2:59746 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:988ms clang++-3.9-arm /root/Bela/projects/delay01/render.cpp

in the distcc log.

On host console I get:

echo 'Building render.cpp...'
Building render.cpp...
distcc-clang++  -I/root/Bela/projects/delay01 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/delay01/build/render.d" -o "/root/Bela/projects/delay01/build/render.o" "/root/Bela/projects/delay01/render.cpp" 
distcc[3031] ERROR: compile /root/Bela/projects/delay01/render.cpp on 192.168.7.1 failed
distcc[3031] (dcc_build_somewhere) Warning: remote compilation of '/root/Bela/projects/delay01/render.cpp' failed, retrying locally
distcc[3031] (dcc_build_somewhere) Warning: failed to distribute and fallbacks are disabled
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:434: recipe for target '/root/Bela/projects/delay01/build/render.o' failed
make: *** [/root/Bela/projects/delay01/build/render.o] Error 1
Connection to 192.168.7.2 closed.
gary@audio-workstation:~/faust/examples/bela$

DDigital-Larry · Jun 1, 2019

Think you were talking about this before, but not sure I understood.

ary@audio-workstation:~/faust/examples/bela$ cat /usr/local/bin/*arm
#!/bin/bash
/usr/bin/clang -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $@
#!/bin/bash
/usr/bin/clang++ -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $0
gary@audio-workstation:~/faust/examples/bela$

These don't currently point to clang-3.9. They just point to clang which is actually version 6. I think you were saying that it has to be version 3.9?

However, this step works:

distcc-clang -c ~/Bela/resources/network/udp-client.c

DDigital-Larry · Jun 1, 2019

gary@audio-workstation:~$ ld -V
GNU ld (GNU Binutils for Ubuntu) 2.30
  Supported emulations:
   elf_x86_64
   elf32_x86_64
   elf_i386
   elf_iamcu
   i386linux
   elf_l1om
   elf_k1om
   i386pep
   i386pe
gary@audio-workstation:~$

giuliomoro · Jun 1, 2019

Digital-Larry By "the Bela Makefile" I presume that you mean the file /root/Bela/Makefile.

yes

Digital-Larry On host console I get:

I guess that's technically the "client" console (i.e.: what you see there is while you are logged in on Bela).

So it seems that the compiler is failing, and my guess is that it is because it is being passed -march=armv7-a but no -target armv7l-unknown-linux-gnueabihf, and this - I guess - is because you are not calling the wrapper but you are calling the vanilla compiler directly (through the symlink you created).
The point of the wrapper is exactly to automatically add -target armv7l-unknown-linux-gnueabihf to all calls.

I think what is happening is what I mentioned in the other thread:

giuliomoro But if you do put some symlinks there, they should be pointing NOT to the regular clang, but to the clang-3.9-arm and clang++-3.9-arm files you created in /usr/local/bin/, and they will eventually call clang with the appropriate option -target armv7l-unknown-linux-gnueabihf, to tell it to build code for ARM.

Digital-Larry gary@audio-workstation:/usr/local/lib$ ls -l distcc/
total 0
lrwxrwxrwx 1 root root 14 May 28 17:45 clang-3.9-arm -> /usr/bin/clang
lrwxrwxrwx 1 root root 16 May 28 17:45 clang++-3.9-arm -> /usr/bin/clang++

Is this working? I find it surprising, but I guess it may depend on how you clang has been built. This way you are simply using the regular clang on your computer, but it should need a -target armv7l-unknown-linux-gnueabihf option in order to know it should build for arm (at least, it does with mine, which I installed with macports on Mac).

basically, remove your current symlinks from /usr/local/lib/distcc/ and make some new ones that point NOT to the vanilla clang/clang++, but to the wrapper clang-3.9-arm/clang++-3.9-arm you created:

sudo rm /usr/local/lib/distcc/*
sudo ln -s /usr/local/bin/clang-3.9-arm/ /usr/local/lib/distcc/
sudo ln -s /usr/local/bin/clang++-3.9-arm/ /usr/local/lib/distcc/

Digital-Larry However, this step works:

distcc-clang -c ~/Bela/resources/network/udp-client.c

I see now how that can easily give false positives: it just checks that the server can compile, but does not check the nature of the resulted file. If you were to do file a.out, it would probably say it's an x86 file and not an ARM one.
So the test should probably be amended to :

distcc-clang -c ~/Bela/resources/network/udp-client.c -march=armv7-a

This should then fail similarly to the how the Bela project is failing in your current configuration, and should be fixed with the updated symlinks.

DDigital-Larry · Jun 1, 2019

giuliomoro thanks for your continued help... will check all this!

DDigital-Larry · Jun 2, 2019

OK let's see where we are now.

root@bela:~/Bela# grep distcc Makefile 
  CLANG_PATH=distcc-clang
root@bela:~/Bela#

reference command line build on bela works (without specifying architecture):

root@bela:~/Bela/resources/network# distcc-clang -v -c udp-client.c
clang version 3.9.1-9 (tags/RELEASE_391/rc2)
Target: armv7l-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6
Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6
Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6.3.0
Selected GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
Candidate multilib: .;@m32
Selected multilib: .;@m32
 "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name udp-client.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -fuse-init-array -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include -internal-externc-isystem /usr/include/arm-linux-gnueabihf -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /root/Bela/resources/network -ferror-limit 19 -fmessage-length 80 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o - -x c udp-client.c
clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target armv7l-unknown-linux-gnueabihf
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include
 /usr/include/arm-linux-gnueabihf
 /usr/include
End of search list.
clang version 3.9.1-19ubuntu1 (tags/RELEASE_391/rc2)
Target: armv7l-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/bin
 "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name distccd_ba396596.i -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -coverage-file /tmp/distccd_ba666596.o -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 0 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/distccd_ba666596.o -x cpp-output /tmp/distccd_ba396596.i
clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target x86_64-pc-linux-gnu
#include "..." search starts here:
End of search list.
root@bela:~/Bela/resources/network#

reference build works with specifying architecture:

root@bela:~/Bela/resources/network# distcc-clang -c -v ~/Bela/resources/network/udp-client.c -march=armv7-a
clang version 3.9.1-9 (tags/RELEASE_391/rc2)
Target: armv7l-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6
Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6
Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6.3.0
Selected GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
Candidate multilib: .;@m32
Selected multilib: .;@m32
 "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name udp-client.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -fuse-init-array -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include -internal-externc-isystem /usr/include/arm-linux-gnueabihf -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /root/Bela/resources/network -ferror-limit 19 -fmessage-length 80 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o - -x c /root/Bela/resources/network/udp-client.c
clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target armv7l-unknown-linux-gnueabihf
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include
 /usr/include/arm-linux-gnueabihf
 /usr/include
End of search list.
clang version 3.9.1-19ubuntu1 (tags/RELEASE_391/rc2)
Target: armv7l-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/bin
 "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name distccd_ff4c663b.i -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -coverage-file /tmp/distccd_fcec663b.o -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 0 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/distccd_fcec663b.o -x cpp-output /tmp/distccd_ff4c663b.i
clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target x86_64-pc-linux-gnu
#include "..." search starts here:
End of search list.
root@bela:~/Bela/resources/network#

Here's the distcclog from the Ubuntu machine:

distccd[14752] (dcc_check_client) connection from 192.168.7.2:59770
distccd[14752] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
distccd[14752] (dcc_r_token_int) got DIST00000001
distccd[14752] (dcc_r_token_int) got ARGC00000006
distccd[14752] (dcc_r_argv) reading 6 arguments from job submission
distccd[14752] (dcc_r_token_int) got ARGV0000000d
distccd[14752] (dcc_r_token_string) got 'clang-3.9-arm'
distccd[14752] (dcc_r_argv) argv[0] = "clang-3.9-arm"
distccd[14752] (dcc_r_token_int) got ARGV00000002
distccd[14752] (dcc_r_token_string) got '-v'
distccd[14752] (dcc_r_argv) argv[1] = "-v"
distccd[14752] (dcc_r_token_int) got ARGV00000002
distccd[14752] (dcc_r_token_string) got '-c'
distccd[14752] (dcc_r_argv) argv[2] = "-c"
distccd[14752] (dcc_r_token_int) got ARGV0000000c
distccd[14752] (dcc_r_token_string) got 'udp-client.c'
distccd[14752] (dcc_r_argv) argv[3] = "udp-client.c"
distccd[14752] (dcc_r_token_int) got ARGV00000002
distccd[14752] (dcc_r_token_string) got '-o'
distccd[14752] (dcc_r_argv) argv[4] = "-o"
distccd[14752] (dcc_r_token_int) got ARGV0000000c
distccd[14752] (dcc_r_token_string) got 'udp-client.o'
distccd[14752] (dcc_r_argv) argv[5] = "udp-client.o"
distccd[14752] (dcc_r_argv) got arguments: clang-3.9-arm -v -c udp-client.c -o udp-client.o
distccd[14752] (dcc_scan_args) scanning arguments: clang-3.9-arm -v -c udp-client.c -o udp-client.o
distccd[14752] (dcc_scan_args) found input file "udp-client.c"
distccd[14752] (dcc_scan_args) found object/output file "udp-client.o"
distccd[14752] compile from udp-client.c to udp-client.o
distccd[14752] (dcc_run_job) output file udp-client.o
distccd[14752] (dcc_input_tmpnam) input file udp-client.c
distccd[14752] (dcc_r_token_int) got DOTI00016dac
distccd[14752] (dcc_r_file) received 93612 bytes to file /tmp/distccd_ba396596.i
distccd[14752] (dcc_r_file_timed) 93612 bytes received in 0.011009s, rate 8304kB/s
distccd[14752] (dcc_set_input) changed input from "udp-client.c" to "/tmp/distccd_ba396596.i"
distccd[14752] (dcc_set_input) command after: clang-3.9-arm -v -c /tmp/distccd_ba396596.i -o udp-client.o
distccd[14752] (dcc_set_output) changed output from "udp-client.o" to "/tmp/distccd_ba666596.o"
distccd[14752] (dcc_set_output) command after: clang-3.9-arm -v -c /tmp/distccd_ba396596.i -o /tmp/distccd_ba666596.o
distccd[14752] (dcc_check_compiler_masq) /usr/local/bin/clang-3.9-arm is not a symlink
distccd[14752] (dcc_spawn_child) forking to execute: clang-3.9-arm -v -c /tmp/distccd_ba396596.i -o /tmp/distccd_ba666596.o
distccd[14752] (dcc_spawn_child) child started as pid32524
distccd[32524] (dcc_new_pgrp) entered process group
distccd[32524] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[14752] (dcc_collect_child) cc child 32524 terminated with status 0
distccd[14752] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[14752] (dcc_x_token_int) send DONE00000001
distccd[14752] (dcc_x_token_int) send STAT00000000
distccd[14752] (dcc_x_file) send 1002 byte file /tmp/distcc_b8066596.stderr with token SERR and compression 69
distccd[14752] (dcc_x_token_int) send SERR000003ea
distccd[14752] (dcc_x_file) send 0 byte file /tmp/distcc_b8246596.stdout with token SOUT and compression 69
distccd[14752] (dcc_x_token_int) send SOUT00000000
distccd[14752] (dcc_x_file) send 2192 byte file /tmp/distccd_ba666596.o with token DOTO and compression 69
distccd[14752] (dcc_x_token_int) send DOTO00000890
distccd[14752] clang-3.9-arm udp-client.c on localhost completed ok
distccd[14752] job complete
distccd[14752] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[14752] (dcc_job_summary) client: 192.168.7.2:59770 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:318ms clang-3.9-arm udp-client.c
distccd[14753] (dcc_check_client) connection from 192.168.7.2:59772
distccd[14753] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
distccd[14753] (dcc_r_token_int) got DIST00000001
distccd[14753] (dcc_r_token_int) got ARGC00000006
distccd[14753] (dcc_r_argv) reading 6 arguments from job submission
distccd[14753] (dcc_r_token_int) got ARGV0000000d
distccd[14753] (dcc_r_token_string) got 'clang-3.9-arm'
distccd[14753] (dcc_r_argv) argv[0] = "clang-3.9-arm"
distccd[14753] (dcc_r_token_int) got ARGV00000002
distccd[14753] (dcc_r_token_string) got '-c'
distccd[14753] (dcc_r_argv) argv[1] = "-c"
distccd[14753] (dcc_r_token_int) got ARGV00000029
distccd[14753] (dcc_r_token_string) got '/root/Bela/resources/network/udp-client.c'
distccd[14753] (dcc_r_argv) argv[2] = "/root/Bela/resources/network/udp-client.c"
distccd[14753] (dcc_r_token_int) got ARGV0000000e
distccd[14753] (dcc_r_token_string) got '-march=armv7-a'
distccd[14753] (dcc_r_argv) argv[3] = "-march=armv7-a"
distccd[14753] (dcc_r_token_int) got ARGV00000002
distccd[14753] (dcc_r_token_string) got '-o'
distccd[14753] (dcc_r_argv) argv[4] = "-o"
distccd[14753] (dcc_r_token_int) got ARGV0000000c
distccd[14753] (dcc_r_token_string) got 'udp-client.o'
distccd[14753] (dcc_r_argv) argv[5] = "udp-client.o"
distccd[14753] (dcc_r_argv) got arguments: clang-3.9-arm -c /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
distccd[14753] (dcc_scan_args) scanning arguments: clang-3.9-arm -c /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
distccd[14753] (dcc_scan_args) found input file "/root/Bela/resources/network/udp-client.c"
distccd[14753] (dcc_scan_args) found object/output file "udp-client.o"
distccd[14753] compile from udp-client.c to udp-client.o
distccd[14753] (dcc_run_job) output file udp-client.o
distccd[14753] (dcc_input_tmpnam) input file /root/Bela/resources/network/udp-client.c
distccd[14753] (dcc_r_token_int) got DOTI00016eeb
distccd[14753] (dcc_r_file) received 93931 bytes to file /tmp/distccd_20816628.i
distccd[14753] (dcc_r_file_timed) 93931 bytes received in 0.012232s, rate 7499kB/s
distccd[14753] (dcc_set_input) changed input from "/root/Bela/resources/network/udp-client.c" to "/tmp/distccd_20816628.i"
distccd[14753] (dcc_set_input) command after: clang-3.9-arm -c /tmp/distccd_20816628.i -march=armv7-a -o udp-client.o
distccd[14753] (dcc_set_output) changed output from "udp-client.o" to "/tmp/distccd_20dc6628.o"
distccd[14753] (dcc_set_output) command after: clang-3.9-arm -c /tmp/distccd_20816628.i -march=armv7-a -o /tmp/distccd_20dc6628.o
distccd[14753] (dcc_check_compiler_masq) /usr/local/bin/clang-3.9-arm is not a symlink
distccd[14753] (dcc_spawn_child) forking to execute: clang-3.9-arm -c /tmp/distccd_20816628.i -march=armv7-a -o /tmp/distccd_20dc6628.o
distccd[14753] (dcc_spawn_child) child started as pid32547
distccd[32547] (dcc_new_pgrp) entered process group
distccd[32547] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[14753] (dcc_collect_child) cc child 32547 terminated with status 0
distccd[14753] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[14753] (dcc_x_token_int) send DONE00000001
distccd[14753] (dcc_x_token_int) send STAT00000000
distccd[14753] (dcc_x_file) send 0 byte file /tmp/distcc_27c16628.stderr with token SERR and compression 69
distccd[14753] (dcc_x_token_int) send SERR00000000
distccd[14753] (dcc_x_file) send 0 byte file /tmp/distcc_27996628.stdout with token SOUT and compression 69
distccd[14753] (dcc_x_token_int) send SOUT00000000
distccd[14753] (dcc_x_file) send 2192 byte file /tmp/distccd_20dc6628.o with token DOTO and compression 69
distccd[14753] (dcc_x_token_int) send DOTO00000890
distccd[14753] clang-3.9-arm /root/Bela/resources/network/udp-client.c on localhost completed ok
distccd[14753] job complete
distccd[14753] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[14753] (dcc_job_summary) client: 192.168.7.2:59772 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:286ms clang-3.9-arm /root/Bela/resources/network/udp-client.c
distccd[14754] (dcc_check_client) connection from 192.168.7.2:59774
distccd[14754] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
distccd[14754] (dcc_r_token_int) got DIST00000001
distccd[14754] (dcc_r_token_int) got ARGC00000007
distccd[14754] (dcc_r_argv) reading 7 arguments from job submission
distccd[14754] (dcc_r_token_int) got ARGV0000000d
distccd[14754] (dcc_r_token_string) got 'clang-3.9-arm'
distccd[14754] (dcc_r_argv) argv[0] = "clang-3.9-arm"
distccd[14754] (dcc_r_token_int) got ARGV00000002
distccd[14754] (dcc_r_token_string) got '-c'
distccd[14754] (dcc_r_argv) argv[1] = "-c"
distccd[14754] (dcc_r_token_int) got ARGV00000002
distccd[14754] (dcc_r_token_string) got '-v'
distccd[14754] (dcc_r_argv) argv[2] = "-v"
distccd[14754] (dcc_r_token_int) got ARGV00000029
distccd[14754] (dcc_r_token_string) got '/root/Bela/resources/network/udp-client.c'
distccd[14754] (dcc_r_argv) argv[3] = "/root/Bela/resources/network/udp-client.c"
distccd[14754] (dcc_r_token_int) got ARGV0000000e
distccd[14754] (dcc_r_token_string) got '-march=armv7-a'
distccd[14754] (dcc_r_argv) argv[4] = "-march=armv7-a"
distccd[14754] (dcc_r_token_int) got ARGV00000002
distccd[14754] (dcc_r_token_string) got '-o'
distccd[14754] (dcc_r_argv) argv[5] = "-o"
distccd[14754] (dcc_r_token_int) got ARGV0000000c
distccd[14754] (dcc_r_token_string) got 'udp-client.o'
distccd[14754] (dcc_r_argv) argv[6] = "udp-client.o"
distccd[14754] (dcc_r_argv) got arguments: clang-3.9-arm -c -v /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
distccd[14754] (dcc_scan_args) scanning arguments: clang-3.9-arm -c -v /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
distccd[14754] (dcc_scan_args) found input file "/root/Bela/resources/network/udp-client.c"
distccd[14754] (dcc_scan_args) found object/output file "udp-client.o"
distccd[14754] compile from udp-client.c to udp-client.o
distccd[14754] (dcc_run_job) output file udp-client.o
distccd[14754] (dcc_input_tmpnam) input file /root/Bela/resources/network/udp-client.c
distccd[14754] (dcc_r_token_int) got DOTI00016eeb
distccd[14754] (dcc_r_file) received 93931 bytes to file /tmp/distccd_ff4c663b.i
distccd[14754] (dcc_r_file_timed) 93931 bytes received in 0.011458s, rate 8006kB/s
distccd[14754] (dcc_set_input) changed input from "/root/Bela/resources/network/udp-client.c" to "/tmp/distccd_ff4c663b.i"
distccd[14754] (dcc_set_input) command after: clang-3.9-arm -c -v /tmp/distccd_ff4c663b.i -march=armv7-a -o udp-client.o
distccd[14754] (dcc_set_output) changed output from "udp-client.o" to "/tmp/distccd_fcec663b.o"
distccd[14754] (dcc_set_output) command after: clang-3.9-arm -c -v /tmp/distccd_ff4c663b.i -march=armv7-a -o /tmp/distccd_fcec663b.o
distccd[14754] (dcc_check_compiler_masq) /usr/local/bin/clang-3.9-arm is not a symlink
distccd[14754] (dcc_spawn_child) forking to execute: clang-3.9-arm -c -v /tmp/distccd_ff4c663b.i -march=armv7-a -o /tmp/distccd_fcec663b.o
distccd[14754] (dcc_spawn_child) child started as pid32551
distccd[32551] (dcc_new_pgrp) entered process group
distccd[32551] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[14754] (dcc_collect_child) cc child 32551 terminated with status 0
distccd[14754] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[14754] (dcc_x_token_int) send DONE00000001
distccd[14754] (dcc_x_token_int) send STAT00000000
distccd[14754] (dcc_x_file) send 1002 byte file /tmp/distcc_f203663b.stderr with token SERR and compression 69
distccd[14754] (dcc_x_token_int) send SERR000003ea
distccd[14754] (dcc_x_file) send 0 byte file /tmp/distcc_f22c663b.stdout with token SOUT and compression 69
distccd[14754] (dcc_x_token_int) send SOUT00000000
distccd[14754] (dcc_x_file) send 2192 byte file /tmp/distccd_fcec663b.o with token DOTO and compression 69
distccd[14754] (dcc_x_token_int) send DOTO00000890
distccd[14754] clang-3.9-arm /root/Bela/resources/network/udp-client.c on localhost completed ok
distccd[14754] job complete
distccd[14754] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[14754] (dcc_job_summary) client: 192.168.7.2:59774 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:320ms clang-3.9-arm /root/Bela/resources/network/udp-client.c

DDigital-Larry · Jun 2, 2019

Now trying "faust2bela" from the Ubuntu machine. /usr/Bela/Makefile still references distcc.

gary@audio-workstation:~/faust/examples/bela$ faust2bela -tobela -gui FXChaine2.dsp 
Send to bela
/home/gary/faust/examples/bela/FXChaine2
Start communication with bela
/home/gary/faust/examples/bela/FXChaine2
FXChaine2
Checking the board is up and running at root@192.168.7.2...done
Stop running process...
Run bela now
uploadBuildRun START
Copying new source files to BeagleBone...Using rsync...
Automatically detected PROJECT_TYPE: cpp 
Running on __stretch__ with Xenomai __3__
systemctl stop bela_startup || true
PID=`grep bela-audio  /proc/xenomai/sched/stat | cut -d " " -f 5 | sed s/\s//g`; if [ -z $PID ]; then [ true = true ] || echo "No process to kill"; else [  true = true  ] || echo "Killing old Bela process $PID"; kill -2 $PID; sleep 0.2; kill -9 $PID 2> /dev/null; fi; screen -X -S Bela quit > /dev/null; exit 0;
killall scsynth 2>/dev/null& killall sclang 2>/dev/null& true
echo 'Building render.cpp...'
Building render.cpp...
distcc-clang++  -I/root/Bela/projects/FXChaine2 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/FXChaine2/build/render.d" -o "/root/Bela/projects/FXChaine2/build/render.o" "/root/Bela/projects/FXChaine2/render.cpp" 
distcc[5035] ERROR: compile /root/Bela/projects/FXChaine2/render.cpp on 192.168.7.1 failed
distcc[5035] (dcc_build_somewhere) Warning: remote compilation of '/root/Bela/projects/FXChaine2/render.cpp' failed, retrying locally
distcc[5035] (dcc_build_somewhere) Warning: failed to distribute and fallbacks are disabled
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:434: recipe for target '/root/Bela/projects/FXChaine2/build/render.o' failed
make: *** [/root/Bela/projects/FXChaine2/build/render.o] Error 1
Connection to 192.168.7.2 closed.
gary@audio-workstation:~/faust/examples/bela$

This still seems to be the failure:

/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe

This link takes us to some hints on that error message, which seem to culminate in "install the toolchain". The question is in the context of Android NDK though so I don't know if it applies here.

https://stackoverflow.com/questions/52646860/ld-unrecognised-emulation-mode-armelf-linux-eabi-when-cross-compiling-with-a

Here's the distcc log from this step:

distccd[14755] (dcc_check_client) connection from 192.168.7.2:59776
distccd[14755] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
distccd[14755] (dcc_r_token_int) got DIST00000001
distccd[14755] (dcc_r_token_int) got ARGC00000013
distccd[14755] (dcc_r_argv) reading 19 arguments from job submission
distccd[14755] (dcc_r_token_int) got ARGV0000000f
distccd[14755] (dcc_r_token_string) got 'clang++-3.9-arm'
distccd[14755] (dcc_r_argv) argv[0] = "clang++-3.9-arm"
distccd[14755] (dcc_r_token_int) got ARGV0000000e
distccd[14755] (dcc_r_token_string) got '-march=armv7-a'
distccd[14755] (dcc_r_argv) argv[1] = "-march=armv7-a"
distccd[14755] (dcc_r_token_int) got ARGV0000000a
distccd[14755] (dcc_r_token_string) got '-mfpu=vfp3'
distccd[14755] (dcc_r_argv) argv[2] = "-mfpu=vfp3"
distccd[14755] (dcc_r_token_int) got ARGV0000001c
distccd[14755] (dcc_r_token_string) got '-fasynchronous-unwind-tables'
distccd[14755] (dcc_r_argv) argv[3] = "-fasynchronous-unwind-tables"
distccd[14755] (dcc_r_token_int) got ARGV00000003
distccd[14755] (dcc_r_token_string) got '-O3'
distccd[14755] (dcc_r_argv) argv[4] = "-O3"
distccd[14755] (dcc_r_token_int) got ARGV0000000e
distccd[14755] (dcc_r_token_string) got '-march=armv7-a'
distccd[14755] (dcc_r_argv) argv[5] = "-march=armv7-a"
distccd[14755] (dcc_r_token_int) got ARGV00000010
distccd[14755] (dcc_r_token_string) got '-mtune=cortex-a8'
distccd[14755] (dcc_r_argv) argv[6] = "-mtune=cortex-a8"
distccd[14755] (dcc_r_token_int) got ARGV00000010
distccd[14755] (dcc_r_token_string) got '-mfloat-abi=hard'
distccd[14755] (dcc_r_argv) argv[7] = "-mfloat-abi=hard"
distccd[14755] (dcc_r_token_int) got ARGV0000000a
distccd[14755] (dcc_r_token_string) got '-mfpu=neon'
distccd[14755] (dcc_r_argv) argv[8] = "-mfpu=neon"
distccd[14755] (dcc_r_token_int) got ARGV00000010
distccd[14755] (dcc_r_token_string) got '-ftree-vectorize'
distccd[14755] (dcc_r_argv) argv[9] = "-ftree-vectorize"
distccd[14755] (dcc_r_token_int) got ARGV0000000b
distccd[14755] (dcc_r_token_string) got '-ffast-math'
distccd[14755] (dcc_r_argv) argv[10] = "-ffast-math"
distccd[14755] (dcc_r_token_int) got ARGV0000000a
distccd[14755] (dcc_r_token_string) got '-std=c++11'
distccd[14755] (dcc_r_argv) argv[11] = "-std=c++11"
distccd[14755] (dcc_r_token_int) got ARGV0000000c
distccd[14755] (dcc_r_token_string) got '-Wno-varargs'
distccd[14755] (dcc_r_argv) argv[12] = "-Wno-varargs"
distccd[14755] (dcc_r_token_int) got ARGV00000005
distccd[14755] (dcc_r_token_string) got '-Wall'
distccd[14755] (dcc_r_argv) argv[13] = "-Wall"
distccd[14755] (dcc_r_token_int) got ARGV00000002
distccd[14755] (dcc_r_token_string) got '-c'
distccd[14755] (dcc_r_argv) argv[14] = "-c"
distccd[14755] (dcc_r_token_int) got ARGV00000012
distccd[14755] (dcc_r_token_string) got '-fmessage-length=0'
distccd[14755] (dcc_r_argv) argv[15] = "-fmessage-length=0"
distccd[14755] (dcc_r_token_int) got ARGV00000002
distccd[14755] (dcc_r_token_string) got '-o'
distccd[14755] (dcc_r_argv) argv[16] = "-o"
distccd[14755] (dcc_r_token_int) got ARGV0000002c
distccd[14755] (dcc_r_token_string) got '/root/Bela/projects/FXChaine2/build/render.o'
distccd[14755] (dcc_r_argv) argv[17] = "/root/Bela/projects/FXChaine2/build/render.o"
distccd[14755] (dcc_r_token_int) got ARGV00000028
distccd[14755] (dcc_r_token_string) got '/root/Bela/projects/FXChaine2/render.cpp'
distccd[14755] (dcc_r_argv) argv[18] = "/root/Bela/projects/FXChaine2/render.cpp"
distccd[14755] (dcc_r_argv) got arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/FXChaine2/build/render.o /root/Bela/projects/FXChaine2/render.cpp
distccd[14755] (dcc_scan_args) scanning arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/FXChaine2/build/render.o /root/Bela/projects/FXChaine2/render.cpp
distccd[14755] (dcc_scan_args) found object/output file "/root/Bela/projects/FXChaine2/build/render.o"
distccd[14755] (dcc_scan_args) found input file "/root/Bela/projects/FXChaine2/render.cpp"
distccd[14755] compile from render.cpp to render.o
distccd[14755] (dcc_run_job) output file /root/Bela/projects/FXChaine2/build/render.o
distccd[14755] (dcc_input_tmpnam) input file /root/Bela/projects/FXChaine2/render.cpp
distccd[14755] (dcc_r_token_int) got DOTI0019610a
distccd[14755] (dcc_r_file) received 1663242 bytes to file /tmp/distccd_261c6794.ii
distccd[14755] (dcc_r_file_timed) 1663242 bytes received in 0.150477s, rate 10794kB/s
distccd[14755] (dcc_set_input) changed input from "/root/Bela/projects/FXChaine2/render.cpp" to "/tmp/distccd_261c6794.ii"
distccd[14755] (dcc_set_input) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/FXChaine2/build/render.o /tmp/distccd_261c6794.ii
distccd[14755] (dcc_set_output) changed output from "/root/Bela/projects/FXChaine2/build/render.o" to "/tmp/distccd_27a96794.o"
distccd[14755] (dcc_set_output) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_27a96794.o /tmp/distccd_261c6794.ii
distccd[14755] (dcc_check_compiler_masq) /usr/local/bin/clang++-3.9-arm is not a symlink
distccd[14755] (dcc_spawn_child) forking to execute: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_27a96794.o /tmp/distccd_261c6794.ii
distccd[14755] (dcc_spawn_child) child started as pid32659
distccd[32659] (dcc_new_pgrp) entered process group
distccd[32659] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distccd[14755] (dcc_collect_child) cc child 32659 terminated with status 0x100
distccd[14755] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[14755] (dcc_x_token_int) send DONE00000001
distccd[14755] (dcc_x_token_int) send STAT00000100
distccd[14755] (dcc_x_file) send 248 byte file /tmp/distcc_58bf6794.stderr with token SERR and compression 69
distccd[14755] (dcc_x_token_int) send SERR000000f8
distccd[14755] (dcc_x_file) send 0 byte file /tmp/distcc_5b7e6794.stdout with token SOUT and compression 69
distccd[14755] (dcc_x_token_int) send SOUT00000000
distccd[14755] (dcc_x_token_int) send DOTO00000000
distccd[14755] clang++-3.9-arm /root/Bela/projects/FXChaine2/render.cpp on localhost failed
distccd[14755] job complete
distccd[14755] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[14755] (dcc_job_summary) client: 192.168.7.2:59776 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:1109ms clang++-3.9-arm /root/Bela/projects/FXChaine2/render.cpp

giuliomoro · Jun 2, 2019

This is very confusing to me. The error message you get seems to be about the linker, ld, and it seems to come from the host, because of the list of supported emulations. The weird thing about this is that distcc should never attempt to run the linker on the host (only the compiler), and also that the line that fails really has nothing to do with the linker.

So this is the line that is run on the host and fails:

Digital-Larry distccd[14755] (dcc_spawn_child) forking to execute: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_27a96794.o /tmp/distccd_261c6794.ii

As you see, it tries to compile the pre-processed file /tmp/distccd_261c6794.ii into the object file /tmp/distccd_27a96794.o. The pre-processed file is generated by distcc on the client (Bela), where it runs cpp (the C-preprocessor) on the file to bring in all the includes and apply all the defines and resolve the macros, generating a stand-alone C++ file that has no dependency on any external files.

Let's troubleshoot this with incremental steps. Create a simple C++ file with no includes on the host:

Create a simple C++ file with no includes on the HOST: create a file called test.ii:

int func()
{
    return 0;
}

and then run the same command as above (except for the file paths at the end) :

clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.ii

if this is successful, make sure the generated file test.o is indeed an ARM binary:

$ file test.o
test.o: ELF 32-bit LSB relocatable, ARM, version 1 (SYSV), not stripped

if the above worked fine, then copy the test.ii file to the board and run ON THE BOARD:

distcc-clang++ -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.ii

and if this is successful, check that the generated file is indeed an ARM binary:

$ file test.o
test.o: ELF 32-bit LSB relocatable, ARM, version 1 (SYSV), not stripped

if this is still good, then rename test.ii to test.cpp and run the same line again, this time with test.cpp at the end:

distcc-clang++ -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.cpp

if this is still good, still on the board, do cd /root/Bela/ and there try to re-run the full line that was failing while building your project:

distcc-clang++  -I/root/Bela/projects/FXChaine2 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/FXChaine2/build/render.d" -o "/root/Bela/projects/FXChaine2/build/render.o" "/root/Bela/projects/FXChaine2/render.cpp"

does this successfully generated the file /root/Bela/projects/FXChaine2/build/render.o? And is it an ARM binary?

DDigital-Larry · Jun 2, 2019

giuliomoro first step fails.

gary@audio-workstation:~/test$ clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.ii
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)

gary@audio-workstation:~/test$ cat test.ii
int func()
{
    return 0;
}
gary@audio-workstation:~/test$ 

gary@audio-workstation:~/test$ which clang++-3.9-arm 
/usr/local/bin/clang++-3.9-arm

gary@audio-workstation:~/test$ cat /usr/local/bin/clang++-3.9-arm 
#!/bin/bash
/usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $0

gary@audio-workstation:~/test$ clang++-3.9-arm 
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gary@audio-workstation:~/test$

Calling that compiler throws the error with no parameters given at all! [edit - it's calling the script which does add parameters] So maybe that is broken somehow. I used apt to install it.

gary@audio-workstation:~/test$ sudo apt list | grep ^clang-3.9

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

clang-3.9/bionic,now 1:3.9.1-19ubuntu1 amd64 [installed]
clang-3.9-doc/bionic,bionic 1:3.9.1-19ubuntu1 all
clang-3.9-examples/bionic 1:3.9.1-19ubuntu1 amd64
gary@audio-workstation:~/test$ 

gary@audio-workstation:~/test$ sudo dpkg -s clang-3.9
Package: clang-3.9
Status: install ok installed
Priority: optional
Section: devel
Installed-Size: 140661
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: amd64
Source: llvm-toolchain-3.9
Version: 1:3.9.1-19ubuntu1
Replaces: clang-3.1, clang-3.2, clang-3.3, clang-3.4 (<< 1:3.4.2-7~exp1), clang-3.5 (<< 1:3.5~+rc1-3~exp1), clang-include-fixer-3.9, compiler-rt
Provides: c++-compiler, c-compiler, objc-compiler
Depends: libc6 (>= 2.14), libclang1-3.9 (= 1:3.9.1-19ubuntu1), libgcc1 (>= 1:3.4), libjsoncpp1 (>= 1.7.4), libllvm3.9 (>= 1:3.9.1-6~), libstdc++6 (>= 5.2), libstdc++-7-dev, libgcc-7-dev, libobjc-7-dev, libclang-common-3.9-dev (= 1:3.9.1-19ubuntu1), libc6-dev, binutils
Recommends: llvm-3.9-dev, python
Suggests: gnustep, gnustep-devel, clang-3.9-doc
Breaks: clang-3.1, clang-3.2, clang-3.3, clang-3.4 (<< 1:3.4.2-7~exp1), clang-3.5 (<< 1:3.5~+rc1-3~exp1), clang-include-fixer-3.9, compiler-rt
Description: C, C++ and Objective-C compiler (LLVM based)
 Clang project is a C, C++, Objective C and Objective C++ front-end
 for the LLVM compiler. Its goal is to offer a replacement to the GNU Compiler
 Collection (GCC).
 .
 Clang fully implements all published ISO C++ standards including C++11, as
 well as the upcoming C++14 standard, and some parts of the fledgling C++1z
 standard, and is considered a production-quality C++ compiler.
Original-Maintainer: LLVM Packaging Team <pkg-llvm-team@lists.alioth.debian.org>
Homepage: http://www.llvm.org/
gary@audio-workstation:~/test$

Some other random tests to see what exactly we are calling and the simplest command to generate the error.

gary@audio-workstation:~/test$ which clang++-3.9-arm 
/usr/local/bin/clang++-3.9-arm
gary@audio-workstation:~/test$ cat /usr/local/bin/clang++-3.9-arm 
#!/bin/bash
/usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $0
gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 
clang: error: no input files
gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf
clang: error: no input files
gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf ./test.ii
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gary@audio-workstation:~/test$ /usr/bin/clang++-6.0 -target armv7l-unknown-linux-gnueabihf ./test.ii
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gary@audio-workstation:~/test$ /usr/bin/clang++ -target armv7l-unknown-linux-gnueabihf ./test.ii
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gary@audio-workstation:~/test$