Next question, when I ran the faust2bela script while watching ~/distcc.log, nothing is appended to the log file (this is on the host of course). The file is actually compiled and running on the Bela though. What do I need to do to enable the distcc to run when I run faust2bela -tobela also from the host?

    PLEASE format your posts properly.
    Enclose blocks of code or terminal logs with ``` , .e.g:
    ```
    /tmp/distcc/distccd_502b6055.ii🙁.text+0x3e8): undefined reference to MHD_create_response_from_fd'
    /tmp/distcc/distccd_502b6055.ii:(.text+0x414): undefined reference to
    MHD_add_response_header'
    /tmp/distcc/distccd_502b6055.ii🙁.text+0x42c): undefined reference to MHD_add_response_header'
    ```

    so that it gets visualized as

    /tmp/distcc/distccd_502b6055.ii:(.text+0x3e8): undefined reference to `MHD_create_response_from_fd'
    /tmp/distcc/distccd_502b6055.ii:(.text+0x414): undefined reference to `MHD_add_response_header'
    /tmp/distcc/distccd_502b6055.ii:(.text+0x42c): undefined reference to `MHD_add_response_header'

    I have been editing your posts to be able to read them.

    Digital-Larry This does not agree with the directions given. They hint that there should be some .so* files also:

    libHTTPDFaust.so*

    some files are missing there, I will fix that shortly.

    Digital-Larry ok I see many references to MHD_stop_daemon for example, what is that I wonder, well I found this, next question is how do I integrate it with this project?

    https://www.gnu.org/software/libmicrohttpd/manual/libmicrohttpd.html

    The instructions from here actually say:

    Install libmicrohttpd-dev on the board:
    connect the board to the internet and run
    apt-get install libmicrohttpd-dev

    you may have skipped that?

    Digital-Larry ext question, when I ran the faust2bela script while watching ~/distcc.log, nothing is appended to the log file (this is on the host of course). The file is actually compiled and running on the Bela though. What do I need to do to enable the distcc to run when I run faust2bela -tobela also from the host?

    You need to enable it to build all Bela projects. It is described in the distcc-on-bela documentation:

    Using distcc by default on Bela
    Make the following edit to the Bela Makefile:

     endif
    
     ifeq ($(COMPILER), clang)
    -  CLANG_PATH?=/usr/bin/clang
    + CLANG_PATH=distcc-clang
       CC=$(CLANG_PATH)
       CXX=$(CLANG_PATH)++
       DEFAULT_CPPFLAGS += -DNDEBUG # Maybe we should add back in -no-integrated-as?

    have you done these steps yet? It doesn't look like it, because your linker line starts with /usr/bin/clang++ instead of distcc-clang+

      giuliomoro Thank you!!!! Sorry about the bad formatting, it wasn't obvious what I should be doing, but I suppose I should be aware this is a markdown environment. My building of the httplib locally was probably unnecessary as I had previously used the apt-get step. Using apt-get gives you 0.9.51 while building from source gives you a version that is a couple of years more recent, not that I think it matters.

      I did not edit the Bela makefile.

      I'll try this later today. I'm sick with the flu and have a pounding headache. 8( But I trust that if this succeeds, I will be singing in the sunshine. 8^) with pitch shifted delays

      giuliomoro THANK YOU!!!!! The updated library files made a bunch of difference. However I had to copy them to /usr/lib as /usr/local/lib did not work. OMG I can see the GUI! I'm even using remote access software so I'm viewing the Bela attached to my Linux box downstairs on USB while I'm upstairs using Windows. Tonight I will try out some audio tests!

        Digital-Larry However I had to copy them to /usr/lib as /usr/local/lib did not work.

        hmmm not sure why that would be the case. Possibly it just needed running ldconfig ... Or possibly, the linker would find the file libHTTPDFaust.a you previously had in /usr/local/lib and looked no further? Not sure.

          not sure really. However I have an explanation as to why it worked with the shared library (.so) and not with the static (.a) one:

          a static library is simply an archive of object files, while a shared library - among other things - has been linked with all the libraries it depends upon. So when using the static library, in order for that to work, you need to explicitly pass to the linker the library(ies) that your your static library depends upon (in this case: -lmicrohttpd), so that the external symbols in the objects files that make up the .a file can be resolved. When using the shared library, instead, these symbols have been already resolved when it was linked together in a .so file, so there is no need to explicitly state it again.

          Still having trouble at this step.

          Using distcc by default on Bela
          Make the following edit to the Bela Makefile:

          By "the Bela Makefile" I presume that you mean the file /root/Bela/Makefile. Sorry, it wasn't obvious to me.

          I found the lines and made the change.

          echo 'Building render.cpp...'
          Building render.cpp...
          distcc-clang++  -I/root/Bela/projects/delay01 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/delay01/build/render.d" -o "/root/Bela/projects/delay01/build/render.o" "/root/Bela/projects/delay01/render.cpp" 
          distcc[2855] ERROR: nonblocking connect to 192.168.7.1:3632 failed: Connection refused
          distcc[2855] (dcc_build_somewhere) Warning: failed to distribute and fallbacks are disabled
          Makefile:434: recipe for target '/root/Bela/projects/delay01/build/render.o' failed
          make: *** [/root/Bela/projects/delay01/build/render.o] Error 103
          Connection to 192.168.7.2 closed.
          gary@audio-workstation:~/faust/examples/bela$ 

            OK the distccd wasn't running. Now that I started that, I get:

            distccd[14751] (dcc_standalone_server) not detaching
            distccd[14751] (dcc_new_pgrp) already a process group leader
            distccd[14751] (dcc_log_daemon_started) preforking daemon started (3.3.2 x86_64-pc-linux-gnu, built May 26 2019 14:11:45)
            distccd[14751] (dcc_create_kids) up to 1 children
            distccd[14751] (dcc_create_kids) up to 2 children
            distccd[14751] (dcc_create_kids) up to 3 children
            distccd[14751] (dcc_create_kids) up to 4 children
            distccd[14751] (dcc_create_kids) up to 5 children
            distccd[14751] (dcc_create_kids) up to 6 children
            distccd[14752] (dcc_check_client) connection from 192.168.7.2:59746
            distccd[14752] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
            distccd[14752] (dcc_r_token_int) got DIST00000001
            distccd[14752] (dcc_r_token_int) got ARGC00000013
            distccd[14752] (dcc_r_argv) reading 19 arguments from job submission
            distccd[14752] (dcc_r_token_int) got ARGV0000000f
            distccd[14752] (dcc_r_token_string) got 'clang++-3.9-arm'
            distccd[14752] (dcc_r_argv) argv[0] = "clang++-3.9-arm"
            distccd[14752] (dcc_r_token_int) got ARGV0000000e
            distccd[14752] (dcc_r_token_string) got '-march=armv7-a'
            distccd[14752] (dcc_r_argv) argv[1] = "-march=armv7-a"
            distccd[14752] (dcc_r_token_int) got ARGV0000000a
            distccd[14752] (dcc_r_token_string) got '-mfpu=vfp3'
            distccd[14752] (dcc_r_argv) argv[2] = "-mfpu=vfp3"
            distccd[14752] (dcc_r_token_int) got ARGV0000001c
            distccd[14752] (dcc_r_token_string) got '-fasynchronous-unwind-tables'
            distccd[14752] (dcc_r_argv) argv[3] = "-fasynchronous-unwind-tables"
            distccd[14752] (dcc_r_token_int) got ARGV00000003
            distccd[14752] (dcc_r_token_string) got '-O3'
            distccd[14752] (dcc_r_argv) argv[4] = "-O3"
            distccd[14752] (dcc_r_token_int) got ARGV0000000e
            distccd[14752] (dcc_r_token_string) got '-march=armv7-a'
            distccd[14752] (dcc_r_argv) argv[5] = "-march=armv7-a"
            distccd[14752] (dcc_r_token_int) got ARGV00000010
            distccd[14752] (dcc_r_token_string) got '-mtune=cortex-a8'
            distccd[14752] (dcc_r_argv) argv[6] = "-mtune=cortex-a8"
            distccd[14752] (dcc_r_token_int) got ARGV00000010
            distccd[14752] (dcc_r_token_string) got '-mfloat-abi=hard'
            distccd[14752] (dcc_r_argv) argv[7] = "-mfloat-abi=hard"
            distccd[14752] (dcc_r_token_int) got ARGV0000000a
            distccd[14752] (dcc_r_token_string) got '-mfpu=neon'
            distccd[14752] (dcc_r_argv) argv[8] = "-mfpu=neon"
            distccd[14752] (dcc_r_token_int) got ARGV00000010
            distccd[14752] (dcc_r_token_string) got '-ftree-vectorize'
            distccd[14752] (dcc_r_argv) argv[9] = "-ftree-vectorize"
            distccd[14752] (dcc_r_token_int) got ARGV0000000b
            distccd[14752] (dcc_r_token_string) got '-ffast-math'
            distccd[14752] (dcc_r_argv) argv[10] = "-ffast-math"
            distccd[14752] (dcc_r_token_int) got ARGV0000000a
            distccd[14752] (dcc_r_token_string) got '-std=c++11'
            distccd[14752] (dcc_r_argv) argv[11] = "-std=c++11"
            distccd[14752] (dcc_r_token_int) got ARGV0000000c
            distccd[14752] (dcc_r_token_string) got '-Wno-varargs'
            distccd[14752] (dcc_r_argv) argv[12] = "-Wno-varargs"
            distccd[14752] (dcc_r_token_int) got ARGV00000005
            distccd[14752] (dcc_r_token_string) got '-Wall'
            distccd[14752] (dcc_r_argv) argv[13] = "-Wall"
            distccd[14752] (dcc_r_token_int) got ARGV00000002
            distccd[14752] (dcc_r_token_string) got '-c'
            distccd[14752] (dcc_r_argv) argv[14] = "-c"
            distccd[14752] (dcc_r_token_int) got ARGV00000012
            distccd[14752] (dcc_r_token_string) got '-fmessage-length=0'
            distccd[14752] (dcc_r_argv) argv[15] = "-fmessage-length=0"
            distccd[14752] (dcc_r_token_int) got ARGV00000002
            distccd[14752] (dcc_r_token_string) got '-o'
            distccd[14752] (dcc_r_argv) argv[16] = "-o"
            distccd[14752] (dcc_r_token_int) got ARGV0000002a
            distccd[14752] (dcc_r_token_string) got '/root/Bela/projects/delay01/build/render.o'
            distccd[14752] (dcc_r_argv) argv[17] = "/root/Bela/projects/delay01/build/render.o"
            distccd[14752] (dcc_r_token_int) got ARGV00000026
            distccd[14752] (dcc_r_token_string) got '/root/Bela/projects/delay01/render.cpp'
            distccd[14752] (dcc_r_argv) argv[18] = "/root/Bela/projects/delay01/render.cpp"
            distccd[14752] (dcc_r_argv) got arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/delay01/build/render.o /root/Bela/projects/delay01/render.cpp
            distccd[14752] (dcc_scan_args) scanning arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/delay01/build/render.o /root/Bela/projects/delay01/render.cpp
            distccd[14752] (dcc_scan_args) found object/output file "/root/Bela/projects/delay01/build/render.o"
            distccd[14752] (dcc_scan_args) found input file "/root/Bela/projects/delay01/render.cpp"
            distccd[14752] compile from render.cpp to render.o
            distccd[14752] (dcc_run_job) output file /root/Bela/projects/delay01/build/render.o
            distccd[14752] (dcc_input_tmpnam) input file /root/Bela/projects/delay01/render.cpp
            distccd[14752] (dcc_r_token_int) got DOTI0018d9d3
            distccd[14752] (dcc_r_file) received 1628627 bytes to file /tmp/distccd_1db50d83.ii
            distccd[14752] (dcc_r_file_timed) 1628627 bytes received in 0.146217s, rate 10877kB/s
            distccd[14752] (dcc_set_input) changed input from "/root/Bela/projects/delay01/render.cpp" to "/tmp/distccd_1db50d83.ii"
            distccd[14752] (dcc_set_input) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/delay01/build/render.o /tmp/distccd_1db50d83.ii
            distccd[14752] (dcc_set_output) changed output from "/root/Bela/projects/delay01/build/render.o" to "/tmp/distccd_1dc50d83.o"
            distccd[14752] (dcc_set_output) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_1dc50d83.o /tmp/distccd_1db50d83.ii
            distccd[14752] (dcc_check_compiler_masq) /usr/local/bin/clang++-3.9-arm is not a symlink
            distccd[14752] (dcc_spawn_child) forking to execute: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_1dc50d83.o /tmp/distccd_1db50d83.ii
            distccd[14752] (dcc_spawn_child) child started as pid14971
            distccd[14971] (dcc_new_pgrp) entered process group
            distccd[14971] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
            distccd[14752] (dcc_collect_child) cc child 14971 terminated with status 0x100
            distccd[14752] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
            distccd[14752] (dcc_x_token_int) send DONE00000001
            distccd[14752] (dcc_x_token_int) send STAT00000100
            distccd[14752] (dcc_x_file) send 248 byte file /tmp/distcc_161f0d83.stderr with token SERR and compression 69
            distccd[14752] (dcc_x_token_int) send SERR000000f8
            distccd[14752] (dcc_x_file) send 0 byte file /tmp/distcc_16d70d83.stdout with token SOUT and compression 69
            distccd[14752] (dcc_x_token_int) send SOUT00000000
            distccd[14752] (dcc_x_token_int) send DOTO00000000
            distccd[14752] clang++-3.9-arm /root/Bela/projects/delay01/render.cpp on localhost failed
            distccd[14752] job complete
            distccd[14752] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
            distccd[14752] (dcc_job_summary) client: 192.168.7.2:59746 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:988ms clang++-3.9-arm /root/Bela/projects/delay01/render.cpp

            in the distcc log.

            On host console I get:

            echo 'Building render.cpp...'
            Building render.cpp...
            distcc-clang++  -I/root/Bela/projects/delay01 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/delay01/build/render.d" -o "/root/Bela/projects/delay01/build/render.o" "/root/Bela/projects/delay01/render.cpp" 
            distcc[3031] ERROR: compile /root/Bela/projects/delay01/render.cpp on 192.168.7.1 failed
            distcc[3031] (dcc_build_somewhere) Warning: remote compilation of '/root/Bela/projects/delay01/render.cpp' failed, retrying locally
            distcc[3031] (dcc_build_somewhere) Warning: failed to distribute and fallbacks are disabled
            /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
            Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
            clang: error: linker command failed with exit code 1 (use -v to see invocation)
            Makefile:434: recipe for target '/root/Bela/projects/delay01/build/render.o' failed
            make: *** [/root/Bela/projects/delay01/build/render.o] Error 1
            Connection to 192.168.7.2 closed.
            gary@audio-workstation:~/faust/examples/bela$ 

              Think you were talking about this before, but not sure I understood.

              ary@audio-workstation:~/faust/examples/bela$ cat /usr/local/bin/*arm
              #!/bin/bash
              /usr/bin/clang -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $@
              #!/bin/bash
              /usr/bin/clang++ -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $0
              gary@audio-workstation:~/faust/examples/bela$ 

              These don't currently point to clang-3.9. They just point to clang which is actually version 6. I think you were saying that it has to be version 3.9?

              However, this step works:

              distcc-clang -c ~/Bela/resources/network/udp-client.c
                gary@audio-workstation:~$ ld -V
                GNU ld (GNU Binutils for Ubuntu) 2.30
                  Supported emulations:
                   elf_x86_64
                   elf32_x86_64
                   elf_i386
                   elf_iamcu
                   i386linux
                   elf_l1om
                   elf_k1om
                   i386pep
                   i386pe
                gary@audio-workstation:~$ 

                Digital-Larry By "the Bela Makefile" I presume that you mean the file /root/Bela/Makefile.

                yes

                Digital-Larry On host console I get:

                I guess that's technically the "client" console (i.e.: what you see there is while you are logged in on Bela).

                So it seems that the compiler is failing, and my guess is that it is because it is being passed -march=armv7-a but no -target armv7l-unknown-linux-gnueabihf, and this - I guess - is because you are not calling the wrapper but you are calling the vanilla compiler directly (through the symlink you created).
                The point of the wrapper is exactly to automatically add -target armv7l-unknown-linux-gnueabihf to all calls.

                I think what is happening is what I mentioned in the other thread:

                giuliomoro But if you do put some symlinks there, they should be pointing NOT to the regular clang, but to the clang-3.9-arm and clang++-3.9-arm files you created in /usr/local/bin/, and they will eventually call clang with the appropriate option -target armv7l-unknown-linux-gnueabihf, to tell it to build code for ARM.

                Digital-Larry gary@audio-workstation:/usr/local/lib$ ls -l distcc/
                total 0
                lrwxrwxrwx 1 root root 14 May 28 17:45 clang-3.9-arm -> /usr/bin/clang
                lrwxrwxrwx 1 root root 16 May 28 17:45 clang++-3.9-arm -> /usr/bin/clang++

                Is this working? I find it surprising, but I guess it may depend on how you clang has been built. This way you are simply using the regular clang on your computer, but it should need a -target armv7l-unknown-linux-gnueabihf option in order to know it should build for arm (at least, it does with mine, which I installed with macports on Mac).

                basically, remove your current symlinks from /usr/local/lib/distcc/ and make some new ones that point NOT to the vanilla clang/clang++, but to the wrapper clang-3.9-arm/clang++-3.9-arm you created:

                sudo rm /usr/local/lib/distcc/*
                sudo ln -s /usr/local/bin/clang-3.9-arm/ /usr/local/lib/distcc/
                sudo ln -s /usr/local/bin/clang++-3.9-arm/ /usr/local/lib/distcc/

                Digital-Larry However, this step works:

                distcc-clang -c ~/Bela/resources/network/udp-client.c

                I see now how that can easily give false positives: it just checks that the server can compile, but does not check the nature of the resulted file. If you were to do file a.out, it would probably say it's an x86 file and not an ARM one.
                So the test should probably be amended to :

                distcc-clang -c ~/Bela/resources/network/udp-client.c -march=armv7-a

                This should then fail similarly to the how the Bela project is failing in your current configuration, and should be fixed with the updated symlinks.

                  OK let's see where we are now.

                  root@bela:~/Bela# grep distcc Makefile 
                    CLANG_PATH=distcc-clang
                  root@bela:~/Bela#

                  reference command line build on bela works (without specifying architecture):

                  root@bela:~/Bela/resources/network# distcc-clang -v -c udp-client.c
                  clang version 3.9.1-9 (tags/RELEASE_391/rc2)
                  Target: armv7l-unknown-linux-gnueabihf
                  Thread model: posix
                  InstalledDir: /usr/bin
                  Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6
                  Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
                  Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6
                  Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6.3.0
                  Selected GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
                  Candidate multilib: .;@m32
                  Selected multilib: .;@m32
                   "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name udp-client.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -fuse-init-array -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include -internal-externc-isystem /usr/include/arm-linux-gnueabihf -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /root/Bela/resources/network -ferror-limit 19 -fmessage-length 80 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o - -x c udp-client.c
                  clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target armv7l-unknown-linux-gnueabihf
                  ignoring nonexistent directory "/include"
                  #include "..." search starts here:
                  #include <...> search starts here:
                   /usr/local/include
                   /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include
                   /usr/include/arm-linux-gnueabihf
                   /usr/include
                  End of search list.
                  clang version 3.9.1-19ubuntu1 (tags/RELEASE_391/rc2)
                  Target: armv7l-unknown-linux-gnueabihf
                  Thread model: posix
                  InstalledDir: /usr/bin
                   "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name distccd_ba396596.i -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -coverage-file /tmp/distccd_ba666596.o -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 0 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/distccd_ba666596.o -x cpp-output /tmp/distccd_ba396596.i
                  clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target x86_64-pc-linux-gnu
                  #include "..." search starts here:
                  End of search list.
                  root@bela:~/Bela/resources/network# 

                  reference build works with specifying architecture:

                  root@bela:~/Bela/resources/network# distcc-clang -c -v ~/Bela/resources/network/udp-client.c -march=armv7-a
                  clang version 3.9.1-9 (tags/RELEASE_391/rc2)
                  Target: armv7l-unknown-linux-gnueabihf
                  Thread model: posix
                  InstalledDir: /usr/bin
                  Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6
                  Found candidate GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
                  Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6
                  Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/6.3.0
                  Selected GCC installation: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0
                  Candidate multilib: .;@m32
                  Selected multilib: .;@m32
                   "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name udp-client.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -fuse-init-array -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include -internal-externc-isystem /usr/include/arm-linux-gnueabihf -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /root/Bela/resources/network -ferror-limit 19 -fmessage-length 80 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o - -x c /root/Bela/resources/network/udp-client.c
                  clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target armv7l-unknown-linux-gnueabihf
                  ignoring nonexistent directory "/include"
                  #include "..." search starts here:
                  #include <...> search starts here:
                   /usr/local/include
                   /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/include
                   /usr/include/arm-linux-gnueabihf
                   /usr/include
                  End of search list.
                  clang version 3.9.1-19ubuntu1 (tags/RELEASE_391/rc2)
                  Target: armv7l-unknown-linux-gnueabihf
                  Thread model: posix
                  InstalledDir: /usr/bin
                   "/usr/lib/llvm-3.9/bin/clang" -cc1 -triple armv7-unknown-linux-gnueabihf -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name distccd_ff4c663b.i -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -target-cpu cortex-a8 -target-abi aapcs-linux -mfloat-abi hard -v -dwarf-column-info -debugger-tuning=gdb -coverage-file /tmp/distccd_fcec663b.o -resource-dir /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1 -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 0 -fallow-half-arguments-and-returns -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/distccd_fcec663b.o -x cpp-output /tmp/distccd_ff4c663b.i
                  clang -cc1 version 3.9.1 based upon LLVM 3.9.1 default target x86_64-pc-linux-gnu
                  #include "..." search starts here:
                  End of search list.
                  root@bela:~/Bela/resources/network# 

                  Here's the distcclog from the Ubuntu machine:

                  distccd[14752] (dcc_check_client) connection from 192.168.7.2:59770
                  distccd[14752] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
                  distccd[14752] (dcc_r_token_int) got DIST00000001
                  distccd[14752] (dcc_r_token_int) got ARGC00000006
                  distccd[14752] (dcc_r_argv) reading 6 arguments from job submission
                  distccd[14752] (dcc_r_token_int) got ARGV0000000d
                  distccd[14752] (dcc_r_token_string) got 'clang-3.9-arm'
                  distccd[14752] (dcc_r_argv) argv[0] = "clang-3.9-arm"
                  distccd[14752] (dcc_r_token_int) got ARGV00000002
                  distccd[14752] (dcc_r_token_string) got '-v'
                  distccd[14752] (dcc_r_argv) argv[1] = "-v"
                  distccd[14752] (dcc_r_token_int) got ARGV00000002
                  distccd[14752] (dcc_r_token_string) got '-c'
                  distccd[14752] (dcc_r_argv) argv[2] = "-c"
                  distccd[14752] (dcc_r_token_int) got ARGV0000000c
                  distccd[14752] (dcc_r_token_string) got 'udp-client.c'
                  distccd[14752] (dcc_r_argv) argv[3] = "udp-client.c"
                  distccd[14752] (dcc_r_token_int) got ARGV00000002
                  distccd[14752] (dcc_r_token_string) got '-o'
                  distccd[14752] (dcc_r_argv) argv[4] = "-o"
                  distccd[14752] (dcc_r_token_int) got ARGV0000000c
                  distccd[14752] (dcc_r_token_string) got 'udp-client.o'
                  distccd[14752] (dcc_r_argv) argv[5] = "udp-client.o"
                  distccd[14752] (dcc_r_argv) got arguments: clang-3.9-arm -v -c udp-client.c -o udp-client.o
                  distccd[14752] (dcc_scan_args) scanning arguments: clang-3.9-arm -v -c udp-client.c -o udp-client.o
                  distccd[14752] (dcc_scan_args) found input file "udp-client.c"
                  distccd[14752] (dcc_scan_args) found object/output file "udp-client.o"
                  distccd[14752] compile from udp-client.c to udp-client.o
                  distccd[14752] (dcc_run_job) output file udp-client.o
                  distccd[14752] (dcc_input_tmpnam) input file udp-client.c
                  distccd[14752] (dcc_r_token_int) got DOTI00016dac
                  distccd[14752] (dcc_r_file) received 93612 bytes to file /tmp/distccd_ba396596.i
                  distccd[14752] (dcc_r_file_timed) 93612 bytes received in 0.011009s, rate 8304kB/s
                  distccd[14752] (dcc_set_input) changed input from "udp-client.c" to "/tmp/distccd_ba396596.i"
                  distccd[14752] (dcc_set_input) command after: clang-3.9-arm -v -c /tmp/distccd_ba396596.i -o udp-client.o
                  distccd[14752] (dcc_set_output) changed output from "udp-client.o" to "/tmp/distccd_ba666596.o"
                  distccd[14752] (dcc_set_output) command after: clang-3.9-arm -v -c /tmp/distccd_ba396596.i -o /tmp/distccd_ba666596.o
                  distccd[14752] (dcc_check_compiler_masq) /usr/local/bin/clang-3.9-arm is not a symlink
                  distccd[14752] (dcc_spawn_child) forking to execute: clang-3.9-arm -v -c /tmp/distccd_ba396596.i -o /tmp/distccd_ba666596.o
                  distccd[14752] (dcc_spawn_child) child started as pid32524
                  distccd[32524] (dcc_new_pgrp) entered process group
                  distccd[32524] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
                  distccd[14752] (dcc_collect_child) cc child 32524 terminated with status 0
                  distccd[14752] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
                  distccd[14752] (dcc_x_token_int) send DONE00000001
                  distccd[14752] (dcc_x_token_int) send STAT00000000
                  distccd[14752] (dcc_x_file) send 1002 byte file /tmp/distcc_b8066596.stderr with token SERR and compression 69
                  distccd[14752] (dcc_x_token_int) send SERR000003ea
                  distccd[14752] (dcc_x_file) send 0 byte file /tmp/distcc_b8246596.stdout with token SOUT and compression 69
                  distccd[14752] (dcc_x_token_int) send SOUT00000000
                  distccd[14752] (dcc_x_file) send 2192 byte file /tmp/distccd_ba666596.o with token DOTO and compression 69
                  distccd[14752] (dcc_x_token_int) send DOTO00000890
                  distccd[14752] clang-3.9-arm udp-client.c on localhost completed ok
                  distccd[14752] job complete
                  distccd[14752] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
                  distccd[14752] (dcc_job_summary) client: 192.168.7.2:59770 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:318ms clang-3.9-arm udp-client.c
                  distccd[14753] (dcc_check_client) connection from 192.168.7.2:59772
                  distccd[14753] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
                  distccd[14753] (dcc_r_token_int) got DIST00000001
                  distccd[14753] (dcc_r_token_int) got ARGC00000006
                  distccd[14753] (dcc_r_argv) reading 6 arguments from job submission
                  distccd[14753] (dcc_r_token_int) got ARGV0000000d
                  distccd[14753] (dcc_r_token_string) got 'clang-3.9-arm'
                  distccd[14753] (dcc_r_argv) argv[0] = "clang-3.9-arm"
                  distccd[14753] (dcc_r_token_int) got ARGV00000002
                  distccd[14753] (dcc_r_token_string) got '-c'
                  distccd[14753] (dcc_r_argv) argv[1] = "-c"
                  distccd[14753] (dcc_r_token_int) got ARGV00000029
                  distccd[14753] (dcc_r_token_string) got '/root/Bela/resources/network/udp-client.c'
                  distccd[14753] (dcc_r_argv) argv[2] = "/root/Bela/resources/network/udp-client.c"
                  distccd[14753] (dcc_r_token_int) got ARGV0000000e
                  distccd[14753] (dcc_r_token_string) got '-march=armv7-a'
                  distccd[14753] (dcc_r_argv) argv[3] = "-march=armv7-a"
                  distccd[14753] (dcc_r_token_int) got ARGV00000002
                  distccd[14753] (dcc_r_token_string) got '-o'
                  distccd[14753] (dcc_r_argv) argv[4] = "-o"
                  distccd[14753] (dcc_r_token_int) got ARGV0000000c
                  distccd[14753] (dcc_r_token_string) got 'udp-client.o'
                  distccd[14753] (dcc_r_argv) argv[5] = "udp-client.o"
                  distccd[14753] (dcc_r_argv) got arguments: clang-3.9-arm -c /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
                  distccd[14753] (dcc_scan_args) scanning arguments: clang-3.9-arm -c /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
                  distccd[14753] (dcc_scan_args) found input file "/root/Bela/resources/network/udp-client.c"
                  distccd[14753] (dcc_scan_args) found object/output file "udp-client.o"
                  distccd[14753] compile from udp-client.c to udp-client.o
                  distccd[14753] (dcc_run_job) output file udp-client.o
                  distccd[14753] (dcc_input_tmpnam) input file /root/Bela/resources/network/udp-client.c
                  distccd[14753] (dcc_r_token_int) got DOTI00016eeb
                  distccd[14753] (dcc_r_file) received 93931 bytes to file /tmp/distccd_20816628.i
                  distccd[14753] (dcc_r_file_timed) 93931 bytes received in 0.012232s, rate 7499kB/s
                  distccd[14753] (dcc_set_input) changed input from "/root/Bela/resources/network/udp-client.c" to "/tmp/distccd_20816628.i"
                  distccd[14753] (dcc_set_input) command after: clang-3.9-arm -c /tmp/distccd_20816628.i -march=armv7-a -o udp-client.o
                  distccd[14753] (dcc_set_output) changed output from "udp-client.o" to "/tmp/distccd_20dc6628.o"
                  distccd[14753] (dcc_set_output) command after: clang-3.9-arm -c /tmp/distccd_20816628.i -march=armv7-a -o /tmp/distccd_20dc6628.o
                  distccd[14753] (dcc_check_compiler_masq) /usr/local/bin/clang-3.9-arm is not a symlink
                  distccd[14753] (dcc_spawn_child) forking to execute: clang-3.9-arm -c /tmp/distccd_20816628.i -march=armv7-a -o /tmp/distccd_20dc6628.o
                  distccd[14753] (dcc_spawn_child) child started as pid32547
                  distccd[32547] (dcc_new_pgrp) entered process group
                  distccd[32547] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
                  distccd[14753] (dcc_collect_child) cc child 32547 terminated with status 0
                  distccd[14753] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
                  distccd[14753] (dcc_x_token_int) send DONE00000001
                  distccd[14753] (dcc_x_token_int) send STAT00000000
                  distccd[14753] (dcc_x_file) send 0 byte file /tmp/distcc_27c16628.stderr with token SERR and compression 69
                  distccd[14753] (dcc_x_token_int) send SERR00000000
                  distccd[14753] (dcc_x_file) send 0 byte file /tmp/distcc_27996628.stdout with token SOUT and compression 69
                  distccd[14753] (dcc_x_token_int) send SOUT00000000
                  distccd[14753] (dcc_x_file) send 2192 byte file /tmp/distccd_20dc6628.o with token DOTO and compression 69
                  distccd[14753] (dcc_x_token_int) send DOTO00000890
                  distccd[14753] clang-3.9-arm /root/Bela/resources/network/udp-client.c on localhost completed ok
                  distccd[14753] job complete
                  distccd[14753] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
                  distccd[14753] (dcc_job_summary) client: 192.168.7.2:59772 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:286ms clang-3.9-arm /root/Bela/resources/network/udp-client.c
                  distccd[14754] (dcc_check_client) connection from 192.168.7.2:59774
                  distccd[14754] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
                  distccd[14754] (dcc_r_token_int) got DIST00000001
                  distccd[14754] (dcc_r_token_int) got ARGC00000007
                  distccd[14754] (dcc_r_argv) reading 7 arguments from job submission
                  distccd[14754] (dcc_r_token_int) got ARGV0000000d
                  distccd[14754] (dcc_r_token_string) got 'clang-3.9-arm'
                  distccd[14754] (dcc_r_argv) argv[0] = "clang-3.9-arm"
                  distccd[14754] (dcc_r_token_int) got ARGV00000002
                  distccd[14754] (dcc_r_token_string) got '-c'
                  distccd[14754] (dcc_r_argv) argv[1] = "-c"
                  distccd[14754] (dcc_r_token_int) got ARGV00000002
                  distccd[14754] (dcc_r_token_string) got '-v'
                  distccd[14754] (dcc_r_argv) argv[2] = "-v"
                  distccd[14754] (dcc_r_token_int) got ARGV00000029
                  distccd[14754] (dcc_r_token_string) got '/root/Bela/resources/network/udp-client.c'
                  distccd[14754] (dcc_r_argv) argv[3] = "/root/Bela/resources/network/udp-client.c"
                  distccd[14754] (dcc_r_token_int) got ARGV0000000e
                  distccd[14754] (dcc_r_token_string) got '-march=armv7-a'
                  distccd[14754] (dcc_r_argv) argv[4] = "-march=armv7-a"
                  distccd[14754] (dcc_r_token_int) got ARGV00000002
                  distccd[14754] (dcc_r_token_string) got '-o'
                  distccd[14754] (dcc_r_argv) argv[5] = "-o"
                  distccd[14754] (dcc_r_token_int) got ARGV0000000c
                  distccd[14754] (dcc_r_token_string) got 'udp-client.o'
                  distccd[14754] (dcc_r_argv) argv[6] = "udp-client.o"
                  distccd[14754] (dcc_r_argv) got arguments: clang-3.9-arm -c -v /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
                  distccd[14754] (dcc_scan_args) scanning arguments: clang-3.9-arm -c -v /root/Bela/resources/network/udp-client.c -march=armv7-a -o udp-client.o
                  distccd[14754] (dcc_scan_args) found input file "/root/Bela/resources/network/udp-client.c"
                  distccd[14754] (dcc_scan_args) found object/output file "udp-client.o"
                  distccd[14754] compile from udp-client.c to udp-client.o
                  distccd[14754] (dcc_run_job) output file udp-client.o
                  distccd[14754] (dcc_input_tmpnam) input file /root/Bela/resources/network/udp-client.c
                  distccd[14754] (dcc_r_token_int) got DOTI00016eeb
                  distccd[14754] (dcc_r_file) received 93931 bytes to file /tmp/distccd_ff4c663b.i
                  distccd[14754] (dcc_r_file_timed) 93931 bytes received in 0.011458s, rate 8006kB/s
                  distccd[14754] (dcc_set_input) changed input from "/root/Bela/resources/network/udp-client.c" to "/tmp/distccd_ff4c663b.i"
                  distccd[14754] (dcc_set_input) command after: clang-3.9-arm -c -v /tmp/distccd_ff4c663b.i -march=armv7-a -o udp-client.o
                  distccd[14754] (dcc_set_output) changed output from "udp-client.o" to "/tmp/distccd_fcec663b.o"
                  distccd[14754] (dcc_set_output) command after: clang-3.9-arm -c -v /tmp/distccd_ff4c663b.i -march=armv7-a -o /tmp/distccd_fcec663b.o
                  distccd[14754] (dcc_check_compiler_masq) /usr/local/bin/clang-3.9-arm is not a symlink
                  distccd[14754] (dcc_spawn_child) forking to execute: clang-3.9-arm -c -v /tmp/distccd_ff4c663b.i -march=armv7-a -o /tmp/distccd_fcec663b.o
                  distccd[14754] (dcc_spawn_child) child started as pid32551
                  distccd[32551] (dcc_new_pgrp) entered process group
                  distccd[32551] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
                  distccd[14754] (dcc_collect_child) cc child 32551 terminated with status 0
                  distccd[14754] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
                  distccd[14754] (dcc_x_token_int) send DONE00000001
                  distccd[14754] (dcc_x_token_int) send STAT00000000
                  distccd[14754] (dcc_x_file) send 1002 byte file /tmp/distcc_f203663b.stderr with token SERR and compression 69
                  distccd[14754] (dcc_x_token_int) send SERR000003ea
                  distccd[14754] (dcc_x_file) send 0 byte file /tmp/distcc_f22c663b.stdout with token SOUT and compression 69
                  distccd[14754] (dcc_x_token_int) send SOUT00000000
                  distccd[14754] (dcc_x_file) send 2192 byte file /tmp/distccd_fcec663b.o with token DOTO and compression 69
                  distccd[14754] (dcc_x_token_int) send DOTO00000890
                  distccd[14754] clang-3.9-arm /root/Bela/resources/network/udp-client.c on localhost completed ok
                  distccd[14754] job complete
                  distccd[14754] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
                  distccd[14754] (dcc_job_summary) client: 192.168.7.2:59774 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:320ms clang-3.9-arm /root/Bela/resources/network/udp-client.c

                  Now trying "faust2bela" from the Ubuntu machine. /usr/Bela/Makefile still references distcc.

                  gary@audio-workstation:~/faust/examples/bela$ faust2bela -tobela -gui FXChaine2.dsp 
                  Send to bela
                  /home/gary/faust/examples/bela/FXChaine2
                  Start communication with bela
                  /home/gary/faust/examples/bela/FXChaine2
                  FXChaine2
                  Checking the board is up and running at root@192.168.7.2...done
                  Stop running process...
                  Run bela now
                  uploadBuildRun START
                  Copying new source files to BeagleBone...Using rsync...
                  Automatically detected PROJECT_TYPE: cpp 
                  Running on __stretch__ with Xenomai __3__
                  systemctl stop bela_startup || true
                  PID=`grep bela-audio  /proc/xenomai/sched/stat | cut -d " " -f 5 | sed s/\s//g`; if [ -z $PID ]; then [ true = true ] || echo "No process to kill"; else [  true = true  ] || echo "Killing old Bela process $PID"; kill -2 $PID; sleep 0.2; kill -9 $PID 2> /dev/null; fi; screen -X -S Bela quit > /dev/null; exit 0;
                  killall scsynth 2>/dev/null& killall sclang 2>/dev/null& true
                  echo 'Building render.cpp...'
                  Building render.cpp...
                  distcc-clang++  -I/root/Bela/projects/FXChaine2 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/FXChaine2/build/render.d" -o "/root/Bela/projects/FXChaine2/build/render.o" "/root/Bela/projects/FXChaine2/render.cpp" 
                  distcc[5035] ERROR: compile /root/Bela/projects/FXChaine2/render.cpp on 192.168.7.1 failed
                  distcc[5035] (dcc_build_somewhere) Warning: remote compilation of '/root/Bela/projects/FXChaine2/render.cpp' failed, retrying locally
                  distcc[5035] (dcc_build_somewhere) Warning: failed to distribute and fallbacks are disabled
                  /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
                  Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
                  clang: error: linker command failed with exit code 1 (use -v to see invocation)
                  Makefile:434: recipe for target '/root/Bela/projects/FXChaine2/build/render.o' failed
                  make: *** [/root/Bela/projects/FXChaine2/build/render.o] Error 1
                  Connection to 192.168.7.2 closed.
                  gary@audio-workstation:~/faust/examples/bela$ 

                  This still seems to be the failure:

                  /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
                  Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe

                  This link takes us to some hints on that error message, which seem to culminate in "install the toolchain". The question is in the context of Android NDK though so I don't know if it applies here.

                  https://stackoverflow.com/questions/52646860/ld-unrecognised-emulation-mode-armelf-linux-eabi-when-cross-compiling-with-a

                  Here's the distcc log from this step:

                  distccd[14755] (dcc_check_client) connection from 192.168.7.2:59776
                  distccd[14755] (check_address_inet) match client 0x207a8c0, value 0x207a8c0, mask 0xffffffff
                  distccd[14755] (dcc_r_token_int) got DIST00000001
                  distccd[14755] (dcc_r_token_int) got ARGC00000013
                  distccd[14755] (dcc_r_argv) reading 19 arguments from job submission
                  distccd[14755] (dcc_r_token_int) got ARGV0000000f
                  distccd[14755] (dcc_r_token_string) got 'clang++-3.9-arm'
                  distccd[14755] (dcc_r_argv) argv[0] = "clang++-3.9-arm"
                  distccd[14755] (dcc_r_token_int) got ARGV0000000e
                  distccd[14755] (dcc_r_token_string) got '-march=armv7-a'
                  distccd[14755] (dcc_r_argv) argv[1] = "-march=armv7-a"
                  distccd[14755] (dcc_r_token_int) got ARGV0000000a
                  distccd[14755] (dcc_r_token_string) got '-mfpu=vfp3'
                  distccd[14755] (dcc_r_argv) argv[2] = "-mfpu=vfp3"
                  distccd[14755] (dcc_r_token_int) got ARGV0000001c
                  distccd[14755] (dcc_r_token_string) got '-fasynchronous-unwind-tables'
                  distccd[14755] (dcc_r_argv) argv[3] = "-fasynchronous-unwind-tables"
                  distccd[14755] (dcc_r_token_int) got ARGV00000003
                  distccd[14755] (dcc_r_token_string) got '-O3'
                  distccd[14755] (dcc_r_argv) argv[4] = "-O3"
                  distccd[14755] (dcc_r_token_int) got ARGV0000000e
                  distccd[14755] (dcc_r_token_string) got '-march=armv7-a'
                  distccd[14755] (dcc_r_argv) argv[5] = "-march=armv7-a"
                  distccd[14755] (dcc_r_token_int) got ARGV00000010
                  distccd[14755] (dcc_r_token_string) got '-mtune=cortex-a8'
                  distccd[14755] (dcc_r_argv) argv[6] = "-mtune=cortex-a8"
                  distccd[14755] (dcc_r_token_int) got ARGV00000010
                  distccd[14755] (dcc_r_token_string) got '-mfloat-abi=hard'
                  distccd[14755] (dcc_r_argv) argv[7] = "-mfloat-abi=hard"
                  distccd[14755] (dcc_r_token_int) got ARGV0000000a
                  distccd[14755] (dcc_r_token_string) got '-mfpu=neon'
                  distccd[14755] (dcc_r_argv) argv[8] = "-mfpu=neon"
                  distccd[14755] (dcc_r_token_int) got ARGV00000010
                  distccd[14755] (dcc_r_token_string) got '-ftree-vectorize'
                  distccd[14755] (dcc_r_argv) argv[9] = "-ftree-vectorize"
                  distccd[14755] (dcc_r_token_int) got ARGV0000000b
                  distccd[14755] (dcc_r_token_string) got '-ffast-math'
                  distccd[14755] (dcc_r_argv) argv[10] = "-ffast-math"
                  distccd[14755] (dcc_r_token_int) got ARGV0000000a
                  distccd[14755] (dcc_r_token_string) got '-std=c++11'
                  distccd[14755] (dcc_r_argv) argv[11] = "-std=c++11"
                  distccd[14755] (dcc_r_token_int) got ARGV0000000c
                  distccd[14755] (dcc_r_token_string) got '-Wno-varargs'
                  distccd[14755] (dcc_r_argv) argv[12] = "-Wno-varargs"
                  distccd[14755] (dcc_r_token_int) got ARGV00000005
                  distccd[14755] (dcc_r_token_string) got '-Wall'
                  distccd[14755] (dcc_r_argv) argv[13] = "-Wall"
                  distccd[14755] (dcc_r_token_int) got ARGV00000002
                  distccd[14755] (dcc_r_token_string) got '-c'
                  distccd[14755] (dcc_r_argv) argv[14] = "-c"
                  distccd[14755] (dcc_r_token_int) got ARGV00000012
                  distccd[14755] (dcc_r_token_string) got '-fmessage-length=0'
                  distccd[14755] (dcc_r_argv) argv[15] = "-fmessage-length=0"
                  distccd[14755] (dcc_r_token_int) got ARGV00000002
                  distccd[14755] (dcc_r_token_string) got '-o'
                  distccd[14755] (dcc_r_argv) argv[16] = "-o"
                  distccd[14755] (dcc_r_token_int) got ARGV0000002c
                  distccd[14755] (dcc_r_token_string) got '/root/Bela/projects/FXChaine2/build/render.o'
                  distccd[14755] (dcc_r_argv) argv[17] = "/root/Bela/projects/FXChaine2/build/render.o"
                  distccd[14755] (dcc_r_token_int) got ARGV00000028
                  distccd[14755] (dcc_r_token_string) got '/root/Bela/projects/FXChaine2/render.cpp'
                  distccd[14755] (dcc_r_argv) argv[18] = "/root/Bela/projects/FXChaine2/render.cpp"
                  distccd[14755] (dcc_r_argv) got arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/FXChaine2/build/render.o /root/Bela/projects/FXChaine2/render.cpp
                  distccd[14755] (dcc_scan_args) scanning arguments: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/FXChaine2/build/render.o /root/Bela/projects/FXChaine2/render.cpp
                  distccd[14755] (dcc_scan_args) found object/output file "/root/Bela/projects/FXChaine2/build/render.o"
                  distccd[14755] (dcc_scan_args) found input file "/root/Bela/projects/FXChaine2/render.cpp"
                  distccd[14755] compile from render.cpp to render.o
                  distccd[14755] (dcc_run_job) output file /root/Bela/projects/FXChaine2/build/render.o
                  distccd[14755] (dcc_input_tmpnam) input file /root/Bela/projects/FXChaine2/render.cpp
                  distccd[14755] (dcc_r_token_int) got DOTI0019610a
                  distccd[14755] (dcc_r_file) received 1663242 bytes to file /tmp/distccd_261c6794.ii
                  distccd[14755] (dcc_r_file_timed) 1663242 bytes received in 0.150477s, rate 10794kB/s
                  distccd[14755] (dcc_set_input) changed input from "/root/Bela/projects/FXChaine2/render.cpp" to "/tmp/distccd_261c6794.ii"
                  distccd[14755] (dcc_set_input) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /root/Bela/projects/FXChaine2/build/render.o /tmp/distccd_261c6794.ii
                  distccd[14755] (dcc_set_output) changed output from "/root/Bela/projects/FXChaine2/build/render.o" to "/tmp/distccd_27a96794.o"
                  distccd[14755] (dcc_set_output) command after: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_27a96794.o /tmp/distccd_261c6794.ii
                  distccd[14755] (dcc_check_compiler_masq) /usr/local/bin/clang++-3.9-arm is not a symlink
                  distccd[14755] (dcc_spawn_child) forking to execute: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_27a96794.o /tmp/distccd_261c6794.ii
                  distccd[14755] (dcc_spawn_child) child started as pid32659
                  distccd[32659] (dcc_new_pgrp) entered process group
                  distccd[32659] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
                  distccd[14755] (dcc_collect_child) cc child 32659 terminated with status 0x100
                  distccd[14755] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
                  distccd[14755] (dcc_x_token_int) send DONE00000001
                  distccd[14755] (dcc_x_token_int) send STAT00000100
                  distccd[14755] (dcc_x_file) send 248 byte file /tmp/distcc_58bf6794.stderr with token SERR and compression 69
                  distccd[14755] (dcc_x_token_int) send SERR000000f8
                  distccd[14755] (dcc_x_file) send 0 byte file /tmp/distcc_5b7e6794.stdout with token SOUT and compression 69
                  distccd[14755] (dcc_x_token_int) send SOUT00000000
                  distccd[14755] (dcc_x_token_int) send DOTO00000000
                  distccd[14755] clang++-3.9-arm /root/Bela/projects/FXChaine2/render.cpp on localhost failed
                  distccd[14755] job complete
                  distccd[14755] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
                  distccd[14755] (dcc_job_summary) client: 192.168.7.2:59776 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:1109ms clang++-3.9-arm /root/Bela/projects/FXChaine2/render.cpp

                    This is very confusing to me. The error message you get seems to be about the linker, ld, and it seems to come from the host, because of the list of supported emulations. The weird thing about this is that distcc should never attempt to run the linker on the host (only the compiler), and also that the line that fails really has nothing to do with the linker.

                    So this is the line that is run on the host and fails:

                    Digital-Larry distccd[14755] (dcc_spawn_child) forking to execute: clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o /tmp/distccd_27a96794.o /tmp/distccd_261c6794.ii

                    As you see, it tries to compile the pre-processed file /tmp/distccd_261c6794.ii into the object file /tmp/distccd_27a96794.o. The pre-processed file is generated by distcc on the client (Bela), where it runs cpp (the C-preprocessor) on the file to bring in all the includes and apply all the defines and resolve the macros, generating a stand-alone C++ file that has no dependency on any external files.

                    Let's troubleshoot this with incremental steps. Create a simple C++ file with no includes on the host:

                    • Create a simple C++ file with no includes on the HOST: create a file called test.ii:
                      int func()
                      {
                          return 0;
                      }
                      and then run the same command as above (except for the file paths at the end) :
                      clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.ii
                      if this is successful, make sure the generated file test.o is indeed an ARM binary:
                      $ file test.o
                      test.o: ELF 32-bit LSB relocatable, ARM, version 1 (SYSV), not stripped
                    • if the above worked fine, then copy the test.ii file to the board and run ON THE BOARD:
                      distcc-clang++ -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.ii
                      and if this is successful, check that the generated file is indeed an ARM binary:
                      $ file test.o
                      test.o: ELF 32-bit LSB relocatable, ARM, version 1 (SYSV), not stripped
                    • if this is still good, then rename test.ii to test.cpp and run the same line again, this time with test.cpp at the end:
                      distcc-clang++ -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.cpp
                    • if this is still good, still on the board, do cd /root/Bela/ and there try to re-run the full line that was failing while building your project:
                      distcc-clang++  -I/root/Bela/projects/FXChaine2 -I./include -I./build/pru/ -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -march=armv7-a -mfpu=vfp3 -D_GNU_SOURCE -D_REENTRANT -fasynchronous-unwind-tables -D__COBALT__ -D__COBALT_WRAP__ -DXENOMAI_SKIN_posix -DXENOMAI_MAJOR=3 -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -DNDEBUG -DBELA_USE_RTDM -I/root/Bela/resources/stretch/include -std=c++11 -Wno-varargs -DNDEBUG  -Wall -c -fmessage-length=0 -U_FORTIFY_SOURCE -MMD -MP -MF"/root/Bela/projects/FXChaine2/build/render.d" -o "/root/Bela/projects/FXChaine2/build/render.o" "/root/Bela/projects/FXChaine2/render.cpp" 
                      does this successfully generated the file /root/Bela/projects/FXChaine2/build/render.o? And is it an ARM binary?

                      giuliomoro first step fails.

                      gary@audio-workstation:~/test$ clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.ii
                      /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
                      Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
                      clang: error: linker command failed with exit code 1 (use -v to see invocation)
                      
                      gary@audio-workstation:~/test$ cat test.ii
                      int func()
                      {
                          return 0;
                      }
                      gary@audio-workstation:~/test$ 
                      
                      gary@audio-workstation:~/test$ which clang++-3.9-arm 
                      /usr/local/bin/clang++-3.9-arm
                      
                      gary@audio-workstation:~/test$ cat /usr/local/bin/clang++-3.9-arm 
                      #!/bin/bash
                      /usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $0
                      
                      gary@audio-workstation:~/test$ clang++-3.9-arm 
                      /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
                      Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
                      clang: error: linker command failed with exit code 1 (use -v to see invocation)
                      gary@audio-workstation:~/test$ 

                      Calling that compiler throws the error with no parameters given at all! [edit - it's calling the script which does add parameters] So maybe that is broken somehow. I used apt to install it.

                      gary@audio-workstation:~/test$ sudo apt list | grep ^clang-3.9
                      
                      WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
                      
                      clang-3.9/bionic,now 1:3.9.1-19ubuntu1 amd64 [installed]
                      clang-3.9-doc/bionic,bionic 1:3.9.1-19ubuntu1 all
                      clang-3.9-examples/bionic 1:3.9.1-19ubuntu1 amd64
                      gary@audio-workstation:~/test$ 
                      
                      gary@audio-workstation:~/test$ sudo dpkg -s clang-3.9
                      Package: clang-3.9
                      Status: install ok installed
                      Priority: optional
                      Section: devel
                      Installed-Size: 140661
                      Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
                      Architecture: amd64
                      Source: llvm-toolchain-3.9
                      Version: 1:3.9.1-19ubuntu1
                      Replaces: clang-3.1, clang-3.2, clang-3.3, clang-3.4 (<< 1:3.4.2-7~exp1), clang-3.5 (<< 1:3.5~+rc1-3~exp1), clang-include-fixer-3.9, compiler-rt
                      Provides: c++-compiler, c-compiler, objc-compiler
                      Depends: libc6 (>= 2.14), libclang1-3.9 (= 1:3.9.1-19ubuntu1), libgcc1 (>= 1:3.4), libjsoncpp1 (>= 1.7.4), libllvm3.9 (>= 1:3.9.1-6~), libstdc++6 (>= 5.2), libstdc++-7-dev, libgcc-7-dev, libobjc-7-dev, libclang-common-3.9-dev (= 1:3.9.1-19ubuntu1), libc6-dev, binutils
                      Recommends: llvm-3.9-dev, python
                      Suggests: gnustep, gnustep-devel, clang-3.9-doc
                      Breaks: clang-3.1, clang-3.2, clang-3.3, clang-3.4 (<< 1:3.4.2-7~exp1), clang-3.5 (<< 1:3.5~+rc1-3~exp1), clang-include-fixer-3.9, compiler-rt
                      Description: C, C++ and Objective-C compiler (LLVM based)
                       Clang project is a C, C++, Objective C and Objective C++ front-end
                       for the LLVM compiler. Its goal is to offer a replacement to the GNU Compiler
                       Collection (GCC).
                       .
                       Clang fully implements all published ISO C++ standards including C++11, as
                       well as the upcoming C++14 standard, and some parts of the fledgling C++1z
                       standard, and is considered a production-quality C++ compiler.
                      Original-Maintainer: LLVM Packaging Team <pkg-llvm-team@lists.alioth.debian.org>
                      Homepage: http://www.llvm.org/
                      gary@audio-workstation:~/test$ 

                      Some other random tests to see what exactly we are calling and the simplest command to generate the error.

                      gary@audio-workstation:~/test$ which clang++-3.9-arm 
                      /usr/local/bin/clang++-3.9-arm
                      gary@audio-workstation:~/test$ cat /usr/local/bin/clang++-3.9-arm 
                      #!/bin/bash
                      /usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $0
                      gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 
                      clang: error: no input files
                      gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf
                      clang: error: no input files
                      gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf ./test.ii
                      /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
                      Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
                      clang: error: linker command failed with exit code 1 (use -v to see invocation)
                      gary@audio-workstation:~/test$ /usr/bin/clang++-6.0 -target armv7l-unknown-linux-gnueabihf ./test.ii
                      /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
                      Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
                      clang: error: linker command failed with exit code 1 (use -v to see invocation)
                      gary@audio-workstation:~/test$ /usr/bin/clang++ -target armv7l-unknown-linux-gnueabihf ./test.ii
                      /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
                      Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
                      clang: error: linker command failed with exit code 1 (use -v to see invocation)
                      gary@audio-workstation:~/test$