giuliomoro first step fails.

gary@audio-workstation:~/test$ clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.ii
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)

gary@audio-workstation:~/test$ cat test.ii
int func()
{
    return 0;
}
gary@audio-workstation:~/test$ 

gary@audio-workstation:~/test$ which clang++-3.9-arm 
/usr/local/bin/clang++-3.9-arm

gary@audio-workstation:~/test$ cat /usr/local/bin/clang++-3.9-arm 
#!/bin/bash
/usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $0

gary@audio-workstation:~/test$ clang++-3.9-arm 
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gary@audio-workstation:~/test$ 

Calling that compiler throws the error with no parameters given at all! [edit - it's calling the script which does add parameters] So maybe that is broken somehow. I used apt to install it.

gary@audio-workstation:~/test$ sudo apt list | grep ^clang-3.9

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

clang-3.9/bionic,now 1:3.9.1-19ubuntu1 amd64 [installed]
clang-3.9-doc/bionic,bionic 1:3.9.1-19ubuntu1 all
clang-3.9-examples/bionic 1:3.9.1-19ubuntu1 amd64
gary@audio-workstation:~/test$ 

gary@audio-workstation:~/test$ sudo dpkg -s clang-3.9
Package: clang-3.9
Status: install ok installed
Priority: optional
Section: devel
Installed-Size: 140661
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: amd64
Source: llvm-toolchain-3.9
Version: 1:3.9.1-19ubuntu1
Replaces: clang-3.1, clang-3.2, clang-3.3, clang-3.4 (<< 1:3.4.2-7~exp1), clang-3.5 (<< 1:3.5~+rc1-3~exp1), clang-include-fixer-3.9, compiler-rt
Provides: c++-compiler, c-compiler, objc-compiler
Depends: libc6 (>= 2.14), libclang1-3.9 (= 1:3.9.1-19ubuntu1), libgcc1 (>= 1:3.4), libjsoncpp1 (>= 1.7.4), libllvm3.9 (>= 1:3.9.1-6~), libstdc++6 (>= 5.2), libstdc++-7-dev, libgcc-7-dev, libobjc-7-dev, libclang-common-3.9-dev (= 1:3.9.1-19ubuntu1), libc6-dev, binutils
Recommends: llvm-3.9-dev, python
Suggests: gnustep, gnustep-devel, clang-3.9-doc
Breaks: clang-3.1, clang-3.2, clang-3.3, clang-3.4 (<< 1:3.4.2-7~exp1), clang-3.5 (<< 1:3.5~+rc1-3~exp1), clang-include-fixer-3.9, compiler-rt
Description: C, C++ and Objective-C compiler (LLVM based)
 Clang project is a C, C++, Objective C and Objective C++ front-end
 for the LLVM compiler. Its goal is to offer a replacement to the GNU Compiler
 Collection (GCC).
 .
 Clang fully implements all published ISO C++ standards including C++11, as
 well as the upcoming C++14 standard, and some parts of the fledgling C++1z
 standard, and is considered a production-quality C++ compiler.
Original-Maintainer: LLVM Packaging Team <pkg-llvm-team@lists.alioth.debian.org>
Homepage: http://www.llvm.org/
gary@audio-workstation:~/test$ 

Some other random tests to see what exactly we are calling and the simplest command to generate the error.

gary@audio-workstation:~/test$ which clang++-3.9-arm 
/usr/local/bin/clang++-3.9-arm
gary@audio-workstation:~/test$ cat /usr/local/bin/clang++-3.9-arm 
#!/bin/bash
/usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $0
gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 
clang: error: no input files
gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf
clang: error: no input files
gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf ./test.ii
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gary@audio-workstation:~/test$ /usr/bin/clang++-6.0 -target armv7l-unknown-linux-gnueabihf ./test.ii
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gary@audio-workstation:~/test$ /usr/bin/clang++ -target armv7l-unknown-linux-gnueabihf ./test.ii
/usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gary@audio-workstation:~/test$ 

    right, what about ls -l /usr/bin/clang++-3.9 ? is it a symlink to something?
    Further to this, you could try running

    clang++-3.9-arm -march=armv7-a -mfpu=vfp3 -fasynchronous-unwind-tables -O3 -march=armv7-a -mtune=cortex-a8 -mfloat-abi=hard -mfpu=neon -ftree-vectorize -ffast-math -std=c++11 -Wno-varargs -Wall -c -fmessage-length=0 -o ./test.o ./test.ii

    and remove one option at a time until it succeeds (or better: start with just clang++-3.9-arm -march=armv7-a ./test.o ./test.ii, is it successful? Then add more and more options until it fails .

    I am wondering whether maybe your clang-3.9 does not support hard-float, in which case the failing option should be -mfloat-abi=hard. Can you dpkg -l | grep clang and perhaps then dpkg --info <names of the 3.9 packages from the previous result>?

      I used the -### command with just clang++ (the 6.0 version installed by default, but the error is the same so far regardless of the compiler version used, 3.9 or 6.0):

      gary@audio-workstation:~/test$ /usr/bin/clang++ -### -target armv7l-unknown-linux-gnueabihf ./test.ii
      clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
      Target: armv7l-unknown-linux-gnueabihf
      Thread model: posix
      InstalledDir: /usr/bin
       "/usr/lib/llvm-6.0/bin/clang" "-cc1" "-triple" "armv7-unknown-linux-gnueabihf" "-emit-obj" "-mrelax-all" "-disable-free" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "test.ii" "-mrelocation-model" "static" "-mthread-model" "posix" "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases" "-fuse-init-array" "-target-cpu" "generic" "-target-abi" "aapcs-linux" "-mfloat-abi" "hard" "-fallow-half-arguments-and-returns" "-dwarf-column-info" "-debugger-tuning=gdb" "-resource-dir" "/usr/lib/llvm-6.0/lib/clang/6.0.0" "-fdeprecated-macro" "-fdebug-compilation-dir" "/home/gary/test" "-ferror-limit" "19" "-fmessage-length" "80" "-fno-signed-char" "-fobjc-runtime=gcc" "-fcxx-exceptions" "-fexceptions" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "/tmp/test-dc97e4.o" "-x" "c++-cpp-output" "./test.ii"
       "/usr/bin/ld" "-z" "relro" "-X" "--hash-style=gnu" "--eh-frame-hdr" "-m" "armelf_linux_eabi" "-dynamic-linker" "/lib/ld-linux-armhf.so.3" "-o" "a.out" "crt1.o" "crti.o" "crtbegin.o" "-L/usr/lib/llvm-6.0/bin/../lib" "-L/lib/../lib" "-L/usr/lib/../lib" "-L/usr/lib/llvm-6.0/bin/../lib" "-L/lib" "-L/usr/lib" "/tmp/test-dc97e4.o" "-lstdc++" "-lm" "-lgcc_s" "-lgcc" "-lc" "-lgcc_s" "-lgcc" "crtend.o" "crtn.o"
      gary@audio-workstation:~/test$ 

      giuliomoro yes there is a symlink there that appears to be from the installation (I didn't do it).

      gary@audio-workstation:~/test$ ls -l /usr/bin/clang++-3.9 
      lrwxrwxrwx 1 root root 27 Dec 26  2017 /usr/bin/clang++-3.9 -> ../lib/llvm-3.9/bin/clang++
      gary@audio-workstation:~/test$
      gary@audio-workstation:~/test$ clang++-3.9-arm -march=armv7-a ./test.o ./test.ii
      /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
      Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
      clang: error: linker command failed with exit code 1 (use -v to see invocation)
      gary@audio-workstation:~/test$
      
      gary@audio-workstation:~/test$ dpkg -l | grep clang
      ii  clang                                      1:6.0-41~exp5~ubuntu1                        amd64        C, C++ and Objective-C compiler (LLVM based)
      ii  clang-3.9                                  1:3.9.1-19ubuntu1                            amd64        C, C++ and Objective-C compiler (LLVM based)
      ii  clang-6.0                                  1:6.0-1ubuntu2                               amd64        C, C++ and Objective-C compiler
      ii  libclang-common-3.9-dev                    1:3.9.1-19ubuntu1                            amd64        clang library - Common development package
      ii  libclang-common-6.0-dev                    1:6.0-1ubuntu2                               amd64        clang library - Common development package
      ii  libclang1-3.9:amd64                        1:3.9.1-19ubuntu1                            amd64        C interface to the clang library
      ii  libclang1-6.0:amd64                        1:6.0-1ubuntu2                               amd64        C interface to the clang library
      gary@audio-workstation:~/test$ 

      dpkg --info doesn't show anything but apt show does:

      gary@audio-workstation:~/test$ apt show clang-3.9
      Package: clang-3.9
      Version: 1:3.9.1-19ubuntu1
      Priority: optional
      Section: universe/devel
      Source: llvm-toolchain-3.9
      Origin: Ubuntu
      Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
      Original-Maintainer: LLVM Packaging Team <pkg-llvm-team@lists.alioth.debian.org>
      Bugs: https://bugs.launchpad.net/ubuntu/+filebug
      Installed-Size: 144 MB
      Provides: c++-compiler, c-compiler, objc-compiler
      Depends: libc6 (>= 2.14), libclang1-3.9 (= 1:3.9.1-19ubuntu1), libgcc1 (>= 1:3.4), libjsoncpp1 (>= 1.7.4), libllvm3.9 (>= 1:3.9.1-6~), libstdc++6 (>= 5.2), libstdc++-7-dev, libgcc-7-dev, libobjc-7-dev, libclang-common-3.9-dev (= 1:3.9.1-19ubuntu1), libc6-dev, binutils
      Recommends: llvm-3.9-dev, python
      Suggests: gnustep, gnustep-devel, clang-3.9-doc
      Breaks: clang-3.1, clang-3.2, clang-3.3, clang-3.4 (<< 1:3.4.2-7~exp1), clang-3.5 (<< 1:3.5~+rc1-3~exp1), clang-include-fixer-3.9, compiler-rt
      Replaces: clang-3.1, clang-3.2, clang-3.3, clang-3.4 (<< 1:3.4.2-7~exp1), clang-3.5 (<< 1:3.5~+rc1-3~exp1), clang-include-fixer-3.9, compiler-rt
      Homepage: http://www.llvm.org/
      Download-Size: 37.4 MB
      APT-Manual-Installed: yes
      APT-Sources: http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages
      Description: C, C++ and Objective-C compiler (LLVM based)
       Clang project is a C, C++, Objective C and Objective C++ front-end
       for the LLVM compiler. Its goal is to offer a replacement to the GNU Compiler
       Collection (GCC).
       .
       Clang fully implements all published ISO C++ standards including C++11, as
       well as the upcoming C++14 standard, and some parts of the fledgling C++1z
       standard, and is considered a production-quality C++ compiler.
      
      gary@audio-workstation:~/test$ 
      
      gary@audio-workstation:~/test$ apt show libclang-common-3.9-dev
      Package: libclang-common-3.9-dev
      Version: 1:3.9.1-19ubuntu1
      Priority: optional
      Section: universe/libdevel
      Source: llvm-toolchain-3.9
      Origin: Ubuntu
      Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
      Original-Maintainer: LLVM Packaging Team <pkg-llvm-team@lists.alioth.debian.org>
      Bugs: https://bugs.launchpad.net/ubuntu/+filebug
      Installed-Size: 39.1 MB
      Depends: libc6 (>= 2.15), libgcc1 (>= 1:3.4), libstdc++6 (>= 5.2), libtinfo5 (>= 6), libllvm3.9 (= 1:3.9.1-19ubuntu1)
      Homepage: http://www.llvm.org/
      Download-Size: 2,575 kB
      APT-Manual-Installed: no
      APT-Sources: http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages
      Description: clang library - Common development package
       Clang project is a C, C++, Objective C and Objective C++ front-end
       for the LLVM compiler. Its goal is to offer a replacement to the GNU Compiler
       Collection (GCC).
       .
       Clang fully implements all published ISO C++ standards including C++11, as
       well as the upcoming C++14 standard, and some parts of the fledgling C++1z
       standard, and is considered a production-quality C++ compiler.
       .
       This package contains the clang generic headers and some libraries
       (profiling, etc).
      
      gary@audio-workstation:~/test$ 
      
      gary@audio-workstation:~/test$ apt show libclang1-3.9
      Package: libclang1-3.9
      Version: 1:3.9.1-19ubuntu1
      Priority: optional
      Section: universe/devel
      Source: llvm-toolchain-3.9
      Origin: Ubuntu
      Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
      Original-Maintainer: LLVM Packaging Team <pkg-llvm-team@lists.alioth.debian.org>
      Bugs: https://bugs.launchpad.net/ubuntu/+filebug
      Installed-Size: 22.9 MB
      Depends: libc6 (>= 2.14), libgcc1 (>= 1:3.4), libllvm3.9 (>= 1:3.9.1-6~), libstdc++6 (>= 5.2)
      Homepage: http://www.llvm.org/
      Download-Size: 5,950 kB
      APT-Manual-Installed: no
      APT-Sources: http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages
      Description: C interface to the clang library
       Clang project is a C, C++, Objective C and Objective C++ front-end
       for the LLVM compiler. Its goal is to offer a replacement to the GNU Compiler
       Collection (GCC).
       .
       Clang fully implements all published ISO C++ standards including C++11, as
       well as the upcoming C++14 standard, and some parts of the fledgling C++1z
       standard, and is considered a production-quality C++ compiler.
       .
       This package contains the clang library.
       .
       The C Interface to Clang provides a relatively small API that exposes
       facilities for parsing source code into an abstract syntax tree (AST),
       loading already-parsed ASTs, traversing the AST, associating physical source
       locations with elements within the AST, and other facilities that support
       Clang-based development tools.
      
      gary@audio-workstation:~/test$ 

      Digital-Larry gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf ./test.ii

      you nee -c otherwise it will attempt to use the linker.

      giuliomoro
      or better: start with just clang++-3.9-arm -march=armv7-a ./test.o ./test.ii, is it successful?

      I forgot -c, too, and also -o.

      so try:

      clang++-3.9-arm -march=armv7-a -c -o ./test.o ./test.ii

      if that succeeds, add one option at a time until it fails ..

        giuliomoro

        gary@audio-workstation:~/test$ clang++-3.9-arm -march=armv7-a -c -o ./test.o ./test.ii
        /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
        Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
        gary@audio-workstation:~/test$

        giuliomoro

        gary@audio-workstation:~/test$ /usr/bin/clang++-3.9 -###  -march=arm7-a -c -o ./test.o ./test.ii
        clang version 3.9.1-19ubuntu1 (tags/RELEASE_391/rc2)
        Target: x86_64-pc-linux-gnu
        Thread model: posix
        InstalledDir: /usr/bin
         "/usr/lib/llvm-3.9/bin/clang" "-cc1" "-triple" "x86_64-pc-linux-gnu" "-emit-obj" "-mrelax-all" "-disable-free" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "test.ii" "-mrelocation-model" "static" "-mthread-model" "posix" "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases" "-munwind-tables" "-fuse-init-array" "-target-cpu" "arm7-a" "-dwarf-column-info" "-debugger-tuning=gdb" "-coverage-file" "/home/gary/test/./test.o" "-resource-dir" "/usr/lib/llvm-3.9/bin/../lib/clang/3.9.1" "-fdeprecated-macro" "-fdebug-compilation-dir" "/home/gary/test" "-ferror-limit" "19" "-fmessage-length" "80" "-fobjc-runtime=gcc" "-fcxx-exceptions" "-fexceptions" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "./test.o" "-x" "c++-cpp-output" "./test.ii"
        gary@audio-workstation:~/test$ 

        sorry, hang on, swimming in stuff here...

        gary@audio-workstation:~/test$ clang++-3.9-arm -### -march=armv7-a -c -o ./test.o ./test.ii
        /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
        Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
        gary@audio-workstation:~/test$ 

        I added the "-###" right in the clang++-3.9-arm script itself (I also tried adding "-fno-autolink"):

        gary@audio-workstation:~/test$ sh -x /usr/local/bin/clang++-3.9-arm -### -march=armv7-a -c -o ./test.o ./test.ii
        + /usr/bin/clang++-3.9 -### -target armv7l-unknown-linux-gnueabihf -fno-autolink --sysroot /home/gary/arm /usr/local/bin/clang++-3.9-arm
        clang version 3.9.1-19ubuntu1 (tags/RELEASE_391/rc2)
        Target: armv7l-unknown-linux-gnueabihf
        Thread model: posix
        InstalledDir: /usr/bin
         "/usr/bin/ld" "--sysroot=/home/gary/arm" "-X" "--eh-frame-hdr" "-m" "armelf_linux_eabi" "-dynamic-linker" "/lib/ld-linux-armhf.so.3" "-o" "a.out" "crt1.o" "crti.o" "crtbegin.o" "/usr/local/bin/clang++-3.9-arm" "-lstdc++" "-lm" "-lgcc_s" "-lgcc" "-lc" "-lgcc_s" "-lgcc" "crtend.o" "crtn.o"
        gary@audio-workstation:~/test$ 

        OK after that last one I found it! It wasn't until I ran the script with "sh -x" that I saw the wrong command line substitution. The last thing on the command line should be the source file name, but it is the script file name!

        /usr/bin/clang++-3.9 -### -target armv7l-unknown-linux-gnueabihf -fno-autolink --sysroot /home/gary/arm /usr/local/bin/clang++-3.9-arm

        In this file, I had "$0" instead of "$@" - I changed it and it works!!!!

        gary@audio-workstation:~/faust/examples/bela$ cat /usr/local/bin/clang++-3.9-arm 
        #!/bin/bash
        /usr/bin/clang++-3.9 -target armv7l-unknown-linux-gnueabihf --sysroot ~/arm $@
        gary@audio-workstation

        Giuilo thanks so much for your patient help. Sorry it was so difficult but maybe it helps the next guy. At this point I think Faust on Bela is a very viable idea. Hopefully my next questions and comments will be more on the implementation side.

        giuliomoro I think so. At least, I see it shown in some of the things I posted a day or two ago. Just very hard to see! One question I have which I will defer answering for now is whether clang 3.9 is really needed on the Ubuntu system or whether "clang" as installed (which by now is at 6.0) is OK. I understand that the distcc as being called by faust2bela may be making the specific reference to 3.9. Lots of different interacting parts here.

        Linker problem

        One issue of 3.9 vs 6.0 is that linking is always done on the board by calling clang++-3.9, and so if there is any incompatibility between the .o files generated by 6.0 and the way clang++-3.9 calls the ld linker, or the way the older ld linker on the board deals with object files, then you could have some issues. I guess this is unlikely to happen, but I am not sure what guarantees clang and ld give about interoperability of different versions.

        Compiler problem

        Another issue is that different versions of clang may decide to pack structs (or at least ARM intrinsics) in different ways, for instance:

        giuliomoro What is surprising is that while alignof(float32x4_t) is 16 bytes on clang-3.9, it is actually 8 bytes on gcc-6.3 and clang-6.0.

        and

        giuliomoro However, I am wondering where that difference comes from, as the arm_neon.h files for the two versions of clang provide identical definition for all the neon vector types.

        More in general, there would be a problem if you have object files that both include some .h file that is interpreted differently by the two versions of the compiler, and you compile the two objects with different versions of the compiler. Again, I am not sure what backward/forward compatibility guarantees clang provides. Clearly, if you ALWAYS use distcc to build all the files on Bela, there would be no problem. The problem only comes up (potentially) if you are intermixing .o files compiled with different versions of the compiler.
        For instance - in you current situation - you built all the Bela core/ files with the local version of clang-3.9 the first time you built the first program. If you switch to using clang-6.0, I'd recommend you do a make -C ~/Bela clean on the board, so that those files are re-built next time you build a project.

          giuliomoro ah well never mind, all that stuff gives me a headache! I'm OK leaving 3.9 installed. Thanks for the detailed explanation.