cross compilation for bela

thetechnobear

I wanted and easy way to cross compile my bela/salt/pepper projects,

a variety of reasons:
- speed (much faster compilations on desktop/laptop)
- workflow improvements (e.g. using my favourite code edit),
- integrate into my normal build projects based on cmake

finally, have v1.0 working... Id be happy for any feedback, or pull requests

https://github.com/TheTechnobear/xcBela

from README...

goals

This project provides a simple to install and use cross compiler environment

simple installation of cross compiling environment
e.g ./install.sh
easy to workflow with target board
e.g xcCompileRun render.cpp
support for cmake, with cmake toolchain
use modern tools
clang-10, llvm, cmake

requirements

supported OS are:
- macOS
- Linux (debian based e.g. deb/ubuntu)

it would be easy to add other linux variants
windows could probably be done using gnu/cygwin tools
however, I dont have the time.... but Im happy to accept pull requests from others 🙂

installation

see install.md

usage

installs and runs by default using
location: ~/xcBela
bela hostname : 192.168.7.2 , default for usb

you can override with environment variables, if using a different bela, or scripts placed somewhere else
e.g.

export XC_IP=salt.local
export XC_ROOT=~/projects/xcBela

note: when these are stored in ~/.xcBela.config, by install.sh, and xcSetEnv, so only need to be done once, or if you are changing target etc.

setup environment

. $XC_ROOT/xcSetEnv

usage note: . (dot) - do not run as a script directly

commands

xcExec remotecmd - execute command on remote
xcCopy localfile remotefile - copy local file to remote
xcCompile patch.cpp - compile single c++ file
xcRun patch - run (local executable) patch on remote
xcCompileRun patch.cpp - compile local file and run on remote
xcScope - bring up scope
xcGui - bring up gui

using with cmake

included in this project is a template which can be used as a cmake toolchain,
see cmakefaq.md

using Bela tools

Bela commands

the 'usual' bela command line scripts are also available (and on the path),
e.g. we can disable the ide with

ide.sh stop

Accessing Bela IDE

my main intention is not to use the IDE for development so editing and compiling. but it still has a couple of useful features I will use
(assuming, Ive not disabled ide at that time 😉 )

note: shortcuts assume chromium-browser on linux

Oscilloscope
shortcut: xcScope
this is available directly in a browser using: http://bela.local/scope

Bela Patch GUI
bela now supports web user interfaces
shortcut: xcGui
this can be access in the browser using: http://bela.local/gui

Credits & Resources

article on cross-compiling on mac

cross compile script from by Andrew Capon, can be found here:
https://github.com/AndrewCapon/OSXBelaCrossCompiler

Bela wiki on remote compilation with Eclipse
https://github.com/BelaPlatform/Bela/wiki/Compiling-Bela-projects-in-Eclipse

giuliomoro

Great!

The existence of arm-linux-gnueabihf-binutils from brew is really helpful. I always got stuck when setting up cross-compilation as to obtain ld there was always need for someone (e.g.: Andrew Capon) to build the full gcc cross-compiler, which was a pain to maintain.

thetechnobear

giuliomoro

ive a couple of problems with the changes you made to install_bela.sh

a) it relies on a newer version of git.... for --show-current
thats no problem, we can install git from homebrew

b) the merge is always failing for me.
this is because there are a ton of file that are not ignored (as Ive pointed out before), as so git creates a merge conflict
I could do a

git reset --hard
git clean -fxd

seems a bit drastic, but would get to the state where the two images are aligned...
(which I think was the purpose of your change)

thoughts?

EDIT: another slight problem.... but perhaps is ok

so you are doing

git push $XC_SSH:Bela $BRANCH:tmp-$BRANCH

which you then merge....

first this seems for some reason to create a lot of commits on the bela board,
e.g. even though I reset both on mac and on bela, after this merge bela board things its ahead by 1102 commits on master!?

Im not sure this matters, as I never push from the bela board to github

also....
git push will only push committed changes
so its still possible that you have change in your local ~/xcBela that will not push to bela

(I found this out the hard way, when I was editing the Makefile locally, then was confused why make lib still buidl the old library)

perhaps if we really want the ~/Bela and ~/xcBela/Bela to be the same, we should rsync the changes over?
(excluding ~/Bela/projects )

thetechnobear

yeah, I've setup cross-compiling setup on machines, but had to always spend a while 'googling' what to do, what to compile... then at some point id need to move to a different machine, and id forget how i did it, so had to do again... it just got a bit irritating...so stopped doing it.

so, I just wanted a simple way to get an environment running - and seemed like a good time to go investigate newer tools in this area - particularly clang and llvm. also, i have to say the way cmake integrates is nothing short of amazing!

the added benefit for me, is all of this works for other platforms (like rPI ,organelle) too, so all my cross-platform stuff is simplified. this is turn makes it easier for me to integrate bela, which encourages me to use it more - so win/win situation.

anyway, all I've done here is combine the amazing work others have done on these modern tools...
so thanks to all those contributions from around the world!

thanks for your PR, nice refinements.

p.s goes without saying i hope, this all gpl, so if you feel any of it is useful to bela , feel free to take and use.
or alternatively reference from bela wiki if you prefer - completely up to you, and what/if you find most useful

giuliomoro

thetechnobear a) it relies on a newer version of git.... for --show-current
thats no problem, we can install git from homebrew

sorry didn't know that. It seems silly to add a dependency for something so simple. We can just git branch | grep "^*" | sed "s/\* \(.*\)/\1/" instead.

thetechnobear this is because there are a ton of file that are not ignored (as Ive pointed out before), as so git creates a merge conflict

Right, I initially had git checkout . && git clean -fd . before doing the merge, but in the end I purposely skipped that step: I didn't want to cause any data loss for the user.

thetechnobear e.g. even though I reset both on mac and on bela, after this merge bela board things its ahead by 1102 commits on master!?

This is 1102 commits ahead of origin/master (not of master). According to the board, origin/master is at exactly the same state as it was when the image was created, unless it had a chance to connect to the internet and pull, fetch or push since.

thetechnobear first this seems for some reason to create a lot of commits on the bela board,

If your repo on the board is untainted (that is you have not committed anything on the board itself, and/or it has been "cleaned" as above), then there should be no merge commit, as the merge will be a fast-forward.

thetechnobear git push will only push committed changes

yes, my understanding was that your local repo would be read-only (i.e.: it's just a dependency for your build scripts, after all).

thetechnobear perhaps if we really want the ~/Bela and ~/xcBela/Bela to be the same, we should rsync the changes over?

You are right. Actually the best thing would be to call update_board from the locally-checked out repo, as this will also install whatever needs to be installed (e.g.: updated .deb packages in resources/stretch/deb) and re-build some tools in resources/tools), and of course ignore projects/. It is slower than it would be with git but safer, in that it ensures all the proper steps are performed. With --no-frills it will be faster (it won't call make clean, so it will be faster to build (if not all core files have changed).

thetechnobear

giuliomoro yeah I like the idea of calling update_board , as it’s consistent with using the makefile on the client

thetechnobear

hmm, something in the latest changes on bela/master has broken this....

Building lib/libbelafull.so
/usr/bin/ld: /usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0/../../../../lib/libmathneon.a(math_sinf.o): relocation R_ARM_MOVW_ABS_NC against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/usr/bin/../lib/gcc/arm-linux-gnueabihf/6.3.0/../../../../lib/libmathneon.a: error adding symbols: Bad value
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:798: recipe for target 'lib/libbelafull.so' failed

if I manually link without ibmathneon.a it works...

not quite sure what was changed recently in master that has caused this...
any ideas giuliomoro

libbela.so and libbelaextra.so are compiling ok.

edit: ok, is libraries/Oscillator

thetechnobear

hmm, issue appears to be libraries/Oscillator is new (?) and is using mathneon for math_sinf()
unfortunately mathneon is not compiled for relocatable code... so cannot be brought into a shared library.

not quite sure best step forward...

I guess I'll have to rollback or delete it or something because I wanted to get on with coding the patch, not this kind of stuff 🙁

giuliomoro

That's interesting, let me have a look. I was fairly sure I had already compiled math_neon with -fPIC, but maybe I am thinking about ne10 instead. So, just delete the Oscillator library for now (or revert my recent commit), and I will re-build math_neon later today

thetechnobear

ok, Ive got a new solution - which is perhaps closer to what usually happens on the bela board 🙂

Ive removed the libbelafull.so compilation, and just compile a static library, which I then link against.

given the patch files are usually linked statically to all these things (hence why Oscillator doesn't complain when compiled in the ide) , this approach is closer that.

this is possibly safe guards from other vagaries occurring due to static vs dynamic builds.

giuliomoro

Ok, here is a commit (on dev) with a new build of libmathneon which provides both a .a and a .so (the latter built with -fPIC) https://github.com/BelaPlatform/Bela/commit/8e7c626d2b592c3dc11dda08dc76aa51330aea6d .

It occurred to me that libbelaextra was working with lmathneon even though the latter was only available as .a (and built without -fPIC) because actually none of its symbols were requested by the object files in libbelaextra.so, so it was effectively left out (I THINK).

thetechnobear

libbelaextra did not include oscillator, so indeed did not need to resolve mathneon, its only when it resolves it that it realizes it cannot relocate the code which is required for an .so - so yeah, you never notice until it gets used - and as the ide is building everything static its not going to see this.

there are some other subtle differences for code in static lib vs dynamic, its not always completely transparent e.g. I've had issues with static c++ initializers when moving stuff into dynamic libs... so perhaps better to use static if you want to be 'close' to whats done in the IDE.

is dev relatively close to master and stable?

for now, i've removed dyanmic lib for libbelafull, and static link it - so that fixes the issues.

giuliomoro

thetechnobear is dev relatively close to master and stable?

now merged. I will get to your PR in a while. Ultimately it could be good to provide hooks for people to inject their custom Makefile at the beginning and end of our Makefile, without need for mainlining all such things.

I for one would find it very useful because I often have custom options that I have to stash/apply for every branch switch.

thetechnobear

giuliomoro yeah, ability to add a per project make step would be useful and/or 'site settings'

of course, as such I could probably make libbelafull in my own projects too.

one thing that crossed my mind this morning, is that perhaps what I could do on my projects is, copy the libbelafull and bela headers into my own projects (kind of like a sub project) - given there is then no runtime dependancy (except sys libs) that project would then compile regardless of what version the bela software was on.
... so i reduce my dependancy on the 'state' of the bela board, in turn this means I can just release a binary.

imho, reducing dependancies on other projects is always a good thing... 🙂

of course, then at 'leisure', i can update that projects copy of the bela headers/libs as required e.g. if the patch is requiring new features.

with cross-compilation, Ive been quite happily running binaries outside of the bela ide/makefile system, so should work well.
the only 'issue' I've encountered (and had to adapt xcBela appropriately), is that if you want to use P5 JS, then there is a dependancy that it has to be in ~/Bela/projects/projectname - otherwise sketch.js does not work. (iirc, a path dependancy?)

oh... and another small oddity /bug(?) with P5 JS...
for some reason it doesn't work if you don't included any command line parameters...it doesn't seem to matter what parameter you use, but it needs at least one ... I usually just tag -X 0 on the end 😉
(non p5 js patches work fine with no parameters sent)

giuliomoro

thetechnobear the only 'issue' I've encountered (and had to adapt xcBela appropriately), is that if you want to use P5 JS, then there is a dependancy that it has to be in ~/Bela/projects/projectname - otherwise sketch.js does not work. (iirc, a path dependancy?)

The sketch.js file is passed to the client through the node server. The latter has a route to allow access to all the /root/Bela/projects folder. The Gui class therefore sends only the _projectName to the client via websocket, and then the client requests the actual sketch.js file through node. It is not straightforward to do differently (e.g.: sending the js down the websocket, but then the Gui class should be passed a path, as it couldn't automatically grab it from ,e.g.: argv[0], so a new global variable and/or Bela API function should be added, so... the current solution is fine enough until when (if) we start integrating this with other applications). I think you can override it somehow by placing your js files directly in the /root/Bela/IDE/public/gui/ folder.

giuliomoro

thetechnobear oh... and another small oddity /bug(?) with P5 JS...
for some reason it doesn't work if you don't included any command line parameters...it doesn't seem to matter what parameter you use, but it needs at least one ... I usually just tag -X 0 on the end 😉
(non p5 js patches work fine with no parameters sent)

This should have been fixed as of

commit bf01eaf60b1f1049d360ade256761f124a9349e4
Author: Giulio Moro <giuliomoro@yahoo.it>
Date:   Wed Dec 11 13:23:18 2019 +0000

    Fixed projectName when no command line arguments are provided

(which has been merged into master only last week).

HjalteBestedMoeller

@thetechnobear This setup sounds interesting. Is there some kind of guide to install and workflow ?

I would like to try it 🙂

giuliomoro

here

HjalteBestedMoeller

Now I have installed xcBela on Ubuntu 18.04... All seemed to install correctly, but I'm confused about how to use the scripts...

Where do I place my project ?
I have tried to place it in xcBela/Bela/projects/myProject

and cd in to that folder and do xcCompile, result:

xcCompile: command not found

Then I tried
../../../scripts/xcCompile render.cpp

But then I get a lot of fatal error's, like:

fatal error: 'Bela.h' file not found
fatal error: 'libraries/Midi/Midi.h' file not found

Am I supposed to make the scripts globally available commands somehow ?
Or what am I missing ?

Would be happy with an example project, showing location of files and how to compile, run

Best,
Hjalte

giuliomoro

HjalteBestedMoeller and cd in to that folder and do xcCompile, result:

for that to work you have to follow the instructions in readme:

export XC_IP=bela.local
export XC_ROOT=/path/to/xcBela
. $XC_ROOT/xcSetEnv

HjalteBestedMoeller But then I get a lot of fatal error's, like:

can you show the full output, including the first lines, which are the actual command being executed? That will show where it is looking for the include , and you can then double check that the folder/file exist.

giuliomoro

did you do git submodule init; git submodule update? Is the folder xcBela/Bela empty?

HjalteBestedMoeller

giuliomoro
I did not do those commands, I just did what is stated in the install.md that you linked me to....

But the install.sh scripts fills the Bela folder, so it's not empty... the folder looks fine and
xcBela/Bela/include/Bela.h does exist...