• [MOVED: ]Managing complex projects

sorry LiamDonovan for not getting back to you sooner...

Im integrating bela into a 'bigger project' at the moment, which allows Eigenlabs Eigenharps and Madrona Labs Soundplanes to be connected to it, so Im still 'feeling my way around' , how best to do this in terms of project management (makefiles/repos etc)

for now, Ive moved away from using the GIT support in the UI.

whilst I love the web ui for quick development, the main issue I have is its not really flexible enough for bigger projects, there are a couple of things that would help me 🙂

  • allow 'project hierarchy', not just browsing the top level of ~/Bela/projects
  • allow easy linking in of shared libraries
  • move built 'assets' into a build directly, so that it can be routinely added to .gitignore
  • allow webui to trigger a 'make' in any directory

let me explain how my project is structured.. and you can might be able to see how things might work
this is all in one repo (I dont really want

tb project
       - my-api (built into a shared library, via cmake)
           - many subdirectories 
       - my-bela-app-1
       - my-bela-app-2
       - builds (created by cmake)
            - release/lib (make in builds directory)

ok, Id like to put this in
~/Bela/projects/tb project

then browse in the web-ui to my-bela-app1 .. 2 etc, and compile bela apps using it
ideally Id like to add so's in builds/release/lib to the projects, and it would be great if , when I open 'builds' in webui, I can just build it, using the make file that is there (default target is fine)

then of course, git commit is done on the entire project (my project), rather than individual projects.

of course, this is perhaps overkill for many... but its an example of how a bigger project might be structured, also Id say that its over kill to have one git repo per bela projects (for my needs at least

what Im doing now is putting my project in ~/projects/tb project, and then doing a symbolic link to form my-bela-app-1 to ~/Bela/projects... so I can edit in bela, but then I just log in to bela to commit via git/build the libs etc.

probably mid-term, Im going to look into using cross compilation anyway, so this might change again.
(useful when you want to move the image to multiple belas)

anyway, just food for thought... perhaps others can comment on how they are structure their projects.

      btw a couple of tips for others... if you want to link to shared libraries in a different directory you will need something like the following make flags (in bela settings) , (my lib is called limbed-api.so

      AT=;CPPFLAGS=-I/root/projects/MEC/mec-api;
LDFLAGS=-L/root/projects/MEC/build/release/lib;
LDLIBS=-lmec-api -Wl,-rpath,/root/projects/MEC/build/release/lib

      (this all goes on one line, and dont forget the semicolons)

      AT= gives you 'verbose' compile (would be useful as a 'checkbox' on build settings I think. (as very useful!)

      rests, sets include for C++ compiler, a link library path, and library to link, and finally set a path for when running the bela executable to find the so....
      (took me a while to figure which LINK FLAGS to set to get the relevant link flags where i needed them on the command line!)

      oh, can you also make the make flags edit field in the UI much bigger, with something like this, I had to resort to cut n pasting it elsewhere, so I could see what was going on 🙂

      Thanks very useful feedback, points noted.

      I see how, for your use, the space in the Make parameters box is way too small. It surely would be good to make it user-adjustable. Also, I have been thinking about the Makefile detecting makefiles in the project folder, to allow for custom makefiles for a project. How do you think we should handle that? Simply discard all our recipes and run the Makefile found in the root of a project (~/Bela/projects/projectname/Makefile) ? That would be easy but it would most likely lead to lots of duplicate code in this secondary Makefile.
      Alternatively, the main Makefile would only -include $(PROJECT_DIR)/Makefile. You could then override recipes for a given target with something like this (I think):

      # add a dependency to the existing target
existing-target-file: mycustomrule

mycustomrule:
         # your custom rule

      Would this work fine?
      This allows both fine granularity control on individual file (e.g.: you want to specify certain optimizations for certain files), and if you want to run some of your sub-sub-makefiles, then you can do that in mycustomrule.

      Not sure I fully understand your issues with git but I think you could have a repo in each subfolder and handle them from the console in the IDE without having to log into the board.

      If you want to move the built file to a different folder, then I think you could do that by adding a dependency for the .phony target Bela, which currently looks like this :

      Bela: $(OUTPUT_FILE)

      so if you add another dependency in your included makefile, with the system described above, that would look like:

      Bela: moveAsset

moveAsset:
       # move the file to the assets folder

      Regarding libraries, we have been thinking about how to make it easier to handle them, but we ended up figuring that the effort it would take to add LDFLAGS=-L/root/projects/MEC/build/release/lib; is comparable to typing /root/projects/MEC/build/release/lib into a dedicated field labelled "Library include path", but much less trouble for us to implement. While a neater support will come in the future, it is not top priority just now.

      If anyone else is interested in using external libraries, I put together a blog post on how to do so which hopefully also gives a bit of insight on the logic behind it: http://blog.bela.io/2017/01/04/using-an-external-library-with-bela/ .

      yeah, perhaps having separate fields for LDFLAG/LDLIBS , might make it a little more friendly....
      but I do understand, that more complex projects need more complex setups, thats cool by me 🙂

      building non-bela directories... well the 'easy' option, i guess would be just run make, if you load a makefile up, but perhaps more 'advanced' would be for the webui to all create a bela-settings.json, which could have some settings including build target, command line option. its only used by bela, but no harm it being in the projects.
      ... alternatively, perhaps the settings, should be in the bela project, i.e. it knows to build other parts of the project, by triggering makefiles. (this would be ideal)

      repo - i dont like too many , as they get difficult to manage, as things like GitHub show them as a big list 🙂 - sure in a particular project you can link them as sub-projects, but that has its own pitfalls/overhead.

      but perhaps this is all 'out of scope' and thats cool too.
      in practice, I tend to always have an ssh session open to my bela when developing anyway...

      as i said, perhaps Id might get more mileage, out of integrating a cross compile and upload build/run cycle, which I could integrate to my editor (sublime) a mac/linux ...

        thetechnobear thanks for this excellent and detailed feedback, it's really interesting to get an idea of how the IDE is being used to manage more complex projects!

        A resizable make parameters box is certainly easy to do, and makes sense. Project hierarchies would be nice, but there are a few practical implementation issues - e.g how does the IDE differentiate between a Bela project containing subfolders and a directory of Bela projects? Actually the way the IDE manages directory structure in general needs to be refined, so maybe I can look into this in a bit more detail. Library management is going to be looked at too.

        I also like the idea of being able to optionally add a custom Makefile to each project to allow users to customise the build process if they need to, though as Giulio pointed out the rules for building a Bela project are not exactly simple, and replicating our complex centralised Makefile in each project is not going to help anyone, so this is something to think about.

        Of course it's important that anything we add does not make the experience of using the IDE more difficult or confusing for new and inexperienced users, but I'm all for doing anything we can to increase its flexibility without affecting that experience.

          a month later

          thetechnobear do you think $(LDLIBS) should go on the command line BEFORE Bela's default included libs? I recently got caught by including with LDLIBS=-lmylib a library which in depended on libne10 and the linker could not resolve the symbols in libne10 because of the order, so I had to do LDLIBS=-lmylib /root/Bela/lib/libne10.a which is suboptimal.

              giuliomoro yeah, ordering can be a bit of a pain. as you say generally you'd expect the 'core libs' to be after the 'application libs'