Tensorflow Lite for Microcontrollers

SEDur

Hi,
I've messed about with Tensorflow Lite for Microcontrollers on the Daisy, and I was wondering if anyone has tried it on Bela?
Theres a makefile build system that allows you to precompile the TFL4M library for your arch, and you can include that in your C/C++ project. However, as whatever ARM core in the BBB is not going to be one of the supported ones for the codebase, I am looking for help from someone who can help me figure out the bits I need to do to cross-compile the lib such that it is compatible. There is some documentation in the TFL4M repo that describes that this process is possible, but i'm not sure I have enough of a complete picture to do it with what I know now.

If anyone would be willing to help me out, it would be much appreciated. The process would be fully documented so that other people can do it too!

I'm interested in doing some soundscape analysis, building on someone elses network and hopefully being able to learn some lessons about how soundscapes change in a place over time.

Cheers
Simon

giuliomoro

haven't tried it myself, but have you looked at this https://github.com/tensorflow/tensorflow/issues/42906 ?
It would seem that you can build it from source with the instructions provided.

SEDur

Ah I had not seen this, thats right!
But Tensforflow Lite and Tensorflow Lite for Microcontrollers are not exactly the same thing.
TFL4MCU is like a TFLite subset that is specifically designed to be run embedded on microcontrollers in (sort of) real time.

Its like a c++ lib that you include in your build system, and when you compile your application you do so with a pre-processed trained network that is optimised to be small in memory footprint and really fast and liteweight to execute. This means you can run the thing without all the massive ugly overhead of tensorflow lite, but instead have just the stuff needed to run the thing on something like an STM32. This is attractive even on the BBB because you are still resource constrained, running in something reminiscent of real time with a fairly finite memory footprint.
Heres the part of the TF git repo where the bit for microcontrollers starts:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/micro

Because ARM cortex M7s are already officially supported it wasn't so bad to get it to build for the Daisy, being an STM32H750, but there is still loads of stuff on that project that needs to be finished before I could use that as a platform to do exploration. Bela is such a mature (relatively) ecosystem that I was thinking, if I could compile for the Arm A(7?) arch that is in bela, I could do the same kinds of things, though probably as more of a proof of concept

giuliomoro

SEDur A(7?)

Cortex A8. armv7l, neon, vfpv3

adanlbenito

I haven't tried myself, but it seems possible to build TFlite Micro for different target platforms as this blog post seems to indicate. There are also some instructions on Tensorflow's github page.

However, I am not sure you will see many advantages using TF Lite Micro vs TF Lite on the beaglebone black. My understanding (which is limited) is that there is some overhead on TF Lite due to calculating the size of the intermediate FlatBuffers during runtime (see paper), but other than that I'd be curious to see any other improvements in terms of performance.

SEDur

adanlbenito Amazing thanks for these!

jamesstaub

@SEDur did you have any luck running TensorFlow Lite on Bela? I'm wondering if it would be feasible to run Basic PItch on Bela.

https://github.com/spotify/basic-pitch

SEDur

jamesstaub no, though I cannot remember how far I got. It would be interesting to have another go though!

jamesstaub

So far I was able to install and configure python 3.10 but I'm having trouble with openssl so I can't yet run pip install basic pitch

I built and configured the python install with

cd /path/to/Python-3.10.0
./configure --prefix=/root/Bela/projects/basic-pitch/python-3.10 --with-openssl=/usr/local/ssl --enable-optimizations
make
make install

but I get errors that ssl cannot properly configure, and it prevents me from installing anything with pip.

I tried downloading the modules in a different linux environment and copying them over but this fails with

    ModuleNotFoundError: No module named '_ctypes'
``` 
I think because I am unable to install ` libffi` , which fails with

Err:1 https://deb.debian.org/debian stretch/main armhf libffi-dev armhf 3.2.1-6
404 Not Found
E: Failed to fetch https://deb.debian.org/debian/pool/main/libf/libffi/libffi-dev_3.2.1-6_armhf.deb 404 Not Found


so a bit of dependency hell trying to get the package installed.

giuliomoro

jamesstaub but I get errors that ssl cannot properly configure, and it prevents me from installing anything with pip.

I would recommend you use the 0.5.0 image before attempting to install any modern toolchain: https://github.com/BelaPlatform/bela-image-builder/releases/tag/v0.5.0alpha2

jamesstaub

thanks. I loaded up the new image but seem to be stuck with the same open ssl problem when installing python modules

giuliomoro

make sure the date on the board is correct . I'd assume that to happen automatically when connecting it to the internet or it would happen when opening the IDE.

jamesstaub

@giuliomoro when I run date on the board it is about 25 hours behind the correct current time. is this a possible culprit for the problems installing/configuring pythion with openssl?

giuliomoro

I think 25 hours should be fine? Open the IDE and it should fix itself. If it doesn't appear to fix itself, it likely already is fixed and you may be reading it wrong. Time on the board is UTC, FWIW.