• Hardware
  • Bela UART Connection to DJI Drone

Hi all,

I am thinking of connecting Bela to the UART connection on a DJI Matrice 100 drone, in order to retrieve data (such as telemetry and motor speed) from the internal system during flight. This will require additional code on both Bela and within the DJI SDKs.

I am aware that Bela is capable of utilising UART from reading through the following: (https://learn.bela.io/using-bela/advanced-topics/bela-as-a-uart-adapter/#picking-a-uart-port)

And: (https://github.com/BelaPlatform/Bela/wiki/Load-a-device-tree-overlay)

I assume that through this connection, it can act as a serial port where data can be transferred in real-time from the internal system of the drone? What would be the best type of connection? E.g. USB to UART.

It would be great to get some advice on the potential approach to this. I look forward to your response.

If the drone has a UART port available, then UART to UART without involving USB would probably come with less overhead. You can use the Serial library from C++: library https://github.com/BelaPlatform/Bela/tree/master/libraries/Serial and example https://github.com/BelaPlatform/Bela/tree/master/examples/Communication/Serial

You'll also want to make sure that the drone's UART operates at 3.3V or add appropriate level shifting circuitry

  • mpc replied to this.

    giuliomoro

    Ok great thank you Giulio, the drone does have a UART port and actually comes with a UART cable which you can see attached.

    alt text

    I believe that the drone's UART operates at 3.3V but will double check this. I'll incorporate some of the code for serial communication into my current code and see how it goes.

    giuliomoro

    I will be installing the DJI onboard SDK onto the Bela CTAG in order to enable communication between the drone and Bela. I wanted to double check but am I correct in thinking that the following is the way to install it on the Bela system after downloading the SDK file to my computer? In the event that I'm not connected to the internet and cloning from the board:

    scp -r Onboard-SDK-3.2.zip root@192.168.7.2:

    then run the command:

    ssh root@192.168.7.2 "cd Onboard-SDK-3.2 && make ARCH=arm && make ARCH=arm install"

      well I don't know how you'd install this specific program.

      mpc scp -r Onboard-SDK-3.2.zip root@192.168.7.2:

      this seems OK, however it will result in a zip archive, not a folder, therefore the next line will fail:

      mpc "cd Onboard-SDK-3.2 && make ARCH=arm && make ARCH=arm install"

      because you cannot cd into it yet. First you need to unzip it. Try:

      ssh root@192.168.7.2 "unzip  Onboard-SDK-3.2.zip  && cd Onboard-SDK-3.2 && make ARCH=arm && make ARCH=arm install"

      ultimately you are probably better off logging onto the board with

      ssh root@192.168.7.2 

      and then run each of the other programs interactively and verify each works well, otherwise if any of those fails, you may have to do some extra cleanup (e.g.: remove the folder) in order to be able to run the chainf of commands wit h && again.

      • mpc replied to this.

        giuliomoro

        Ah yes I forgot to remove the .zip but I'll make sure that is unzipped prior to installation. The program is compatible with linux, so I think it should be ok. It will be worth trying out.

        With regards to the following statement:

        giuliomoro ultimately you are probably better off logging onto the board with

        ssh root@192.168.7.2

        and then run each of the other programs interactively and verify each works well, otherwise if any of those fails, you may have to do some extra cleanup (e.g.: remove the folder) in order to be able to run the chainf of commands wit h && again.

        Just to clarify when you say 'run each of the other programs', are you referring to the example serial communication examples that you mentioned before? Or do you mean programs that are included in the Onboard SDK?

          no I guess I meant each of the other commands. Like, run them interactively, one at a time, so you'd do

          ssh root@192.168.7.2 

          verify you are in
          then

          cd Onboard-SDK-3.2

          verify you are in the correct folder (e.g: that you didn't mistpye it)
          then

          make ARCH=arm

          verify it doesn't cause any errors
          then install

          make install

          mpc The program is compatible with linux, so I think it should be ok.

          well the installation steps for programs under Linux varies greatly. You'll want to check the build instructions that come with it. My guess is you won't need the ARCH=arm, but possibly you'll need some configuration steps before make, who knows...

          • mpc replied to this.
            11 days later

            giuliomoro

            Ah ok no worries I understand what you mean. I worked out how to get the onboard SDK installed and running by using a different installation approach. The drone onboard SDK now runs on Bela and when I run the telemetry sample file, the data is printed to the terminal.

            I’m now working on incorporating the serial communication to the current code that I have. At present the programs are run within the onboard SDK (installed on Bela). I would like this data to be received by Bela within the audio recording code that I’ve written. Rather than only within the SDK on Bela.

            So that the program I’m running is receiving the serial data from the drone and essentially logging it. It also gives me options to use the data later. I have modified the current code to incorporate serial communication but wasn't too sure what would go in the file1.log();. I assume it is possible to log the drone data coming in from the serial connection? In this case it would be logging the drone telemetry and motor speed data.

            Also out of curiosity, how are the values determined within the brackets in setFormat: file1.setFormat("%f %f %f %f %f %f %f %f\n");?

            Here is my current code with the serial communication and part of the logging added:

            
            #include <Bela.h>
            #include <libraries/Pipe/Pipe.h>
            #include <libraries/Serial/Serial.h>
            #include <libraries/sndfile/sndfile.h>
            #include <libraries/WriteFile/WriteFile.h>
            #include <cmath>
            
            const char* path = "/mnt/usb/audioCh.wav";
            
            SNDFILE * outfile2;
            char originalFilename[] = "/mnt/usb/audioCh.wav";
            char* uniqueFilename = WriteFile::generateUniqueFilename(originalFilename);
            
            WriteFile file1;
            
            //Audio
            AuxiliaryTask gFillBufferTask;
            SNDFILE * outfile;
            unsigned int gAudioFrames;
            unsigned int gAudioInChannels;
            float gAudioSampleRate;
            Pipe gPipe;
            Serial gSerial;
            
            //Open files audio
            void openFile() {
                SF_INFO sfinfo;
                sfinfo.channels = gAudioInChannels;
                sfinfo.samplerate = gAudioSampleRate;
                sfinfo.format = SF_FORMAT_WAV | SF_FORMAT_PCM_16;
                //outfile = sf_open(path, SFM_WRITE, &sfinfo);
                outfile2 = sf_open(uniqueFilename, SFM_WRITE, &sfinfo);
            }
            
            //close audio files
            void closeFile() {
                sf_write_sync(outfile2);
                sf_close(outfile2);
                //sf_write_sync(outfile);
                //sf_close(outfile);
                printf(".wav file written and closed\n");
            }
            
            void writeBuffer(void*) {
              unsigned int numItems = gAudioFrames * gAudioInChannels;
              float buf[numItems];
              int ret;
              while((ret = gPipe.readNonRt(buf, numItems) ) > 0)
              {
                sf_write_float(outfile2, &buf[0], ret);
              }
            }
            
            void startSerial(void* arg)
            {
                unsigned int maxLen = 256;
                char serialBuffer[maxLen];
                // read from the serial port with a timeout of 100ms
                int ret2 = gSerial.read(serialBuffer, maxLen, 100);
                if (ret2 > 0) {
                    printf("Received: %.*s\n", ret2, serialBuffer);
                } else {
                    printf("UART4 not available\n");
                }
            }
            
            
            bool setup(BelaContext* context, void* arg)
            {
             
              gSerial.setup ("/dev/ttyS4", 230400);
              AuxiliaryTask serialCommsTask = Bela_createAuxiliaryTask(startSerial, 0, "serial-thread", NULL);
              Bela_scheduleAuxiliaryTask(serialCommsTask);
                
              file1.setup("droneData.txt"); //set the file name to write to
              file1.setFileType(kText);
              //file1.setFileType(kBinary); //this file type reduces the size of the file
              file1.setFormat("%f %f %f %f %f %f %f %f\n");
                
              //setup audio frames and channels
              gAudioSampleRate = context->audioSampleRate;
              gAudioFrames = context->audioFrames;
              gAudioInChannels = context->audioInChannels;
              
              gPipe.setup("sndfile-write", 65536, false, false);
              openFile();
              
              if((gFillBufferTask = Bela_createAuxiliaryTask(&writeBuffer, 90, "writeBuffer")) == 0) {
                return false;
              }
              return true;
              
            }
            
            void render(BelaContext* context, void* arg)
            {
              // context->audioIn is the float* that points to all the input samples, stored as interleaved channels)
              // audioIn is an array of 4 frames * 2 channels = 8 audio input samples.
              gPipe.writeRt(context->audioIn, context->audioFrames * context->audioInChannels);
              
              // analogIn is an array of 2 frames * 8 channels = 16 analog input samples. 
              // (2 frames because analog I/O runs at half the sample rate of audio I/O.) 
              //gPipe.writeRt(context->analogIn, context->analogFrames * context->analogInChannels);
              
              Bela_scheduleAuxiliaryTask(gFillBufferTask);
                
              //file1.log();
              
            }
            
            void cleanup(BelaContext* context, void* arg)
            {
                closeFile();
                free(uniqueFilename);
            }

              mpc how are the values determined within the brackets in setFormat

              Those formats (which all need to be %f with modifiers) are used in a snprintf statement internally using as values - in turn - those passed to log().

              mpc I would like this data to be received by Bela within the audio recording code that I’ve written.

              How does the SDK work? Is it a shared library that you can link a program to? Is it a couple of .h and .c/.cpp files that you could drop in the Bela project?

              mpc Here is my current code with the serial communication and part of the logging added:

              what sort of feedback are you looking for here? I see that serialCommsTask is scheduled only once, so it will only read from the serial port once.

              • mpc replied to this.

                giuliomoro

                giuliomoro How does the SDK work? Is it a shared library that you can link a program to? Is it a couple of .h and .c/.cpp files that you could drop in the Bela project?

                I'm still fairly new to using the SDK but it seems like you can link a application to it according to this link: https://developer.dji.com/onboard-sdk/documentation/development-workflow/integrate-sdk.html

                The SDK folder includes various .hpp and .cpp files for running programs. Here is a link to the onboard SDK folder that I have, so its bit easier to explain the structure: https://we.tl/t-ew4k0dYgPk

                So at present the SDK is installed on Bela and I run the telemetry code within the SDK. The telemetry code also consists of .hpp and .cpp files. The path to the file within the SDK folder that I sent is sample/platform/linux/telemetry.

                giuliomoro what sort of feedback are you looking for here? I see that serialCommsTask is scheduled only once, so it will only read from the serial port once.

                So I would be looking for feedback throughout the duration of a flight, so it would need to read from the serial port frequently if possible.

                  mpc So I would be looking for feedback

                  Sorry, I meant what feedback did you want from me on the code you sent.

                  mpc I'm still fairly new to using the SDK but it seems like you can link a application to it according to this link: https://developer.dji.com/onboard-sdk/documentation/development-workflow/integrate-sdk.html

                  it seems like you should be able to build a Bela project that links with this library by adding:

                  CPPFLAGS=-I OBS/api/inc  -I OBS/utility/inc -I OBS/hal/inc OBS/protocol/inc -I OBS/platform/linux/inc; LDLIBS=-ldjiosdk-core

                  to the Make Parameters for your project in the Bela IDE, after replacing OBS with the path to the sdk's root folder.
                  Then put these lines at the top of your Bela render.cpp file and see if it builds successfully.

                  (what's their bloody problem? Why do they put code as images in their docs????)

                  If this fails to link, for missing symbols, you may also need to add the files in common/dji_linux_environment.cpp and common/dji_linux_helpers.cpp to your Bela project.

                  • mpc replied to this.

                    giuliomoro

                    giuliomoro Sorry, I meant what feedback did you want from me on the code you sent.

                    Ah sorry so I'm wondering how to implement logging the serial data that is received (telemetry and motor speed data). I wasn't too sure what would go in the file1.log(); part of my code in the render function. I would also need the serial port to read frequently as the data values constantly change throughout a flight.

                    giuliomoro after replacing OBS with the path to the sdk's root folder.

                    Just to clarify, this would be the path to the sdk's root folder located on Bela? All mentions of OBS would be replaced by the path to sdk root folder?

                    giuliomoro (what's their bloody problem? Why do they put code as images in their docs????)

                    I know for a company of their size, the documentation is pretty terrible.

                      mpc Just to clarify, this would be the path to the sdk's root folder located on Bela? All mentions of OBS would be replaced by the path to sdk root folder?

                      yes

                      mpc I would also need the serial port to read frequently as the data values constantly change throughout a flight.

                      my understanding is that this part would be handled by the SDK?

                      • mpc replied to this.

                        giuliomoro

                        giuliomoro mpc Just to clarify, this would be the path to the sdk's root folder located on Bela? All mentions of OBS would be replaced by the path to sdk root folder?

                        yes

                        Ok great, thanks Giulio!

                        giuliomoro my understanding is that this part would be handled by the SDK?

                        That's true actually the SDK would handle that but I would still like to log the data that comes in within my Bela code. I am slightly stuck on this as I thought that maybe I access the analogIn buffers directly, then have a pointer to it and pass it to WriteFile::log() directly. However I wasn't too sure especially with regards to serial data through UART.

                        Did you write some code you to interface to the SDK? There (I assume) you should have the variables you want to log to WriteFile

                        • mpc replied to this.

                          giuliomoro

                          If you mean interface between my Bela code and the SDK then I haven't written anything at present. Purely on the basis that I started on this section of development not long ago and working out how it all connects together.

                          giuliomoro it seems like you should be able to build a Bela project that links with this library by adding:

                          CPPFLAGS=OBS/api/inc OBS/utility/inc OBS/hal/inc OBS/protocol/inc OBS/platform/linux/inc; LDLIBS=-ldjiosdk-core
                          to the Make Parameters for your project in the Bela IDE, after replacing OBS with the path to the sdk's root folder.
                          Then put these lines at the top of your Bela render.cpp file and see if it builds successfully.

                          I've not quite got it working yet with regards to the above suggestion. It seems to be having some issues with the last parameter:

                          /usr/bin/ld: cannot find -ldjiosdk-core
                          clang: error: linker command failed with exit code 1 (use -v to see invocation)

                          Although I believe that is a case of changing where its looking for the library.

                          giuliomoro Then put these lines at the top of your Bela render.cpp file and see if it builds successfully.

                          If this fails to link, for missing symbols, you may also need to add the files in common/dji_linux_environment.cpp and common/dji_linux_helpers.cpp to your Bela project.

                          I also noticed once I started including files (in code), most of the .hpp or .cpp files require another file to be added to the Bela project due to other dependencies. I'll keep developing it to see what I can get working.

                            mpc I also noticed once I started including files (in code), most of the .hpp or .cpp files require another file to be added to the Bela project due to other dependencies.

                            according to their documentation, you shouldn't need to include any hpp file into the Bela project. If you set the CPPFLAGS= correctly, then they should all be included there. I just noticed that I got the CPPFLAGS wrong above and now I fixed it (needs -I in front of each path).

                            mpc /usr/bin/ld: cannot find -ldjiosdk-core

                            this is because the library is probably not installed in a standard path? When you build the sdk is there a command to install it ?

                            mpc If you mean interface between my Bela code and the SDK then I haven't written anything at present. Purely on the basis that I started on this section of development not long ago and working out how it all connects together.

                            I mean your stand-alone app using the SDK, what code does it run?

                            • mpc replied to this.

                              giuliomoro

                              giuliomoro according to their documentation, you shouldn't need to include any hpp file into the Bela project. If you set the CPPFLAGS= correctly, then they should all be included there. I just noticed that I got the CPPFLAGS wrong above and now I fixed it (needs -I in front of each path).

                              No worries, thank you for making the edit. I'll give this a try and hopefully get it working.

                              giuliomoro this is because the library is probably not installed in a standard path? When you build the sdk is there a command to install it ?

                              So after copying the SDK to Bela I used the following commands to install it:

                              cd Onboard-SDK-3.9
                              mkdir build
                              cd build
                              cmake ..
                              make
                              cp ../sample/linux/common/UserConfig.txt bin/

                              Once I've edited the UserConfig.txt file using sudo nano UserConfig.txt, I use the following commands to run the telemetry code within the SDK:

                              cd bin
                              ./djiosdk-telemetry-sample UserConfig.txt

                              Here is a link to the telemetry code that I'm currently using that utilises the SDK: https://we.tl/t-5INSKFNWLy

                                mpc cd Onboard-SDK-3.9
                                mkdir build
                                cd build
                                cmake ..
                                make
                                cp ../sample/linux/common/UserConfig.txt bin/

                                probably after make you could try make install which should (hopefully!) install the shared library in the appropriate place.

                                I looked at the code in telemetry_sample.cpp . It looks like the actual serial reading is happening elsewhere and the main thread here is just polling that data which is actually read from another thread behind the hood. This matches their threading model which is explained under "The OSDK Threading Model" in https://developer.dji.com/onboard-sdk/documentation/development-workflow/integrate-sdk.html . Now, ideally you'd like to process this data directly from the serial polling thread, or at least be notified when new data is ready (either with your thread waiting on a lock / condition variable or with the serial thread calling a callback function defined by you). You'll have to dig deeper into their API to see if any of these are exposed to the user, otherwise you'll have to modify their implementation and implement this behaviour yourself.

                                Just to clarify: the Serial library from Bela allows to read and write raw data on the serial port, but I am not sure it will be particularly useful here, because it's the SDK's job to parse that raw data into variables you can access. If the SDK API allows you to pass in raw data and obtain back parsed data, then it could be useful to use the Serial library, but - again - it's up to you to investigate whether the API does that. Clearly there's always the option to ditch the SDK altogether: use Serial to write and read the data and then parse them yourself, assuming that documentation on the format is available, but - depending on the complexity of the format and/or the need for specific sequence of commands - it may be quite an effort.

                                • mpc replied to this.
                                  6 days later

                                  giuliomoro

                                  giuliomoro probably after make you could try make install which should (hopefully!) install the shared library in the appropriate place.

                                  Thanks Giulio, that seems to have solved the problem. Current Bela code links correctly with the libraries/api etc and compiles without issue.

                                  giuliomoro I looked at the code in telemetry_sample.cpp . It looks like the actual serial reading is happening elsewhere and the main thread here is just polling that data which is actually read from another thread behind the hood. This matches their threading model which is explained under "The OSDK Threading Model" in https://developer.dji.com/onboard-sdk/documentation/development-workflow/integrate-sdk.html . Now, ideally you'd like to process this data directly from the serial polling thread, or at least be notified when new data is ready (either with your thread waiting on a lock / condition variable or with the serial thread calling a callback function defined by you). You'll have to dig deeper into their API to see if any of these are exposed to the user, otherwise you'll have to modify their implementation and implement this behaviour yourself.

                                  So it seems after doing some digging into the SDK code, I found some code that seems to be dealing with the linux serial devices, threads and thread management. Here is a link to it for reference: https://we.tl/t-cwRpXxGOMJ

                                  It seems from the code (link above) that it could potentially be possible to process data directly from the serial polling thread, or at least implement this type of behaviour. Although I'm not too sure where to start with that, as I haven't done too much development with serial communication before. Any suggestions to point me in the right direction?

                                  giuliomoro Just to clarify: the Serial library from Bela allows to read and write raw data on the serial port, but I am not sure it will be particularly useful here, because it's the SDK's job to parse that raw data into variables you can access. If the SDK API allows you to pass in raw data and obtain back parsed data, then it could be useful to use the Serial library, but - again - it's up to you to investigate whether the API does that. Clearly there's always the option to ditch the SDK altogether: use Serial to write and read the data and then parse them yourself, assuming that documentation on the format is available, but - depending on the complexity of the format and/or the need for specific sequence of commands - it may be quite an effort.

                                  So it could be an issue logging any of the telemetry data using the serial library unless the SDK API allows for the passing in of raw data or obtaining parsed data. In this case, I require the SDK for running the telemetry code and communicating with the drone's internal system. To ditch it altogether would require a lot of effort in order to get everything functioning as intended. So I'll keep the SDK for now.