Hi Everyone,

Just thought I would share a nice way to interface Bela with Arduino using Websockets. Pretty new to posting so feel free to add any corrections/insight.

This was tested on an Arduino Nano 33 iot. The library used is the following:
Built by Khoi Hoang https://github.com/khoih-prog/Websockets2_Generic
Licensed under MIT license

The library is designed to work with many wireless wifi boards so opens a lot of possibilities. I modified the Arduino websockets2 client example so that it connects directly to the bela Gui library and there is almost no noticeable latency. For running external lights in real time for level indicators or toggles of any sort it should work exceptionally well, in use as a live instrument there were some occasional noticeable lags but I'm not sure if it was due to the actual button registering or wifi latency. Multiple overlapping wifi signals or other reduncancies should be enough to make it fully useable realtime-ish.

Here's the Arduino code, which is the base websocket2 library mentioned above modified with button debounce and state detection:


#include "defines.h"
#include <Arduino_JSON.h>
#include <WebSockets2_Generic.h>

using namespace websockets2_generic;
WebsocketsClient client;

const int IPLastByte = 130;   //set last byte of ip address 

//BUTTONS////
unsigned long debounceDelay = 50;    // the debounce time; increase if the output flickers
const int arraySize = 4;

int buttonPin[arraySize];    // the pin that the pushbutton is attached to
// Variables will change:
int buttonPushCounter[arraySize];   // counter for the number of button presses
int buttonState[arraySize];         // current state of the button
int lastButtonState[arraySize];     // previous state of the button
unsigned long lastDebounceTime[arraySize];  // the last time the output pin was toggled

void setup() 
{
  Serial.begin(115200);
  //while (!Serial);

  pinMode(2, INPUT_PULLUP);
  pinMode(3, INPUT_PULLUP);

    //initialize button arrays
  for(int i=0; i<arraySize; i++) {
    buttonPushCounter[i]=0;   // counter for the number of button presses
    buttonState[i]=0;         // current state of the button
    lastButtonState[i]=0; // previous state of the button
    buttonPin[i] = i;
    lastDebounceTime[i] = 0;
  }

  Serial.println("\nStarting SAMD-Client with WiFiNINA on " + String(BOARD_NAME));
  Serial.println(WEBSOCKETS2_GENERIC_VERSION);
  
  // check for the WiFi module:
  if (WiFi.status() == WL_NO_MODULE) 
  {
    Serial.println("Communication with WiFi module failed!");
    // don't continue
    return;
  }

  String fv = WiFi.firmwareVersion();
  if (fv < WIFI_FIRMWARE_LATEST_VERSION) 
  {
    Serial.println("Please upgrade the firmware");
  }

  Serial.print("Attempting to connect to SSID: ");
  Serial.println(ssid);

   WiFiConnect();

  if (WiFi.status() == WL_CONNECTED)
  {
    Serial.print("Connected to Wifi, IP address: ");
    Serial.println(WiFi.localIP());
    Serial.print("Connecting to WebSockets Server @");
  }
  else
  {
    Serial.println("\nNo WiFi");
    return;
  }

  // run callback when messages are received
  client.onMessage(onMessageCallback);

  // run callback when events are occuring
  client.onEvent(onEventsCallback);

  // Connect to server
  client.connect(websockets_server_host, websockets_server_port, "/");

  // Send a message
  String WS_msg = String("Hello to Server from ") + BOARD_NAME;
  client.send(WS_msg);

  // Send a ping
  client.ping();

}

void loop() 
{
  //Try to reconnect WiFi if connection was lost
  if (WiFi.status() != WL_CONNECTED) {
    digitalWrite(LED_BUILTIN, LOW);
    Serial.println("\n--Lost WiFi connection");
    WiFi.end();
    WiFiConnect();
  }
  // let the websockets client check for incoming messages
  if (client.available()) 
  {
    client.poll();
  }

  button();   //scan for button state changes
}                                          //////////////////End Main Loop

void button(void) {
  for (int b=2; b<arraySize; b++){
        // read the pushbutton input pin:
      int reading = digitalRead(buttonPin[b]);
    
      // compare the buttonState to its previous state
      if (buttonState[b] != lastButtonState[b]) {
          //if state changed, start debounce timer
          lastDebounceTime[b] = millis();
      }    
      if ((millis() - lastDebounceTime[b]) > debounceDelay) {
            if (reading != buttonState[b]) {
                buttonState[b] = reading;

                Serial.println("pushed");                        
                if (buttonState[b] == LOW) {
                  // if the current state is LOW(input pullup) then the button went from off to on:
                  buttonPushCounter[b]++;
                  Serial.println("on");
                  Serial.print("number of button pushes: ");
                  Serial.println(buttonPushCounter[b]);
                  sendMessage("p");   //if pushed, send char
                } else {
                  // if the current state is LOW then the button went from on to off:
                  Serial.println("off");
                }
                // Debounce        
          }
          // save the current state as the last state, for next time through the loop
          lastButtonState[b] = buttonState[b];
      }
  }
}


void socketConnect() {
bool connected = client.connect(websockets_server_host, websockets_server_port, "/gui_data");
while(!connected) {
connected = client.connect(websockets_server_host, websockets_server_port, "/gui_data");
Serial.println("Connnecting sockets...sockeConnect");
delay(5000);
}
}

void sendMessage(const char* button)
{
  client.sendBinary(button,2);
}

void onMessageCallback(WebsocketsMessage message) 
{
  Serial.print("Got Message: ");
  Serial.println(message.data());
}

void onEventsCallback(WebsocketsEvent event, String data) 
{
  if (event == WebsocketsEvent::ConnectionOpened) 
  {
    Serial.println("Connnection Opened");
  } 
  else if (event == WebsocketsEvent::ConnectionClosed) 
  {
    Serial.println("Connnection Closed");
    socketConnect();
  } 
  else if (event == WebsocketsEvent::GotPing) 
  {
    Serial.println("Got a Ping!");
  } 
  else if (event == WebsocketsEvent::GotPong) 
  {
    Serial.println("Got a Pong!");
  }
}

void WiFiConnect() {
  while (WiFi.status() != WL_CONNECTED) {
    Serial.println("Connecting to ");
    Serial.println(ssid);
    Serial.println(" ...");
    WiFi.begin(ssid, password);
    delay(5000);
    
  }
  Serial.println(websockets_server_host);
   // run callback when messages are received
  client.onMessage([&](WebsocketsMessage message) 
  {
    Serial.print("Got Message: ");
    Serial.println(message.data());
  });

  // run callback when events are occuring
  client.onEvent(onEventsCallback);

  //sendMessage();
  IPAddress IP = WiFi.localIP();
  IP[3] = IPLastByte;
  WiFi.config(IP, WiFi.gatewayIP(), WiFi.gatewayIP(), WiFi.subnetMask());
  Serial.println("Connected to ");
  Serial.println(ssid);
  int tries = 5;
  WiFi.lowPowerMode();
  digitalWrite(LED_BUILTIN, HIGH);
}

and the defines.h file:

/****************************************************************************************************************************
  defines.h
  For SAMD21/SAMD51 with WiFiNINA module/shield.
  
  Based on and modified from Gil Maimon's ArduinoWebsockets library https://github.com/gilmaimon/ArduinoWebsockets
  to support STM32F/L/H/G/WB/MP1, nRF52 and SAMD21/SAMD51 boards besides ESP8266 and ESP32
  
  The library provides simple and easy interface for websockets (Client and Server).
  
  Example first created on: 10.05.2018
  Original Author: Markus Sattler
  
  Built by Khoi Hoang https://github.com/khoih-prog/Websockets2_Generic
  Licensed under MIT license
 *****************************************************************************************************************************/

#ifndef defines_h
#define defines_h

#if    ( defined(ARDUINO_SAMD_ZERO) || defined(ARDUINO_SAMD_MKR1000) || defined(ARDUINO_SAMD_MKRWIFI1010) \
      || defined(ARDUINO_SAMD_NANO_33_IOT) || defined(ARDUINO_SAMD_MKRFox1200) || defined(ARDUINO_SAMD_MKRWAN1300) || defined(ARDUINO_SAMD_MKRWAN1310) \
      || defined(ARDUINO_SAMD_MKRGSM1400) || defined(ARDUINO_SAMD_MKRNB1500) || defined(ARDUINO_SAMD_MKRVIDOR4000) || defined(__SAMD21G18A__) \
      || defined(ARDUINO_SAMD_CIRCUITPLAYGROUND_EXPRESS) || defined(__SAMD21E18A__) || defined(__SAMD51__) || defined(__SAMD51J20A__) || defined(__SAMD51J19A__) \
      || defined(__SAMD51G19A__) || defined(__SAMD51P19A__) || defined(__SAMD21G18A__) )
#if defined(WEBSOCKETS_WIFININA_USE_SAMD)
#undef WEBSOCKETS_WIFININA_USE_SAMD
#endif
#define WEBSOCKETS_USE_WIFININA           true
#define WEBSOCKETS_WIFININA_USE_SAMD      true
#else
#error This code is intended to run only on the SAMD boards ! Please check your Tools->Board setting.
#endif

#if defined(WEBSOCKETS_WIFININA_USE_SAMD)

#if defined(ARDUINO_SAMD_ZERO)
#define BOARD_TYPE      "SAMD Zero"
#elif defined(ARDUINO_SAMD_MKR1000)
#define BOARD_TYPE      "SAMD MKR1000"
#elif defined(ARDUINO_SAMD_MKRWIFI1010)
#define BOARD_TYPE      "SAMD MKRWIFI1010"
#elif defined(ARDUINO_SAMD_NANO_33_IOT)
#define BOARD_TYPE      "SAMD NANO_33_IOT"
#elif defined(ARDUINO_SAMD_MKRFox1200)
#define BOARD_TYPE      "SAMD MKRFox1200"
#elif ( defined(ARDUINO_SAMD_MKRWAN1300) || defined(ARDUINO_SAMD_MKRWAN1310) )
#define BOARD_TYPE      "SAMD MKRWAN13X0"
#elif defined(ARDUINO_SAMD_MKRGSM1400)
#define BOARD_TYPE      "SAMD MKRGSM1400"
#elif defined(ARDUINO_SAMD_MKRNB1500)
#define BOARD_TYPE      "SAMD MKRNB1500"
#elif defined(ARDUINO_SAMD_MKRVIDOR4000)
#define BOARD_TYPE      "SAMD MKRVIDOR4000"
#elif defined(ARDUINO_SAMD_CIRCUITPLAYGROUND_EXPRESS)
#define BOARD_TYPE      "SAMD ARDUINO_SAMD_CIRCUITPLAYGROUND_EXPRESS"
#elif defined(ADAFRUIT_FEATHER_M0_EXPRESS)
#define BOARD_TYPE      "SAMD21 ADAFRUIT_FEATHER_M0_EXPRESS"
#elif defined(ADAFRUIT_METRO_M0_EXPRESS)
#define BOARD_TYPE      "SAMD21 ADAFRUIT_METRO_M0_EXPRESS"
#elif defined(ADAFRUIT_CIRCUITPLAYGROUND_M0)
#define BOARD_TYPE      "SAMD21 ADAFRUIT_CIRCUITPLAYGROUND_M0"
#elif defined(ADAFRUIT_GEMMA_M0)
#define BOARD_TYPE      "SAMD21 ADAFRUIT_GEMMA_M0"
#elif defined(ADAFRUIT_TRINKET_M0)
#define BOARD_TYPE      "SAMD21 ADAFRUIT_TRINKET_M0"
#elif defined(ADAFRUIT_ITSYBITSY_M0)
#define BOARD_TYPE      "SAMD21 ADAFRUIT_ITSYBITSY_M0"
#elif defined(ARDUINO_SAMD_HALLOWING_M0)
#define BOARD_TYPE      "SAMD21 ARDUINO_SAMD_HALLOWING_M0"
#elif defined(ADAFRUIT_METRO_M4_EXPRESS)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_METRO_M4_EXPRESS"
#elif defined(ADAFRUIT_GRAND_CENTRAL_M4)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_GRAND_CENTRAL_M4"
#elif defined(ADAFRUIT_FEATHER_M4_EXPRESS)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_FEATHER_M4_EXPRESS"
#elif defined(ADAFRUIT_ITSYBITSY_M4_EXPRESS)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_ITSYBITSY_M4_EXPRESS"
#elif defined(ADAFRUIT_TRELLIS_M4_EXPRESS)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_TRELLIS_M4_EXPRESS"
#elif defined(ADAFRUIT_PYPORTAL)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_PYPORTAL"
#elif defined(ADAFRUIT_PYPORTAL_M4_TITANO)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_PYPORTAL_M4_TITANO"
#elif defined(ADAFRUIT_PYBADGE_M4_EXPRESS)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_PYBADGE_M4_EXPRESS"
#elif defined(ADAFRUIT_METRO_M4_AIRLIFT_LITE)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_METRO_M4_AIRLIFT_LITE"
#elif defined(ADAFRUIT_PYGAMER_M4_EXPRESS)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_PYGAMER_M4_EXPRESS"
#elif defined(ADAFRUIT_PYGAMER_ADVANCE_M4_EXPRESS)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_PYGAMER_ADVANCE_M4_EXPRESS"
#elif defined(ADAFRUIT_PYBADGE_AIRLIFT_M4)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_PYBADGE_AIRLIFT_M4"
#elif defined(ADAFRUIT_MONSTER_M4SK_EXPRESS)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_MONSTER_M4SK_EXPRESS"
#elif defined(ADAFRUIT_HALLOWING_M4_EXPRESS)
#define BOARD_TYPE      "SAMD51 ADAFRUIT_HALLOWING_M4_EXPRESS"
#elif defined(SEEED_WIO_TERMINAL)
#define BOARD_TYPE      "SAMD SEEED_WIO_TERMINAL"
#elif defined(SEEED_FEMTO_M0)
#define BOARD_TYPE      "SAMD SEEED_FEMTO_M0"
#elif defined(SEEED_XIAO_M0)
#define BOARD_TYPE      "SAMD SEEED_XIAO_M0"
#elif defined(Wio_Lite_MG126)
#define BOARD_TYPE      "SAMD SEEED Wio_Lite_MG126"
#elif defined(WIO_GPS_BOARD)
#define BOARD_TYPE      "SAMD SEEED WIO_GPS_BOARD"
#elif defined(SEEEDUINO_ZERO)
#define BOARD_TYPE      "SAMD SEEEDUINO_ZERO"
#elif defined(SEEEDUINO_LORAWAN)
#define BOARD_TYPE      "SAMD SEEEDUINO_LORAWAN"
#elif defined(SEEED_GROVE_UI_WIRELESS)
#define BOARD_TYPE      "SAMD SEEED_GROVE_UI_WIRELESS"
#elif defined(__SAMD21E18A__)
#define BOARD_TYPE      "SAMD21E18A"
#elif defined(__SAMD21G18A__)
#define BOARD_TYPE      "SAMD21G18A"
#elif defined(__SAMD51G19A__)
#define BOARD_TYPE      "SAMD51G19A"
#elif defined(__SAMD51J19A__)
#define BOARD_TYPE      "SAMD51J19A"
#elif defined(__SAMD51J20A__)
#define BOARD_TYPE      "SAMD51J20A"
#elif defined(__SAM3X8E__)
#define BOARD_TYPE      "SAM3X8E"
#elif defined(__CPU_ARC__)
#define BOARD_TYPE      "CPU_ARC"
#elif defined(__SAMD51__)
#define BOARD_TYPE      "SAMD51"
#else
#define BOARD_TYPE      "SAMD Unknown"
#endif

#endif

#ifndef BOARD_NAME
  #define BOARD_NAME    BOARD_TYPE
#endif

#include <WiFiNINA_Generic.h>

#define DEBUG_WEBSOCKETS_PORT     Serial
// Debug Level from 0 to 4
#define _WEBSOCKETS_LOGLEVEL_     3

const char* ssid = "<yourSSIDHere"; //Enter SSID
const char* password = "<yourSuperSecretPasswordHere>"; //Enter Password

const char* websockets_server_host = "<bela'sIPHere"; //xxx.xxx.xxx.xxx format
//const char* websockets_server_host = "serverip_or_name"; //Enter server address

#define WEBSOCKETS_PORT     5555  //bela's port

const uint16_t websockets_server_port = WEBSOCKETS_PORT; // Enter server port

#endif      //defines_h

To receive messages I just made some ugly hard code directly into the Gui library, for a code wizard it should be easy to make a nice implementation or library of some sort for simplicity.

that's interesting, thanks for sharing ... so if I understand correctly, on the Bela side you use the Gui library, but without actually visualising the HTML/Js GUI in the browser, rather you establish a websocket connection with the Arduino, right? Out of curiosity, what motivated the use of the Arduino here? Was it to leverage the expanded connectivity or was it mostly about the opportunities of the wireless link? What did you connect to the Arduino?

    The connection is directly to the Bela and bypasses the browser, thereby allowing the Arduino to be used as an instrument and keeps the latency down (I think). When the Arduino is booted it keeps searching indefinitely until it finds the wifi signal (which I'm using a wifi dongle on the Bela setup as an AP) then does the same with the websockets. I've only used actual push buttons so far. Next I plan on connecting some Trill sensors to the Arduino. Also I plan on using arduino's gyroscope and accelerometer as toggles/instruments. It just makes sense to me to use websockets to connect it all since the framework is already there, it's bi-directional, reliable, etc. There's almost no noticeable overhead on the Bela processor. I tried this project using UDP first but it was very cpu intensive.

      giuliomoro I have a usb hub connected to the embedded bela mini I'm using and it's running 3 wifi dongles, one ad-hoc, one ap and one client. I would love to have some usb 3.0 functionality just for the increased throughput and I'm guessing reduced latency, is there a beaglebone I could attach a cape to for easy realization?

        not really 🙁 Either way, I doubt you'll be constrained by the USB 2 bandwidth ... my gut feeling would be that the bottleneck is in the wireless link and/or the co-existence if several antennas in such a constrained space.

        What are you using each dongle for?

        thanks for the clarification.

        ZekeNorris I tried this project using UDP first but it was very cpu intensive.

        that's unexpected... I am wondering there was anything wrong with your implementation ....

            giuliomoro There was possibly error in implementation of UDP. I seem to recall not significantly negligible increase in cpu (~10%) when using bela as a UDP client then something substantial (30-40%) cpu load as a server. Before spending too much time on it I researched websockets more and it seemed on paper to be a non-substantial latency increase with solid error reduction built in. The websockets over wifi have just performed impressively well- and as you mentioned the biggest bottle neck would theoretically seem to be in the usb2, although I haven't noticed much of a hit. I do however see extremely fast connections over the statically coded ad-hoc network along with steady ~ 0.7-2.4 ms max pings vs intermittant 4-5ms pings on the access point network connection along with it taking longer to connect initally and sometimtes not even starting. The wpa_supplicant client connection is pretty stable and in the middle for speed.

                ZekeNorris steady ~ 0.7-2.4 ms max pings vs intermittant 4-5ms pings o

                You'll probably find that you may have much larger peak delays especially under system load. While this specific setup was not analysed, the paper below goes through some of the issues with jitter also related to wireless networks: even if the average latency seems acceptable, the worst-case one is unlikely to be good enough http://www.eecs.qmul.ac.uk/~andrewm/mcpherson_nime2016.pdf

                Also, check out
                S. Madgwick and T. J. Mitchell. x-OSC: A versatile wireless I/O device for creative/music applications. In
                Proc. SMC, 2013.

                ZekeNorris and as you mentioned the biggest bottle neck would theoretically seem to be in the usb2, a

                I actually suggested the bottleneck would be in the wireless link. By the way, what are you using 3 separate dongles at the same time for? What is the purpose of each?

                      giuliomoro The reason behind multiple dongles is mainly versatility when it's embedded and difficult to reach physically. One wifi dongle connects to my home router and there are high ping delays due to cross traffic but it reaches the outside internet and is a fail safe in case I screw up one of the other two while tinkering. One is setup as an access point but that one intermittently doesn't even come up and I have to ssh through the first and ifdown/up it. The third is an ad-hoc setup I just started using but it's very quick and stable, my next goal will be to utilize that with arduio and maybe combine two together for more stability/coverage or just get rid of the extra connections to reduce power consumption and resource utilization, still just messing around with it all and having fun.

                        giuliomoro Thanks for the read:

                        I see there is not much focus on the ad-hoc network, and it actually took a while for me to get it working properly. Also the for the access point I did follow your article:

                        and had a hard time getting it to consistently bring itself up, potentially due to my lack of linux experience as well, but after a whole lot of trial and error, it seems the following addition to /etc/network/interfaces helps it remain consistent on reboot:

                        allow-hotplug wlan2
iface wlan2 inet static
 address 192.168.30.1
 netmask 255.255.255.0
 up hostapd /etc/hostapd/hostapd.conf ifup wlan2

                        I had tried some other things involving systemd and scripts, but had no luck.

                        As for the ad hoc, now that I have it consistently working it is rock solid and very fast. I followed the following article:
                        [https://wiki.debian.org/WiFi/AdHoc]
                        I'm using ubuntu on my laptop with a wifi usb dongle and it is just very stable and fast. I'm next going to figure out a way if possible to enable ad-hoc on the arduino and see if the jitter is low enough for a solid instrument. If not, I may as mentioned before combine multiple wifi devices for further latency and jitter reduction.

                                giuliomoro I'm thinking also maybe using virtual wifi devices over the same connection for redundancy, haven't researched it much but in theory I would imagine it would help with jitter reduction, if anyone has experience it would be welcome to lessen the research/time it will take to learn it lol.

                                  giuliomoro I don't know if it's the case anytime a new connection is made in linux/ubuntu, but for some reason I think it didn't work until I actually pinged from the laptop to the bela. But after that initial ping, it's been up immediately and very consistent with no additional work required and after any reboot.