Hello. Just getting into this so apologies if this question is answered elsewhere. I'm wondering if I can receive a sysex message into a pure data patch using libpd. If I send a sysex message, the bela console confirms that it has been received, but I'm not getting anything out of a [sysexin] object in my patch. Am I limited to [notein] and [ctlin] as far as MIDI goes with libpd?

My goal is to alter patch parameters in real time with some kind of MIDI message, so maybe the regular ctlin messages are the way to go? New to MIDI and pd, so any advice is appreciated. Thanks

There is no support for sysex yet. I can look into adding it. However, for the purpose of what you are describing, ctlin or pgmin could indeed be more straightforward to use

Thanks for the response! I think I'll work it out with ctlin.

3 years later

Hello. Is the sysexin object now supported ?

It would be nice but it's anyway not a big limitation, as I suppose that one easy workaround would be to modify the C++ wrapper (as explained here) to transmit the received bytes to the pd patch.

Not there yet ... one needs to call setSysexCallback() on each of the Midi objects and then from the callback, which runs on the MIDI receiver thread , call writeNonRt() on a Pipe object, so that the audio thread can call readRt() on the same object and pass the bytes to libpd_sysex(int port, int byte). Alternatively, some lighter thread-safe circular buffer can be used.

Wanna give it a try? Or shall I do it?

Although not being a C++ expert, I want to give a try myself first and will let you know the results soon.

Thanks for your help!

Hi !

I could make it working by simply sending the bytes directly from the callback function. Is it that wrong?

Following your recommendations I also implemented a version with a Pipe object so the bytes are read and sent to pd from the audio thread. To what extent it would be safer/preferable over the first version?

Thanks again for your precious help.

    alexmich I could make it working by simply sending the bytes directly from the callback function. Is it that wrong?

    yes. The callback function is executed in a different thread from the audio thread where Pd runs. This means that if the audio thread preempts the callback function halfway through a libpd function call, then libpd will crash or misbehave, libpd is single-threaded in this application and so all libpd calls need to be made from the audio thread. Wanna share some code?

    Understood, thanks.
    Here are the few very simple lines I added to my custom render.cpp file:

    In the setup function, I setup the pipe object gSysexPipe (declared globally) and call setSysexCallback on the newMidi Midi object:

    #ifdef BELA_LIBPD_MIDI
        gSysexPipe.setup("sysexPipe", 16);  // add this line
        unsigned int n = 0;
        while(n < gMidiPortNames.size()) {
            Midi* newMidi = openMidiDevice(gMidiPortNames[n], false, false);
            if(newMidi) {
    	    newMidi->getParser()->setSysexCallback(sysexCallback, NULL); // add this line
                midi.push_back(newMidi);
                ++n;
            } else {
    	    gMidiPortNames.erase(gMidiPortNames.begin() + n);
    	}
        }
        dumpMidi();	
    #endif // BELA_LIBPD_MIDI
    

    In the callback function, I simply push the received bytes into the pipe object:

    void sysexCallback(midi_byte_t byte, void*arg) {
        gSysexPipe.writeNonRt(byte);
    }
    

    ...to finally call them from the audio thread and send them to pd:

        midi_byte_t byte;
        while(gSysexPipe.readRt(byte) > 0) {
            libpd_sysex(0, byte);	
        }
    

    Is this implementation acceptable?

    That probably works OK in many cases when you only have one MIDI port and for 99% of Sysex users (i.e.: 99.99% of total users would already be happy with this), so let's work out how to make it work for the remaining 1% / 0.01%.

    However,

    while(gSysexPipe.readRt(byte) > 0) {
    libpd_sysex(0, byte);
    }

    here you are hardcoding 0 as the port number, but what if one has more than one Midi port (i.e.: if midi.size() > 1) ? That first argument becomes the output of the second outlet of [sysexin] and with the code as it is, that will always be 0, regardless of how many MIDI devices are connected and regardless of which receives the data. So there is need to pass the information about the device on which this was received to the libpd_sysex() call.

    Now, there are two ways of doing this. One is to use a separate Pipe object per each MIDI device that is created, the other is to send not just the byte but also the MIDI device index down the pipe. The former option requires sending less data overall, but the audio thread would have to read not from one but from several pipes for every callback. The latter option would require sending more data down a single pipe (e.g.: one-byte idx, one-byte data). I suspect the latter option is more efficient (as sysex data is normally pretty infrequent) and it is surely more fun to implement, so let's see how to do this.

    To send structured data down a pipe, you will need - well - a struct:

    struct SysexMsg {
        unsigned char idx;
        midi_byte_t data;
    }

    Then the callback would send both midi port idx and data:

    void sysexCallback(midi_byte_t byte, void*arg) {
        SysexMsg msg;
        msg.idx = midiPortIdx;
        msg.data = byte
        gSysexPipe.writeNonRt(msg);
    }

    and the audio thread can retrieve the full msg and use both of its elements:

        SysexMsg msg;
        while(gSysexPipe.readRt(msg) > 0) {
            libpd_sysex(msg.idx, msg.data);	
        }

    Nice. However, this won't compile because midiPortIdx in the callback is not defined. How to retrieve - from within the callback - what's the index of the Midi object that triggered the call? One could look at this and compare it with the addresses of the elements of midi, but that wouldn't look particularly good. The canonic solution is to leverage the callback's argument. Remember where in setup() you call

    	    newMidi->getParser()->setSysexCallback(sysexCallback, NULL);

    ? Well, the second argument to setSysexCallback is passed as an argument to sysexCallback when this gets called (that void* arg that is currently unused). So we could pass the index in there:

    	    newMidi->getParser()->setSysexCallback(sysexCallback, midi.size());

    And then in the callback:

    void sysexCallback(midi_byte_t byte, void*arg) {
        unsigned char midiPortIdx = arg;
        SysexMsg msg;
        msg.idx = midiPortIdx;
       ...
    }

    If your compiler is lenient enough, this may even compile, but for good measure one should be explicitly casting data to and from the void* on both sides:

    // in setup()
    	    newMidi->getParser()->setSysexCallback(sysexCallback, (void*)midi.size());

    and

    // in the callback
        unsigned char midiPortIdx = (unsigned char)arg;

    Note: if you had to pass more data to the callback, you wouldn't pass it by value, but pass a pointer to a struct of arbitrary size and sufficient lifetime.
    Right, I suspect the above will work.

    Finally,

        gSysexPipe.setup("sysexPipe", 16);

    you probably want a much larger size for the pipe (especially if using the 2-byte per data byte approach and using a single Pipe). Memory is basically free, make it at least 16384!

    I indeed wrote the minimum number of lines that suits my basic needs in an acceptable way, selfishly omitting the scenarios where multiple MIDI devices 1) are used and 2) should be differentiated within the pd patch!

    Thanks to your very detailed explanations I can see that implementing properly sysex support wasn't actually that hard and that the code still remains light and straightforward.

    If the [sysexin] object now reacts adequately to input sysex messages, could those few lines be added to the default libpd wrapper for future users ? This is a naive question, as I have no idea about how is managed the development of the official release.

    you probably want a much larger size for the pipe (especially if using the 2-byte per data byte approach and using a single Pipe). Memory is basically free, make it at least 16384!

    In my very limited case, I am sending sysex messages from a Yamaha QY300 whose maximum sysex message length is...16.
    I assumed that I would hardly reach a situation where sysex bytes are sent at a rate that the audio thread cannot handle. I might be wrong, and increasing the size of the pipe is indeed not a big deal in any case.

      alexmich I assumed that I would hardly reach a situation where sysex bytes are sent at a rate that the audio thread cannot handle

      That's a bold assumption!

      alexmich is indeed not a big deal in any case.

      Keep in mind that increasing the pipe size doesn't increase the minimum latency, but allows for higher latencies without data loss.

      alexmich If the [sysexin] object now reacts adequately to input sysex messages, could those few lines be added to the default libpd wrapper for future users ? This is a naive question, as I have no idea about how is managed the development of the official release.

      yes I'll try to do that soon and send it to you for testing

        giuliomoro yes I'll try to do that soon and send it to you for testing

        Is Bela's github repository open to public contributions?
        If so, I can eventually ease a little bit your workload by implementing these few lines myself and proposing it for review.

        Sure, why don't you send a PR against the dev branch? I wanted to add an #ifdef that allows to keep Midi enabled but disable sysex only (see the style of other #ifdefs there at the top of the file). This is to remove the dependency from Pipe when running on systems which do not support it.