I'm getting disconnected from the IDE quite often. I think it happens when the CPU gets overloaded: is that plausible?

yes: high CPU usage will make the IDE less responsive, potentially disconnecting you. When this is the case, just press the button on the Bela cape and it will stop the running program, so the IDE should become available again.

What if the button is out of reach due to the Bela being inside an instrument enclosure?

You can remote it: it's "P9.27" (Bela) or "P2.34" (BelaMini). Short it to ground briefly to stop the currently running program. If you hold it for 2 seconds or more, you will get the action associated with it (default: graceful shutdown of the board, can be overridden by creating an executable file in /opt/Bela/local/bela_button_hold.sh to be called upon press hold). You can also change the behaviour of the button (or even the GPIO pin in use) by editing /lib/systemd/system/bela_button.service.

a month later

When I press the button with run on boot enabled it starts again within a second. Can this be disabled? I can't connect to the IDE when the program runs, so I shut it by pressing the button however if it then immediatly boots again I'm still can't reach the IDE.

My current solution to be able to open the IDE or even SSH into the Bela is to take out the SD card and delete the program executable so it won't run on boot. Not really a nice solution because I have to boot into linux instead of windows to be able to open the correct partition to delete the executable..

The default behaviour is considered desirable because if the program crashes while in headless mode, you would want it to start again automatically.
This behaviour can be changed or disabled by editing /lib/systemd/system/bela_startup.service . There you can chose to avoid restarting the service, or set a longer timeout.

Here are some instructions in order to recover froma similar situation: https://github.com/BelaPlatform/Bela/wiki/Recover-from-stuck-in-reboot-loop