The use of the BB-UART4 overlay is alternative to the use of cape universala. This should mean that with this overlay loaded the relevant pins of UART4 should already be enabled without need to use config-pin. Try using the UART4 peripheral to verify that

OMG yes! Thank you so much 🙂
I was looking in all the wrong places, but your answer helped me shift my focus to the actual problem:
It wasn't really a problem with the serial connection as I had assumed but with the ttymidi.service..
I had used @ebai101 's build.sh script before. Compiling manually did the Trick!