Nice idea. I think the speed of that solution would be fine. Reading from USB would need to go through the Linux kernel so it wouldn't necessarily have the same hard-real-time performance as the digital or analog I/O, but in practice the delay ought to be very small, maybe a couple milliseconds at most.
The way to handle it will be to set up an AuxiliaryTask (i.e. a second thread) which handles reading from the keyboard. When you receive a character, use it to set a global variable which communicates back to the audio thread. You will have some jitter in this arrangement (i.e. timing will be quantised to the audio block size) but in practice I don't think the difference will be at all audible, especially if your block size is one of the default small sizes like 16 samples.
The filter-IIR example in the "terminal-only" is a starting place for working with the keyboard. Look at the function read_input()
.