I think I found a way to implement the submodular synthesizer architecture that I have been thinking about. Following discussions with my good friend Dave, I believe that I could use the AD75019 analog crosspoint switch as a foundation. This device is essentially a matrix of 256 switches that lets you route an analog signal from any one of 16 inputs to any one of 16 outputs, with minimal distortion (no gain, 150 Ω on resistance), and a very simple serial interface for digital control.
What’s more, multiple devices can be cascaded in order to extend the number of switches in the array, while using a single serial interface. As a result, a 32 × 32 matrix could be created with four devices, and a 64 × 64 with 16. At $20 a piece, this would remain relatively affordable. The device uses the 44 ld PLCC package, which could be interfaced to a prototyping breadboard with this adapter.