Hi,
I can't tell much about the GUI crashes. I've never had any problems om my test systems. The only time GUI crashes happened when I was using a system with an atom processor. I'm testing on a intel nuc i5-5250 and i5-6250 + 8GB memory. So maybe a processor problem? You could do a test by reducing the samplerate to 48000. It won't be stereo but the CPU takes about a 3rd of processing power, if crashes are gone it's most likely the the CPU capacity.
I use a composite peak control and it controls the full baseband 40Hz-53kHz but it isn't actual clipping just reducing gain on the parts that are to loud and therefor not producing many 3e harmonics. I'm gonna do some testing of splitting up the baseband as you suggested. The fm peak contol only affects the 40Hz-15kHz mono and stereo diff. signal before DBS modulation.
Cor