Buffer features for generative effects about audiolib.js

[jussi-kalliokoski]

Implemented. Now needs to be specced.

[jussi-kalliokoski]

Specced. Review?

[davidgovea]

I just got around to playing with this functionality today - made it way easier to plug into dsp.js.

Some comments:

1. Maybe default to channelCount=1 in generator.append:
   • Calling G.append(buffer) like you do with an effect fails silently - default value would fix this
2. Might be nice to be able to add the generated signal to an existing buffer, rather than overwriting only
   • Makes the mix parameter more useful to generators:
     g1.mix = g2.mix = 0.5; g1.append(g2.append(buf,1), 1, add_to_existing_please);
3. Not sure if it makes sense to do this, but what about independently changing generation parameters on each channel? This is in place for effects already.. something like g[0].fm=lfo1.getMix(); g[1].fm = 440;

I think the last one would require a sizable change to the layout.. lowest priority for sure.

I think being able to generate on top of an existing buffer would be really helpful for my note tracker.. preserving note overflows and such.

I really like all this flexibility.

[davidgovea]

Edit: ^ this guy beat me to it! :P
I've implemented comments 1&2 here: a7974ff9b042bbdf9083
Let me know what you think

[jussi-kalliokoski]

Oh, that's actually a bug, it's always supposed to mix the original signal with the effects signal according to the mix parameter ( wet * mix + dry * (1 - mix) ), I can fix that today.

[davidgovea]

It seems like dynamic frequency modulation cannot be applied with osc.append(). During normal buffer fill, fmOsc.generate(); osc.fm = fmOsc.getMix(); would be used, but this cannot be done during append.

Possible solution: allow Oscillator.fm to be a function:
osc.fm = function(){ lfo.generate(); return lfo.getMix()}
Dont want to add any fm-specific code to append, since it wouldn't apply to all generators necessarily. This solution might slow down the Oscillator's generate stage.. I don't like it.

Another:
Add a genAppend() method to GeneratorClass.prototype, which by default is set to this.generate().
In the append loop, this.genAppend() would be called instead of this.generate()
If fm during append is desired, this function can be set to something like:
osc.genAppend = function(){ this.generate(); lfo.generate(); this.fm = lfo.getMix();}

[davidgovea]

how's that?

Now you can do

    osc.onGenerate(function(){
        lfo.generate();
        osc.fm = lfo.getMix();
    }, function(){
        // do some stuff after generate, too
    })

useful outside of the append() use case as well.

Also got the default channels=1 and the mixing in there

[davidgovea]

Obviously didn't want those if checks in the generate fn.. changed it a little but it's still not the greatest. I'll bet you have a better idea of how to do this. Maybe separate onGenerate into preGenerate and postGenerate? Maybe omit postGenerate altogether? Not sure there's any real use for it..

I'll stop spamming this thread :P

[davidgovea]

OK, think i got something here.
Here is an example which fades the oscillator in and out (100 sample linear fades), and applies FM:

var buf = new Float32Array(300),
    osc = new audioLib.Oscillator(44100, 440);
    lfo = new audioLib.Oscillator(44100, 15);
    len = buf.length;
    i=0;

osc.setPreGenerate(function(){
    lfo.generate();
    osc.fm = lfo.getMix();

    if(i<=100){
        osc.mix= i*1/100;
    } else if(i >= len-100) {
        osc.mix = (len-i)*1/100;
    } else osc.mix = 1;

    i++;
});
osc.append(buf); 

Removed postGenerate, couldn't think of a good reason to have it in there..

[jussi-kalliokoski]

I like the idea there, and the optimization too, so that no checks or anything happen if it's not defined.

I am planning to implement some automation too, just haven't made up my mind about the design yet, I was thinking that the prototype of effect and generator could have an array called automation, containing the automation instructions for the object, and that generator classes would have a function called generateBuffer (sampleCount), that would create a new (Float32) array containing the samples generated and attach it to the generator as generatedBuffer, and append functions would use this data as automation parameters. This is probably best illustrated with an example, let's say I have two oscillators, and the other one is a LFO controlling the FM of the other osc at 2 semitones magnitude, kinda similar to your example.

var buf = new Float32Array(300),
    osc = new audioLib.Oscillator(44100, 440),
    lfo = new audioLib.Oscillator(44100, 15),
    len = buf.length,
    i   = 0;

osc.setAutomation('fm' /* parameter */, lfo /* automation */, 0.121481 * 2 /* automation amount */);
lfo.generateBuffer();
osc.append(buf);

It's not as efficient and not as low-level as your idea, but I think it's more elegant and easy. However, I was really worried about the overhead that might cause, so I haven't implemented it yet, but actually it could be implemented pretty easily with no overhead at all, using your idea as a base. That said, your idea needs some polishing:

• It should be applied to both effects and generators.
• Therefore, the name should be something like addPreProcessing().
• Like you might see in the name I suggested, it needs to be chainable.

You're a real asset, you know?

[davidgovea]

VERY COOL! Elegant code..
Makes it way more modular-synth-like.. next up - lfo control of the page title, location bar, and window size! Just kidding..

Fixed a couple things in the test.

Also, we need to be able to remove the preprocessing steps. This could be linked with xyz.reset(), but since that's pretty instance-specific, maybe a resetPreProcessing() prototype method would be best.

[jussi-kalliokoski]

Hahaha, nice!

I'm a big modular synth lover! The first "sane" DSP thing I did was a modular synth, I was a bit too ambitious with that.

Yeah, I thought of that too, the function could do something like

while (this.generate.generate) this.generate = this.generate.generate;

For more sophisticated stuff, you could pass the reference of a function to the tear down function to remove just a single step.

[jussi-kalliokoski]

Well, I think this is done.