Building Adaptive Plugins

Adaptive plugins are defined as plugins which automatically change their parameters based on the incomming audio stream. For instance, the RMS amplitude may be used to set the gain (similar to a compressor, which can be considered an adaptive effect). This quick guide will demonstrate how to build an adaptive Wah-Wah effect.

The Wah-Wah

The Wah-Wah is a bandpass filter where the center frequency is modulated over time, usually through an LFO or user interaction. In this example, the center frequency of the biquad filter will be controlled by the spectral centroid of the signal.

/*
    SpectralWah
    An adaptive Wah-Wah effect
*/
var SpectralWah = function (factory, owner) {
    /* 
        Each plugin is passed two arguments on construction:
            1 - > Factory: The factory that built this plugin
            2 - > Owner: The SubFactory that this plugin is registered too (if given)
    */

    // This attaches the base plugin items to the Object
    BasePlugin.call(this, factory, owner);

    /* USER MODIFIABLE BEGIN */
    // Only modify between this line and the end of the object!

    // The current web audio API context is available to the plugin through the this.context object
    // We are using it to create a web audio API gain node.
    var filter = this.context.createBiquadFilter();
    filter.type = "peaking";
    filter.Q.value = 2.0;
    filter.gain.value = 24;
    var input = this.context.createGain(),
        output = this.context.createGain();

    input.connect(filter);
    filter.connect(output);

    // Set the gain node as the input point. All connections to the plugin are made
    // to this node.
    this.addInput(input);
    // Set the gain node as the output point. All connections from the plugin are
    // made from this node
    this.addOutput(output);
    /* USER MODIFIABLE END */
}

// Also update the prototype function here!
SpectralWah.prototype = Object.create(BasePlugin.prototype);
SpectralWah.prototype.constructor = SpectralWah;
SpectralWah.prototype.name = "SpectralWah";

The above excerpt shows the layout of the effect. It has one input and output and no parameters (it's adaptive!). The filter is attached in parallel to the procesor to make sure that not all the content is filtered out.

Getting the features

For this effect, we want to know the spectral centroid of the incomming signal. This is achieved by getting the features from the plugin before this one, or the SubFactory if this is the first plugin. The feature object to create for the request would look something like:

{
    "outputIndex": 0,
    "frameSize": 1024,
    "features": [{
        "name": "spectrum",
        "features": [{
            "name": "spectral_centroid"
        }]
    }]
}

The plugin can obtain its position in the processing chain through its owning SubFactory function getPluginIndex(PluginInstance):

var plugin_index = this.owner.getPluginIndex(this.pluginInstance);

Then the request for features must be made through the featureMap object:

var request = {
    "outputIndex": 0,
    "frameSize": 1024,
    "features": [{
        "name": "spectrum",
        "features": [{
            "name": "spectral_centroid"
        }]
    }]
}
if (index == 0) {
    this.featureMap.Receiver.requestFeaturesFromPlugin(this.owner.featureSender, request);
} else {
    this.featureMap.Receiver.requestFeaturesFromPlugin(this.owner.getPlugins()[index-1], request);
}

Now we need to collect the features being passed back. There is a callback function applied to the Receiver node which must be modified:

this.featureMap.onfeatures = function(message) {
    // Get the results of each channel, for simplicity we will assume a mono system
    var channelresults = message.features.results;
    var spectral_centroid = channelresults[0].spectrum.spectral_centroid;
    // Now some processing
}

Applying the features

The features need to be applied internally to the system, therefore a private plugin parameter must be made:

function updateFrequency(f) {
    // This can be called to update the filter information
    // First, check the number we are getting is valid
    if (!isFinite(f)) {
        return;
    }
    // Ensure it fits within some sensible bounds
    f = Math.min(f, 5000);
    f = Math.max(f, 100);
    // Apply to the filter
    filter.frequency = f;
}

Tying it all together

/*
    SpectralWah
    An adaptive Wah-Wah effect
*/
var SpectralWah = function (factory, owner) {
    /* 
        Each plugin is passed two arguments on construction:
            1 - > Factory: The factory that built this plugin
            2 - > Owner: The SubFactory that this plugin is registered too (if given)
    */

    // This attaches the base plugin items to the Object
    BasePlugin.call(this, factory, owner);

    /* USER MODIFIABLE BEGIN */
    // Only modify between this line and the end of the object!

    // The current web audio API context is available to the plugin through the this.context object
    // We are using it to create a web audio API gain node.
    var filter = this.context.createBiquadFilter();
    filter.type = "peaking";
    filter.Q.value = 2.0;
    filter.gain.value = 24;
    var input = this.context.createGain(),
        output = this.context.createGain();

    input.connect(filter);
    filter.connect(output);

    this.onloaded = function () {
        var plugin_index = this.owner.getPluginIndex(this.pluginInstance);

        // Register the Features:
        var request = {
            "outputIndex": 0,
            "frameSize": 8192,
            "features": [{
                "name": "spectrum",
                "features": [{
                    "name": "spectral_centroid"
            }]
        }]
        }
        console.log(plugin_index);
        if (plugin_index == 0) {
            this.featureMap.Receiver.requestFeaturesFromPlugin(this.owner.featureSender, request);
        } else {
            this.featureMap.Receiver.requestFeaturesFromPlugin(this.owner.getPlugins()[plugin_index - 1], request);
        }
    }

    this.onunloaded = function() {
        this.featureMap.Receiver.cancelAllFeatures();
    }

    // Create the function for the callback and setting the filter:
    function updateFrequency(f) {
        // y[n] = x[n]*(1-a) + y[n-1]*(a);
        // This can be called to update the filter information
        // First, check the number we are getting is valid
        if (!isFinite(f)) {
            return;
        }
        var a = 0.7;
        // Ensure it fits within some sensible bounds
        f = Math.min(f, 5000);
        f = Math.max(f, 100);
        // Apply to the filter
        f = f * (1 - a) + filter.frequency.value * a;
        filter.frequency.value = f;
    }

    this.featureMap.onfeatures = function (message) {
        // Get the results of each channel, for simplicity we will assume a mono system
        var channelresults = message.features.results;
        var spectral_centroid = channelresults[0].spectrum.spectral_centroid;
        // Now some processing
        updateFrequency(Number(spectral_centroid));
    }

    // Set the gain node as the input point. All connections to the plugin are made
    // to this node.
    this.addInput(input);
    // Set the gain node as the output point. All connections from the plugin are
    // made from this node
    this.addOutput(output);
    /* USER MODIFIABLE END */
}

// Also update the prototype function here!
SpectralWah.prototype = Object.create(BasePlugin.prototype);
SpectralWah.prototype.constructor = SpectralWah;
SpectralWah.prototype.name = "SpectralWah";
SpectralWah.prototype.version = "1.0.0";
SpectralWah.prototype.uniqueID = "JSPW";

We have to use the supplied onloaded event when requesting features. This is because until the plugin is actually inserted into a chain, it cannot determine what plugin comes before it. We should also use the unloaded event to cancel the features if the plugin is moved or destroyed.

The updateFrequency function also includes a simple one-pole filter to smooth the data out into a more pleasent stream.