Feature Extraction

All JSAP plugins are built with a JS-Xtract feature extraction unit on each output. This allows each plugin instance to perform feature extraction. There are three types of plugins where feature extraction is important:

Accessing the extractors

Extractors exists on each output of your plugin and are placed there as soon as the plugin is built. Another extractor is placed on the beginning of each SubFactory chain. Therefore every plugin can access audio features before and after their processing cycle.

Access is controlled through a module in the plugin called featureMap. This module has two further objects, the Sender and Receiver. The Sender is controlled directly by the PluginFactory object and therefore should not be touched. The Receiver allows a plugin to receive audio features from any plugin / extractor in the chain.

\\ The interface of the featureMap.Receiver object

pluginInstance.featureMap.Receiver.requestFeatures(featureList);
pluginInstance.featureMap.Receiver.requestFeaturesFromPlugin(source, featureList);

pluginInstance.featureMap.Receiver.cancelFeaturesFromPlugin(source, featureList);
pluginInstance.featureMap.Receiver.cancelAllFeaturesFromPlugin(featureList);
pluginInstance.featureMap.Receiver.cancelAllFeatures();

requestFeatures

This function takes an object as its argument and has the following structure:

var featureList = [
    {
        plugin: Object(),
        outputIndex: Number(),
        frameSize: Number(),
        features: []
    }
]
pluginInstance.featureMap.Receiver.requestFeatures(featureList);

It is an array of objects, each object for a specific plugin | output | frameSize combination. For instance, to get the mean every 1024 samples from the plugin at the top of the chain, the object would be:

var featureList = [
    {
        plugin: SubFactory.getPlugins()[0],
        outputIndex: 0,
        frameSize: 1024,
        features: [{
            'name': "mean"
        }]
    }
]
pluginInstance.featureMap.Receiver.requestFeatures(featureList);

To access data from the same plugin at a different output index and/or frame size, a separate object would be required:

var featureList = [
    {
        plugin: SubFactory.getPlugins()[0],
        outputIndex: 0,
        frameSize: 1024,
        features: [{
            'name': "mean"
        }]
    },
    {
        plugin: SubFactory.getPlugins()[0],
        outputIndex: 0,
        frameSize: 2048,
        features: [{
            'name': "mean"
        }]
    }
]
pluginInstance.featureMap.Receiver.requestFeatures(featureList);

requestFeaturesFromPlugin

This function takes two parameters, the plugin to get features from and the featureList object. This is very similar to the requestFeatures plugin, but instead takes only one of the objects from the list:

var featureList = {
    outputIndex: 0,
    frameSize: 1024,
    features: [{
        'name': "mean"
    }]
}
pluginInstance.featureMap.Receiver.requestFeaturesFromPlugin(sourcePlugin, featureList);

To do the two object data, we would have to perform the following lines:

var source = SubFactory.getPlugins()[0];
pluginInstance.featureMap.Receiver.requestFeaturesFromPlugin(source, {
    outputIndex: 0,
    frameSize: 1024,
    features: [{
        'name': "mean"
    }]
});

pluginInstance.featureMap.Receiver.requestFeaturesFromPlugin(source, {
    outputIndex: 0,
    frameSize: 2048,
    features: [{
        'name': "mean"
    }]
});