Spectral Features

This lists all the currently implemented spectral features with both procedural and object-oriented function calls. To get the spectrum data, use the following functions:

// Procedural
Float64Array() = xtract_spectrum(array, sample_rate, withDC, normalise);

// Object-Oriented

SpectrumData() = TimeData().spectrum(withDC, normalise);

For ALL procedural calls, the spectrum, peakSpectrum or harmonicSpectrum arguments require the data to include the magnitude spectrum AND the frequency bin centre values. These are automatically computed when using the internal conversion features (such as xtract_spectrum or xtract_peak_spectrum). However if using other scripts to generate your magnitude spectrum, ensure you also append to the data the frequency calculations.

Scalar Features

These features return a single scalar value per function call representing the desired feature.

Spectral Centroid

Return the 'centre of gravity' of the energy spectrum by frequency.

Code

Procedural:

Number() = xtract_spectral_centroid(spectrum);

Object-Oriented:

Number() = SpectrumData().spectral_centroid();
Number() = PeakSpectrumData().spectral_centroid();
Number() = HarmonicSpectrumData().spectral_centroid();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA

Spectral Mean

Return the mean magnitude value

Code

Procedural:

Number() = xtract_spectral_mean(spectrum);

Object-Oriented:

Number() = SpectrumData().spectral_mean();
Number() = PeakSpectrumData().spectral_mean();
Number() = HarmonicSpectrumData().spectral_mean();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA

Spectral Variance

Calculate the spread of the spectrum indicated by the frequency spread about the spectrum 'centre of gravity' / centroid.

Code

Procedural:

Number() = xtract_spectral_variance(spectrum, spectral_centroid)

Object-Oriented:

Number() = SpectrumData().spectral_variace();
Number() = PeakSpectrumData().spectral_variance();
Number() = HarmonicSpectrumData().spectral_variance();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
spectral_centroid Number No spectral_centroid = xtract_spectral_centroid(spectrum)

Spectral Spread

Included for conformity with LibXtract conversions. Same function as spectral variance under alternative name. ONLY in procedural view.

Code

Procedural:

Number() = xtract_spectral_spread(spectrum, spectral_centroid);

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
spectral_centroid Number No spectral_centroid = xtract_spectral_centroid(spectrum)

Spectral Standard Deviation

Calculate the standrad deviation of the magnitude bins about the 'centre of gravity'.

Code

Procedural:

Number() = xtract_spectral_standard_deviation(spectrum, spectral_variance);

Object-Oriented:

Number() = SpectrumData().spectral_standard_deviation();
Number() = PeakSpectrumData().spectral_standard_deviation();
Number() = HarmonicSpectrumData().spectral_standard_deviation();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
spectral_variance Number No spectral_variance = xtract_spectral_variance(spectrum)

Spectral Skewness

Calculate the skewness of the distribution of the magnitude array as a frequency

Code

Procedural:

Number() = xtract_spectral_skewness(spectrum, spectral_mean, spectral_standard_deviation);

Object-Oriented:

Number() = SpectrumData().spectral_skewness();
Number() = PeakSpectrumData().spectral_skewness();
Number() = HarmonicSpectrumData().spectral_skewness();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
spectral_mean Number No spectral_mean = xtract_spectral_mean(spectrum);
spectral_standard_deviation Number No spectral_standard_deviation = xtract_spectral_standard_deviation(spectrum);

Spectral Kurtosis

Calculate the flatness of the distribution of the magnitude array by frequency

Code

Procedural:

Number() = xtract_spectral_kurtosis(spectrum, spectral_mnea, spectral_standard_deviation);

Object-Oriented:

Number() = SpectrumData().spectral_kurtosis();
Number() = PeakSpectrumData().spectral_kurtosis();
Number() = HarmonicSpectrumData().spectral_kurtosis();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
spectral_mean Number No spectral_mean = xtract_spectral_mean(spectrum);
spectral_standard_deviation Number No spectral_standard_deviation = xtract_spectral_standard_deviation(spectrum);

Irregularity K

Calculate the K irregularity of the magnitude bins

Code

Procedural:

Number() = xtract_irregularity_k(spectrum);

Object-Oriented:

Number() = SpectrumData().irregularity_k();
Number() = PeakSpectrumData().irregularity_k();
Number() = HarmonicSpectrumData().irregularity_k();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA

Irregularity J

Calculate the J irregularity of the magnitude bins

Code

Procedural:

Number() = xtract_irregularity_j(spectrum);

Object-Oriented:

Number() = SpectrumData().irregularity_j();
Number() = PeakSpectrumData().irregularity_j();
Number() = HarmonicSpectrumData().irregularity_j();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA

Tristimulus

Extract the tristimulus functions (1, 2 or 3)

Code

Procedural:

Number() = xtract_tristimulus_1(spectrum, f0);
Number() = xtract_tristimulus_2(spectrum, f0);
Number() = xtract_tristimulus_3(spectrum, f0);

Object-Oriented:

// For conciseness, only showing tristimulus_1, however change the ending number to get the other tristimulus values
Number() = SpectrumData().tristimulus_1();
Number() = PeakSpectrumData().tristimulus_1();
Number() = HarmonicSpectrumData().tristimulus_1();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
f0 Number Yes NA

Smoothness

Calculate the smoothness of the magnitude spectrum

Code

Procedural:

Number() = xtract_smoothness(spectrum)

Object-Oriented:

Number() = SpectrumData().smoothness();
Number() = PeakSpectrumData().smoothness();
Number() = HarmonicSpectrumData().smoothness();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA

Rolloff

Determine the pivot point where threshold amount of energy is located below. Returns the frequency of the bin at the pivot point.

Code

Procedural:

Number() = xtract_rolloff(spectrum, sampleRate, threshold);

Object-Oriented:

Number() = SpectrumData().rolloff(threshold);
Number() = PeakSpectrumData().rolloff(threshold);
Number() = HarmonicSpectrumData().rolloff(threshold);

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
sampleRate Number Yes NA
threshold Number (0 to 1) Yes NA

Flatness

Calculate the flatness of the spectrum by dividing the product of the magnitudes by the sum of the magnitudes.

Code

Procedural:

Number() = xtract_flatness(spectrum);

Object-Oriented:

Number() = SpectrumData().flatness();
Number() = PeakSpectrumData().flatness();
Number() = HarmonicSpectrumData().flatness();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA

Flatness dB

Represent the flatness as a dB value (with a minimum value of -60.0dB)

Code

Procedural:

Number() = xtract_flatness_db(spectrum, flatness);

Object-Oriented:

Number() = SpectrumData().flatness_db();
Number() = PeakSpectrumData().flatness_db();
Number() = HarmonicSpectrumData().flatness_db();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
flatness Number No flatness = xtract_flatness(spectrum)

Tonality

Determine the tonality of the spectrum (peakiness).

Code

Procedural:

Number() = xtract_tonality(spectrum, flatness_db);

Object-Oriented:

Number() = SpectrumData().tonality();
Number() = PeakSpectrumData().tonality();
Number() = HarmonicSpectrumData().tonality();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
flatness_db Number No flatness_db = xtract_flatness_db(spectrum);

Spectral Inharmonicity

Return the relationship between harmonic and inharmonic peaks

Code

Procedural:

Number() = xtract_spectral_inharmonicity(peakSpetrum, f0);

Object-Oriented:

Number() = PeakSpectrumData().spectral_inharmonicity();

Arguments

Argument Type Mandatory Auto-calculation
peakSpectrum Array, TypedArray Yes NA
f0 Number Yes NA

Odd Even Ratio

Return the ratio between the energy of the odd and even harmonics from the fundamental

Code

Procedural:

Number() = xtract_odd_even_ratio(harmonicSpectrum, f0);

Object-Oriented:

Number() = HarmonicSpectrumData().odd_even_ratio();

Arguments

Argument Type Mandatory Auto-calculation
harmonicSpectrum Array, TypedArray Yes NA
f0 Number Yes NA

Spectral Slope

Return the slope of the magnitude spectrum

Code

Procedural:

Number() = xtract_spectral_slope(spectrum);

Object-Oriented:

Number() = SpectrumData().spectral_slope();
Number() = PeakSpectrumData().spectral_slope();
Number() = HarmonicSpectrumData().spectral_slope();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA

HPS

Calculate the HPS

Code

Procedural:

Number()  = xtract_hps(spectrum);

Object-Oriented:

Number() = SpectrumData().hps();
Number() = PeakSpectrumData().hps();
Number() = HarmonicSpectrumData().hps();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA

Vector Features

These features return a vector (as a Float64Array) of the computed features.

MFCC

Calculate the MFC coefficients from the spectrum.

Code

Procedural:

// The mfcc object must be built first
var mfcc = xtract_init_mfcc(N, nyquist, style, freq_min, freq_max, freq_bands);

Float64Array() = xtract_mfcc(spectrum, mfcc);

Object-Oriented:

// The mfcc object is built internally automatically
Float64Array() = SpectrumData().mfcc();
Float64Array() = PeakSpectrumData().mfcc();
Float64Array() = HarmonicSpectrumData().mfcc();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
mfcc Object Yes NA

Peak Spectrum

Extract the peak spectrum from the spectrum using peak-picking and a given threshold

Code

Procedural:

Float64Array() = xtract_peak_spectrum(spectrum, q, threshold);

Object-Oriented:

PeakSpectrumData() = Spectrum.peak_spectrum(threshold);

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
q = sampleRate / N Number Yes NA
threshold Number between 0 & 100 Yes NA

Harmonic Spectrum

Extract the harmonic spectrum from the peak spectrum by removing all peaks not related to the fundamental frequency

Code

Procedural:

Float64Array() = xtract_harmonic_spectrum(peakSpectrum, f0, threshold);

Object-Oriented:

HarmonicSpectrumData() = PeakSpectrumData().harmonic_spectrum(threshold);

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
f0 Number Yes NA
threshold Number between 0 and 1 Yes NA

Bark Bands

These use the Bark Band coefficients to calculate futher spectral features. To get the coefficients, firstly the bark bands filter bank must be designed. In the SpectrumData(), PeakSpectrumData() and HarmonicSpectrumData() this is automatically calculated.

Procedural:

var barkBandsFilter = xtract_init_bark(N, samplerate);

Then the filter must be applied using:

var barkBands = xtract_bark_coefficients(spectrum, barkBandsFilter);

Again, in SpectrumData(), PeakSpectrumData() and HarmonicSpectrumData() this is automatically calculated.

Loudness

Calculate the loudness from the bark band coefficients

Code

Procedural:

Number() = xtract_loudness(barkBandsArray);

Object-Oriented:

Number() = SpectrumData().loudness();
Number() = PeakSpectrumData().loudness();
Number() = HarmonicSpectrumData().loudness();

Arguments

Argument Type Mandatory Auto-calculation
barkBandsArray Array, TypedArray Yes NA

Sharpness

Calculate the sharpness of the spectrum from the bark band coefficients

Code

Procedural:

Number() = xtract_sharpness(barkBandsArray);

Object-Oriented:

Number() = SpectrumData().sharpness();
Number() = PeakSpectrumData().sharpness();
Number() = HarmonicSpectrumData().sharpness();

Arguments

Argument Type Mandatory Auto-calculation
barkBandsArray Array, TypedArray Yes NA

Fundamental Frequency

Calculate the fundamental frequency of the spectrum using the autocorrelation function

Code

Procedural:

Number() = xtract_spectral_fundamental(spectrum, sample_rate);

Object-Oriented:

Number() = SpectrumData.spectral_fundamental();
Number() = PeakSpectrumData.spectral_fundamental();
Number() = HarmonicSpectrumData.spectral_fundamental();

Arguments

Argument Type Mandatory Auto-calculation
spectrum Array, TypedArray Yes NA
sample_rate Number Yes NA