Maaate

Maaate is an audio analysis toolkit that supports the extraction of structure and content of MPEG-encoded audio files as well as raw files. Maaate was implemented by Silvia Pfeiffer and other researchers of the Commonwealth Scientific and Industrial Research Organization (CSIRO), Mathematical and Information Sciences (CMIS) in Australia. It is offered as Free Software under the GPL license for GNU/Linux and Windows. Maaate is implemented in C++ but a C wrapper interface is also offered. The framework is functional but not updated regularly, with the author's focus turning in other directions.

There are several reasons for which the authors justify the use of MPEG encoded files. First, MPEG is a de facto standard not only for internet audio for also for audio encoding in video formats. Second, the encoding process usually removes non-perceivable sounds that are not interesting either for content description applications. And finally, MPEG encoded files already have frequency domain information that can be used to extract some descriptors more easily and efficiently as the library allows to access directly the fields of the MPEG encoded file without having to decode it.

Apart for the general infrastructure, Maaate also includes modules with analysis algorithms such as loudness approximation, segmentation, silence or background noise level.

Maaate's architecture allows to easily add new algorithms. It is designed in tiers in order to separate functionalities and offer a cleaner API.

Tier 1 is in charge of parsing MPEG streams and offers access to the encoded fields. The most important class in this tier is the MPEGfile class.

Tier 2 offers two generic data containers (SegmentData and SegmentTable) that can be used from the analysis modules. SegmentData: contains analysis data for one segment (continuous time range) of the audio file. A SegmentTable contains a collection of segment of an audio file. These segments are ordered by their start time. Tier 2 also offers a module interface to plugin analysis routines that are stored in dynamic libraries. By using the offered plugin interface modules can be developed and compiled separately from Maaate and the toolkit can be extended without ever having to recompile it. The boundary between Maaate and the analysis modules is explicit so the author can clearly keep his/her authorship.

A module is simply a collection of related functions that offer a particular functionality. Modules that analyze content usually collect information on several MPEG audio frames and calculate information from these. Tier 2 was constructed for such modules but other kinds are also possible. Modules can be broadly classified into feature extraction, feature analysis and content analysis. Feature extraction modules make use of tier 1 access functions and store the result in tier 2 containers. Feature analysis modules use the already extracted features for further (usually statistical) analysis such as clustering or segmentation. Finally content analysis modules calculate higher level information such as silence/music/speech discrimination using feature extraction and analysis modules and storing the result in convenience containers.

A module is in fact an instance of the Module class, which also offers convenience functions to access the information, handle parameters, check constraints, and call the module functions. The apply method contains the implementation of all the analysis functionality, taking a list of parameters as input and producing another list of parameters as output.

Any module must offer an appropriate constructor and a function to add itself to the list of available modules. The other important operations present in the Module class and all its subclasses are the following:

• init (required): sets up basic module information (name, author...) and the input/output parameter specification.
• default (required): sets default values for input parameters and returns the parameter list.
• suggest (optional, recommended): takes an input parameter list, suggests parameter values based on information from other parameters and changes constraints on input parameters.
• reset (optional): resets a module by reseting internal processing information or parameter values.
• apply (required): takes an input parameter list, performs the analysis function and returns the calculated output parameters.
• destroy (optional): cleans up allocated memory and deletes parameter specification.

A parameter is an instance of the ModuleParam class. The init function sets up the list of input/output parameter specification and the application calls the default function to set default values. From there on, the input parameters may be modified by the application. It can also call the suggest function at any time for filling-in necessary parameter values and constraints and performing sanity checks. Finally, the application may then call the apply function, which first checks for parameters to be within the constraints.

There is a short list of allowed simple or complex data types for parameters. Basic types may be: boolean, integer, real and string (all of them defined in Maaate with particular conditions). Complex types may be: a pointer to an opened MPEG file, a pointer to a segment data structure, or a pointer to a segment table. All of them have their particular MAAATE_ define.

There are three types of constraints: no constraints, a list of single values that allow the parameter to take as value any on the list, and a list of ranges according to which parameters may take any value in any range on the list. A constraint is an instance of the ModuleParameterConstraint class that can handle either single values or a single range. For a single value there is usually a list of constraints realized by instantiating the MaateConstraint class, a class that provides functions for such things as adding constraints or checking whether values satisfy them.

On the other hand, a parameter specification is an instance of the ModuleParamSpec class, which contains the specification of a single parameter. The specification consists of a name, a description, a data type, default values and constraints.

The Plugins class provides functionality to load and unload single modules and whole libraries, and administrates the list of available modules. For building a shared library a function loadModules must be supplied that instantiates the modules and returns their list to Maaate. For dynamically loading modules into an application an instance of the Plugins class must be declared and the particular library must be loaded using the class' AddLibrary function. Then the different loaded modules can be accessed with the GetModule(string) function.

Bewdy is a sample graphical application that demonstrates Maaate by using a few of its modules. It runs on GNOME and needs the OSS sound system.