Max

Max [Puckette, 1991a,Puckette, 2002,Zicarelli, 2002], together with PD and jMax that are particular instances of the same model, is, apart from CSound (see section 2.6.1), probably the most widely used computer music environment and with one of the longest development life. Max is, according to its author Miller Puckette, ``a graphical programming environment for developing real-time musical applications'' [Puckette, 1991a]. Max was first written for Macintosh, then ported to NeXT workstations and it is now available in any of its forms for most operating systems and platforms. At the time Max was proposed, graphical applications existed but they did not address the real-time issue. On the other hand, other systems that had advanced real-time strategies did not have graphical interface. Max was born as a compromise in between both approaches.

Although Max has been classified as a graphical language/application, its definition is a matter of controversy. According to the author Max was not intended to be a programming language [Puckette, 1996]. On the other hand, in [Déchelle, 2000] we find that Max can be seen as ``an imperative graphical programming language'' or a ``graphical interface generator''. Our opinion is that Max is not more than a graphical application although its graphical model has had so much success that it has almost given place to the Max paradigm. As a matter of fact Max can now be seen as a conceptual model that has had and still has many different implementations, before explaining any of them it is interesting to understand its development history.

Max's predecessor was called the Patcher and was presented by Miller Puckette in 1988 [Puckette, 1988]. The Patcher was a Macintosh software for treating and controlling MIDI. The Patcher was acquired by the Opcode American company where David Zicarelli added many new features and enhancements and converted into the popular Max/Opcode.

In 1989 Ircam started the Ircam Sound Processing Workstation (ISPW) project in order to develop a complete workstation for audio and music working on a NeXTSTEP environment. This project enabled to develop a new version of the Patcher known as Max/ISPW. The main novelty of this version was its client server architecture: on one hand the graphical interface and on the other a real-time sound processing environment called FTS.

Once the ISPW started to evolve, it became clear that Max/ISPW had to become more portable to different platforms. For that reason a new version was written called Max/FTS. The version was distributed for Silicon Graphics computers.

Miller Puckette started about the same time with the development of the free PD environment [Puckette, 1996,Puckette, 1997a,Puckette, 1997b] in order to give answer to some Max limitations, namely its lack of support for dynamic data structures. PD used many of the results of the Animal project and introduced a portable graphical interface based on Tcl/Tk. In 1997, he enhanced PD's audio processing module and named it MSP (Max Signal Processing). MSP was added to Max/Opcode, introducing therefore real-time audio processing.

The graphical interface in Max/FTS was re-implemented in Java and that gave rise to jMax, a new implementation of the Max language. At the moment of this writing the Java interface in jMax is being re-written in Python as it depended on some proprietary libraries.

Although from the beginning Max was intended to be a unified environment both for signal and control flow, it was historically developed as a MIDI program. Later on, Max was extended to include signal processing capabilities [Puckette, 1991a], these capabilities are now in Max through the MSP extension [Dobrian et al., 2000].

It is beyond our scope to give a complete view of Max and related environments. For this reason we will mostly concentrate in its signal processing capabilities and on PD, its free version.

The most important issue in MAX is its graphical model. The fundamental item is the patch, a collection of boxes connected by lines. A patch can either be in ``run'' or ``edit'' mode. The boxes represent objects that wait for messages to be passed to them. They may respond by taking an action or by passing messages to other boxes. Boxes may have inlets and outlets, which are represented as dark rectangles. Lines connect inlets to outlets and a message passed to an outlet is transferred to all inlets connected to it. The messages are an ordered list of atoms, each of which may be a number or a symbol.

Apart from sending/receiving messages through the outlets/inlets, objects can access the clock and the MIDI I/O. The clock is accessed with a callback mechanism. In order to receive input MIDI messages the object must be inserted in the MIDI callback list, output MIDI messages are sent by calling a library function.

By convention, if an object has more than one inlet, its leftmost one is the ``active'' one. Passing a message to it causes something to happen, passing it to the other inlets just changes the state of the object. In a similar way, output messages are always written to outlets from right to left.

Before MSP was introduced signal processing in MAX was carried out by a collection of tilde classes implemented in the FTS engine. These objects communicated through inlets and outlets using the signal message. But the calculations on signals required more communication bandwidth than the message passing mechanism can offer [Puckette, 1991b]. For that reason they used a special scheduling mechanism based on a duty cycle that is carried out regularly to compute a new set of output signals.

Tilde objects executed processing tasks on fixed-size vectors typically between 16 and 64 samples. The DSP computation period was set to the sampling rate divided by the buffer size. Tilde objects intercommunicated at setup time in order to determine a calling order and the addresses of input/output signals to be used. This communication was done through the signal message using regular inlets/outlets. The signal message took two integers: a selector (COUNTINPUTS, COUNTOUTPUTS or DOIT) and the address of a signal object. DSP objects cannot initiate messages when in the duty cycle. This may introduce a small but non-zero delay between message and actual processing. Tilde classes communicated with control objects through their member attributes.

For building the duty cycle call list the signal message was used. The signal message acted like a token used by tilde objects to simulate data-driven dataflow networks. Each tilde object could ``run'' only when it had data in all its inputs and then produced into all its outputs. Each time a tilde object was run, it appended itself to the call list.

When a dac object received the start message, it traversed the list of all tilde objects causing them to send the COUNTINPUTS message to all its outlets. Each inlet also counted the number of outputs connected to it. The list was traversed a second time and any tilde object that had no inlets or whose inlets count to zero was added to the call list. A change in the network is reflected as a change in the DSP duty cycle call list.

Each time a tilde object was added to the list, new signals were allocated for all its outlets and the outlets were first passed a COUNTOUTPUTS message and then a DOIT message. The first message was simply passed to count the number of inlets connected to each inlet. The DOIT message passed the address of the newly allocated signal and informed the patch that the signal was ready to be used. Therefore when a tilde object received a DOIT message it could determine whether all its inputs were available. After decrementing the inlet's count, if all inlets count were set to zero the object could be added to the call list, knowing already the addresses of all its inputs and outputs.

The signal allocated for a signal output could be freed as soon as the last tilde object having it as input was put on the call list. This way, a chain of tilde objects each with one input and one output would typically use the same signal inplace (important for processors with limited memory).

If at the end of the call list building process, a tilde object was not on the list, it meant that a signal loop had been detected. This was an unwanted situation as if a signal loop was actually needed a delay read/write pair had to be used (set to the minimum of one duty cycle).

Tilde objects carried out their DSP actions synchronously so execution order of inputs and outputs did not matter. But control parts of a patch could be activated in different ways yielding different results.

While the previous explanation is basically about Miller Puckette's design, David Zicarelli started working in parallel with another implementation of Max signal processing capabilities. This new design finally gave place to MSP (Max Signal Processing). MSP is now a basic extension of Max, up to the point that the current environment is known as Max/MSP.

MSP [Dobrian et al., 2000] includes over 170 Max objects for digital signal processing. In the graphical environment MSP objects are very similar to Max objects with the only difference that their name ends with a '~'. MSP objects are connected the same way than Max objects but intercommunication is conceptually different. Instead of establishing a path for messages, MSP connections establish a relationship between objects and that relationship is used to calculate the audio information necessary at a particular instant. The configuration of MSP objects is known as the signal network.

MSP are in constant communication. Max objects sit idle waiting for a message to occur, but MSP objects are always active, constantly computing the current output samples. For that reason, an MSP signal network can be understood as a portion of a patch that runs at a faster (audio) rate than Max. Max (and thus the user) can only affect the signal portion of the patch every millisecond. What happens in between those milliseconds is calculated and performed by MSP.

Some MSP objects provide a link between Max and MSP and to translate between control rate and audio rate. MSP inlets can accept both signal and Max messages (e.g. they can be turned on and off with the Max messages start and stop). A Max patch can contain both Max and MSP objects. Nevertheless, for organization, MSP objects in a signal network are often encapsulated in a subpatch.

Pure Data (or Pd for short) [Puckette, 1996,Puckette, 1997a,Puckette, 1997b,Puckette, 2004,www-PD, ] is a real-time graphical programming environment for audio and graphical processing. It is very similar to the Max/MSP system but is simpler and more portable. Pd also has two basic features that are not available in Max/MSP: first, via the GEM package, Pd can be used for simultaneous computer graphics animation and computer audio; second, an experimental facility is provided for defining and accessing data structures.

Pd is an effort to solve some problems in Max while keeping its strength. Pd was designed by Max's author Miller Puckette and although he mentions that it was not his intention to make a Max clone he recognizes that whenever there was no real reason to make something different, the solution available in Max was used [Puckette, 2004]. In parallel the GEM [Danks, 1997] project started to develop a real-time graphical synthesis/processing/rendering environment which would run with Pd.

The main weakness in Max are reported to be its difficulty of maintaining compound data structures and of integrating non-audio signals like video or audio spectra [Puckette, 1996]. In order to use Max to process data, the project Animal started [Lindermann, 1991]. Many ideas in PD come from this program. The first thing introduced in PD was the ability to plot graphics and figures.

The main goals when starting the Pd and GEM projects are summarized in [Puckette, 1997b]: (1) a real-time patchable environment ala Max: (2) management of audio and image processing in the same environment; (3) adaptability to a wide range of platforms; (4) long-term stability; (5) newer and more flexible set of tools to manipulate data.

In Pd, any data structure (called ``Pure Datum'') can become a message handled the usual way. Also, users may create their own ``DSP blocks'' in which sample rate and vector size vary.

The clock allows users to attach a symbolic name to a specific combination of sample rate and vector size. Any clock can be turned on/off. The reclock object simply converts any signal to the desired clock (for instance, preparing an input signal for overlapped FFT; the overlap-add process in the synthesis is accomplished by simply reclocking to the original clock).

Pd is designed in two parts: the ``real'' Pd and the Pd-gui. The real Pd, does real-time computations using a Max-like interpreter and scheduler. The Pd-GUI talks to the computer window system through the tk toolkit.

Pd shares with MSP a few objects for audio analysis [Puckette et al., 1998]. The fiddle and bonk objects, for instance, are two basic implementations, the former for pitch detection and the latter for bounded-Q analysis. Their main goal is to get predictable and acceptable behavior with easy-to-understand techniques that will not place a heavy load on the machine. The output of both objects appears as Max-style control messages.

Fiddle is a maximum-likelihood pitch detector similar that can also be used to obtain a raw list of sinusoidal components. Bonk performs a bounded-Q analysis of the input signal to obtain onsets of percussion instruments.

Finally jMax [Déchelle et al., 1998,Déchelle et al., 1999b,Déchelle et al., 1999a,Déchelle, 2000,Déchelle et al., 2000,Déchelle and Tisserand, 2003] is a new implementation of Max/FTS in which the graphical interface is re-implemented in Java. jMax reuses Max/FTS and PD's two component architecture (client/server). This architecture allows decoupling the two components and executing the processing component independently from the user interface (for example inside a VST plug-in). Communication between the two components is done through TCP/IP or UDP.

jMax's interface offers basically the same functionalities than the other Max versions: a patch editor for constructing and controlling program patches and a set of specialized editors for objects with complex data structures (such as tables or sequences).

jMax introduces a textual scripting language for controlling structures that were difficult to represent graphically such as operator banks and other repetitive operations. jMax used Tcl scripting. But because of integration problems of Tcl and Java it was decided to change to Scheme (a Lisp dialect). Currently, and because of some problems with proprietary Java libraries, the interface is being ported to Python.

In any case jMax is available for Silicon Graphics over the IRIX system, PC or Macintosh over Linux (using either OSS or ALSA sound devices).