M4CAVE - Sound Visualization in a VR Environment

M4CAVE is a program for the visualization of sounds in a virtual-reality (VR) environment. It takes a score file from the sound synthesis program DIASS and renders the sounds represented by the score in visual images in a CAVE or Immersadesk. These visual images serve to enhance the aural perception of the sounds and their properties. Thus, M4CAVE is a useful tool for the sonification of scientific data. The code is written in C++.

Table of Contents

1. General Information

1.1 Project Background
1.2 Mappings in the Graphical Representations
1.3 Features
1.4 M4CAVE Images

2. Running M4CAVE

2.1 M4CAVE Specifications
2.2 Necessary Files and Directories
2.3 Running M4CAVE in the CAVE
2.4 Running M4CAVE on the ImmersaDesk
2.5 Running M4CAVE on the SGI Desktop
2.6 Running M4CAVE Elsewhere
2.7 Adding New Sound Files
2.8 Improving M4CAVE Performance

3. Modifying M4CAVE

3.1 Bugs and Other Problems

4. Documentation

4.1 External Source Documentation
4.2 Notes on the GUI Menu
4.3 Adding a New Graphics Module
4.4 Links to Outside Documentation

5. Project Contacts


1. General Information

1.1 Project Background

M4CAVE is part of the
"Environment for Music Composition" developed at the University of Illinois at Urbana-Champaign (UIUC) and Argonne National Laboratory (ANL) in the framework of the Scientific Sonification project. The objective is to enable the simultaneous representation of complex data sets as visual and aural images in a virtual-reality (VR) environment. The figure below diagrams the "Environment for Music Composition," showing data entry points for composition (C) and sonification (S).

M4CAVE handles the visualization aspects of the Environment. Using the CAVE VR Environment, the sound parameters are mapped to visual parameters to create images completely based upon the corresponding sounds.

1.2 Mappings in the Graphical Representations

Currently, three graphical representations of sound exist as part of M4CAVE: spheres, cloud, and planes. The spheres representation is the most developed and incorporates more parameters of sound into the visualization than the other two representations. The strength of the cloud representation is in showing tremolo and vibrato in the sound, while the planes representation is unique in that it limits the visualization to only the fundamentals of each sound.

In the spheres representation, sounds are visualized as stacks of spheres. The fundamental of a sound is usually the largest sphere in a stack and is generally at the bottom. Smaller spheres representing the sound's partials or overtones are stacked on top of the fundamental sphere.

A person standing in the center of the CAVE facing the back wall is considered to be at the origin of the CAVE in both the spheres representation and the CAVE in general. The x axis runs, therefore, from the left to right wall of the CAVE. The y axis is considered to be from the floor to the ceiling, and the z axis is considered to run from the back wall to the front of the CAVE. A more detailed description of the CAVE coordinate system is available in the
CAVE User's Guide.

The position of each sphere along the y axis is determined by the frequency of the sound that the sphere represents. Visualization of each sphere's position is enhanced by the presence of an optional grid in the representation. As with all of the representations, visual parameters are mapped proportionally to the sound parameters and vary accordingly. The chart below further explains the mappings of the parameters in the spheres representation.


The Spheres Representation

Graphical Representation
Sound Parameter
x coordinate Channel
y coordinate Frequency
z coordinate Reverb. mix
Radius Amplitude
Vertical stretching or shrinking Tremolo magnitude
Rotation Tremolo rate
Pulsing of spheres Vibrato magnitude
Base color Reverb. mix
Lightness of color Hall size

In the cloud representation, the density of a large particle cloud is determined by the number of sounds active at any one time. The cloud becomes more agitated as the tremolo and vibrato in the sounds increase, and the position of the cloud reflects the position of the sound in relation to the viewer. The table below shows mappings of the parameters.


The Cloud Representation

Graphical Representation
Sound Parameter
Number of particles affected by changes in any other sound parameter Amplitude
x coordinate Channel
y coordinate Hall size/Decay time
z coordinate Reverb. mix
Size of particles Amplitude
Distance of particles from the center of the cloud Hall size
Rotation of particles around the x axis Tremolo rate
Rotation of particles around the y axis Vibrato rate
Color Frequency

The planes representation does not consider the overtones of a sound in the visualization. Each large plane represents the fundamental of each currently active sound. Parameter mappings are explained below.


The Planes Representation

Graphical Representation
Sound Parameter
y coordinate Amplitude
z coordinate Reverb. mix
Angle of tilt of the plane Tremolo magnitude
Color Frequency
Grayscale accents on the plane Vibrato magnitude

1.3 Features

All features of M4CAVE can be accessed through the three wand buttons. The leftmost button acts as a switch to activate the sound file and start the simulation. The center button is a toggle that switches the grid in the spheres representation on or off. All other features of M4CAVE are accessed through the menu that is made available by pressing the right wand button. The user points with the wand, highlights the desired option, and presses the right wand button.

Submenus allow the user to change sound files or switch between graphical representations. The user is also able to toggle certain parameters on and off. For example, if a user were to turn off the frequency option, the resulting visualization would not be affected by the frequencies of the sounds. Visual parameters that depend upon frequency would be set to default values until frequency was toggled back on.

1.4 M4CAVE Images

The following are still images of M4CAVE visualizations of sounds. Click on any image to see a larger version.



2. Running M4CAVE

2.1 M4CAVE Specifications

Currently, M4CAVE is a C++ application that utilizes the 2.6 version of the CAVE libraries. Graphics use OpenGL, while the audio portion of the program uses the 2.3 version of the NCSA sound server VSS. M4CAVE also links libraries for a
GUI menu created by Daniel Heath.

2.2 Necessary Files and Directories

In order to run M4CAVE, the following must be present in the working directory:
In the ANL CAVE, the startcave.appinfo file is needed. Also, a link to the startcave script is useful, but not necessary.

The file run_m4cave_idesk is needed for running on the ANL ImmersaDesk.

2.3 Running M4CAVE in the CAVE

In the ANL CAVE, M4CAVE is run by using the startcave script. The user must be logged into all SGI terminals controlling the CAVE. If a link to the startcave script exists in the working directory, M4CAVE is executed from the working directory with the following command:
startcave
If the link to startcave is absent, M4CAVE can be executed with the full path as follows:
/usr/local/CAVE/bin/startcave
The sound server will be started automatically by the startcave script. Note that as paths and programs change, the startcave.appinfo script will need to be altered to account for these changes.

Also note that if a .caverc file in one's home directory, the lines referring to the simulator will need to be commented out. The .caverc file is discussed more extensively in
Section 2.5, Running M4CAVE on the SGI Desktop.

2.4 Running M4CAVE on the ImmersaDesk

To run M4CAVE on the ANL ImmersaDesk, the user must be logged into the SGI that controls it. The ImmersaDesk projector is activated by using the projector remote control. The application is then started with the command
run_m4cave_idesk
This script will start the both the sound server and the executable. As these variables change, the script will need to be altered accordingly.

2.5 Running M4CAVE on the SGI Desktop

A simulated version of M4CAVE can be run on the desktop of an SGI running IRIX 6.2 or above. A .caverc file must be present in the user's home directory containing the following lines:
simulator y
DisplayMode mono
HideCursor n
If this file is present in the user's home directory, it will be necessary to comment (#) out the above three lines to run M4CAVE in the CAVE or on the ImmersaDesk. The comments should be removed for simulated runs.

To manually start the sound server, one issues the following commands on the SGI, replacing the x's with the IP address of the sound server:
SOUNDSERVER=xxx.xxx.xx.xx
export SOUNDSERVER
On the machine that is the sound server, one moves to the directory where VSS is stored. Currently at ANL it is /usr/local/CAVE/snd/vss23/vssBin. From that directory, one issues the following command:
vssRelease -chans 2 -srate 22050
A GUI audio panel should appear on the screen. To eliminate the audio panel, one adds the commandline flag -nopanel.

Finally, one executes the program with an m4c file as a command line argument:
m4cave foo.m4c
Complete instructions for using the CAVE simulator are available in the
CAVE User's Guide. Running M4CAVE in simulator mode depends upon the individual environment. The user may wish to write a script to handle desktop execution.

2.6 Running M4CAVE Elsewhere

Environments vary from CAVE to CAVE. As noted, the above instructions are specific to the ANL CAVE. M4CAVE has, however, been successfully run in the NCSA CAVE. To run in another CAVE environment, the files listed under 2.2 Necessary Files and Directories must be present. The machine that is the sound server must be identified, VSS 2.3 must be running, and the program must be executed with one m4c file as a command line argument as follows:
m4cave foo.m4c

2.7 Adding New Sound Files

Three files are be necessary for any new visualization: an m4c file, an aiff file, and a score file. The m4c file is an ASCII file listing the names of the score file and the aiff file. It should look like:
foo.sc
foo.aiff
The aiff file should be placed in the audio directory off the working directory. Similarly, the score and m4c files should be placed in the score and m4c directories, respectively.

The files that are listed in the m4c directory will be listed by the menu when the option to select a new sound file is chosen. Because of limitations of the menu interface, no more than eight files may be displayed. The program will sort the files in the directory alphabetically, listing only the first eight valid m4c files. Extra m4c files may be stored in the store directory located in the m4c directory.

2.8 Improving M4CAVE Performance

As a result of environmental factors, namely, other users on CAVE machines and network traffic, slight interruptions in the sound file playback may occur if M4CAVE is run with the above instructions. To improve M4CAVE performance, one may wish to move all aiff files to the /usr/tmp directory of the machine acting as the sound server. If this change is made, it will be necessary to change the audio line of the m4c files by adding the path. Thus, an m4c file would look like:
foo.sc
/usr/tmp/foo.aiff

3. Modifying M4CAVE

3.1 Bugs and Other Problems

Currently, there are no known bugs in M4CAVE.

A limitation does exist, however, in the menu interface. As noted earlier, no more than eight items can appear on any one menu. At this time, the only solution is to create submenus that offer choices above and beyond the first eight.

4. Documentation

4.1 External Source Documentation

The following is a link to external documentation for the M4CAVE source code.

4.2 Notes on the GUI Menu

The GUI menu used by M4CAVE was originally written by
Daniel Heath for use in the ANL CAVE application BoilerMaker. Currently, there is no available documentation for the menu. The MenuSetup and menuCallBack functions in the M4CAVE source code hold all code directly related to the M4CAVE implementation of the menu.

4.3 Adding a New Graphics Module

Any new graphics module that is added to M4CAVE must be a class that inherits from the class GModule. It must contain the following virtual functions: preinit_graphics, init_graphics, draw_graphics, update_graphics, and kill_graphics.

The init_graphics and draw_graphics routines contain the main initialization and drawing routines that are called by the CAVE init and draw functions. The update_graphics function is called once per main loop, and any calculations based on the active waves list should be performed here. The preinit_graphics and kill_graphics routines should contain any preliminary or final setup or cleanup commands.

Additionally, if it has not already been done, the init_graphics function must call an initialization routine for the menu. Existing classes contain a pointer to a flag that indicates whether the routine has been called. Examples are in the class definitions and initialization routines.

4.4 Links to Outside Documentation

The following are links to documentation for other libraries and programs used to create M4CAVE and its documentation.

5. Project Contacts

Hans G. Kaper
Mathematics and Computer Science Division, Argonne National Laboratory
kaper@mcs.anl.gov

Sever Tipei
School of Music, University of Illinois, Urbana/Champaign
Mathematics and Computer Science Division, Argonne National Laboratory
s-tipei@uiuc.edu

Elizabeth Wiebel
Student, St. Norbert College
Mathematics and Computer Science Division, Argonne National Laboratory
wiebep@sncac.snc.edu

[ Scientific Sonification | MCS Division | UIUC Computer Music Project | Hans G. Kaper | Sever Tipei ]

Last updated: October 22, 1998 (HGK)

Original program and documentation prepared by Elizabeth Wiebel, wiebep@sncac.snc.edu

Elizabeth Wiebel's participation in this project was made possible through the Student Research Participation Program sponsored by Argonne National Laboratory's Department of Educational Programs.