M4CAVE - Sound Visualization in a VR Environment
M4CAVE is a program for the visualization
of sounds in a virtual-reality (VR) environment.
It takes a score file from the sound synthesis program
DIASS
and renders the sounds represented by the score
in visual images in a CAVE or Immersadesk.
These visual images serve to enhance the aural perception
of the sounds and their properties.
Thus, M4CAVE is a useful tool for the sonification
of scientific data.
The code is written in C++.
Table of Contents
-
- 1.1 Project Background
- 1.2 Mappings in the Graphical
Representations
- 1.3 Features
- 1.4 M4CAVE Images
-
- 2.1 M4CAVE Specifications
- 2.2 Necessary Files and Directories
- 2.3 Running M4CAVE in the CAVE
- 2.4 Running M4CAVE on the ImmersaDesk
- 2.5 Running M4CAVE on the SGI Desktop
- 2.6 Running M4CAVE Elsewhere
- 2.7 Adding New Sound Files
- 2.8 Improving M4CAVE Performance
-
- 3.1 Bugs and Other Problems
-
- 4.1 External Source Documentation
- 4.2 Notes on the GUI Menu
- 4.3 Adding a New Graphics Module
- 4.4 Links to Outside Documentation
-
1. General Information
1.1 Project Background
M4CAVE is part of the
"Environment for Music Composition"
developed at the
University of Illinois at Urbana-Champaign (UIUC)
and Argonne National Laboratory (ANL)
in the framework of the
Scientific Sonification project.
The objective is to enable the simultaneous representation
of complex data sets as visual and aural images in a
virtual-reality (VR) environment.
The figure below diagrams the
"Environment for Music Composition,"
showing data entry points for composition (C)
and sonification (S).
M4CAVE handles the visualization aspects of the Environment.
Using the CAVE VR Environment, the sound parameters are mapped to visual
parameters to create images completely based upon the corresponding sounds.
1.2 Mappings in the Graphical Representations
Currently, three graphical representations of sound exist as part of M4CAVE: spheres,
cloud, and planes. The spheres representation is the most developed and incorporates
more parameters of sound into the visualization than the other two representations.
The strength of the cloud representation is in showing tremolo and vibrato in the sound,
while the planes representation is unique in that it limits the visualization to only
the fundamentals of each sound.
In the spheres representation, sounds are visualized as stacks of spheres. The fundamental
of a sound is usually the largest sphere in a stack and is generally at the bottom.
Smaller spheres representing the sound's partials or overtones are stacked on top of
the fundamental sphere.
A person standing in the center of the CAVE facing the back wall is considered to be at the
origin of the CAVE in both the spheres representation and the CAVE in general. The x axis
runs, therefore, from the left to right wall of the CAVE. The y axis is considered to
be from the floor to the ceiling, and the z axis is considered to run from the
back wall to the front of the CAVE. A more detailed description of the CAVE coordinate
system is available in the
CAVE User's Guide.
The position of each sphere along the y axis is determined by the frequency of the
sound that the sphere represents. Visualization of each sphere's position is enhanced
by the presence of an optional grid in the representation. As with
all of the representations, visual parameters are mapped proportionally to the sound
parameters and vary accordingly. The chart below further explains the mappings of the
parameters in the spheres representation.
The Spheres Representation |
Graphical Representation |
Sound Parameter |
x coordinate |
Channel |
y coordinate |
Frequency |
z coordinate |
Reverb. mix |
Radius |
Amplitude |
Vertical stretching or shrinking |
Tremolo magnitude |
Rotation |
Tremolo rate |
Pulsing of spheres |
Vibrato magnitude |
Base color |
Reverb. mix |
Lightness of color |
Hall size |
In the cloud representation, the density of a large particle cloud is determined by
the number of sounds active at any one time. The cloud becomes more agitated as
the tremolo and vibrato in the sounds increase, and the position of the cloud
reflects the position of the sound in relation to the viewer. The table below
shows mappings of the parameters.
The Cloud Representation |
Graphical Representation |
Sound Parameter |
Number of particles affected by changes in any other sound
parameter |
Amplitude |
x coordinate |
Channel |
y coordinate |
Hall size/Decay time |
z coordinate |
Reverb. mix |
Size of particles |
Amplitude |
Distance of particles from the center of the cloud |
Hall size |
Rotation of particles around the x axis |
Tremolo rate |
Rotation of particles around the y axis |
Vibrato rate |
Color |
Frequency |
The planes representation does not consider the overtones of a sound in the visualization.
Each large plane represents the fundamental of each currently active sound. Parameter
mappings are explained below.
The Planes Representation |
Graphical Representation |
Sound Parameter |
y coordinate |
Amplitude |
z coordinate |
Reverb. mix |
Angle of tilt of the plane |
Tremolo magnitude |
Color |
Frequency |
Grayscale accents on the plane |
Vibrato magnitude |
1.3 Features
All features of M4CAVE can be accessed through the three wand buttons. The
leftmost button acts as a switch to activate the sound file and start the
simulation. The center button is a toggle that switches the grid in the
spheres representation on or off. All other features of M4CAVE are accessed
through the menu that is made available by pressing the right wand button.
The user points with the wand, highlights the desired option, and presses the
right wand button.
Submenus allow the user to change sound files or switch between graphical representations.
The user is also able to toggle certain parameters on and off. For example, if a
user were to turn off the frequency option, the resulting visualization would not be
affected by the frequencies of the sounds. Visual parameters that depend upon frequency
would be set to default values until frequency was toggled back on.
1.4 M4CAVE Images
The following are still images of M4CAVE visualizations of sounds. Click on any
image to see a larger version.
2. Running M4CAVE
2.1 M4CAVE Specifications
Currently, M4CAVE is a C++ application that utilizes the 2.6 version of the CAVE
libraries. Graphics use OpenGL, while the audio portion of the program uses the 2.3
version of the NCSA sound server VSS. M4CAVE also links libraries for a
GUI menu created by Daniel Heath.
2.2 Necessary Files and Directories
In order to run M4CAVE, the following must be present in the working directory:
- The m4cave executable.
- The textures directory, containing
- The wall texture file tilegrr.bw.
- The m4c directory, containing
- The audio directory, containing
- The file swish.aiff
- One <name>.aiff file for each sound file to be visualized.
These aiff files should have the same name as the aiff files listed
in each of the m4c files.
- The score directory, containing
- One <name>.sc file for each sound file to be
visualized. These score files should have the same name as the score
files listed in each of the m4c files.
In the ANL CAVE, the startcave.appinfo file is needed. Also, a link to the
startcave script is useful, but not necessary.
The file run_m4cave_idesk is needed for running on the ANL ImmersaDesk.
2.3 Running M4CAVE in the CAVE
In the ANL CAVE, M4CAVE is run by using the startcave script. The user must
be logged into all SGI terminals controlling the CAVE. If a link to the startcave
script exists in the working directory, M4CAVE is executed from the working directory
with the following command:
startcave
If the link to startcave is absent, M4CAVE can be executed with the full path as
follows:
/usr/local/CAVE/bin/startcave
The sound server will be started automatically by the startcave script. Note that as
paths and programs change, the startcave.appinfo script will need to be
altered to account for these changes.
Also note that if a .caverc file in one's home directory,
the lines referring to the simulator will need to be commented out. The .caverc
file is discussed more extensively in
Section 2.5, Running M4CAVE on the SGI Desktop.
2.4 Running M4CAVE on the ImmersaDesk
To run M4CAVE on the ANL ImmersaDesk, the user must be logged into the SGI that controls
it. The ImmersaDesk projector is activated by using the projector remote control. The
application is then started with the command
run_m4cave_idesk
This script will start the both the sound server and the executable. As these
variables change, the script will need to be altered accordingly.
2.5 Running M4CAVE on the SGI Desktop
A simulated version of M4CAVE can be run on the desktop of an SGI running IRIX 6.2 or
above. A .caverc file must be present in the user's home directory containing
the following lines:
simulator y
DisplayMode mono
HideCursor n
If this file is present in the user's home directory, it will be necessary to comment (#)
out the above three lines to run M4CAVE in the CAVE or on the ImmersaDesk. The comments
should be removed for simulated runs.
To manually start the sound server, one issues the following commands on the SGI,
replacing the x's with the IP address of the sound server:
SOUNDSERVER=xxx.xxx.xx.xx
export SOUNDSERVER
On the machine that is the sound server, one moves to the directory where VSS is stored.
Currently at ANL it is /usr/local/CAVE/snd/vss23/vssBin. From that
directory, one issues the following command:
vssRelease -chans 2 -srate 22050
A GUI audio panel should appear on the screen. To eliminate the audio panel,
one adds the commandline flag -nopanel.
Finally, one executes the program with an m4c file as a command line argument:
m4cave foo.m4c
Complete instructions for using the CAVE simulator are available in the
CAVE User's Guide.
Running M4CAVE in simulator mode depends upon the individual environment.
The user may wish to write a script to handle desktop execution.
2.6 Running M4CAVE Elsewhere
Environments vary from CAVE to CAVE. As noted, the above instructions are specific to
the ANL CAVE. M4CAVE has, however, been successfully run in the NCSA CAVE. To run in
another CAVE environment, the files listed under 2.2 Necessary
Files and Directories must be present. The machine that is the sound server must
be identified, VSS 2.3 must be running, and the program must be executed with one m4c
file as a command line argument as follows:
m4cave foo.m4c
2.7 Adding New Sound Files
Three files are be necessary for any new visualization: an m4c file, an aiff file, and
a score file. The m4c file is an ASCII file listing the names of the score file and the
aiff file. It should look like:
foo.sc
foo.aiff
The aiff file should be placed in the audio directory off the working directory.
Similarly, the score and m4c files should be placed in the score and m4c
directories, respectively.
The files that are listed in the m4c directory will be listed by the menu when
the option to select a new sound file is chosen. Because of limitations of the
menu interface, no more than eight files may be displayed. The program will
sort the files in the directory alphabetically, listing only the first eight
valid m4c files. Extra m4c files may be stored in the store directory located
in the m4c directory.
2.8 Improving M4CAVE Performance
As a result of environmental factors, namely, other users on CAVE machines and network traffic,
slight interruptions in the sound file playback may occur if M4CAVE is run with the
above instructions. To improve M4CAVE performance, one may wish to move
all aiff files to the /usr/tmp directory of the machine acting as the
sound server. If this change is made, it will be necessary to change the audio line
of the m4c files by adding the path. Thus, an m4c file would look like:
foo.sc
/usr/tmp/foo.aiff
3. Modifying M4CAVE
3.1 Bugs and Other Problems
Currently, there are no known bugs in M4CAVE.
A limitation does exist, however, in the menu interface. As noted earlier,
no more than eight items can appear on any one menu. At this time, the only
solution is to create submenus that offer choices above and beyond the first
eight.
4. Documentation
4.1 External Source Documentation
The following is a link to external documentation for the M4CAVE source code.
4.2 Notes on the GUI Menu
The GUI menu used by M4CAVE was originally written by
Daniel Heath for use in the ANL
CAVE application BoilerMaker. Currently, there is no available
documentation for the menu. The MenuSetup and menuCallBack functions in the
M4CAVE source code hold all code directly related to the M4CAVE implementation
of the menu.
4.3 Adding a New Graphics Module
Any new graphics module that is added to M4CAVE must be a class that inherits from
the class GModule. It must contain the following virtual functions: preinit_graphics,
init_graphics, draw_graphics, update_graphics, and kill_graphics.
The init_graphics and draw_graphics routines contain the main initialization and
drawing routines that are called by the CAVE init and draw functions. The update_graphics
function is called once per main loop, and any calculations based
on the active waves list should be performed here. The preinit_graphics and kill_graphics
routines should contain any preliminary or final setup or cleanup commands.
Additionally, if it has not already been done, the init_graphics
function must call an initialization routine for the menu. Existing classes contain a
pointer to a flag that indicates whether the routine has been called. Examples
are in the class definitions and initialization routines.
4.4 Links to Outside Documentation
The following are links to documentation for other libraries and programs used
to create M4CAVE and its documentation.
5. Project Contacts
Hans G. Kaper
Mathematics and Computer Science Division, Argonne National Laboratory
[email protected]
Sever
Tipei
School of Music, University of Illinois, Urbana/Champaign
Mathematics and Computer Science Division, Argonne National Laboratory
[email protected]
Elizabeth Wiebel
Student, St. Norbert College
Mathematics and Computer Science Division, Argonne National Laboratory
[email protected]
[
Scientific Sonification |
MCS Division |
UIUC
Computer Music Project |
Hans G. Kaper |
Sever
Tipei
]
Last updated: October 22, 1998 (HGK)
Original program and documentation prepared by Elizabeth Wiebel,
[email protected]
Elizabeth Wiebel's participation
in this project was made possible through
the Student Research Participation Program
sponsored by Argonne National Laboratory's
Department of Educational Programs.