Spectrum3d

1. dependencies

FOR UBUNTU from 10.10 : All dependencies are available in the Ubuntu repositories.

• gcc, pkg-config, automake;
• libgtk-2.0-dev or libgtk-3.0-dev;
• libgstreamer0.10-dev;
• lidsdl1.2-dev (alternatively GtkGlExt can be chosen instead (see downstairs);
• (optional) gstreamer0.10-plugins-bad, gstreamer0.10-plugins-bad-multiverse, gstreamer0.10-plugins-ugly et gstreamer0.10-plugins-ugly-multiverse (for reading files such as mp3);
• (optional) JACK support : libjack-dev (for jack1) or libjack-jackd2-dev for jack2; note that you don't need to have the 'jackd' package installed, but only the '-dev' library;
• (optional) multitouch support : libutouch-geis-dev version 2.

For DISPLAY (OpenGL extension): SDL is used by default; GtkGlExt can be used instead and its version should match the GTK version (GTKGLEXT1 for GTK2, GTKGLEXT3 for GTK3); however, GTKGLEXT3 is still in development; the development version should be installed then (see later); the display of harmonics itself will look exactly the same regardless of the library that is used.

NOTE : For Ubuntu 11.04, the repositories version will work, but filtering on number of touches won't work (Bug#769859 in libutouch-geis); to solve this you need a version of uTouch-geis that is at least 2.1.0; you can have it either by installing the '.deb' packages from Oneiric repositories (type in a terminal 'sudo apt-get build-dep libutouch-geis-dev', then install 'libutouch-geis1' first, then 'libutouch-geis-dev'), or by compiling the latest version from source (https://launchpad.net/utouch-geis); if you compile from sources, you can type in a terminal 'sudo apt-get build-dep libutouch-geis-dev' to have all dependencies installed. For Ubuntu 10.10, do not use the repositories version; use one of the previous solutions.

FOR OTHER DISTRIBUTIONS : For the other distributions, the dependencies should be very similar. Only the multitouch features maybe won't work since uTouch-Geis has been build for Ubuntu. However, the uTouch Suite (including uTouch-Geis) starts beeing ported in other distributions such as Gentoo (https://sabotageandi.wordpress.com/2011/04/20/utouch-gentoo/) or Fedora. More info on this in the documentation directory.

2. compilation and installation

1) in a terminal, go to the directory where the sources are; for example, if the source package is in the HOME directory :

2) type :

IMPORTANT NOTE for GTK VERSION :
Both GTK2 and GTK3 are supported; if GTK3 is installed, it will be used by default but you can override this by forcing the use of GTK2 by typing :

or

NOTE for DISPLAY (OpenGL extension):
SDL is used by default and a error message will be delivered if it is not found. However you can choose to use GTKGLEXT library instead of SDL (GTKGLEXT for GTK2, GTKGLEXT3 for GTK3). Since the GTKGLEXT3 version (appropriate for GTK3) is still in development, use of GTKGLEXT is not yet the default choice and this has to be specifically enabled at the time of compilation. However the GTKGLEXT3 version (for GTK3) is already supported in Spectrum3d (see GtkGlExt support). The display itself will look exactly the same regardless of the library that is used.
If you want GTKGLEXT to be used instead of SDL, type:

NOTE for JACK and MULTITOUCH SUPPORT:
If Jack library is found, JACK support will be enabled. Also, if libbamf and libutouch-geis version 2 libraries are found, MULTITOUCH support will be enabled. If those libraries are not installed, JACK and MULTITOUCH support won't be enabled. However, if you have those libraries installed but don't want those features enabled, you have to type '--disable-jack' or '--disable-geis' after 'configure' ('--enable-jack=no' and '--enable-geis=no' also works).
For example :

will deactivate JACK and MULTITOUCH support if those libraries are installed. You can combine many arguments. For example :

If you want Spectrum3d to be installed in another directory than the default one ('/usr/local/bin'), you can add it at 'configure' step according to the 'Autoconf' standards. For example :

will install the executable 'spectrum3d' file in '/usr/bin' directory instead of '/usr/local/bin'.

3) type :

4) type :

5) to completely remove Spectrum3d, type :

Type 'spectrum3d' in a terminal or run it from 'Menu->Applications->Sound & Video->Spectrum3d'.

The 2 most important things to do first are :
- to select the type of audio source : either microphone or audio file;
- set if analyse will be done in real time or not : if 'analyze in real time' is checked, harmonics will be retrieved and displayed on the fly, while the sound is being played; if 'Analyse in real time' is unchecked, harmonics of the whole audio file will be retrieved and displayed first, then the file can be played afterwards. When source is set to microphone and 'Analyse in real time' is unchecked, recording can be made; it will be analysed and displayed like for an audio file; similarly it can be also played afterward.

Selection of multiple audio files is not possible anymore. The 'reload' button allows to reload (i.e. analyse and display its harmonics) without reselecting the file.

You can record from the microphone and have the generated audio file analysed and displayed by selecting 'microphone' as the audio source, then unchecking the 'analyse in real-time' button; then the record button will be usable; a 'record window' will then pop up; after recording, a click on 'OK' will load the file as for an audio file.

If 'Use JACK' is checked, everything explained above can be done with Jack (Jack-Audio-Connection-Kit).

A new window that commands different filters can be displayed :
- a 10 bands equalizer; the filtered frequencies and their range can be adjusted; the amplification goes from -72dB to +36dB;
- a band-reject or band-pass filter;

The lowest displayed frequency is 0 Hz by default, and it can be increased. The default range is 2000 Hz. It can be decreased to 40 Hz or raised to 20000Hz. To change this, you can either change it by using the 'Range of Display' scale, or increase it by changing the factor that is on its right : this factor multiplies the frequency range. The smallest possible step between 2 analyzed and displayed frequencies is 2 Hz.

From the menu :

• you can choose between 3 types of view :
  - a 3D view that shows frequency, intensity and time; It is possible to rotate or translate the display along the X, Y and Z axis, either by keyboard commands, or by mouse (+/- keyboard) commands, or by touch gestures (if enabled). All the analyzed values are kept in memory, which implies that even if zoom or perspective changes, any value can be retrieved at any time, even when playing is paused.
  - a 2D 'flat' view : that shows time versus frequency; the intensity however is represented by the amount of red color;
  - a 3D 'flat' view that is somehow a compromise between the two previous views.
• the perspective can be reset, set to a 'front' view (frequency versus intensity), or set to values preset by the user. The text can be displayed or not, as well as the line scale. A 'pointer' can be displayed, showing a location on the scale and its exact numerical value as well as the intensity of the frequency (fot the 3D view)
• you can chose to display text and lines or not and also a pointer that will point to a specific frequency; it will display it frequency value, and for the 3D perspective its intensity value;
• you can play a test sound which is a sine wave from 1 to 20000 Hz;
• you can set, in 'preferences' :
  • the distance between frames : the biggest this value is, the 'deeper' the image will go;
  • the number of displayed frames, the biggest this value is, the 'deeper' the image will go and the more cpu will be used;
  • the interval of time (in milliseconds) between each refreshing of the display; the smallest this value will be, the nicer the display will look, but the more demanding it will be on the cpu; every change in this will require Spectrum3d to be restarted to be effective;
  • the color of the display (when analyse in real-time is selected);
  • whether you want the display in the same window as the GUI or in a separate window;
  • whether you want that the actual display is saved as preset;
  • the interval of time (in milliseconds) between each new analysis of the spectral data; as for display, the smallest this value will be, the more demanding it will be on the cpu; this value should match the interval of time between 2 displays for better display but this is not mandatory;
  • whether you want Spectrum3d to run in realtime mode (when JACK is not used); if so, you can choose between SCHED_RR or SCHED_FIFO policies, and set the priority; please see important note at the end of this file.

All those preferences are kept in a 'preferences' file that is created at first use of Spectrum3d and placed in the home directory : ~/.spectrum3d/spectrum3d.pref. Everytime Spectrum3d starts, it check some values of this 'preferences' file for consistency; if out of range value is found, a new 'preferences' file with default values is created.

Keyboard and/or mouse shortcuts also exist, as well as gestures commands (if you have a multi-touch screen). You can also find a summary of this in the menu, at the 'Help'->'Shortcuts' and 'Gestures Shortcuts' items :

1. Although it is 3D with OpenGL, the drivers for 3D acceleration are not needed (no need for proprietary drivers here!), but it can help to have a better rendering.
2. Spectrum 3D can use a lot of resources on your machine, given the real-time analysis and 3D display of the data's. 3 aspects are important : the power of the processor, the performance of the graphic card (with or without the 3D drivers) and the type of kernel that is being used (a preempt or real-time will allow better performance). The amount of needed resources is proportional to the width of the window. If your machine has difficulty, you can use a smaller window size (in Menu -> Edit -> Preferences), which decreases the needed resources.
3. RealTime mode will allow Spectrum3d to have priority on other running processes, in order to not be slowed down by them; it won't decrease the amount of CPU and GPU resources needed to perform the task. IMPORTANT WARNING : If you want to take advantage of this you should edit or create an '/etc/security/limits.d/audio.conf' file and put this inside : @audio - rtprio 95 , then you should add yourself as a member of the 'audio' group : for this, either go in System->Preferences->Users and Groups->Manage Groups, then select the 'audio' groups, click on 'Properties' and add yourself, or by typing in a terminal : sudo adduser ${LOGNAME} audio then reboot. If you don't do this, you'll have to run Spectrum3d as root, which is not recommended at all.
4. Multitouch support is provided by the uTouch-geis API. For now, the whole screen receives fingers input for Spectrum3d while Spectrum3d is running (input is not limited to the Spectrum3d window), unless you use a version of uTouch-geis that is at least 2.1.0. On the other hand, when Sprectrum3d is running, the mouse pointer cannot be used from the touchpoint on the screen, but the mouse pointer is still driven by the mouse. This behavior could be the same in Natty or later, if you have a touchpad that supports multitouch : the touchpad only receives input as multitouch gestures, and you cannot use it anymore to drive the mouse as long as Spectrum 3d is running.

RealTime mode will allow Spectrum3d to have priority on other running processes, in order to not be slowed down by them. In any case, it won't decrease the amount of CPU and GPU ressources needed to perform the task. IMPORTANT WARNING : If you enable RealTime mode and if you don't do the following, you'll have to run Spectrum3d as root.

Edit audio.conf file and make yourself a member of 'audio' group

If you want to take advantage of this you should edit or create an '/etc/security/limits.d/audio.conf' file :

and put this inside :

then you should add yourself as a member of the 'audio' group : either go in System->Preferences->Users and Groups->Manage Groups, then select the 'audio' groups, click on 'Properties' and add yourself, or type in a terminal :

then reboot. IMPORTANT WARNING : If you enable RealTime mode and if you don't do this, you'll have to run Spectrum3d as root:

which is not recommendended at all.

How to check that you are using realtime policy?

If realtime mode is chosen, gstreamer will play the sound with the RR ('Round Robin') policy with a priority of 50. You can check that the RR policy is used by typing in a terminal :

while Spectrum3d is playing; you'll see something like this :

The thread called "alsasrc:src" is running with the RR policy ('RR' in the 3d column).

What about Jack?

This is not needed for Jack, since, if Jack is run in realtime mode, all its clients (including the Gstreamer thread of Spectrum3d) are run in realtime mode also. So for Jack, realtime mode depends on Jack, since Spectrum3d connects to Jack as a client.

Multitouch support in Spectrum3d is provided by the uTouch-geis API (Geis2).

Where to get general information about multitouch?

For general information about multitouch in Ubuntu, you can refer to : https://wiki.ubuntu.com/Multitouch

Where to get it?

It is available from Natty's repositories. For Maverick, it has to be installed from the uTouch-geis website(the libutouch-geis-dev in the Maverick's repositories is the Geis1 API and we are using Geis2 here). A convenient way to do that on Maverick is :

1. uninstall libutouch-geis-dev and libutouch-geis if you have them installed;
2. type in a terminal : sudo apt-get build-dep libutouch-geis-dev (this will get all the needed dependencied to build Geis2);
3. then download the source package from uTouch-Geis website (https://launchpad.net/canonical-multitouch/utouch-geis) and follow the instruction from the INSTALL file; steps should include go in the source package directory, then ./configure, make, make install.

How is it implemented in Spectrum3d?

Only one and two fingers gestures have been implemented untill now since most of the touchscreen supports only 2 fingers gestures. On the other hand, as gestures can be used at a system wide level, it seems that 2 fingers gestures are more application-specific, while 3 or 4 fingers gestures apply to window or system management.

The following is important for Ubuntu 11.04 and before (from 11.10, there is no need to change anything for complete and fully working multitouch support). When 'Touch' function is activated, everything seems 'captured' by the 'Touch' function : mouse pointer cannot be controlled anymore by a single touch on the screen or on the touchpad. Any time a gesture is detected, no X interaction will occur. For example, if you listen for one finger drag on a trackpad you won't be able to move the cursor while it resides within the window you are listening to gesture events from. This is to prevent one physical interaction from firing two separate events through the system. It is be possible to "filter" gestures so that only 2 fingers gestures are taken in charge, and the mouse can still be controlled by one finger touch. However this is not possible for libutouch-geis version inferior or equal to 2.0.11 (Bug#769859, https://bugs.launchpad.net/utouch-geis/+bug/769859), and this is fixed in the 2.1.0 version. However, if you use Ubuntu 11.04 Natty Narwhal, that has the 2.0.10 version in the repositories, you will be affected, unless replace it by the 2.1.0 version (or higher); you have to install it directly from the utouch-geis website (https://launchpad.net/utouch-geis).

I don't know if multitouch support can easily be implemented in Ubuntu Lucid Lynx (10.04) or earlier.

A possible future alternative : Ginn

If you don't want to use uTouch-geis, it should be possible to use the excellent 'Ginn' software (https://launchpad.net/canonical-multitouch/ginn) that allows to "inject" gestures without installing the geis API. However, Ginn cannot yet "inject" gestures in a 'SDL' window such as spctrum3d's main window (Bug#751354, https://bugs.launchpad.net/ginn/+bug/751354), but the developpers are planning to implement it. When it will be possible, Ginn will have to be started as root.

GTKGLEXT1, used with GTK2, has been released since years and is in the repositories of the main Linux distributions. GTKGLEXT3, necessary for usage with GTK3, is in development.

Installation of GtkGlExt development version

The developpment version can only be found on Thomas Zimmermann's (tdz) github website; it should be soon available soon on GtkGlExt web site and should be downloaded via git. GtkGLExt's GNOME Git repository can be checked out anonymously:

Since it is a developpment version, it should be installed this way :

1. install dependencies :
• libgtk-3.0-dev;
• gtk documentation package : in Ubuntu, the 'libgtk-3-doc' package is not enough : you have to install the 'gtk-doc-tools' package;
2. go to the directory where the source of gtkglext are :
   $ cd ~/gtkglext-VERSION
3. since we are working on the Git version, we can use bootstrap for generating configure script and Makefiles.
   $ env ACLOCAL_FLAGS='-I /pkgconfig/share/aclocal -I /gtk/share/aclocal' ./bootstrap --prefix=${HOME}/gtkglext --enable-debug=yes
   Do not use 'autogen.sh' as it is recommended in the README since this has been changed and the README has not been updated.
4. then type :
   $ ./configure
   $ make
   $ make install
   
   GtkGlExt-3.0 should then be installed in /usr/local/include.