"5. Forum Medientechnik" at FH St.Poelten (Austria) - Keynote

On Thursday, 22.November 2012 I will give a keynote presentation at the "Forum Medientechnik", organised by the FH St.Poelten (Austria) about my work on orchestral renditions using sequencers, instrument sample libraries, and synthesizers. The talk (given in German) titled "The Virtual Philharmonic Orchestra - Produktion klassischer Orchestermusik am Computer" will cover the fundamental techniques applied in creating such renditions with simple and affordable technology means at home. It will feature the Prelude (1st movement) of Maurice Ravel's orchestral version of "Le Tombeau de Couperin", which will be used to illustrate the application of various techniques for making such renditions "musical".

The website of this conference is at http://fmt2012.fhstp.ac.at/

Mahler: "Resurrection", 1st Movement

Today, 18.May 2012, is the 101st anniversary of Gustav Mahler's death. Just in time for this commemoration I was able to complete my rendition of the first movement of Symphony #2. This has been my most ambitious project so far: it requires 66 independent instrument tracks in the sequencer, and it almost brings my audio system to its knees, with an occasional glitch in the audio engine. But the recording could be created ok.

The image to the left is a painting by artist Paula Arciniega. She has created a whole series of paintings, inspired by Mahler's symphonies, and I find that these express his music very well. The one shown here is the painting which relates to Symphony #2.

My recording if the first movement is on my website, but for direct access it can be also played through the player below. I plan to continue working on this symphony, but it will take a while until I have it completed.

2D Fourier Transform on Music

For a long time, I have thought about what to do to visualize music. The need for this comes when posting music recordings onto YouTube, which is a very nice platform for distributing music. Just showing a black screen does not really do justice to the visual medium, and so one has to think on what to put there to the music. Many music pieces have there a still image, or a collection of images which are relevant to the music. This is fine, but does not use fully the possibilities which video is offering. Ideally, one would have a video that is tightly synchronized to the music and complements the acoustic impression with a visual one - as it is done in popular music videos, since the time MTV has pushed this forward ("video killed the radio star" comes to mind). But for classical music there is really not very much in terms of appropriate music visualizations: there are videos of the performers, there are those series of smartly arranged still images, but rarely is there a video which truly complements the audio and the music in terms of visualizing directly the music and its elements.

In the past there has been some work on this, and I am especially fascinated by the work that Oskar Fischinger has done in the 1920-40s. Sometimes a few of his films have been shown on Turner Classic Movies in between regular movies. He really tried to take music and visualize it with abstract shapes which represent the music.

On most mediaplayers, there are automatic visualization tools which show some automatically generated animations, based on the music audio. However, I have never found a satisfying visualizer, because most of the visualizations appear to be some random motions, on top of a bit of wave analysis.

That kept me thinking, and then in December 2011 I had the idea to use the Fourier Transform to detect periodicities in music. The Fourier Transform (FT) is a well known approach for analyzing audio: it is generally used to create spectrograms and show the frequency content in music. FT detects periodicities in signals, and this is usually used to detect the frequencies which are present in an acoustic signal. Music is based on such frequencies, which relate to musical notes. But music has also "slower" periodicities", not related to the high frequency of sound (>20 Hz), but related to the measures and the beat in music. So I decided to apply the FT on music with a larger window: for a sound spectrum, the FT is usually applied for a time window of 100 ms. But for revealing longer-period periodicities in music, I would apply the FT for a time window of several seconds.

To make the computations easier, I decided to not use the sound wave itself, but the abstract notation of music as input. This is a series of discrete note events, which have pitch and volume. In music sequencers, the notes are usually shown on a piano plot roll, which has as horizontal axis the time, and as vertical axis the pitch resp. the note frequency. A FT can be applied on this piano plot roll, using the 2-dimensional approach for FT that is used in image processing (noise reduction, feature detection).

The figure to the left here shows the 2D Fourier Transform of the piano roll plot of Gustav Mahler's "Urlicht" (4th movement of Symphony No.2). In this plot, the horizontal axis shows spatial frequencies of the rhythm and the beat distribution. The vertical axis indicates the distribution of intervals and chords. There are clearly some peaks to be seen, which indicate predominant intervals in this music. Most of the relevant data are around the central axis (x=0) in the FT plot, which is the axis with small rhythm elements, indicating long notes (or pauses). Along this central vertical axis, the musical chord and interval elements can be seen as distinct points or regions.

This 2D FT can be re-transformed back into a piano roll plot, by again applying the Fourier Transform algorithm. I have tried this, and indeed it comes out correctly. However, this back-transformation also needs the phase of the FT (something that is also numerically computed as part of the transformation, using complex numbers). If this phase is not present, then the resulting piano plot roll is overlaid with its mirror image, due to symmetry. It would be quite beneficial to find a way to remove this mirror image, without using the phase of the FT. Then, the inverse transformation could be used to edit the music in its Fourier space. This would provide completely new ways of creating music.

Overall I think that this 2D FT has quite a significance for music: it is invariant to pitch and tempo (at least if the music time is used as reference, instead of real time), and this can lead to unique fingerprinting of music or musical phrases, which can be used for identifying music. Also, it can provide new ways of creating music by editing its Fourier Transform. And last but not least, I have found a new way of visualizing music. At least, the FT algorithm provides a new dataset (2D array) as a basis for an interesting visualization that directly represents the musical content.

I have published this approach recently at the Music, Mind, Invention workshop in Ewing, NJ. The officially published paper for this is here, and the presentation which I gave is here, although that presentation does not have me speaking, so it may be a bit hard to understand the context and background.

I plan to develop this further, make a few videos with this approach as visual accompaniment of my music, and I will work towards releasing the software as a toolkit so that it can be used by others. Lots of work to do then.

Opening of Workshop "Music, Mind, Invention"

At the evening before the workshop "Music, Mind, Invention" began (held at The College of New Jersey in Ewing), a concert was given in the Center for the Arts, where pianist Joanna Chao performed music by mostly contemporary composers. Here is the program she played:

Joshua Fineberg: "Tremors" (1997)
J.S.Bach: Aria to Goldberg Variations
Fred Lerdahl (2004, 2009): "Three Diatonic Studies", with the movements "Chasing Goldberg", "Cyclic Descent", and "Scalar Rhythms"
Robert Young McMahan: From the "Six Miniatures for Piano" (2011-12) the first two: "Andante" and "Crisp, Strident". This was a world premiere.
Joshua Fineberg: "Veils" (2001)

A very appropriate opening of the workshop, followed by a reception with the opportunity to chat with the participants.

Ravel's "Ma mère l'oye" (Mother Goose) recorded by VPO

Maurice Ravel's orchestral suite "Ma mère l'oye" (Mother Goose) is based on children's stories. It was first written as a piece for piano, but Ravel later created an orchestral version. There is a long orchestral version which was intended as a ballet music, and there is a shorter suite in 5 movements. I have dedicated the past month to work on the latter 5-movement suite and have created a rendition with the Garritan Personal Orchestra samples. Work on this suite was not easy: although the instrumentation is set much lighter than for example a Gustav Mahler symphony, there are several technical difficulties in it: the strings have many parts with fast notes, 1/32 and 1/64. Overall, the instruments need to be played very light-hearted, to keep that airy feel of the music. There are several wind solo instruments which need to be played "with emotion". There is lots of flagolet in the strings, and some of the instruments appear to exceed their natural range (or is this just sloppiness of the editors at Durand, who did not properly assign the right instruments?).

The recordings of all five movements of this suite are now complete, and I have bundled this suite with another suite by Maurice Ravel: "Le Tombeau de Couperin" into the album "Maurice Ravel - Vol.One". My recording of the Le Tombeau suite had already been completed in November 2011, but its 20 minutes duration was too short for a full album.

Since a few days this album has become available on the commercial music outlets. This time Spotify was first, followed by iTunes a few days later. I am now waiting for the album to appear on Amazon.

Happy 137th birthday, Monsieur Joseph-Maurice Ravel!

It is not really a big anniversary - 137 years is kind of an odd number. But there is always a good reason to take the opportunity to celebrate the music of Maurice Ravel. Many people know his "Bolero", which has become a kind of musical icon even in popular culture. Also his orchestration of Modest Mussorgksy's "Pictures at an Exhibition" is well known. But he wrote many other works, which are popular among classic music enthusiasts, but are not so well known in the general public.

I have liked his music since the late 1970s and have learned about many of his other compositions. His special style of instrumentation and orchestration is very recognizable. So it was for me a special challenge to recreate this "sound color" with my synthesizer/sampler instruments. In 1998-1999 I was able to create a rendition of all four movements from his orchestral suite "Le Tombeau de Couperin". Using the Garritan Personal Orchestra 4 samples I then re-recorded this suite a few months ago - the four movements are accessible here on my website. Also in 1998 I recorded the "Pavane for a Dead Princess", and I recently revised that rendition with the Garritan Personal Orchestra 4 as well.

Since quite a while I wanted to record the suite "Ma mère l'oye" (Mother Goose), which is about children's stories. It has a peculiar fine and delicate quality, and one has to be careful not to overpower it. The instrumentation appears quite sparse, but with an array of unusual instruments. The technical abilities required from the instrumentalists in the orchestra appear quite high: several instruments have solo lines which require very precise and expressive playing. The strings have fast 1/32 and several flagolet notes. These challenges are also hard to master with my computer/sampler setup, as the sound must have this special "light and airy" quality. I had planned to complete the whole suite by this birthday on 7.March, but this proved to be too ambitious: so far I only have the first three movements of this suite recorded. I hope to complete this suite in the next few weeks.

A playlist with all my 8 current recording "tracks" of Ravel's music is here on my new site at Indaba.

Another music by Ravel is still waiting for me: the technically very challenging homage to Vienna waltzes: "La Valse".

Let us celebrate the composer of such wonderful music, and let us ignore the scandalous situation related to the royalties of his music.

Redesigned Website

Currently I am working on redesigning the Website of the Virtual Philharmonic Orchestra. Most of the work has now been done, but there are a few more things to complete.

The main change is that the streaming of the audio (MP3 files) which I host on my site, is now done through HTML5 and its <audio> tag, as explained here on my other blog. This allows playing the files on any current browser, without the user having to install a plugin. Also, this method gives me a better control of the access: I can implement access counters which keep track of how often an MP3 file is played, and I can provide authorized access to files which are otherwise protected. One general problem is that the web server where the site is hosted, is not really suitable for large volume streaming; therefore I did have to limit the access of many files, so as not to cause too much bandwidth allocation.

I also cleaned up the layout to make it easier to navigate. The main feature of this layout is that it is also mobile-friendly: all the content adapts to a smaller size (try this out by re-sizing the browser window, and you see what I mean), as it is based on flexible and floating <DIV>s (blocks of HTML parts which have a variable size and can arrange themselves to fit on the screen). The site now includes links to Wikipedia for composers and music pieces on the site. Social networking links are now provided, allowing a more active user feedback and engagement. Music recordings are now called "tracks", in accordance with the usual lingo in the music publishing industry. When available on commercial online music distributors, the direct links to those sites are provided. On Spotify, streaming is for free. iTunes, Amazon, and CDBaby allow download of the tracks and albums for a fee. My music is also on many more online distributors, which would be too many to list.

Still missing in the site re-organisation are pages with the albums, the videos, and my own compositions. These will be done as soon as I find more time. Furthermore, not all of the individual music tracks do yet have their full own page - but this will also be completed soon.

I hope that you enjoy this new site layout with all the new features!

Slow Mahler - Out of Context

The last two years, 2010 and 2011, have seen many performances and new recordings of the Symphonies by Gustav Mahler, due to the fact, that in 2010 his 150th birthday was celebrated, and in 2011 the 100th anniversary of his death was commemorated. Since these symphonies are among the music that I cherish most, I felt compelled to contribute with my own recordings, which, as always, are renditions of the complete orchestral scores by using computer-controlled sampled instruments. My first album release of a complete Mahler Symphony happened on 7.July 2010: his Symphony #1 which I subtitled as "From the Life of a Lonely One" ("Aus dem Leben eines Einsamen"), the original subtitle of Jean Paul's novel "Titan" which was Mahler's inspiration for this Symphony #1. Most often, the Symphony #1 is titled "Titan", but I think that the longer subtitle much better reflects the mood and musical content. The complete symphony is available on iTunes.

But since that release of this symphony in 2010 I had not been able to commit time to complete the rendition of another symphony. These works are just too gigantic. But I did have time to work on a few new individual movements from his symphonies, and I reworked some of the previous recordings which I had created. All of these were slow movements, because of a simple and trivial reason: I can get them done faster, because they contain not so many notes. Even the last movement of Symphony #9 was quite quickly recorded. What took longer was to shape the expression until it sounded acceptable to me. And then in September 2011 I decided to combine these into a new "temporary" album: "SLOW MAHLER - OUT OF CONTEXT". I am fully aware that these movements normally need the context of a whole symphony, and taking them out of this context may not be the most sensible thing to do. But I put them together under the common theme of slowness: they show the development of musical expressiveness, from the "Blumine" in Symphony #1 through the emotional rollercoaster of the 4th movement of Symphony #4, to the final resting in the Finale of Symphony #9. And therefore, this album does show a certain cohesiveness.

I plan of course to complete all symphonies eventually, and work is currently done on creating a rendition of Symphonies #2 and #4 - but this may take a while. In the meantime I invite you to enjoy the album that I have created for you. It is here on iTunes at the address: http://itunes.apple.com/gb/album/slow-mahler-out-of-context/id476151493
or on Amazon UK: http://www.amazon.co.uk/Slow-Mahler-Out-of-Context/dp/B00608G1B8/ref=sr_shvl_album_1?ie=UTF8&qid=1328484020&sr=301-1

Plans for 2012

Almost 1/12 of the new year 2012 has already passed... The Virtual Philharmonic Orchestra (VPO) has big projects for this year: all pre-2005 renditions will be re-recorded, using Garritan Personal Orchestra 4, and all these new recordings will have completely new expression and timing mapping. Also, some of the music excerpts from suites/symphonies which are currently only available in a subset of movements, will be completed, so that these work are no longer there in isolation.

But there are also plans for new renditions, and this weekend I started working on one: although the Mahler year is over, I have begun to record the first movement of Symphony #2. Got already the first minute or so. Now this is a big one, and I am already dreading how to work on the huge final movement (which requires a choir and two orchestras...). Also the first movement requires quite an orchestral setup. In Sonar, I did create 4 sets of ARIA players, with a total of 64 different orchestra instrument samples. This appears to reach the limit of my PC: the audio play is a bit choppy, although only few instruments are playing during this first minute. However, the off-line "bounce-to-track" seemed to work without glitches, so I will carry on. In the 2nd minute there is a large "unisono" which will use most of the instruments simultaneously - if I get this working ok, then the rest should be fine.

You may wonder what this image to the top left is: this is a glimpse into results of my recent work on automatic music visualization. The algorithm takes into account periodicities in music and is based on a 2D Fourier Transform. More will be in a paper which I am currently preparing. This snapshot is a visualization of a moment in Smetana's Moldau.