Organizing a sound library

Over time I have built up a significant library of sounds, mostly single-note samples, but with some loops as well. All these sounds were collected legally, usually through online registration to loop and samples vendors websites, by following the process they propose so that we can test the quality of what they offer.

Collecting samples along the road was not a massive chore, and yes it really helps asserting the quality of the vendors. Although they all claim to be the best, of course some are better than others, at least for your preferred genre.

Last time I ran a little count over the collection there was over 23000 samples.

Lost in sounds

Even though having many sounds sounds good, it leads to some problems: many of these sounds are not very good (since they come from just anywhere), and they must be organized to find them quickly. I have no idea on how to help with the first problem (except simply listen to each sound). The second problem, how to organize sounds to make them easy to retrieve, is not that easy. If I was in classical music it would be easier, but it is more complicated for electronic genres, that have their own sonic vocabulary, very different than the classic, academic orchestra or the jazz/rock band. I need a way to classify sounds in the context of electronic music genres, from house to hip-hop.

A quick search on the topic yields partial answers, none of them being ready to use…

Traditional classifications

Traditional classifications tend to focus on how the sound is originated:

Symphonic classification

The book Garritan Interactive Principles of Orchestration by Nikolay Rimsky-Korsakov describes a classical classification for the symphonic orchestra: Stringed, Wood wind and Brass Instruments, along with “Instruments of Little Sustaining Power”. The later include the harp, pizzicato strings, piano and percussion instruments “producing indefinite sounds”. Here we find a classification that takes into account whether a sound is loud or not, while the book also explains how many violins you need to match one trumpet loudwise. It also considers whether a sound has a clear pitch or not.

Scientific classification

More scientific, the Hornbostel-Sachs musical instrument classification classifies instruments by the physical process that makes sound:  Idiophone (the simplest of all, just hit the body), Membranophone, Chordophone, Aerophone, Electrophone. Each category has hierarchic sub categories, and they are all numbered, e-g 111242 = bells. This is totally cumbersome for my needs!

Alternate classifications

Classify by function

An appealing idea is to classify sounds according to their typical function within a song: “perfect for melodic lines”, “rhythmic”, “harmonic”. The problem with this approach is that except for obvious cases (bass sounds, kicks, grand piano…) it is hard to tell in advance what a sound can be used for.

Connotative classification

Kicks, snares and hi-hats are so numerous they need further sub-classification. This is where considering the emotional connotations, mood and other symbolic evocations can help: happy/sad, serious/light, asian/african/exotic/western, “dark”, “tribal”, “asian-ish”, references to well known song or soundtrack (“cartoon style”). The problem here is to find connotations that can be shared between different people, or that are at least constant in someone’s mind.

Classify by attributes

Going further, the MIR community (music information retrieval) routinely classifies sounds by their time-domain or frequency-domain features, which can sometime enable to deduce the mood, whether it is melodic or not etc. The problem is that this is still research, and the prime classification is indeed a vector of features values, e-g. (spectral centroid=0.68, rms=0.98, zerocross=3051…). The exploitation of these features vectors requires adaptative machine learning, not something that can be done quickly.

Synthesizer-based

A similar but more intuitive approach to describe sounds in order to classify them would be to consider the synthesizer parameters that would be required to reproduce the sound approximately: Spectrum (single pitched triangle-ish, square-ish, saw-ish or sine, pitched/inharmonic/noise), Amplitude envelope (sustained or not, short/long attack, short/long decay), plus the usual synth tricks (detune, oscillators hard sync, filter envelope, pitch envelope, pitch lfo, filter lfo, amplitude lfo, mono/poly play mode, portamento, ring modulation, high resonance…); again the problem is that each sound would require a full vector of parameters, a kind of simplified old synth “patch”.

Pragmatic classifications

Elemental categories

One particularly obvious way to organize sounds is by their elemental nature: single-hit, loop.

Then single-hits chords, for instance a Em7 Rhodes chord, could also be classified in its own category just because it is not at all as all-purpose as a single note Rhodes C#3. The same would apply to instrumental loops that are harmonically constrained.

Preset categories

Synth makers have long introduced instruments categories to help find a particular preset, for instance in my E-MU Mo Phatt the categories are: Bass (bas), Guitar (gtr), Hit (hit), Lead (led), Synth (syn), Special effects (sfx), Brass (brs), Strings (str,) Keybords (key), Voices (vox), Percussions (prc), plus some special categories: Beats (bts), Drum Kits (kit), Basic waveshapes (wav and rom). This is not bad and already helps a lot, but this is not enough for thousands of sounds

Samples vendors classifications

Samples vendors have their own way to classify their sounds: most are purely marketing driven (by the name of the “famous” creator of the sounds packs, or usually by something related to the targeted genre, e-g. “Bombastic Kicks”), while some others do classify by instruments: “Kick, Snare, Claps, HiHats, Industrial Percussions, Fx etc.” each of them being either “Classic, Rugged, Heavy, Combo, Future, Acoustic, Celebrity, Industry etc.”, or “Keys, Chords, Stabs, Licks, Riffs, Bends”, or even more precise: “Squicks, Glitches, Bits, Swooshes, Scratches etc.”. These classifications are completely specific to each vendor, however they do provide a richer vocabulary for electronic genres.

Keywords

If your Operating System enables it, an obvious way to classify sounds would be to tag them manually with ‘relevant’ keywords. The problem is that it would have to be done from the beginning, and even in this case it would incur a serious effort, not to mention how to define the ‘relevant’ set of keywords!

Tracing the terms of use

Whenever we download sounds from a sample vendor, we have to abide to its terms of use everytime we use the sample. Commercial vendors usually do not require much, you just cannot use their material if you were to create another sample pack to redistribute. However many creative commons-like material, though free, require you credit their use. Of course this is fair, but this is just hard! After several months of work on a song, how do you know you used samples from this website that requires credit at the time of putting it to myspace?

Since I had no solution, I had decided to classify every downloaded sound by their provider. Yes, yet another classification!

Criteria of a good classification

We must be able to access a list of consistent sounds by going no deeper than one, two or three folders deep, and the list of sounds should not exceed something like 100-300 samples, in other words a number of sounds we can browse through and listen to each to test how well they fit.

Building our own classification

I used to setup directories to organize sounds following an old article from Sound On Sound, and I felt quite happy with it at that time. However it did not work well in practice, because of the terms of use thing, but also because the proposed classification is not precise enough for the most important categories in electronic music: drums and synths.

I tried to build a classification that fits electronic genres as efficiently as possible, by mixing ideas from all of the above-mentioned approaches.

First we can consider that electronic genres mostly derive from the rock/jazz band, which is essentially a drum, bass, guitar and keys plus vocal thing, enriched with back beats, synths effects, background ambiance, and where the role of vocals is often replaced or complemented by synth leads, samples, loops and sound-design hooks.

Electronic-oriented categories

Drums remain the primary category. It is made of low frequency elements (Kicks), mid-frequency elements (Snares and Claps) and high frequency rhythmic elements (Hi Hats, Shakers, Cabasa).

Together with drums, basses are absolutely vital. Bass sounds are rather easy to classify: acoustic, electric, fretless, finger, slap, acid, moog, etc.

Instruments playing the harmony (Keys, Pads, possibly Strings, harmonic Loops or single chord samples, arpeggios, harmonic Soundtracks) form another clear category. However everyone knows that in some cases a lead or even a bass sound can be used to create harmonic pads. Categories are never that strict.

Short sounds can then be grouped into one consistent and important category. Their important quality is that we can trigger them in a rhythmic fashion, hence they are secondary rhythmic elements. Percussions of course, and Bleeps, Stabs, Hits, Pizz belong to this category. In this category sounds can be pitched or not, which can help further sub classify.

Brass instruments can be considered either as lead instruments or short sounds.

Long sounds (Ambiances, Zippers, Sirens, Crowd, Applause, Swells, Speech parts) are similar in use, often in the background,  as X-factor elements. Hip Tree, Crash, Reverse crash could be put into this category even though they play a low frequency rhythm.

Special effects (FX) can be considered yet another category, as they usually have a narrative role, just like on the soundtrack of a movie (fowler): they are exceptional, non-rhythmic effects, in other words narrative elements.

Finally, melodic elements form yet another category: Leads, Loops, Bends, Licks, Riffs, Vocals, Adlibs, even though many other sounds can also play a melodic role.

Refining the categories

Looking at the above categories, they are not all as important as each other. Typically, Kicks (at least) would deserve their own first-level category.

Sub category classification

To classify more accurately crowded categories, consider naming the building blocks logically: kick fat, kick attack, kick tail etc. Breathe noise, key noise, clicks, etc: they are pure sounds that need to be mixed or that can help tweak another sample.

Within each category, we can then make each name more expressive by using connotative words. Looking at sample vendors wording helps here. Words like “Industrial”, “Whispering Winds”, “Atmospheric”, “Ethereal”, “Ephemeral”,Meditative” can be used for that purpose. For kicks, snares and hi hats, they can also be sorted out by typical genre: a techno kick is obviously different than a hip-hop kick.

Conclusion

The above classification is nothing more than a goal, since I still don’t know how to assign each sample to the right category, except manually… I would love a tool that could do that automatically on my behalf, definitely something to investigate.

How you classify your sounds reveal a lot about the way you approach making music. Sticking to the classic bass/drum/keys/synths categories is not enough if you view electronic genres as having the freedom to use sounds are raw material, just a painter uses raw colours to do whatever result.

For example, considering that every short sound can be used in a rhythmic fashion depends on your perspective on music, and this is what can decide whether to put guitars chops and synth bleeps together with orchestra hits, something a classical composer would hardly do!

Read More

Akai MPC program editor in Java

I juste wanted to automate the creation of programs for my Akai MPC500 sampler/groove machine, for my personal needs, and I ended up releasing a piece of software to sourceforge. It is called MPC Maid, read “assistant of the MPC”, here is a screenshot of the program editor:

MPC Maid program editor view
MPC Maid program editor view

The MPC file format was freely available on the website of Stephen Norum, very well documented. However my day job is to design and code big financial server-side systems, hence coding at the byte-level was refreshing…

Chop loop into slices

I had been playing around with audio/Music Information Retrieval (MIR) recently for fun, so I took the opportunity to integrate some of that, in the form of a really simple loop slicer.

Audio Slice Editor view
Audio Slice Editor view

You just drag and drop a WAV file from the file explorer into its waveform editor, then it analyzes the audio samples to detects the beats. There is a slider to adjust the sensitivity of the detection, which means this is not a totally automatic algorithm.

The idea for the detection comes from the article “Beat Detection Algorithms” from Frédéric Patin, with some personal modifications. In particular I have added a toggle to prevent a detection immediately after another, and a zero-cross detection to cut the slices on a zero-crossing, in order to reduce the glitches (but of course this is not enough to be perfect, fortunately the MPC has a convenient small decay at the very end of every sample, which solves the problem).

The idea is to compute the energy in sliding time-domain windows,then compare the energy of each window against the average in the surrounding; when it exceeds it enough, there is a beat.

Since many loops available are commercial quality, their length gives the tempo with an extreme accuracy. I used that to generate a MIDI file of the groove of the loop. Coupled with the program edition this allows for exporting the slices, the MPC program that uses them, and the MIDI sequence of their placements, in other words this is roughly equivalent to the REX file idea.

Native-looking but fully portable

Making sure that the software is fully portable and looks native in Mac OS X and in Windows requires careful attention, especially on the Mac side, where the menu is on top of the screen, not the window, the shortcuts are different, and even worst, the menu do not follow the same conventions! For instance the “About” menu is in the Application special menu, not in the Help menu as on Windows. To achieve that in Java requires the Apple OSXAdapter to call by reflection an API that will not be available on non Mac platforms!

Again I hardly do any user interface usually at work, so this little project was overal a good experience to me, and now the users have begun to play with it it is soon to become even more interesting!

Update: here is a video of how to chop a loop almost automatically into slices, then how to play it back at a different tempo on the MPC:

Read More