Musical Instrument Recognition Program help

[rhijas]

I am doing something pretty close to that. I convert the harmonics to amplitudes relative to the harmonic of greatest magnitude. That way, the highest harmonic has a value of 1, and all others are relative to that value. Those are the values that I have stored to use in comparing the harmonics of instrument samples. The purpose of that is so that if I had two instrument samples that have different volumes, both of their arrays of harmonics will be converted so that their amplitudes are in the same range, 0 to 1.

[chilly_charly]

May be you should also try to exchange idea with Moh, who had a similar project of bird song recognition ?

[alkazaa]

Hi rhijas,

I stumbled across this thread because of the interesting subject. For curiosity I googled for some more information and came across this link )which you maybe know already):

http://publica.fhg.de/documents/PX-88541.html

where a patent is briefly described which seems to do exactly what you're after.

BTW: One of the patent holders is K. Brandenburg, the guy who invented the MP3 encoding...


-Franz

[Henrik_Volkers]

How about using a neronal network ?

Inputs, at a first shot:

Harmonics, normalized

envelope parameters , total and by harmonics

just my 2c

[rhijas]

Thanks for the ideas everyone. That Bird Song recognition program that Moh is planning looks a little similar, though I think bird calls could more easily be recognised by the specific notes they sing over time, than analysing their harmonics over a constant period.

That patent you linked to is all in german. Is there an english version?

In the Bird Song recognition program, Chilly mentioned using "multivariate analysis". Is that just using a bunch of different characteristics to analyze the harmonics?

This weekend I made recordings of trumpet notes all throughout the scale from 233 Hz (Bd3) to 1035 Hz (around C6). I found that in the trumpet recordings, as the frequency rises, the number of large harmonics diminish. The lower notes, such as B3 (246.94 Hz) may be made of around 7 harmonics. As the frequency approaches 400 hz (around G4), there are 4 large harmonics, and when you get to the very high notes around the 800-1000 Hz area there may only be one or two large harmonics. The Trumpet waveforms seem to become more pure (closer to a single sine wave) as the frequency increases. If this is true with other brass instruments, it would lead me to believe that instruments would be easier to recognize at lower frequencies since they contain more prominent harmonics.

It would probably be a good idea to store samples of different notes throughout the scale, so that if you play a recording of an instrument, there will be a stored sample of a nearby note with which to match it up.

Can someone give me a simple explanation of what an Artificial Neural Network is? Is it a program that learns how to do things, the more you run it? Is it self programming? Would this be something that could be used to recognise similarities in the amplitudes of harmonics?

[DFGray]

A few "gotchas" you may want to look out for:

1. The same instrument can easily change harmonic structure.  For example, I can completely change the sound of my mandolin by playing near the bridge (lots of high harmonics) or near the center of the string (more harplike tone).  All stringed instruments are like this.  Wind instruments will have changes when they go from one "octave" to another.
2. Some different instruments sound very similar.  For example, a clarinet has almost the same harmonic structure as a violin.
3. Different instruments of the same class can sound very different, depending on how "good" they are.  I recently purchased a new violin, and the differences I found while shopping were amazing (I also found that good violins are very expensive).  Check out the Nagyvary website for some discussion on this (as well as a fair amount of marketing).
4. The fundamental in your FFT is not always the highest harmonic.  Found this one the hard way when constructing a LabVIEW tuner and testing with my mandolin.

You may end up with using joint time-frequency analysis to sort all this out.  Good luck.

[alkazaa]

---

@rhijas wrote:
....
That patent you linked to is all in german. Is there an english version?

---

Here's another link to the patent:
http://v3.espacenet.com/textdoc?&DB=EPODOC&IDX=EP1417676&QPN=EP1417676
Content of this patent is a time domain method, as opposed to usually employed frequency domain methods.

A reference to a frequency domain method is e.g.:
B. Kostek and A. Czyzewksi, "Representing Musical Instrument Sounds for their Automatic Classification", J. Audio Eng. Soc., vol 49, no. 9, September 2001

-Franz

[chilly_charly]

One thing you should do is to do a Google search with "chemometrics".

There is really a whole world hidden behind this word. Don't believe that's something just related to chemistry !

[tbob]

Here is a zip file that contains short wav files of various string instruments (electric guitar, acoustic guitar, dulcimer, mandolin,) and a wooden flute.  I hope this is not too late for you to use in your project.

[rhijas]

Thanks everyone for your ideas. I have been real busy recently, but I worked a lot on this project during this past weekend.

As I said in my last post, I had made a lot of samples of my trumpet's notes. On Saturday, I recorded 29 different trumpet notes ranging from 233 to 984 Hz. I had converted and stored as spreadsheet files the prevous samples, and didn't store the harmonics of these new trumpet notes. I used these new .WAV files to test to see if my program could recognise these new notes as being a trumpet. I was able to correctly match all but one recording with the previously stored trumpet harmonics. The one that wasn't successful ended up matching up with a Bass Trombone. When I stored another trumpet harmonic set that was closer to this trumpet, I was able to get it to match with a trumpet.

This showed that by storing multiple sets of harmonics of a type of instrument over a wider range than just using one or two recordings, it allows the program to recognize the instrument at different frequency ranges where its harmonics are different.

I made a wider set of stored harmonics for the Alto Saxophone, Flute, Clarinet, and Trombone, so that now I am able to play an unstored recording of one of those instruments (plus Trumpet) larger range and recognize the instrument that was played. I can recognize a range of around 12 notes for Alto Sax, Flute, Clarinet, and Trombone, and around 22 notes for Trumpet. The frequency range is determined by the range of harmonic samples that I have stored for that instrument, which explains why I have a wider range for the Trumpet which I recorded myself.

I also got access to a French horn, and made a bunch of recordings (around 128 notes over a 20 note range) of different notes. I don't know how to play the Horn, so I just played many different octaves (by just changing my lip tightness) of each fingering position. Apparently, the French Horn can play some notes using different fingerings, because by the time I was done converting the recordings so I know their frequencies, I had anywhere from 2 to 4 of each note.

tbob, thanks again for showing me how to access the files in a folder. I don't even want to think of how large the program would be now if I was still using the same way to access the 82 spreadsheet files I have now, plus the french horn and any other instruments I add in the future. Thanks for the recordings. I'll have to take a look at them the next time I work on the project.

As mentioned previously, I have seen that an instrument's harmonics change over their playing range. In order to recognize the instruments over a wider range, I have stored sets of harmonics for the instrument's notes over a wider range. If I play a trumpet recording that is around 290 Hz, even though it will be different than the 634 Hz trumpet harmonics, it will probably match up with the 293 Hz trumpet harmonics, or ones in that range.

For now, I am just concentrating on getting the program to recognize different types of instruments, so I am only using a single kind of each instrument. I suspect that if you were to store the harmonics characteristic of two different types of violins, you could recognize which one was played (if they were different enough).

I found that the fundamental was not always the highest harmonic fairly early on because I was constantly looking at the frequency spectrum of the instruments. When I store a set of harmonics as a spreadsheet file, I first normalize them to be relative to the amplitide of the greatest harmonic. The greatest harmonic then has the value of 1, and the others are relative to that.

Since my program is currently working sucessfully, I am going to keep adding instrument samples to improve its range and accuracy. If I have everything finished, and have free time, (or hit some other problems) I may take a look at chemometrics. I just think my time would be wiser spent now on continuing to work on a working program than concentrating on something that I know nothing about.