Keyboard trainer

[Frostburn]

I worked a bit on the distortion approach and I've gotten to the point where my algorithm detects the correct pitch of my voice except for the vowel 'ah'.

The idea is to first filter the signal to emphasize low frequencies, then distort it to produce a stronger fundamental and finally filter it again to emphasize the fundamental even more.
So schematically:

[Frostburn]

Here's what I got for now.
Hum in to the microphone and it should play the same frequency with the same amplitude using a TriOsc.

---

pitch_detection.ck
Description:	A filter-distort-filter approach to pitch detection for single harmonic signals.	Download (listen)
Filename:	pitch_detection.ck
Filesize:	3.43 KB
Downloaded:	389 Time(s)

---

[Inventor]

It works pretty well except there is a warbling to the output sound. Still, another useful tool in detecting guitar notes.

My next scenario is as follows: The program has instructed the player to play a chord. The player plays a chord which may or may not be correct. How does the program determine if the correct chord has been played? I am thinking of using multiple comb filters for this, what do you think? Maybe instead I should use this distortion technique? I dunno, there is plenty of time to think about it.

[Frostburn]

I think the comb approach is better for chords and even for single voices.
After all with games like these we are dealing with pitch verification which is easier than pitch detection.

I'll have to try the comb myself to get a feel on it.

[Inventor]

[Frostburn]

I tried pitch verification with feedforward comb filters in series. It was supposed to reduce the ringing compared to a feedback comb.
Not really that successful. Maybe having two feedback combs and alternating between them would help...

Anyway; A screenshot of something I attached to this post thingie:
Well... it's more of a series of screenshots because the display is one dimensional.

[Inventor]

I tried it and it worked! Of course, I can't hold a tune but the indicator definitely changed as I applied audio to the microphone. These little games are fun!

[Frostburn]

I'd like to see someone make an algorithm that can detect/verify this on a real guitar.
Now that would be a live coding performance.

[Inventor]

Yes, that would be cool, perhaps with a bunch of comb filters all firing off in response to the song.

I want to add chords and then songs to Guitar Zero, but maybe that's asking a lot. The good news is that the song is known so that the filters can be set according to the chord or note that is expected. One major difference between Guitar Zero and Guitar Hero is that Hero reduces the instrument to five notes, while Zero preserves the full fretboard. This does, however, conceal future notes so that there is no stream to anticipate like in Hero. That's OK though. I dunno, I'll keep thinking about it and working on it.

[Inventor]

I've got chords added to Guitar Zero now, though it's MAUI only now due to the complexity of maintaining two versions. Come on DEV TEAM, let's get that MAUI release out there! Patience is a virtue, sigh.

Anyway, I've now even added some rudimentary song learning capability and I have part of AC/DC's "Back in Black" song in there (what can I say, I'm a rocker).

The comb filters are working quite well. I switched to your recommendation, Frostburn, of dividing the comb-filtered signal by an unaltered signal to obtain a number that can be compared to a threshold. It's touch-and-go and could use some more work, but for now it's doing quite well.

The latest version can be found on my music web page which is here:

http://www.freedomodds.com/music/

Cheers!

[Inventor]

I've been thinking of a way to display correct/incorrect results, and then it occurred to me to make a bullseye display, as shown in the image below. I'm not sure why the theta values all align like that - how does ChucK's cos and sin function ranges work? I have them going from 0 to 2pi. Anyway, it's getting to be a mature piece of software now and I even had one person give it a try - he/she enjoyed it a lot! I'll keep you posted.

---

Guitar_Zero19.jpg
Description:	The current Guitar Zero user interface
Filesize:	94.12 KB
Viewed:	290 Time(s)
This image has been reduced to fit the page. Click on it to enlarge.

---

[Inventor]

I figured out the theta thing, i was casting to an int instead of a float, duh! I also added a vertical LED meter that continuously displays how close your sound is to passing the threshold of acceptance for a good note/chord. I also adjusted the filters a little and did some cleanup stuff. Aside from adding more chords and more songs, I'm not sure what to do with the program's development right now, so I'm asking for suggestions.

That goes for a whole new program too. If anyone can imagineer a good guitar-driven game that teaches, I might take a try at coding it up. Just let me know if you have any ideas. Can you tell I'm a little bored?

Oh, p.s. I found that playing some songs will actually trigger the note/chord detection, be it due to incorrect detection or due to the fact that the songs actually contain the note/chord being asked for by the program. "Pick up the Pieces" by Average White Band from the 70's is particularly good at playing Guitar Zero, and interestingly "Back in Black" by AC/DC does not play its own song very well in Guitar Zero, most likely because of timing delays. There I go rambling again... Cheers!

[Frostburn]

[Inventor]

Frostburn, thanks for your time in looking at my code! You really have an eye for detail. I fixed the string problem (or potential future problem) by changing all references to str instead of string. As for blowing into the microphone, it does cause false positives if i get real close like two inches away and blow hard on it, or if i rub it with my thumb. Also, I did the other test and yes, just plucking one string of a chord will complete the chord! Darn darn darn!

I multiplied the acceptance threshold of the chords by the number of strings in the chord, but I can see that wasn't enough. The chord harmonics are all mixed together and the comb filter must not be being selective enough.

I have attached the latest source code. The comb filters begin at line 140. If you can spare a moment, would you mind just glancing over the filter implementation and/or thinking about it a little to offer some suggestions?

I know the thing sort of works because when I pluck incorrect strings it rejects them usually, even just one fret away, but that's individual notes. Chords are a different beast altogether. Thanks in advance if you can spare a moment to take a look, and thanks for your help already!

---

Guitar_Zero23.ck
Description:	The lates Guitar Zero source code	Download (listen)
Filename:	Guitar_Zero23.ck
Filesize:	30.57 KB
Downloaded:	246 Time(s)

---

[Frostburn]

Well for one thing you're using Delay instead of DelayL or DelayA, I suggest DelayA. You also forgot that feedback patches have one extra frame of delay.
So that's: second/freq - samp => my_DelayA.delay;

I need to study this pitch verification a bit more. I'll write a class that (hopefully) does it right and try to document it so that you can use it.

EDIT: I'm running to the problem where short bursts of noise create false positives... I'll work on it tomorrow.

[Frostburn]

I've dealt with the burst induced false positives by using 12 different feedback combs, each tuned to a note from low E to D sharp, and comparing their outputs.
Still no reliable chord detection though...
Each note is detected correctly but they still leak to the other banks because my combs cannot differentiate between a voice and the same voice a harmonic multiple higher.
Ie. combs think that freq == n*freq where n is a positive integer.

---

freqverifier.ck
Description:	What I've come up with so far. Detects individual notes but fails on chords.	Download (listen)
Filename:	freqverifier.ck
Filesize:	3.17 KB
Downloaded:	253 Time(s)

---

[Inventor]

Wow, I tried it and it works great! I see what you mean about combs not differentiating between frequency multiples. Let's say we have your 12-comb filter running and we get a positive indication on one comb. Can we then post-process the output of that comb with band-pass filters or better yet an FFT to determine the octave of the note?

You see, I avoided an FFT approach because of frequency selectivity, however once we know the accurate frequency then the FFT peaks can be approximate in frequency bin placement.

Alternatively can we post-process a comb output with your warbling technique that you demonstrated earlier in this thread to enhance the fundamental?

Well, I'm sure you are giving these options some thought already. One thing to keep in mind is that in the particular application we can have a pause between notes to clear out the comb filter's delay line, which is what I am doing in my Guitar Zero program. This might be helpful if you end up needing a computaionally prohibitive amount of circuitry. We could just create a single filter and give it time to clear out between notes.

Thanks so much for the collaboration, Frostburn, this should really help to improve Guitar Zero and be a fine addition to your toolbox as well!

[Frostburn]

All I can say right now that surprisingly a chord acts as if it was more than just the sum of it's parts. But maybe that's the whole point of melodic music. You don't hear individual notes, you hear harmonies. It's just that we need to make our algorithm hear the notes from the harmonies.

EDIT: One part of the problem lies in the fact that the same chords sound different on different instruments.

EDIT: I'm running out of resources here. 12 combs is around the max my laptop can take at 48kHz. If only ChucK would allow different sampling rates in the same piece. :) I think I'd need over 50 combs to do even basic chord detection with a certain harmonic intersection method I had in mind...

I need to think of something more economical with fewer combs online...

[Inventor]

Well, I think that perhaps you are solving the more difficult problem first instead of tackling the easier one first. Let's say that we *know* what note or notes are being provided by the guitar, and just build a detector for that. In this way we only need one comb filter instead of 12. That would be good enough to work with Guitar Zero, and we can later expand it to the general case. Good idea or no? Just a suggestion. Thanks again for the collaboration, you are more theoretical and more academic than I am, so I appreciate your abilities Frostburn.

[Frostburn]

Hmm... maybe I'm approaching this from the wrong angle.
Let's try it in the reverse.

Is this correct?
The input signal is a chord of three notes X, Y and Z if all of the following is true:
1) The input signal has a certain amount of energy so it is above the static noise threshold.
* We take frequencies corresponding to X out of the signal.
2) The signal still has some energy.
* We take Y out.
3) The signal still has some energy.
* We take Z out.
4) The signal has lost all of its energy. (Or is bellow a certain threshold).

If the process fails at 4) then the input is noise or an erroneous chord.
If it fails on 1), 2) or 3) then it didn't have enough notes to make the chord.

All we need is feedforward combs with alpha == -1.0 and some envelope followers.

Gotta try this one out.

EDIT: Oops. A signal of pure Z is considered a chord by the above algorithm sketch. Back to the drawing board.

EDIT2: How about this:
Stage one:
Test if the signal is pure X, Y or Z.
Stage two:
Test if the signal is a pure combination of ( X and Y ), (X and Z) or (Y and Z).
Stage three:
Test if the signal is pure (X, Y and Z)

Stages one and two need to fail but stage three needs to succeed for the signal to be a chord of (X, Y and Z).

[Inventor]

[Frostburn]

I think I got it!

The attached chuck file should congratulate you only if you manage to pull off a major chord of C-E-G.

---

chordverifier.ck
Description:	Chord verification using substracting feedforwad comb filters in series and in parallel.	Download (listen)
Filename:	chordverifier.ck
Filesize:	3.94 KB
Downloaded:	248 Time(s)

---

[Inventor]

[blue hell]

http://www.guitarconsultant.com/chords-cmaj.html any good? The note names are mentioned here along with the finger positions (I googled for "c-maj" without the quotes).

[Inventor]

nope, wasn't on there either. I changed the frequencies to match the open 6th, 5th, and 4th strings on the guitar and it worked like a charm! It recognized all combinations of the any two or any one, plus all three. All possibilities of a three-note chord uniquely identified. Excellent work, Frostburn! You da man!

Now how do we generalize it so that it works with up to 6-string chords? we can't do all combinations of six strings. Maybe we just exclude the single chords and anything else passes? No, that's not it. I am confused. Unfuzzle me brain please!