Key detection and pitch shifting.

[rryan]

This branch adds basic key detection and pitch shifting to Mixxx. It also switches to using librubberband for tempo and pitch shifting instead of libsoundtouch which to my ear is a huge sound quality win in exchange for a moderate CPU increase.

Features / Improvements:

• librubberband for tempo and pitch shifting.
• Key analysis via our VAMP plugin support. Currently uses the Queen Mary Key Detector but libkeyfinder could easily be added.
• User notation preferences so users can switch according to their preferences.
• Global dominant key and the full list of track key changes are saved in the database.
• Key syncing based on the Circle of Fifths.
• Key display widgets.
• Support for display and parsing of OpenKey, Lancelot, and Traditional notations.
• No more mutex locks in the scalers.

Thanks to @ktec for writing the initial version of the preferences (forked from DlgPrefBeats) and Varun Jewalikar for writing the initial key analyser to use the VAMP plugin. Note that this branch is forked from Keith's features_key_preferences branch but I reverted everything but AnalyserKey and DlgPrefKey.

@esbrandt -- I will revert the res/skins change before merge.

[rryan]

Enabling and disabling keylock now adjusts the pitch so that the effective key does not change. This results in the pitch slider jumping around a lot (which will screw up MIDI controllers if soft-takeover is not enabled). Maybe this could become a preference option at some point? I also reverted the skin changes.

[rryan]

Playing with this new behavior, I realize that I often do this:

• Load a track.
• Change the rate to beatmatch it to the other deck.
• Realize it sounds bad with the pitch changed.
• Enable keylock to return it to its natural pitch.

My expectation built up from all previous Mixxx versions has been that hitting keylock returns the track to its natural pitch. The new behavior actually locks the track at the pitch I didn't want.

The new required series of steps (if I only discover I don't want the pitch after beatsyncing):

• Load a track.
• Change the rate to beatmatch it to the other deck.
• Realize it sounds bad with the pitch changed.
• Enable keylock to make the pitch independent of the speed slider. (pitch stays the same)
• Right-click the pitch knob/fader (whichever UI element we end up with) to return the pitch to 0.

[daschuer]

If the key detector returns the number 13.0 there's no way to know if that's 12.0 plus a 100 cent shift. And 12 to 13 crosses the major/minor barrier so information is corrupted.

The key detector must not return 13.0 in this case. It is 1 again.

Key Name         Key value  Cents from Tonic C 
INVALID          0          -     
C_MAJOR          1          0
D_FLAT_MAJOR     2          100
B_MAJOR          12         1100
B_MAJOR +100                           <- invalid its a C_MAJOR
B_MAJOR +49      12.49      1149
B_MAJOR +51                            <- invalid its a C_MAJOR -49 
B_MAJOR +90                            <- invalid its a C_MAJOR -10 
C_MAJOR -49      0.51       1159 
C_MINOR          13         0
C_MINOR -10      12.90      1190
C_Minor -49      12.51      1159 

In this example it wraps at 12.49. We may consider to wrap at 12.99. Its only a case of definition.
We have to be sure, that the chord is not changes by the concert pitch.
Of course you can play C_MINOR notes on an instrument with a concert pitch of A440 - 100 cents -> A415.
But If we translate this to the Mixxx world with A440, it becomes a B_MINOR track.

The question I have is whether the extra resolution in the QM key detector is actually meaningfully predictive of the song's actual concert pitch. If it is then we might want to hack up the QM plugin to expose what it "thinks" the concert pitch of the song is.

Yes that is really the "jumping point" and an interesting question. If we look at how the key detection work, it should be meaningful because the track chroma is turned on the chroma of a reference chord to find a Maximum. The initial tuning value changes the start angel, thats all.

[daschuer]

Thank you @cpill0789

While [tONaRT] key detection is robust across different musical genres etc., the algo-
rithm expects the input signal to be homogenous, i.e. to contain only one key without
(too much) modulations. The standard pitch detection has an accuracy of app. 1-2Hz
at 440Hz

It looks like [tONaRT] could be a nice addition in future version of Mixxx, but it is not a FOSS library :-(

OK, here is what I have learned from our discussion:

1. If we assume a constant concert pitch of a track (this should be the case if the guitar player does not use the fender to much) it seams to be reasonable to store a single global concert pitch independent from the keys.
   This keeps the stored data small and seems to be a good compromise
2. Finally it is up to RJ to deside what is the best way to user all the pitch value in the code. I would like a floating point representation.
3. the QM 33 cent resolution is to big to detect the concert pitch. A Key of e.g 1.33 could be more a hind that the track is not western music or totally off pitch.

[danlin]

some side information:
you talk about [tONaRt] as i know Mixed in Key use this Lib (with some cloud Addons to faster detect stuff)

http://www.djtechtools.com/2012/01/26/key-detection-software-showdown-2012-edition/

and Rapid Evolution is now Open Source! (good key detection)
https://github.com/djqualia/RapidEvolution3?source=cc

[ywwg]

Does anyone know what traktor or serato do when keylock is pushed? I definitely have had cases where I want the pitch to stay the same when switching from vinyl style to keylock, but I can't because the slider isn't at zero. But I also understand not wanting to lose the beatmatch. A tuning knob would fix the problem for sure but adding yet another knob is hard to justify.

Maybe instead of a preference we can have a separate CO, key_maintain -- which holds the current pitch instead of resetting to natural pitch? This is harder for users to configure, I guess.

So all of the options I see:

1. Keep current behavior, do not support going from vinyl-style to keylock and maintaining pitch
2. Always maintain pitch, do not support returning to natural pitch without losing beatmatch
3. A new preference to choose from the above options
4. A new bindable CO that allows second behavior (could be attached to long-press perhaps?)
5. The second behavior plus a new tuning knob so the pitch can be reset to natural if the users wishes it.

[rryan]

Ok, I think we're all agreed on storing concert_pitch as a float alongside the detected key!

My main question now is whether this should be part of the key detection result.

Some key detectors may not detect concert pitch (such as keyfinder and QM keydetect) but some do (such as tonart). Also, there may be some standalone concert pitch detector out there. To make matters more complicated, all key detectors we've seen also take a concert pitch as an input to allow compensating for a known different-than-A440 concert pitch.

So I think it's safe to say these are separate analysis processes. This makes me think we should add a totally separate analysis result for concert pitch and not store it in the key detection results protocol buffer.

For example, we may want to let the user manually enter the concert pitch before running the key detector. This would be awkward if that user-entered concert pitch data was stored in an empty key detection result proto. When the key detector finishes, would we merge it with the existing key detection proto or replace it?

So, I'll leave concert pitch detection/manual-entry and compensation for a future project. Sound good?

[rryan]

RE: Owen -- lets go with the current behavior and see if anyone complains in beta testing. We can add a preference option and manual COs if requested. I just want to avoid adding complexity before it's been requested by a user :).

[cpill0789]

@rryan: sounds good to me.

[daschuer]

I also support the idea of an independent concert_pitch.

It should be limited to +- 50 cents of A440 (the Mixxx concert pitch) to avoid changing keys by concert pitch.

Use cases:

1. improve the key detection by tune it to a known concert pitch (e.g. Boston Symphony Orchestra ;-))
2. It should be easy changeable after you have made two tracks fit by ear
3. It should be set by detection result of concert pitch detectors, but lockable to avoid results from ear.

For me all that is not required for a merge, but the merged version should be able to support it later.

[rryan]

Ugh, adding a preference option probably means adding an "Engine" or "Tempo / Pitch Control" preference tab. The setting doesn't really fit in any existing tab.

Since the branch is stable and doesn't require any more data model changes I'm going to merge it. UI issues and preference options can be fixed in master.

I filed a bug for the preference option: https://bugs.launchpad.net/mixxx/+bug/1258560

Thanks for all the comments! (we just barely made it to centi-thread)

[ywwg]

hooray!