REPET
Implementation
Our motive for using the REPET algorithm was to help us better understand how the CNN was classifying each song. Thus, we chose a subset of songs from the entire dataset and used the REPET algorithm to separate them into foreground and background.
Initially, we tried to automate the process by using Python to split and join different audio files, but we soon realized that the two songs could have mismatched beats per minute, which may have affected their classification. Thus, we decided to choose a fewer number of songs and manually adjust them to minimize the discrepancies between the foreground and background.
We combined the constituent parts of songs of the following genres (never matching foreground and background of the same genre): Electronic, Folk, Hip-Hop, Pop, and Rock. In the process, we matched the beats per minute of the foreground to that of the background in the combination using a music production tool called Ableton. We subsequently classified these mixed music tracks with our CNN and compared the predicted genre to the genres of the foreground and background parts.
Results
We mainly looked at the effects of the selected background on classification and made some interesting discoveries. The most interesting findings are as follow:
First, mixture songs created using background tracks from Electronic music were always classified as the genre of the foreground track, barring one song with a Folk foreground which was classified as Rock. Combinations composed of a Hip-hop background were only ever classified as Hip-hop or Electronic. Next, mixtures created using backgrounds from Folk or Pop music were classified as Rock 75-87.5% of the time, respectively. Lastly, mixtures with a Rock background track were mostly identified as Rock, except when the foreground tracks were from Hip-hop songs. In that case, the songs were always classified as Hip-hop.
When Hip-hop served as the foreground piece of many combinations, it was almost always the case that the CNN classified the song as Hip-hop, likely due to the unique lyrical style not found in other music that we chose.
Mixtures with Pop or Folk backgrounds were classified as Rock most likely due to the mixture of gradual guitar melodies with overlaid drums. Songs with very strong drum noises in the background also classified as Hip-hop or Electronic. We believe this is due to the similarity of songs in the FMA dataset, in which many songs share instruments and musical patterns. Furthermore, the genre descriptors did not necessarily agree with how we believed the music ought to be labelled based on modern trends.