\section{Introduction}
The primary medium for music distribution is rapidly changing from physical
-media to digital media. The \gls{IFPI} stated that about $43\%$ of music
-revenue arises from digital distribution. Another $39\%$ arises from the
+media to digital media. In 2016 the \gls{IFPI} stated that about $50\%$ of
+music revenue arises from digital distribution. Another $34\%$ arises from the
physical sale and the remaining $16\%$ is made through performance and
-synchronisation revenieus. The overtake of digital formats on physical formats
+synchronisation revenues. The overtake of digital formats on physical formats
took place somewhere in 2015. Moreover, ever since twenty years the music
-industry has seen significant growth
+industry has seen significant growth
again\footnote{\url{http://www.ifpi.org/facts-and-stats.php}}.
There has always been an interest in lyrics to music alignment to be used in
-for example karaoke. As early as in the late 1980s karaoke machines were
-available for consumers. While the lyrics for the track are almost always
-available, an alignment is not and it involves manual labour to create such an
-alignment.
+for example karaoke. As early as in the late 1980s, karaoke machines became
+available for consumers. Lyrics for tracks are in almost all cases amply
+available. However, a temporal alignment of the lyrics is not and creating it
+involves manual labour.
-A lot of this musical distribution goes via non-official channels such as
-YouTube\footnote{\url{https://youtube.com}} in which fans of the performers
-often accompany the music with synchronized lyrics. This means that there is an
-enormous treasure of lyrics-annotated music available but not within our reach
-since the subtitles are almost always hardcoded into the video stream and thus
-not directly usable as data. Because of this interest it is very useful to
-device automatic techniques for segmenting instrumental and vocal parts of a
-song, apply forced alignment or even lyrics recognition on the audio file.
+A lot of the current day music distribution goes via non-official channels
+such as YouTube\footnote{\url{https://youtube.com}} in which fans of the
+performers often accompany the music with synchronized lyrics. This means that
+there is an enormous treasure of lyrics-annotated music available. However, the
+data is not within our reach since the subtitles are almost always hardcoded
+into the video stream and thus not directly accessible as data. It sparks the
+ideas for creating automatic techniques for segmenting instrumental and vocal
+parts of a song, apply forced temporal alignment or possible even apply lyrics
+recognition audio data.
These techniques are heavily researched and working systems have been created
-for segmenting audio and even forced alignment (e.g.\ LyricSynchronizer%
-\cite{fujihara_lyricsynchronizer:_2011}). However, these techniques are designed
-to detect a clean singing voice and have not been testen on so-called
-\emph{extended vocal techniques} such as grunting or growling. Growling is
-heavily used in extreme metal genres such as \gls{dm} but it must be noted that
-grunting is not a technique only used in extreme metal styles. Similar or equal
-techniques have been used in \emph{Beijing opera}, Japanese \emph{Noh} and but
-also more western styles like jazz singing by Louis
-Armstrong\cite{sakakibara_growl_2004}. It might even be traced back to viking
-times. For example, an arab merchant visiting a village in Denmark wrote in the
-tenth century\cite{friis_vikings_2004}:
+for segmenting audio and even forced temporal alignment (e.g.\
+LyricSynchronizer~\cite{fujihara_lyricsynchronizer:_2011}). However, these
+techniques are designed to detect a clean singing voice and have not been
+tested on so-called \emph{extended vocal techniques} such as grunting or
+growling. Growling is heavily used in extreme metal genres such as \gls{dm} but
+it must be noted that grunting is not a technique only used in extreme metal
+styles. Similar or equal techniques have been used in \emph{Beijing opera},
+Japanese \emph{Noh} and but also more western styles like jazz singing by Louis
+Armstrong~\cite{sakakibara_growl_2004}. It might even be traced back to
+viking times. For example, an arab merchant visiting a village in Denmark wrote
+in the tenth century~\cite{friis_vikings_2004}:
\begin{displayquote}
Never before I have heard uglier songs than those of the Vikings in
\section{Related work}
Applying speech related processing and classification techniques on music
already started in the late 90s. Saunders et al.\ devised a technique to
-classify audio in the categories \emph{Music} and \emph{Speech}. They was found
-that music has different properties than speech. Music has more bandwidth,
-tonality and regularity. Multivariate Gaussian classifiers were used to
-discriminate the categories with an average performance of $90\%%
-$\cite{saunders_real-time_1996}.
+classify audio in the categories \emph{Music} and \emph{Speech}. They found
+that music has different properties than speech. Music uses a wider spectral
+bandwidth in which events happen. Music contains more tonality and rhythm.
+Multivariate Gaussian classifiers were used to discriminate the categories with
+an average accuracy of $90\%$~\cite{saunders_real-time_1996}.
Williams and Ellis were inspired by the aforementioned research and tried to
-separate the singing segments from the instrumental
-segments\cite{williams_speech/music_1999}. This was later verified by
-Berenzweig and Ellis\cite{berenzweig_locating_2001}. The latter became the de
+separate the singing segments from the instrumental segments~%
+\cite{williams_speech/music_1999}. Their results were later verified by
+Berenzweig and Ellis~\cite{berenzweig_locating_2001}. The latter became the de
facto literature on singing voice detection. Both show that features derived
-from \gls{PPF} such as energy and distribution are highly effective in
-separating speech from non-speech signals such as music. The data used was
-already segmented.
+from \gls{PPF} such as energy are highly effective in separating speech from
+non-speech signals such as music. The data used in the experiments was
+segmented in to segments that only contained data from one class. The
+classifier determined the classper sample.
Later, Berenzweig showed singing voice segments to be more useful for artist
classification and used an \gls{ANN} (\gls{MLP}) using \gls{PLP} coefficients
-to detect a singing voice\cite{berenzweig_using_2002}. Nwe et al.\ showed that
+to detect a singing voice~\cite{berenzweig_using_2002}. Nwe et al.\ showed that
there is not much difference in accuracy when using different features founded
in speech processing. They tested several features and found accuracies differ
-less that a few percent. Moreover, they found that others have tried to tackle
+less than a few percent. Moreover, they found that others have tried to tackle
the problem using myriads of different approaches such as using \gls{ZCR},
\gls{MFCC} and \gls{LPCC} as features and \glspl{HMM} or \glspl{GMM} as
-classifiers\cite{nwe_singing_2004}.
+classifiers~\cite{nwe_singing_2004}.
Fujihara et al.\ took the idea to a next level by attempting to do \gls{FA} on
music. Their approach is a three step approach. The first step is reducing the
accompaniment levels, secondly the vocal segments are separated from the
non-vocal segments using a simple two-state \gls{HMM}. The chain is concluded
by applying \gls{Viterbi} alignment on the segregated signals with the lyrics.
-The system showed accuracy levels of $90\%$ on Japanese music%
-\cite{fujihara_automatic_2006}. Later they improved hereupon%
+The system showed accuracy levels of $90\%$ on Japanese music~%
+\cite{fujihara_automatic_2006}. Later they improved hereupon~%
\cite{fujihara_three_2008} and even made a ready to use karaoke application
-that can do the this online\cite{fujihara_lyricsynchronizer:_2011}.
+that can do the temporal lyrics alignment online~%
+\cite{fujihara_lyricsynchronizer:_2011}.
Singing voice detection can also be seen as a binary genre recognition problem.
Therefore the techniques used in that field might be of use. Genre recognition
has a long history that can be found in the survey by
-Sturm\cite{sturm_survey_2012}. It must be noted that of all the $485$ papers
+Sturm~\cite{sturm_survey_2012}. It must be noted that of all the $485$ papers
cited by Sturm only one master thesis is applying genre recognition on heavy
-metal genres\cite{tsatsishvili_automatic_2011}.
+metal genres~\cite{tsatsishvili_automatic_2011}.
Singing voice detection has been tried on less conventional styles in the past.
Dzhambazov et al.\ proposed to align long syllables in Beijing Opera to the
-audio\cite{dzhambazov_automatic_2016}. Beijing Opera sometimes contains
+audio~\cite{dzhambazov_automatic_2016}. Beijing Opera sometimes contains
growling like vocals. Dzhambazov also tried aligning lyrics to audio in
-classical Turkish music\cite{dzhambazov_automatic_2014}.
+classical Turkish music~\cite{dzhambazov_automatic_2014}.
\section{Research question}
It is debatable whether the aforementioned techniques work because the
spectral properties of a growling voice is different from the spectral
-properties of a clean singing voice. It has been found that growling voices
-have less prominent peaks in the frequency representation and are closer to
-noise than clean singing\cite{kato_acoustic_2013}. This leads us to the
+properties of a clean singing voice. It has been found that growling-like
+vocals have less prominent peaks in the frequency representation and are closer
+to noise than clean singing~\cite{kato_acoustic_2013}. This leads us to the
research question:
\begin{center}\em%
- Are standard \gls{ANN} based techniques for singing voice detection
- suitable for non-standard musical genres like \gls{dm} and \gls{dom}?
+ Are standard techniques for singing voice detection suitable for
+ non-standard musical genres containing extreme vocal styles?
\end{center}
repositories / asr1617.git / blobdiff