%&asr
\usepackage[nonumberlist,acronyms]{glossaries}
\makeglossaries%
\newacronym{ANN}{ANN}{Artificial Neural Network}
\newacronym{HMM}{HMM}{Hidden Markov Model}
\newacronym{GMM}{GMM}{Gaussian Mixture Models}
\newacronym{DHMM}{DHMM}{Duration-explicit \acrlong{HMM}}
\newacronym{HTK}{HTK}{\acrlong{HMM} Toolkit}
\newacronym{FA}{FA}{Forced alignment}
\newacronym{MFC}{MFC}{Mel-frequency cepstrum}
\newacronym{MFCC}{MFCC}{\acrlong{MFC} coefficient}
\newacronym{IFPI}{IFPI}{International Federation of the Phonographic Industry}
\newglossaryentry{dm}{name={Death Metal},
	description={is an extreme heavy metal music style with growling vocals and
	pounding drums}}

\begin{document}
\frontmatter{}

\maketitleru[
	course={(Automatic) Speech Recognition},
	institute={Radboud University Nijmegen},
	authorstext={Author:},
	pagenr=1]
\listoftodos[Todo]

\tableofcontents

%Glossaries
%\glsaddall{}
%\printglossaries

\mainmatter{}
%Berenzweig and Ellis use acoustic classifiers from speech recognition as a
%detector for singing lines.  They achive 80\% accuracy for forty 15 second
%exerpts. They mention people that wrote signal features that discriminate
%between speech and music. Neural net
%\glspl{HMM}~\cite{berenzweig_locating_2001}.
%
%In 2014 Dzhambazov et al.\ applied state of the art segmentation methods to
%polyphonic turkish music, this might be interesting to use for heavy metal.
%They mention Fujihara (2011) to have a similar \gls{FA} system. This method uses
%phone level segmentation, first 12 \gls{MFCC}s. They first do vocal/non-vocal
%detection, then melody extraction, then alignment. They compare results with
%Mesaros \& Virtanen, 2008~\cite{dzhambazov_automatic_2014}. Later they
%specialize in long syllables in a capella. They use \glspl{DHMM} with
%\glspl{GMM} and show that adding knowledge increases alignment (bejing opera
%has long syllables)~\cite{dzhambazov_automatic_2016}.
%


%Introduction, leading to a clearly defined research question
\chapter{Introduction}
\section{Introduction}
The primary medium for music distribution is rapidly changing from physical
media to digital media. The \gls{IFPI} stated that about $43\%$ of music
revenue rises from digital distribution. Another $39\%$ arises from the
physical sale and the remaining $16\%$ is made through performance and
synchronisation revenieus. The overtake of digital formats on physical formats
took place somewhere in 2015. Moreover, ever since twenty years the music
industry has seen significant growth 
again~\footnote{\url{http://www.ifpi.org/facts-and-stats.php}}.

There has always been an interest in lyrics to music alignment to be used in
for example karaoke. As early as in the late 1980s karaoke machines were
available for consumers. While the lyrics for the track are almost always
available, a alignment is not and it involves manual labour to create such an
alignment.

A lot of this musical distribution goes via non-official channels such as
YouTube~\footnote{\url{https://youtube.com}} in which fans of the performers
often accompany the music with synchronized lyrics. This means that there is an
enormous treasure of lyrics-annotated music available but not within our reach
since the subtitles are almost always hardcoded into the video stream and thus
not directly usable as data. Because of this interest it is very useful to
device automatic techniques for segmenting instrumental and vocal parts of a
song, apply forced alignment or even lyrics recognition on the audio file.

Such techniques are heavily researched and working systems have been created.
However, these techniques are designed to detect a clean singing voice and have
not been testen on so-called \emph{extended vocal techniques} such as grunting
or growling. Growling is heavily used in extreme metal genres such as \gls{dm}
but it must be noted that grunting is not a technique only used in extreme
metal styles. Similar or equal techniques have been used in \emph{Beijing
opera}, Japanese \emph{Noh} and but also more western styles like jazz singing
by Louis Armstrong~\cite{sakakibara_growl_2004}. It might even be traced back
to viking times. For example, an arab merchant visiting a village in Denmark
wrote in the tenth century~\cite{friis_vikings_2004}:

\begin{displayquote}
	Never before I have heard uglier songs than those of the Vikings in
	Slesvig. The growling sound coming from their throats reminds me of dogs
	howling, only more untamed.
\end{displayquote}

\section{\gls{dm}}

%Literature overview / related work
\section{Related work}
The field of applying standard speech processing techniques on music started in
the late 90s~\cite{saunders_real-time_1996,scheirer_construction_1997} and it
was found that music has different discriminating features compared to normal
speech.

Berenzweig and Ellis expanded on the aforementioned research by trying to
separate singing from instrumental music\cite{berenzweig_locating_2001}.

\todo{Incorporate this in literary framing}%
~\cite{fujihara_automatic_2006}%
~\cite{fujihara_lyricsynchronizer:_2011}%
~\cite{fujihara_three_2008}%
~\cite{mauch_integrating_2012}%
~\cite{mesaros_adaptation_2009}%
~\cite{mesaros_automatic_2008}%
~\cite{mesaros_automatic_2010}%
~%\cite{muller_multimodal_2012}%
~\cite{pedone_phoneme-level_2011}%
~\cite{yang_machine_2012}%



\section{Research question}
It is discutable whether the aforementioned techniques work because the
spectral properties of a growling voice is different from the spectral
properties of a clean singing voice. It has been found that growling voices
have less prominent peaks in the frequency representation and are closer to
noise then clean singing\cite{kato_acoustic_2013}. This leads us to the
research question:

\begin{center}\em%
	Are standard \gls{ANN} based techniques for singing voice detection
	suitable for non-standard musical genres like \gls{dm}.
\end{center}

\chapter{Methods}
%Methodology

%Experiment(s) (set-up, data, results, discussion)
\section{Data \& Preprocessing}
To run the experiments data has been collected from several \gls{dm} albums.
The exact data used is available in Appendix~\ref{app:data}. The albums are
extracted from the audio CD and converted to a mono channel waveform with the
correct samplerate \emph{SoX}~\footnote{\url{http://sox.sourceforge.net/}}.
When the waveforms are finished they are converted to \glspl{MFCC} vectors
using the \emph{python\_speech\_features}%
~\footnote{\url{https://github.com/jameslyons/python_speech_features}} package.
All these steps combined results in thirteen tab separated features per line in
a file for every source file. Every file is annotated using
Praat~\cite{boersma_praat_2002} where the utterances are manually aligned to
the audio. Examples of utterances are shown in
Figures~\ref{fig:bloodstained,fig:abominations}. It is clearly visible that
within the genre of death metal there are a different spectral patterns
visible.

\begin{figure}[ht]
	\centering
	\includegraphics[width=.7\linewidth]{cement}
	\caption{A vocal segment of the \emph{Cannibal Corpse} song
		\emph{Bloodstained Cement}}\label{fig:bloodstained}
\end{figure}

\begin{figure}[ht]
	\centering
	\includegraphics[width=.7\linewidth]{abominations}
	\caption{A vocal segment of the \emph{Disgorge} song
		\emph{Enthroned Abominations}}\label{fig:abominations}
\end{figure}

The data is collected from two\todo{more in the future}\ studio albums. The first
band is called \emph{Cannibal Corpse} and has been producing \gls{dm} for almost
25 years and have been creating the same type every album. The singer of
\emph{Cannibal Corpse} has a very raspy growls and the lyrics are quite
comprehensible. The second band is called \emph{Disgorge} and make even more
violent music. The growls of the lead singer sound more like a coffee grinder
and are more shallow. The lyrics are completely incomprehensible and therefore
some parts are not annotated with lyrics because it was too difficult to hear
what was being sung.

\section{Methods}
\todo{To remove in final thesis}
The initial planning is still up to date. About one and a half album has been
annotated and a framework for setting up experiments has been created.
Moreover, the first exploratory experiments are already been executed and
promising. In April the experimental dataset will be expanded and I will try to
mimic some of the experiments done in the literature to see whether it performs
similar on Death Metal
\begin{table}[ht]
	\centering
	\begin{tabular}{cll}
		\toprule
		Month & Description\\
		\midrule
		March
			& Preparing the data\\
			& Preparing an experiment platform\\
			& Literature research\\
		April
			& Running the experiments\\
			& Fiddle with parameters\\
			& Explore the possibilities for forced alignment\\
		May
			& Write up the thesis\\
			& Possibly do forced alignment\\
		June
			& Finish up thesis\\
			& Wrap up\\
		\bottomrule
	\end{tabular}
	\caption{Outline}
\end{table}

\todo{Explain why MFCC and which parameters}
\todo{Spectrals might be enough, no decorrelation}

\section{Experiments}

\section{Results}


\chapter{Conclusion \& Discussion}
%Discussion section
\todo{Novelty}
\todo{Weaknesses}
\todo{Dataset is not very varied but\ldots}

\todo{Doom metal}
%Conclusion section
%Acknowledgements
%Statement on authors' contributions
%(Appendices)
\appendix
\chapter{Experimental data}\label{app:data}
\begin{table}[h]
	\centering
	\begin{tabular}{cllll}
		\toprule
		Num. & Artist & Album & Song & Duration\\
		\midrule
		00 & Cannibal Corpse & A Skeletal Domain & High Velocity Impact Spatter & 04:06.91\\
		01 & Cannibal Corpse & A Skeletal Domain & Sadistic Embodiment & 03:17.31\\
		02 & Cannibal Corpse & A Skeletal Domain & Kill or Become & 03:50.67\\
		03 & Cannibal Corpse & A Skeletal Domain & A Skeletal Domain & 03:38.77\\
		04 & Cannibal Corpse & A Skeletal Domain & Headlong Into Carnage & 03:01.25\\
		05 & Cannibal Corpse & A Skeletal Domain & The Murderer's Pact & 05:05.23\\
		06 & Cannibal Corpse & A Skeletal Domain & Funeral Cremation & 03:41.89\\
		07 & Cannibal Corpse & A Skeletal Domain & Icepick Lobotomy & 03:16.24\\
		08 & Cannibal Corpse & A Skeletal Domain & Vector of Cruelty & 03:25.15\\
		09 & Cannibal Corpse & A Skeletal Domain & Bloodstained Cement & 03:41.99\\
		10 & Cannibal Corpse & A Skeletal Domain & Asphyxiate to Resuscitate & 03:47.40\\
		11 & Cannibal Corpse & A Skeletal Domain & Hollowed Bodies & 03:05.80\\
		12 & Disgorge & Parallels of Infinite Torture & Revealed in Obscurity & 05:13.20\\
		13 & Disgorge & Parallels of Infinite Torture & Enthroned Abominations & 04:05.39\\
		14 & Disgorge & Parallels of Infinite Torture & Atonement & 02:57.36\\
		15 & Disgorge & Parallels of Infinite Torture & Abhorrent Desecration of Thee Iniquity & 04:17.20\\
		16 & Disgorge & Parallels of Infinite Torture & Forgotten Scriptures & 02:01.72\\
		17 & Disgorge & Parallels of Infinite Torture & Descending Upon Convulsive Devourment & 04:38.85\\
		18 & Disgorge & Parallels of Infinite Torture & Condemned to Sufferance & 04:57.59\\
		19 & Disgorge & Parallels of Infinite Torture & Parallels of Infinite Torture & 05:03.33\\
		20 & Disgorge & Parallels of Infinite Torture & Asphyxiation of Thee Oppressed & 05:42.37\\
		21 & Disgorge & Parallels of Infinite Torture & Ominous Sigils of Ungodly Ruin & 04:59.15\\
		\bottomrule
	\end{tabular}
	\caption{Songs used in the experiments}
\end{table}

\bibliographystyle{ieeetr}
\bibliography{asr}
\end{document}
