%Experiment(s) (set-up, data, results, discussion)
\section{Data \& Preprocessing}
-To run the experiments data has been collected from several \gls{dm} albums.
-The exact data used is available in Appendix~\ref{app:data}. The albums are
-extracted from the audio CD and converted to a mono channel waveform with the
-correct samplerate \emph{SoX}\footnote{\url{http://sox.sourceforge.net/}}.
-Every file is annotated using
-Praat\cite{boersma_praat_2002} where the utterances are manually aligned to
-the audio. Examples of utterances are shown in
+To answer the research question, several experiments have been performed. Data
+has been collected from several \gls{dm} and \gls{dom} albums. The exact data
+used is available in Appendix~\ref{app:data}. The albums are extracted from the
+audio CD and converted to a mono channel waveform with the correct samplerate
+utilizing \emph{SoX}\footnote{\url{http://sox.sourceforge.net/}}. Every file
+is annotated using Praat~\cite{boersma_praat_2002} where the lyrics are
+manually aligned to the audio. Examples of utterances are shown in
Figure~\ref{fig:bloodstained} and Figure~\ref{fig:abominations} where the
-waveform, $1-8000$Hz spectrals and annotations are shown. It is clearly visible
-that within the genre of death metal there are a different spectral patterns
-visible.
+waveform, $1-8000$Hz spectrogram and annotations are shown. It is clearly
+visible that within the genre of death metal there are different spectral
+patterns visible over time.
\begin{figure}[ht]
\centering
\includegraphics[width=.7\linewidth]{cement}
- \caption{A vocal segment of the \emph{Cannibal Corpse} song
+ \caption{A vocal segment of the Cannibal Corpse song
\emph{Bloodstained Cement}}\label{fig:bloodstained}
\end{figure}
\begin{figure}[ht]
\centering
\includegraphics[width=.7\linewidth]{abominations}
- \caption{A vocal segment of the \emph{Disgorge} song
+ \caption{A vocal segment of the Disgorge song
\emph{Enthroned Abominations}}\label{fig:abominations}
\end{figure}
-The data is collected from three studio albums. The
-first band is called \emph{Cannibal Corpse} and has been producing \gls{dm} for
-almost 25 years and have been creating the same type every album. The singer of
-\emph{Cannibal Corpse} has a very raspy growls and the lyrics are quite
-comprehensible. The vocals produced by \emph{Cannibal Corpse} are bordering
-regular shouting.
-
-The second band is called \emph{Disgorge} and make even more violently sounding
-music. The growls of the lead singer sound like a coffee grinder and are more
-shallow. In the spectrals it is clearly visible that there are overtones
-produced during some parts of the growling. The lyrics are completely
+The data is collected from three studio albums. The first band is called
+\gls{CC} and has been producing \gls{dm} for almost 25 years and has been
+creating albums with a consistent style. The singer of \gls{CC} has a very
+raspy growl and the lyrics are quite comprehensible. The vocals produced by
+\gls{CC} are very close to regular shouting.
+
+The second band is called \gls{DG} and makes even more violently
+sounding music. The growls of the lead singer sound like a coffee grinder and
+are sound less full. In the spectrals it is clearly visible that there are
+overtones produced during some parts of the growling. The lyrics are completely
incomprehensible and therefore some parts were not annotated with the actual
-lyrics because it was not possible what was being sung.
+lyrics because it was impossible to hear what was being sung.
+
+The third band --- originating from Moscow --- is chosen bearing the name
+\gls{WDISS}. This band is a little odd compared to the previous \gls{dm} bands
+because they create \gls{dom}. \gls{dom} is characterized by the very slow
+tempo and low tuned guitars. The vocalist has a very characteristic growl and
+performs in several Muscovite bands. This band also stands out because it uses
+piano's and synthesizers. The droning synthesizers often operate in the same
+frequency as the vocals.
-Lastly a band from Moscow is chosen bearing the name \emph{Who Dies in
-Siberian Slush}. This band is a little odd compared to the previous \gls{dm}
-bands because they create \gls{dom}. \gls{dom} is characterized by the very
-slow tempo and low tuned guitars. The vocalist has a very characteristic growl
-and performs in several moscovian bands. This band also stands out because it
-uses piano's and synthesizers. The droning synthesizers often operate in the
-same frequency as the vocals.
+Additional details about the dataset such are listed in
+Appendix~\ref{app:data}. The data is labeled as singing and instrumental and
+labeled per band. The distribution for this is shown in
+Table~\ref{tbl:distribution}.
+\begin{table}[H]
+ \centering
+ \begin{tabular}{lcc}
+ \toprule
+ Instrumental & Singing\\
+ \midrule
+ 0.59 & 0.41\\
+ \bottomrule
+ \end{tabular}
+ \quad
+ \begin{tabular}{lcccc}
+ \toprule
+ Instrumental & \gls{CC} & \gls{DG} & \gls{WDISS}\\
+ \midrule
+ 0.59 & 0.16 & 0.19 & 0.06\\
+ \bottomrule
+ \end{tabular}
+ \caption{Proportional data distribution}\label{tbl:distribution}
+\end{table}
-\section{\gls{MFCC} Features}
+\section{Mel-frequencey Cepstral Features}
The waveforms in itself are not very suitable to be used as features due to the
-high dimensionality and correlation. Therefore we use the often used
-\glspl{MFCC} feature vectors which has shown to be
-suitable\cite{rocamora_comparing_2007}. The actual conversion is done using the
-\emph{python\_speech\_features}%
-\footnote{\url{https://github.com/jameslyons/python_speech_features}} package.
-
-\gls{MFCC} features are nature inspired and built incrementally in several
-steps.
+high dimensionality and correlation in the temporal domain. Therefore we use
+the often used \glspl{MFCC} feature vectors which have shown to be suitable in
+speech processing~\cite{rocamora_comparing_2007}. It has also been found that
+altering the mel scale to better suit singing does not yield a better
+performance~\cite{you_comparative_2015}. The actual conversion is done using
+the \emph{python\_speech\_features}\footnote{\url{%
+https://github.com/jameslyons/python_speech_features}} package.
+
+\gls{MFCC} features are inspired by human auditory processing and are
+created from a waveform incrementally using several steps:
\begin{enumerate}
\item The first step in the process is converting the time representation
- of the signal to a spectral representation using a sliding window with
- overlap. The width of the window and the step size are two important
- parameters in the system. In classical phonetic analysis window sizes
- of $25ms$ with a step of $10ms$ are often chosen because they are small
- enough to only contain subphone entities. Singing for $25ms$ is
- impossible so it is arguable that the window size is very small.
+ of the signal to a spectral representation using a sliding analysis
+ window with overlap. The width of the window and the step size are two
+ important parameters in the system. In classical phonetic analysis
+ window sizes of $25ms$ with a step of $10ms$ are often chosen because
+ they are small enough to contain just one subphone event.
\item The standard \gls{FT} gives a spectral representation that has
linearly scaled frequencies. This scale is converted to the \gls{MS}
- using triangular overlapping windows.
- \item
+ using triangular overlapping windows to get a more tonotopic
+ representation trying to match the actual representation of the cochlea
+ in the human ear.
+ \item The \emph{Weber-Fechner} law describes how humans perceive physical
+ magnitudes\footnote{Fechner, Gustav Theodor (1860). Elemente der
+ Psychophysik}. They found that energy is perceived in logarithmic
+ increments. This means that twice the amount of energy does not mean
+ twice the amount of perceived loudness. Therefore we take the logarithm
+ of the energy or amplitude of the \gls{MS} spectrum to closer match the
+ human hearing.
+ \item The amplitudes of the spectrum are highly correlated and therefore
+ the last step is a decorrelation step. \Gls{DCT} is applied on the
+ amplitudes interpreted as a signal. \Gls{DCT} is a technique of
+ describing a signal as a combination of several primitive cosine
+ functions.
\end{enumerate}
+The default number of \gls{MFCC} parameters is twelve. However, often a
+thirteenth value is added that represents the energy in the analysis window.
+The $c_0$ is chosen is this example. $c_0$ is the zeroth \gls{MFCC}. It
+represents the average over all \gls{MS} bands. Another option would be
+$\log{(E)}$ which is the logarithm of the raw energy of the sample.
+
+\section{Artificial Neural Network}
+The data is classified using standard \gls{ANN} techniques, namely \glspl{MLP}.
+The classification problems are only binary or four-class problems so it is
+interesting to see where the bottleneck lies; how abstract can the abstraction
+be made. The \gls{ANN} is built with the Keras\footnote{\url{https://keras.io}}
+using the TensorFlow\footnote{\url{https://github.com/tensorflow/tensorflow}}
+backend that provides a high-level interface to the highly technical networks.
+
+The general architecture of the networks is show in Figure~\ref{fig:bcann} and
+Figure~\ref{fig:mcann} for respectively the binary classification and
+multiclass classification. The inputs are fully connected to the hidden layer
+which is fully connected too the output layer. The activation function used is
+a \gls{RELU}. The \gls{RELU} function is a monotonic symmetric one-sided
+function that is also known as the ramp function. The definition is given in
+Equation~\ref{eq:relu}. \gls{RELU} has the downside that it can create
+unreachable nodes in a deep network. This is not a problem in this network
+since it only has one hidden layer. \gls{RELU} was also chosen because of its
+efficient computation and nature inspiredness.
+
+The activation function between the hidden layer and the output layer is the
+sigmoid function in the case of binary classification, of which the definition
+is shown in Equation~\ref{eq:sigmoid}. The sigmoid is a monotonic function that
+is differentiable on all values of $x$ and always yields a non-negative
+derivative. For the multiclass classification the softmax function is used
+between the hidden layer and the output layer. Softmax is an activation
+function suitable for multiple output nodes. The definition is given in
+Equation~\ref{eq:softmax}.
+
+The data is shuffled before fed to the network to mitigate the risk of
+overfitting on one album. Every model was trained using $10$ epochs which means
+that all training data is offered to the model $10$ times. The training set and
+test set are separated by taking a $90\%$ slice of all the data.
+
+\begin{equation}\label{eq:relu}
+ f(x) = \left\{\begin{array}{rcl}
+ 0 & \text{for} & x<0\\
+ x & \text{for} & x \geq 0\\
+ \end{array}\right.
+\end{equation}
+
+\begin{equation}\label{eq:sigmoid}
+ f(x) = \frac{1}{1+e^{-x}}
+\end{equation}
+
+\begin{equation}\label{eq:softmax}
+ \delta{(\boldsymbol{z})}_j = \frac{e^{z_j}}{\sum\limits^{K}_{k=1}e^{z_k}}
+\end{equation}
+
+\begin{figure}[H]
+ \begin{subfigure}{.5\textwidth}
+ \centering
+ \includegraphics[width=.8\linewidth]{bcann}
+ \caption{Binary classifier network architecture}\label{fig:bcann}
+ \end{subfigure}%
+%
+ \begin{subfigure}{.5\textwidth}
+ \centering
+ \includegraphics[width=.8\linewidth]{mcann}
+ \caption{Multiclass classifier network architecture}\label{fig:mcann}
+ \end{subfigure}
+ \caption{Artificial Neural Network architectures.}
+\end{figure}
+
+\section{Experimental setup}
+\subsection{Features}
+The thirteen \gls{MFCC} features are used as the input. The parameters of the
+\gls{MFCC} features are varied in window step and window length. The default
+speech processing parameters are tested but also bigger window sizes since
+arguably the minimal size of a singing voice segment is a lot bigger than the
+minimal size of a subphone component on which the parameters are tuned. The
+parameters chosen are as follows:
+
+\begin{table}[H]
+ \centering
+ \begin{tabular}{lll}
+ \toprule
+ step (ms) & length (ms) & notes\\
+ \midrule
+ 10 & 25 & Standard speech processing\\
+ 40 & 100 &\\
+ 80 & 200 &\\
+ \bottomrule
+ \end{tabular}
+ \caption{\Gls{MFCC} parameter settings}
+\end{table}
+
+\subsection{\emph{Singing}-voice detection}
+The first type of experiment conducted is \emph{Singing}-voice detection. This
+is the act of segmenting an audio signal into segments that are labeled either
+as \emph{Singing} or as \emph{Instrumental}. The input of the classifier is a
+feature vector and the output is the probability that singing is happening in
+the sample. This results in an \gls{ANN} of the shape described in
+Figure~\ref{fig:bcann}. The input dimension is thirteen and the output
+dimension is one.
-\todo{Explain why MFCC and which parameters}
+The \emph{crosstenopy} function is used as the loss function. The
+formula is shown in Equation~\ref{eq:bincross} where $p$ is the true
+distribution and $q$ is the classification. Acurracy is the mean of the
+absolute differences between prediction and true value. The formula is show in
+Equation~\ref{eq:binacc}.
-\section{\gls{ANN} Classifier}
-\todo{Spectrals might be enough, no decorrelation}
+\begin{equation}\label{eq:bincross}
+ H(p,q) = -\sum_x p(x)\log{q(x)}
+\end{equation}
-\section{Model training}
+\begin{equation}\label{eq:binacc}
+ \frac{1}{n}\sum^n_{i=1} abs (ypred_i-y_i)
+\end{equation}
-\section{Experiments}
+\subsection{\emph{Singer}-voice detection}
+The second type of experiment conducted is \emph{Singer}-voice detection. This
+is the act of segmenting an audio signal into segments that are labeled either
+with the name of the singer or as \emph{Instrumental}. The input of the
+classifier is a feature vector and the outputs are probabilities for each of
+the singers and a probability for the instrumental label. This results in an
+\gls{ANN} of the shape described in Figure~\ref{fig:mcann}. The input dimension
+is yet again thirteen and the output dimension is the number of categories. The
+output is encoded in one-hot encoding. This means that the four categories in
+the experiments are labeled as \texttt{1000, 0100, 0010, 0001}.
-\section{Results}
+The loss function is the same as in \emph{Singing}-voice detection.
+The accuracy is calculated a little differenty since the output of the network
+is not one probability but a vector of probabilities. The accuracy is
+calculated of each sample by only taking into account the highest value in the
+one-hot encoded vector. This exact formula is shown in Equation~\ref{eq:catacc}.
+\begin{equation}\label{eq:catacc}
+ \frac{1}{n}\sum^n_{i=1} abs(argmax(ypred_i)-argmax(y_i))
+\end{equation}
repositories / asr1617.git / blobdiff