LATEX:=latex
DOCUMENT:=tt4
-MODELS=model.small.LStar.rand.eps model.partial.LStar.rand.eps model.full.LStar.rand.eps
-TEXS=question1.tex question2.tex question3.tex question4.tex
+MODELS=model.small.LStar.rand.eps model.small.TTT.rand.eps model.small.RS.rand.eps model.small.KV.rand.eps \
+ model.small.LStar.wm.eps model.small.TTT.wm.eps model.small.RS.wm.eps model.small.KV.wm.eps \
+ model.partial.LStar.rand.eps model.partial.TTT.rand.eps model.partial.RS.rand.eps model.partial.KV.rand.eps \
+ model.partial.LStar.wm.eps model.partial.TTT.wm.eps model.partial.RS.wm.eps model.partial.KV.wm.eps \
+ model.full.LStar.rand.eps model.full.TTT.rand.eps model.full.RS.rand.eps model.full.KV.rand.eps \
+ model.full.LStar.wm.eps model.full.TTT.wm.eps model.full.RS.wm.eps
+TEXS=question1.tex question2.tex question3.tex question4.tex models.tex
.SECONDARY: $(DOCUMENT).fmt
.PHONY: clean
--- /dev/null
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.small.LStar.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{small alphabet, LStar, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.small.TTT.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{small alphabet, TTT, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.small.RS.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{small alphabet, RivestSchapire, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.small.KV.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{small alphabet, KearnsVazirani, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.small.LStar.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{small alphabet, LStar, WMethod}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.small.TTT.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{small alphabet, TTT, WMethod}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.small.RS.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{small alphabet, RivestSchapire, WMethod}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.small.KV.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{small alphabet, KearnsVazirani, WMethod}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.partial.LStar.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{partial alphabet, LStar, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.partial.TTT.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{partial alphabet, TTT, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.partial.RS.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{partial alphabet, RivestSchapire, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.partial.KV.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{partial alphabet, KearnsVazirani, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.partial.LStar.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{partial alphabet, LStar, WMethod}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.partial.TTT.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{partial alphabet, TTT, WMethod}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.partial.RS.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{partial alphabet, RivestSchapire, WMethod}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[scale=0.75]{model.partial.KV.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{partial alphabet, KearnsVazirani, WMethod}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[width=\textwidth]{model.full.LStar.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{full alphabet, LStar, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[width=\textwidth]{model.full.TTT.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{full alphabet, TTT, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[width=\textwidth]{model.full.RS.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{full alphabet, RivestSchapire, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[width=\textwidth]{model.full.KV.rand.eps}\r
+% \vspace{5mm}\r
+% \caption{full alphabet, KearnsVazirani, Random}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[width=\textwidth]{model.full.LStar.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{full alphabet, LStar, WMethod}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[width=\textwidth]{model.full.TTT.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{full alphabet, TTT, WMethod}\r
+%\end{figure}\r
+%\r
+%\begin{figure}[H]\r
+% \centering\r
+% \includegraphics[width=\textwidth]{model.full.RS.wm.eps}\r
+% \vspace{5mm}\r
+% \caption{full alphabet, RivestSchapire, WMethod}\r
+%\end{figure}\r
The Candymachine was learned using LearnLib. Figure~\ref{fig:candy} shows the
learned model. In this Figure S0 is the initial state.
-\begin{figure}
+\begin{figure}[H]
\includegraphics[width=1.7\textwidth,natwidth=2389,natheight=891]{1candyFig.png}
\caption{Learned model of the candy machine}
\label{fig:candy}
-In order to be allow learnlib to learn the TCP model it was necessary to have a\r
-deterministic model. We accomplished this by modifying the adapter so it can\r
-reach a \texttt{ERROR} or \texttt{CLOSED} state. In these states all inputs are\r
-discarded and a default output is returned. In the case of a state where an\r
-input results in a non-deterministic output we jump to the \texttt{ERROR} state\r
-for additional this given input. When the connection is successfully closed\r
-using a \texttt{FIN} packet we move the adapter to the \texttt{CLOSED} state.\r
-\r
-We divided the input alphabet into three sets, this way we can control the size\r
-of the model learned by learnlib.\r
-\r
-\begin{table}[H]\r
- \begin{tabular}{cl}\r
- \toprule\r
- Alphabet & Inputs \\\r
- \midrule\r
- small & \texttt{SYN}, \texttt{ACK} \\\r
- partial & \texttt{SYN}, \texttt{ACK}, \texttt{DATA} \\\r
- full & \texttt{SYN}, \texttt{ACK}, \texttt{DATA}, \texttt{RST},\r
- \texttt{FIN} \\\r
- \bottomrule\r
- \end{tabular}\r
- \caption{Different input alphabets used during learning.}\r
-\end{table}\r
-\r
-Just as in our previous assignment the \texttt{DATA} packet is actually a\r
-\texttt{ACK} with an user data payload and the \emph{push} flag set. These\r
-input alphabets will influence the size of the model produced. \emph{small}\r
-will result in a 2 state model, \emph{partial} will be the full model without\r
-the \texttt{CLOSED} state and \emph{full} should result in the full model as\r
-used in the previous assignment.\r
-\r
-\paragraph{Model learned with small input alphabet}\r
-%\includegraphics{model.small.LStar.rand.eps}\r
-\r
-\paragraph{Model learned with partial input alphabet}\r
-%\includegraphics{model.partial.LStar.rand.eps}\r
-\r
-\paragraph{Model learned with full input alphabet}\r
-%\includegraphics{model.full.LStar.rand.eps}\r
+In order to be allow learnlib to learn the TCP model it was necessary to have a
+deterministic model. We accomplished this by modifying the adapter so it can
+reach a \texttt{ERROR} or \texttt{CLOSED} state. In these states all inputs are
+discarded and a default output is returned. In the case of a state where an
+input results in a non-deterministic output we jump to the \texttt{ERROR} state
+for additional this given input. When the connection is successfully closed
+using a \texttt{FIN} packet we move the adapter to the \texttt{CLOSED} state.
+
+We divided the input alphabet into three sets, this way we can control the size
+of the model learned by learnlib.
+
+\begin{table}[H]
+ \begin{tabular}{cl}
+ \toprule
+ Alphabet & Inputs \\
+ \midrule
+ small & \texttt{SYN}, \texttt{ACK} \\
+ partial & \texttt{SYN}, \texttt{ACK}, \texttt{DATA} \\
+ full & \texttt{SYN}, \texttt{ACK}, \texttt{DATA}, \texttt{RST},
+ \texttt{FIN} \\
+ \bottomrule
+ \end{tabular}
+ \caption{Different input alphabets used during learning.}
+\end{table}
+
+Just as in our previous assignment the \texttt{DATA} packet is actually a
+\texttt{ACK} with an user data payload and the \emph{push} flag set. These
+input alphabets will influence the size of the model produced. \emph{small}
+will result in a 2 state model, \emph{partial} will be the full model without
+the \texttt{CLOSED} state and \emph{full} should result in the full model as
+used in the previous assignment.
+%
+%\begin{figure}[H]
+% \centering
+% \includegraphics[scale=0.75]{model.small.LStar.rand.eps}
+% \vspace{5mm}
+% \caption{Model learned with small input alphabet}
+%\end{figure}
+%
+%\begin{figure}[H]
+% \centering
+% \includegraphics[width=\textwidth]{model.partial.LStar.rand.eps}
+% \vspace{5mm}
+% \caption{Model learned with partial input alphabet}
+%\end{figure}
+%
+%\begin{figure}[H]
+% \centering
+% \includegraphics[width=1.2\textwidth]{model.full.LStar.rand.eps}
+% \vspace{5mm}
+% \caption{Model learned with full input alphabet}
+%\end{figure}
-The table below contains some statistics about all the different parameter\r
-configurations we ran learnlib with. All except \emph{RivestSchapire} using\r
-the Random test method result in the correct model being learned. When\r
-\emph{WMethod} is selected as the testing method \emph{RivestSchapire} is also\r
-able to learn the correct model.\r
-\emph{WMethod} does however increase the time needed to learn the model\r
-significantly, when a different learner is used there is no reason not to use\r
-the Random testing method.\r
-\r
-\begin{table}[H]\r
- \begin{tabular}{lllccc}\r
- \toprule\r
- Alphabet & Method & Test method & States & Time \\\r
- \midrule\r
- small & LStar & Random & 2 & 12 sec \\\r
- small & TTT & Random & 2 & 5 sec \\\r
- small & RivestSchapire & Random & 2 & 6 sec \\\r
- small & KearnsVazirani & Random & 2 & 5 sec \\\r
- small & LStar & WMethod & 2 & 35 sec \\\r
- small & TTT & WMethod & 2 & 32 sec \\\r
- small & RivestSchapire & WMethod & 2 & 33 sec \\\r
- small & KearnsVazirani & WMethod & 2 & 33 sec \\\r
- \r
- partial & LStar & Random & 4 & 18 sec \\\r
- partial & TTT & Random & 4 & 16 sec \\\r
- partial & RivestSchapire & Random & 4 & 13 sec \\\r
- partial & KearnsVazirani & Random & 4 & 22 sec \\\r
- partial & LStar & WMethod & 4 & 384 sec \\\r
- partial & TTT & WMethod & 4 & 390 sec \\\r
- partial & RivestSchapire & WMethod & 4 & 384 sec \\\r
- partial & KearnsVazirani & WMethod & 4 & 383 sec \\\r
- \r
- full & LStar & Random & 5 & 44 sec \\\r
- full & TTT & Random & 5 & 25 sec \\\r
- full & RivestSchapire & Random & 4 & 12 sec \\\r
- full & KearnsVazirani & Random & 5 & 19 sec \\\r
- full & LStar & WMethod & 5 & 2666 sec \\\r
- full & TTT & WMethod & 5 & 2632 sec \\\r
- full & RivestSchapire & WMethod & 5 & 2638 sec \\\r
- full & KearnsVazirani & WMethod & - & - \\\r
- \bottomrule\r
- \end{tabular}\r
- \caption{Learning parameters and resulting model size.}\r
-\end{table}\r
+The table below contains some statistics about all the different parameter
+configurations we ran learnlib with. The \emph{RivestSchapire} learner using
+the Random test method resulted in an incorrect model being learned. When the
+\emph{KearnsVazirani} learner using the WMethod tester wasn't able to learn a
+model, this is due the learner hitting a non-deterministic path. This
+problem hasn't anything to do with the actual learner and is the result of a
+uncaught error situation in the adapter. This shows us that a leaner can be
+used to test software, since we discovered a bug in our adapter. Due to time
+constrains we were not able to fix this bug. When \emph{WMethod} is selected
+as the testing method \emph{RivestSchapire} is also able to learn the correct
+model. \emph{WMethod} does however increase the time needed to learn the
+model significantly, when a different learner is used there is no reason not to
+use the Random testing method.
+
+\begin{table}[H]
+ \begin{tabular}{lllccc}
+ \toprule
+ Alphabet & Method & Test method & States & Time \\
+ \midrule
+ small & LStar & Random & 2 & 12 sec \\
+ small & TTT & Random & 2 & 5 sec \\
+ small & RivestSchapire & Random & 2 & 6 sec \\
+ small & KearnsVazirani & Random & 2 & 5 sec \\
+ small & LStar & WMethod & 2 & 35 sec \\
+ small & TTT & WMethod & 2 & 32 sec \\
+ small & RivestSchapire & WMethod & 2 & 33 sec \\
+ small & KearnsVazirani & WMethod & 2 & 33 sec \\
+
+ partial & LStar & Random & 4 & 18 sec \\
+ partial & TTT & Random & 4 & 16 sec \\
+ partial & RivestSchapire & Random & 4 & 13 sec \\
+ partial & KearnsVazirani & Random & 4 & 22 sec \\
+ partial & LStar & WMethod & 4 & 384 sec \\
+ partial & TTT & WMethod & 4 & 390 sec \\
+ partial & RivestSchapire & WMethod & 4 & 384 sec \\
+ partial & KearnsVazirani & WMethod & 4 & 383 sec \\
+
+ full & LStar & Random & 5 & 44 sec \\
+ full & TTT & Random & 5 & 25 sec \\
+ full & RivestSchapire & Random & 4 & 12 sec \\
+ full & KearnsVazirani & Random & 5 & 19 sec \\
+ full & LStar & WMethod & 5 & 2666 sec \\
+ full & TTT & WMethod & 5 & 2632 sec \\
+ full & RivestSchapire & WMethod & 5 & 2638 sec \\
+ full & KearnsVazirani & WMethod & - & - \\
+ \bottomrule
+ \end{tabular}
+ \caption{Learning parameters and resulting model size.}
+\end{table}
\section{Question 4}
\input{question4.tex}
+\appendix
+\section{Models}
+\input{models.tex}
+
\nocite{*}
\bibliographystyle{ieeetr}
\bibliography{tt4}
repositories / tt2015.git / commitdiff