goede dingen

[tt2015.git] / a4 / question3.tex

diff --git a/a4/question3.tex b/a4/question3.tex
index d088a21..d5a1a13 100644 (file)
--- a/a4/question3.tex
+++ b/a4/question3.tex
@@ -1,44 +1,49 @@
-The table below contains some statistics about all the different parameter\r
-configurations we ran learnlib with.  All except \emph{RivestSchapire} using\r
-the Random test method result in the correct model being learned.  When\r
-\emph{WMethod} is selected as the testing method \emph{RivestSchapire} is also\r
-able to learn the correct model.\r
-\emph{WMethod} does however increase the time needed to learn the model\r
-significantly, when a different learner is used there is no reason not to use\r
-the Random testing method.\r
-\r
-\begin{table}[H]\r
-       \begin{tabular}{lllccc}\r
-               \toprule\r
-               Alphabet & Method & Test method & States & Time \\\r
-               \midrule\r
-               small & LStar & Random & 2 & 12 sec \\\r
-               small & TTT & Random & 2 & 5 sec \\\r
-               small & RivestSchapire & Random & 2 & 6 sec \\\r
-               small & KearnsVazirani & Random & 2 & 5 sec \\\r
-               small & LStar & WMethod & 2 & 35 sec \\\r
-               small & TTT & WMethod & 2 & 32 sec \\\r
-               small & RivestSchapire & WMethod & 2 & 33 sec \\\r
-               small & KearnsVazirani & WMethod & 2 & 33 sec \\\r
-               \r
-               partial & LStar & Random & 4 & 18 sec \\\r
-               partial & TTT & Random & 4 & 16 sec \\\r
-               partial & RivestSchapire & Random & 4 & 13 sec \\\r
-               partial & KearnsVazirani & Random & 4 & 22 sec \\\r
-               partial & LStar & WMethod & 4 & 384 sec \\\r
-               partial & TTT & WMethod & 4 & 390 sec \\\r
-               partial & RivestSchapire & WMethod & 4 & 384 sec \\\r
-               partial & KearnsVazirani & WMethod & 4 & 383 sec \\\r
-               \r
-               full & LStar & Random & 5 & 44 sec \\\r
-               full & TTT & Random & 5 & 25 sec \\\r
-               full & RivestSchapire & Random & 4 & 12 sec \\\r
-               full & KearnsVazirani & Random & 5 & 19 sec \\\r
-               full & LStar & WMethod & 5 & 2666 sec \\\r
-               full & TTT & WMethod & 5 & 2632 sec \\\r
-               full & RivestSchapire & WMethod & 5 & 2638 sec \\\r
-               full & KearnsVazirani & WMethod & - & - \\\r
-               \bottomrule\r
-       \end{tabular}\r
-       \caption{Learning parameters and resulting model size.}\r
-\end{table}\r
+The table below contains some statistics about all the different parameter
+configurations we ran learnlib with. The \emph{RivestSchapire} learner using
+the Random test method resulted in an incorrect model being learned. When the
+\emph{KearnsVazirani} learner using the WMethod tester wasn't able to learn a
+model, this is due the learner hitting a non-deterministic path.  This
+problem hasn't anything to do with the actual learner and is the result of a
+uncaught error situation in the adapter. This shows us that a leaner can be
+used to test software, since we discovered a bug in our adapter. Due to time
+constrains we were not able to fix this bug. When \emph{WMethod} is selected
+as the testing method \emph{RivestSchapire} is also able to learn the correct
+model. \emph{WMethod} does however increase the time needed to learn the
+model significantly, when a different learner is used there is no reason not to
+use the Random testing method.
+
+\begin{table}[H]
+       \begin{tabular}{lllccc}
+               \toprule
+               Alphabet & Method & Test method & States & Time \\
+               \midrule
+               small & LStar & Random & 2 & 12 sec \\
+               small & TTT & Random & 2 & 5 sec \\
+               small & RivestSchapire & Random & 2 & 6 sec \\
+               small & KearnsVazirani & Random & 2 & 5 sec \\
+               small & LStar & WMethod & 2 & 35 sec \\
+               small & TTT & WMethod & 2 & 32 sec \\
+               small & RivestSchapire & WMethod & 2 & 33 sec \\
+               small & KearnsVazirani & WMethod & 2 & 33 sec \\
+               
+               partial & LStar & Random & 4 & 18 sec \\
+               partial & TTT & Random & 4 & 16 sec \\
+               partial & RivestSchapire & Random & 4 & 13 sec \\
+               partial & KearnsVazirani & Random & 4 & 22 sec \\
+               partial & LStar & WMethod & 4 & 384 sec \\
+               partial & TTT & WMethod & 4 & 390 sec \\
+               partial & RivestSchapire & WMethod & 4 & 384 sec \\
+               partial & KearnsVazirani & WMethod & 4 & 383 sec \\
+               
+               full & LStar & Random & 5 & 44 sec \\
+               full & TTT & Random & 5 & 25 sec \\
+               full & RivestSchapire & Random & 4 & 12 sec \\
+               full & KearnsVazirani & Random & 5 & 19 sec \\
+               full & LStar & WMethod & 5 & 2666 sec \\
+               full & TTT & WMethod & 5 & 2632 sec \\
+               full & RivestSchapire & WMethod & 5 & 2638 sec \\
+               full & KearnsVazirani & WMethod & - & - \\
+               \bottomrule
+       \end{tabular}
+       \caption{Learning parameters and resulting model size.}
+\end{table}