a4/question3.tex

   1 The table below contains some statistics about all the different parameter
   2 configurations we ran learnlib with.  All except \emph{RivestSchapire} using
   3 the Random test method result in the correct model being learned.  When
   4 \emph{WMethod} is selected as the testing method \emph{RivestSchapire} is also
   5 able to learn the correct model.
   6 \emph{WMethod} does however increase the time needed to learn the model
   7 significantly, when a different learner is used there is no reason not to use
   8 the Random testing method.
   9
  10 \begin{table}[H]
  11         \begin{tabular}{lllccc}
  12                 \toprule
  13                 Alphabet & Method & Test method & States & Time \\
  14                 \midrule
  15                 small & LStar & Random & 2 & 12 sec \\
  16                 small & TTT & Random & 2 & 5 sec \\
  17                 small & RivestSchapire & Random & 2 & 6 sec \\
  18                 small & KearnsVazirani & Random & 2 & 5 sec \\
  19                 small & LStar & WMethod & 2 & 35 sec \\
  20                 small & TTT & WMethod & 2 & 32 sec \\
  21                 small & RivestSchapire & WMethod & 2 & 33 sec \\
  22                 small & KearnsVazirani & WMethod & 2 & 33 sec \\
  23
  24                 partial & LStar & Random & 4 & 18 sec \\
  25                 partial & TTT & Random & 4 & 16 sec \\
  26                 partial & RivestSchapire & Random & 4 & 13 sec \\
  27                 partial & KearnsVazirani & Random & 4 & 22 sec \\
  28                 partial & LStar & WMethod & 4 & 384 sec \\
  29                 partial & TTT & WMethod & 4 & 390 sec \\
  30                 partial & RivestSchapire & WMethod & 4 & 384 sec \\
  31                 partial & KearnsVazirani & WMethod & 4 & 383 sec \\
  32
  33                 full & LStar & Random & 5 & 44 sec \\
  34                 full & TTT & Random & 5 & 25 sec \\
  35                 full & RivestSchapire & Random & 4 & 12 sec \\
  36                 full & KearnsVazirani & Random & 5 & 19 sec \\
  37                 full & LStar & WMethod & 5 & 2666 sec \\
  38                 full & TTT & WMethod & 5 & 2632 sec \\
  39                 full & RivestSchapire & WMethod & 5 & 2638 sec \\
  40                 full & KearnsVazirani & WMethod & - & - \\
  41                 \bottomrule
  42         \end{tabular}
  43         \caption{Learning parameters and resulting model size.}
  44 \end{table}