a4/question3.tex

   1 The table below contains some statistics about all the different parameter configurations we ran learnlib with.
   2 All except \emph{RivestSchapire} using the Random test method result in the correct model being learned.
   3 When \emph{WMethod} is selected as the testing method \emph{RivestSchapire} is also able to learn the correct model.
   4 \emph{WMethod} does however increase the time needed to learn the model significantly, when a different learner is used there is no reason not to use the Random testing method.
   5
   6 \begin{longtable}{| l | l | l | c | c | c |}
   7         \caption{Learning parameters and resulting model size.} \\\hline
   8         Alphabet & Method & Test method & States & Time \\\hline \hline
   9         small & LStar & Random & 2 & 12 sec \\\hline
  10         small & TTT & Random & 2 & 5 sec \\\hline
  11         small & RivestSchapire & Random & 2 & 6 sec \\\hline
  12         small & KearnsVazirani & Random & 2 & 5 sec \\\hline
  13         small & LStar & WMethod & 2 & 35 sec \\\hline
  14         small & TTT & WMethod & 2 & 32 sec \\\hline
  15         small & RivestSchapire & WMethod & 2 & 33 sec \\\hline
  16         small & KearnsVazirani & WMethod & 2 & 33 sec \\\hline
  17
  18         partial & LStar & Random & 4 & 18 sec \\\hline
  19         partial & TTT & Random & 4 & 16 sec \\\hline
  20         partial & RivestSchapire & Random & 4 & 13 sec \\\hline
  21         partial & KearnsVazirani & Random & 4 & 22 sec \\\hline
  22         partial & LStar & WMethod & 4 & 384 sec \\\hline
  23         partial & TTT & WMethod & 4 & 390 sec \\\hline
  24         partial & RivestSchapire & WMethod & 4 & 384 sec \\\hline
  25         partial & KearnsVazirani & WMethod & 4 & 383 sec \\\hline
  26
  27         full & LStar & Random & 5 & 44 sec \\\hline
  28         full & TTT & Random & 5 & 25 sec \\\hline
  29         full & RivestSchapire & Random & 4 & 12 sec \\\hline
  30         full & KearnsVazirani & Random & 5 & 19 sec \\\hline
  31         full & LStar & WMethod & 5 & 2666 sec \\\hline
  32         full & TTT & WMethod & 5 & 2632 sec \\\hline
  33         full & RivestSchapire & WMethod & 5 & 2638 sec \\\hline
  34         full & KearnsVazirani & WMethod & - & - \\\hline
  35 \end{longtable}
  36