marsrover/document/evaluation.tex

   1 \section{Development Plan \& Evaluation}
   2 \subsection{Development Plan}
   3 The development plan is written following the spiral software development model
   4 in which all requirements are divided into small iterations. Each of the
   5 iterations will have an implementation phase, a testing phase, an analysis
   6 phase, and a design phase. Furthermore, each of the iterations will have goals
   7 and results. The iterations and their deliverables are listed in
   8 \autoref{tab:devit}. The schedule for finishing iterations are listed in
   9 \autoref{tab:deadli}. On the 6th of January the must-have should be finished
  10 and if they are we try to complete as many iterations as possible.
  11 \begin{table}[H]
  12         \centering
  13         \begin{tabu} to \linewidth{llX}
  14                 \toprule
  15                 Deadline & Iterations\\
  16                 \midrule
  17                 9 Dec 2015 & 1--2\\
  18                 16 Dec 2015 & 3--5\\
  19                 23 Dec 2015 & 6--9\\
  20                 6 Jan 2015 & 10--11\\
  21                 13 Jan 2015 & Demo\\
  22                 \bottomrule
  23         \end{tabu}
  24         \caption{Deadlines}\label{tab:deadli}
  25 \end{table}
  26 \begin{table}[H]
  27         \centering
  28         \begin{tabu} to \linewidth{rlXX}
  29                 \toprule
  30                 It. & Req. & Description & Deliverables\\
  31                 \midrule
  32                 1 & NR1 & Create a DSL for MarsRover's missions to use available
  33                         sensors and actuators. & \texttt{TaskDSL.xtext}\\
  34                 2 & ER3 & Implement diagnostic functions that print on the \emph{LCD}.
  35                         & A \emph{LCD} class that prints to the \emph{LCD} screen.\\
  36                 3 & -- & Implement functions in the code generation for basic motor the
  37                         motor actions: forward, backward, measure rock, measure lake and
  38                         wait. & Code
  39                         generation for motors\\
  40                 4 & -- & Implement functions in the code generation for communicating
  41                         the sensors from the slave to the master. & Functions in the master
  42                         program to read the slave's sensors.\\
  43                 5 & -- & Create functionality for sensor values and determine the
  44                         treshholds manually. & Code generation for sensors and threshhold
  45                         constants.\\
  46                 6 & CR1 & Create functionality to keep the MarsRover in the planet. &
  47                         A generatable behaviour to stay on the planet.\\
  48                 7 & MR3 & Create functionality for not bumping into rocks. & A
  49                         generatable behaviour to not bump into rocks.\\
  50                 8 & MR1 & Create functionality to find lakes. & A generatable mission
  51                         to find lakes.\\
  52                 9 & MR6 & Create functionality to perform missions in sequence. & A
  53                         generatable main program that performs missions.\\
  54                 10 & MR2, MR4& Create functionality to find rocks. & A generatable
  55                         mission to find rocks.\\
  56                 11 & MR5 & Create functionality to push rocks and detect when the
  57                         robots is pushing. & A generatable behaviour for pushing.\\
  58                 12 & MR7 & Create functionality to find the parking space. & A
  59                         generatable behaviour that can find the parking space.\\
  60                 13 & MR8, ER1 & Create functionality to map the environment and to
  61                         localize. & A generatable behaviour that can map while
  62                         performing.\\
  63                 14 & ER2 & Create functionality to play sounds. & Add a generatable
  64                         function for playing sounds.\\
  65                 15 & NR3, NR4 & Create functionality to calibrate the sensor
  66                         treshholds. & A generatable function to calibrate the sensors.\\
  67                 16 & NR5 & Create functionality that when the robot encounters bugs it
  68                         can restart itself. & Functionality in the main program to do so.\\
  69                 17 & -- & Speed up the behaviour of the robot within safety limits.\\
  70                 \bottomrule
  71         \end{tabu}
  72         \caption{Spiral model iterations}\label{tab:devit}
  73 \end{table}
  74
  75 \subsection{Evaluation}
  76 \subsubsection{Requirements realisation}
  77 %implemented features and mission range
  78 The final implementation satisfies all the \emph{must have} requirements.
  79 Besides all the most important requirements it also supports all \emph{should
  80 have} requirements except \emph{MR8} in which the robot must be able to
  81 remember the locations of the lakes. This is because we encountered some
  82 problems during the development phase that lead to a shortage of time. Because
  83 of that mapping and localization was not implemented. In the end of the process
  84 we had some time left which we used to implement \emph{ER2} since it was
  85 relatively simple.
  86
  87 \paragraph{Mission range}
  88 The range of missions the robot can support is a lot bigger then reflected by
  89 the met functional requirements. All actions specifiable in a behaviour consist
  90 of atomic operations that take very little time leading to the action being
  91 interruptible at all times. Together with the rudimentary variable storage we
  92 can make missions very complex. Possible combinations of behaviour can perform
  93 actions such as: counting, following moving objects and grouping objects.
  94
  95 One could argue that due to the lack of control structures it is very difficult
  96 to program complex missions. This is the case, for example if we would add a
  97 \emph{while} construction it would increase the range of missions very much.
  98 However, it also makes programming the robot much more difficult and will
  99 reduce the usability. With the current structure with the subsumption
 100 architecture and the rudimentary memory you actually can create such control
 101 structures but it takes a little more effort.
 102
 103 \paragraph{Flexibility}
 104 The DSL can very easily be improved in the case of adding new sensors or
 105 actuators. For sensors we can just add clauses in the grammar within the
 106 \emph{StoppingExpression} that will process the values. For actuators the same
 107 thing can be done in the \emph{Action} rule. A change in the robot's
 108 configuration can also be handled very quickly albeit not in the DSL itself. We
 109 specifically chose not to incorporate the hardware configuration in the DSL to
 110 keep is a simple as possible and thus less error prone. This means that if the
 111 hardware configuration changes the underlying library must be changed as well.
 112 However, due to the modularity of the library this is very easy.
 113
 114 \paragraph{Safety}
 115 The DSL by design does not generate \emph{safe} code implicitly. A programmer
 116 can make a program that can destroy the robot very easily. While safety is very
 117 important it would limit the range of missions a lot if some basic behaviours
 118 used in the demonstration for safety would be inherently present.
 119
 120 \subsubsection{Development process}
 121 \paragraph{Feedback loop}
 122 Booting the bricks takes around a minute to complete. When the robot is
 123 programmed launching the application takes another 30 seconds followed by
 124 another 30 seconds waiting on the initialization of the sensors. About once
 125 every 10 times on high battery and about every other time on low battery the
 126 ultrasonic sensor fails to initialize. We did not have the time to investigate
 127 this properly but the fault was hidden somewhere deep in the \emph{LeJOS} code.
 128 When all sensors were initialized the Bluetooth connection could be commenced
 129 by pressing a button on both bricks. Connecting via Bluetooth has about the
 130 same error rate as the ultrasonic sensor has and takes about 30 seconds.
 131
 132 This very long start up led to a rough development process with long feedback
 133 loops. After a long time we and other groups found out that applying power to
 134 the bricks during the initialization and pairing reduced the error frequency a
 135 lot.
 136
 137 \paragraph{Techniques}
 138 Using the combination of all tools such as DSL, XText, XTend and eclipse went
 139 surprisingly well. In the beginning there were some technical difficulties
 140 setting everything up and there was quite some overhead since the tools
 141 required a lot of computing power. But when everything was set up correctly it
 142 worked like a charm.
 143
 144 \paragraph{General lessons learned}
 145 From our experience, the most important thing in the development process was to
 146 work iteratively in small building blocks. Another lesson learned was the fact
 147 that robots are not always reliable and can behave in unexpected ways.
 148
 149 Another lesson learned was that plans mostly take longer than expected. Our
 150 planning was very generous but still we did not have time to implement some
 151 extra functionality. If we would have more time we would have tried to make the
 152 robots autonomous in start up. The current robot needs to have a button pressed
 153 when the bricks can pair. We have experimented with autonomous pairing but we
 154 never got it stable but with more time we could make it stable. Another option
 155 is to try to get more requirements. For example the mapping and internal
 156 representation of the locations of the lakes. This problem is not trivial at
 157 all and because of the lack of time we could unfortunately not spend time on it.