top/int.tex

   1 \documentclass[../thesis.tex]{subfiles}
   2
   3 \input{subfilepreamble}
   4
   5 \setcounter{chapter}{5}
   6
   7 \begin{document}
   8 \input{subfileprefix}
   9 \chapter{Integration of \texorpdfstring{\gls{MTASK}}{mTask} and \texorpdfstring{\gls{ITASK}}{iTask}}%
  10 \label{chp:integration_with_itask}
  11 \begin{chapterabstract}
  12         This chapter shows the integration of \gls{MTASK} and \gls{ITASK} by showing:
  13         \begin{itemize}
  14                 \item an architectural overview of \gls{MTASK} applications;
  15                 \item the interface for connecting devices;
  16                 \item the interface for lifting \gls{MTASK} tasks to \gls{ITASK} tasks;
  17                 \item a interface for lowering \gls{ITASK} \glspl{SDS} to \gls{MTASK} \glspl{SDS};
  18                 \item and a non-trivial home automation example application using all integration mechanisms;
  19         \end{itemize}
  20 \end{chapterabstract}
  21
  22 The \gls{MTASK} language is a multi-view \gls{DSL}, there are multiple interpretations possible for a single \gls{MTASK} term.
  23 When using the byte code compiler (\cleaninline{BCInterpret}) \gls{DSL} interpretation, \gls{MTASK} tasks are fully integrated in \gls{ITASK}.
  24 They execute as regular \gls{ITASK} tasks and they can access \glspl{SDS} from \gls{ITASK}.
  25 Devices in the \gls{MTASK} system are set up with a domain-specific \gls{OS} and become little \gls{TOP} engines in their own respect, being able to execute tasks.
  26
  27 \Cref{fig:mtask_integration} shows the architectural layout of a typical \gls{IOT} system created with \gls{ITASK} and \gls{MTASK}.
  28 The entire system is written as a single \gls{CLEAN} specification where multiple tasks are executed at the same time.
  29 Tasks can access \glspl{SDS} according to many-to-many communication and multiple clients can work on the same task.
  30 The diagram contains three labelled arrows that denote the integration functions between \gls{ITASK} and \gls{MTASK}.
  31 Devices are connected to the system using the \cleaninline{withDevice} function (see \cref{sec:withdevice}).
  32 Using \cleaninline{liftmTask}, \gls{MTASK} tasks are lifted to a device (see \cref{sec:liftmtask}).
  33 \glspl{SDS} from \gls{ITASK} are lowered to the \gls{MTASK} device using \cleaninline{lowerSds} (see \cref{sec:liftsds}).
  34
  35 \begin{figure}
  36         \centering
  37         \includestandalone{mtask_integration}
  38         \caption{An architectural overview of an \imtask{} application.}%
  39         \label{fig:mtask_integration}
  40 \end{figure}
  41
  42 \section{Connecting edge devices}\label{sec:withdevice}
  43 Edge devices in an \gls{MTASK} application are always coordinated by a server.
  44 This means that they wait for a server to take initiative, set up a connection, and send the work.
  45 The heavy lifting of connecting an \gls{MTASK} device to an \gls{ITASK} server is done with the \cleaninline{withDevice} \gls{ITASK} function.
  46 This function has two parameters, a communication specification, and a function using a device handle.
  47 The device handle is required to interact with \gls{MTASK} devices, e.g.\ lift tasks.
  48 By using \gls{HOAS} like this, setting up and tearing down the connection to the device is fully controlled.
  49
  50 All communication with a device happens through a so-called \emph{channels} \gls{SDS}.
  51 The channels contain three fields, a queue of messages that are received, a queue of messages to send, and a stop flag.
  52 Every communication method that implements the \cleaninline{channelSync} class can provide the communication with an \gls{MTASK} device.
  53 At the time of writing, serial port, direct \gls{TCP}, and \gls{MQTT} over \gls{TCP} are supported communication methods (see \cref{lst:connection_types}).
  54 Internally, the \cleaninline{withDevice} task sets up the communication, exchanges specifications with the device, executes the inner task while handling errors, and finally cleans up after closing.
  55 \Cref{lst:mtask_device} shows the types and interface for connecting devices.
  56
  57 \begin{lstClean}[label={lst:mtask_device},caption={Device communication interface in \gls{MTASK}.}]
  58 :: MTDevice //abstract
  59 :: Channels :== ([MTMessageFro], [MTMessageTo], Bool)
  60
  61 class channelSync a :: a (Shared sds Channels) -> Task () | RWShared sds
  62
  63 withDevice :: a (MTDevice -> Task b)
  64         -> Task b | iTask b & channelSync, iTask a
  65 \end{lstClean}
  66
  67 \subsection{Implementation}
  68 \Cref{lst:pseudo_withdevice} shows a pseudocode implementation of the \cleaninline{withDevice} function.
  69 The \cleaninline{MTDevice} abstract type is internally represented as three \gls{ITASK} \gls{SDS} that contain all the current information about the tasks.
  70 The first \gls{SDS} is the information about the \gls{RTS} of the device, i.e.\ metadata on the tasks that are executing, the hardware specification and capabilities, and a list of fresh task identifiers.
  71 The second \gls{SDS} is a map storing downstream \gls{SDS} updates.
  72 When a lowered \gls{SDS} is updated on the device, a message is sent to the server.
  73 This message is initially queued in the map in order to properly handly multiple updates asychronously.
  74 Finally, the \cleaninline{MTDevices} type contains the communication channels.
  75
  76 The \cleaninline{withDevice} task itself first constructs the \glspl{SDS} using the \gls{ITASK} function \cleaninline{withShared}.
  77 Then, it performs the following four tasks in parallel to monitor the edge device.
  78 \begin{enumerate}
  79         \item The channels are synchronised using the overloaded \cleaninline{channelSync} function.
  80                 Errors that occur here are converted to the proper \gls{MTASK} or \gls{ITASK} exception.
  81         \item The shutdown flag of the channels is watched.
  82                 If the connection is lost with the device unexpectedly, an \gls{MTASK} exception is thrown.
  83         \item The received messages in the channels are watched and processed.
  84                 Depending on the type of message, either the device information \gls{SDS} is updated, or the \gls{SDS} update is added to the lowered \gls{SDS} updates \gls{SDS}.
  85         \item A request for a specification is sent.
  86                 Once the specification is received, the device task is run.
  87                 The task value of this device task is then used as the task value of the \cleaninline{withDevice} task.
  88 \end{enumerate}
  89
  90 \begin{lstClean}[caption={Pseudocode for the \texttt{withDevice} function in \gls{MTASK}.},label={lst:pseudo_withdevice}]
  91 withDevice spec deviceTask =
  92         withShared default \dev->parallel
  93         withShared newMap \sdsupdates->
  94         withShared ([], [MTTSpecRequest], False) \channels->
  95                 [ channelSync spec channels
  96                 , watchForShutdown channels
  97                 , watchChannelMessages dev channels
  98                 , waitForSpecification
  99                         >>| deviceTask (MTDevice dev sdsupdates channels)
 100                         >>* [ifStable: issueShutdown]
 101                 ]
 102 \end{lstClean}
 103
 104 If at any stage an unrecoverable device error occurs, an \gls{ITASK} exception is thrown in the \cleaninline{withDevice} task.
 105 This exception can be caught in order to devise fail-safe mechanisms.
 106 For example, if a device fails, the task can be sent to another device as can be seen in \cref{lst:failover}.
 107 This function executes an \gls{MTASK} task on a pool of devices connected through \gls{TCP}.
 108 If a device error occurs during execution, the next device in the pool is tried until the pool is exhausted.
 109 If another type of error occurs, it is rethrown for a parent task to catch.
 110
 111 \begin{lstClean}[caption={An \gls{MTASK} failover combinator.},label={lst:failover}]
 112         failover :: [TCPSettings] (Main (MTask BCInterpret a)) -> Task a
 113         failover []     _     = throw "Exhausted device pool"
 114         failover [d:ds] mtask = try (withDevice d (liftmTask mtask)) except
 115         where except MTEUnexpectedDisconnect = failover ds mtask
 116         except _                       = throw e
 117 \end{lstClean}
 118
 119 \section{Lifting \texorpdfstring{\gls{MTASK}}{mTask} tasks}\label{sec:liftmtask}
 120 Once the connection with the device is established, \gls{MTASK} tasks are lifted to \gls{ITASK} tasks using the \cleaninline{liftmTask} function (see \cref{lst:liftmtask}).
 121 Given an \gls{MTASK} task in the \cleaninline{BCInterpret} view and a device handle obtained from \cleaninline{withDevice}, an \gls{ITASK} task is returned.
 122 This \gls{ITASK} task proxies the \gls{MTASK} task that is executed on the microcontroller.
 123 So, when another task observes the task value, the actual task value from the microcontroller is observed.
 124
 125 \begin{lstClean}[label={lst:liftmtask},caption={The interface for lifting \gls{MTASK} tasks to \gls{ITASK} tasks.}]
 126 liftmTask :: (Main (MTask BCInterpret a)) MTDevice -> Task a | iTask a
 127 \end{lstClean}
 128
 129 \subsection{Implementation}
 130 \Cref{lst:liftmTask_pseudo} shows the pseudocode for the \cleaninline{liftmTask} implementation
 131 The first argument is the task and the second argument is the device which is an \gls{ADT} containing the \glspl{SDS} referring to the device information, the \gls{SDS} update queue, and the channels.
 132 First a fresh identifier for the task is generated using the device state.
 133 With this identifier, the cleanup hook can be installed.
 134 This is done to assure the task is removed from the edge device if the \gls{ITASK} task coordinating it is destroyed.
 135 Tasks in \gls{ITASK} are destroyed when for example it is executed in parallel with another task and the parallel combinator terminates, or when the condition to step holds in a sequential task combination.
 136 Then the \gls{MTASK} compiler is invoked, its only argument besides the task is a function doing something with the results of the compilation, i.e.\ the lowered \glspl{SDS} and the messages containing the compiled and serialised task.
 137 With the result of the compilation, the task can be executed.
 138 First the messages are put in the channels, sending them to the device.
 139 Then, in parallel:
 140 \begin{enumerate}
 141         \item the value is watched by looking in the device state \gls{SDS}, this task also determines the task value of the whole task;
 142         \item the downstream \glspl{SDS} are monitored, i.e.\ the \cleaninline{sdsupdates} \gls{SDS} is monitored and updates from the device are applied to the associated \gls{ITASK} \gls{SDS};
 143         \item the upstream \glspl{SDS} are monitored by spawning tasks that watch these \glspl{SDS}, if one is updated, the novel value is sent to the edge device.
 144 \end{enumerate}
 145
 146 \begin{lstClean}[label={lst:liftmTask_pseudo},caption={Pseudocode implementation for \texttt{liftmTask}.}]
 147 liftmTask task (MTDevice dev sdsupdates channels)
 148         = freshTaskId dev
 149         >>= \tid->withCleanupHook (sendmessage [MTTTaskDel tid] channels) (
 150                 compile task \mrefs msgs->
 151                         sendMessage msgs channels
 152                         >>| waitForReturnAndValue tid dev
 153                         -|| watchSharesDownstream mrefs tid sdsupdates
 154                         -|| watchSharesUpstream mrefs channels tid)
 155 \end{lstClean}
 156
 157 Sending the complete byte code to the device is not always a suitable option.
 158 For example, when the device is connected through an unstable or slow connection, sending the entire byte code induces a lot of delay.
 159 To mitigate this problem, \gls{MTASK} tasks can be preloaded on a device.
 160 Preloading means that the task is compiled and integrated into the device firmware.
 161 On receiving a \cleaninline{TaskPrep}, a hashed value of the task to be sent is included.
 162 The device then checks the preloaded task registry and uses the local preloaded version if the hash matches.
 163 Of course this only works for tasks that are not tailor made for the current work specification and not depend on run time information.
 164 The interface for task preloading can be found in \cref{lst:preload}.
 165 Given an \gls{MTASK} task, a header file is created that should be placed in the source code directory of the \gls{RTS} before building to include it in the firmware.
 166
 167 \begin{lstClean}[label={lst:preload},caption={Preloading tasks in \gls{MTASK}.}]
 168 preloadTask :: (Main (MTask BCInterpret a)) -> Task String
 169 \end{lstClean}
 170
 171 \section{Lowering \texorpdfstring{\gls{ITASK}}{iTask} \texorpdfstring{\glsxtrlongpl{SDS}}{shared data sources}}\label{sec:liftsds}
 172 Lowering \gls{ITASK} \glspl{SDS} to \gls{MTASK} \glspl{SDS} is something that mostly happens at the \gls{DSL} level using the \cleaninline{lowerSds} function (see \cref{lst:mtask_itasksds}).
 173 Lowering \pgls{SDS} proxies the \gls{ITASK} \gls{SDS} for use in \gls{MTASK}.
 174 \Glspl{SDS} in \gls{MTASK} always have an initial value.
 175 For regular \gls{SDS} this value is given in the source code, for lowered \gls{ITASK} \glspl{SDS} this value is obtained by reading the values once just before sending the task to the edge device.
 176 On the device, there is just one difference between lowered \glspl{SDS} and regular \glspl{SDS}: after changing a lowered \gls{SDS}, a message is sent to the server containing this new value.
 177 The \cleaninline{withDevice} task (see \cref{sec:withdevice}) receives and processes this message by writing to the \gls{ITASK} \gls{SDS}.
 178 Tasks watching this \gls{SDS} get notified then through the normal notification mechanism of \gls{ITASK}.
 179
 180 \begin{lstClean}[label={lst:mtask_itasksds},caption={Lowered \gls{ITASK} \glspl{SDS} in \gls{MTASK}.}]
 181 class lowerSds v where
 182         lowerSds :: ((v (Sds t)) -> In (Shared sds t) (Main (MTask v u)))
 183                 -> Main (MTask v u) | RWShared sds
 184 \end{lstClean}
 185
 186 As an example, \cref{lst:mtask_liftsds_ex} shows a light switch function producing an \imtask{} task when given a device handle.
 187 First an \gls{ITASK} \gls{SDS} of the type boolean is created.
 188 This boolean represents the state of the light.
 189 The \gls{MTASK} task uses this \gls{SDS} to turn on or off the light.
 190 The \gls{ITASK} task that runs in parallel allows interactive updating of this state.
 191
 192 \begin{lstClean}[label={lst:mtask_liftsds_ex},caption={Interactive light switch program in \gls{MTASK}.}]
 193 lightswitch :: MTDevice -> Task Bool
 194 lightswitch dev  = withShared False \sh->
 195                     liftmTask (mtask sh) dev
 196                 -|| updateSharedInformation [] sh
 197                 <<@ Hint "Light switch"
 198 where
 199         mtask :: (Shared sds Bool) -> Main (MTask v Bool)
 200                 | mtask, lowerSds v & RWShared sds
 201         mtask sh =
 202                 declarePin D13 PMOutput \d13->
 203                 lowerSds \ls=sh
 204                 In fun \f=(\st->
 205                              getSds ls
 206                         >>*. [IfValue (\v->v !=. st) (\v->writeD d13 v)]
 207                         >>|. f (Not st))
 208                 In {main=f true}
 209 \end{lstClean}
 210
 211 \subsection{Implementation}
 212 The compilation of the code and the serialisation of the data throws away all typing information.
 213 \Glspl{SDS} are stored in the compiler state as a map from identifiers to either an initial value or an \cleaninline{MTLens}.
 214 The \cleaninline{MTLens} is a type synonym for \pgls{SDS} that represents the typeless serialised value of the underlying \gls{SDS}.
 215 This is done so that the \cleaninline{withDevice} task can write the received \gls{SDS} updates to the according \gls{SDS} while the \gls{SDS} is not in scope.
 216 The \gls{ITASK} notification mechanism then takes care of the rest.
 217 Such \pgls{SDS} is created by using the \cleaninline{mapReadWriteError} which, given a pair of read and write functions with error handling, produces \pgls{SDS} with the lens embedded.
 218 The read function transforms converts the typed value to a typeless serialised value.
 219 The write function will, given a new serialised value and the old typed value, produce a new typed value.
 220 It tries to decode the serialised value, if that succeeds, it is written to the underlying \gls{SDS}, an error is thrown otherwise.
 221 \Cref{lst:mtask_itasksds_lens} shows the implementation for this.
 222
 223 % VimTeX: SynIgnore on
 224 \begin{lstClean}[label={lst:mtask_itasksds_lens},caption={Lens applied to lowered \gls{ITASK} \glspl{SDS} in \gls{MTASK}.}]
 225 lens :: (Shared sds a) -> MTLens | type a & RWShared sds
 226 lens sds = mapReadWriteError
 227         ( \r->   Ok (fromString (toByteCode{|*|} r)
 228         , \w r-> ?Just <$> iTasksDecode (toString w)
 229         ) ?None sds
 230 \end{lstClean}
 231 % VimTeX: SynIgnore off
 232
 233 \Cref{lst:mtask_itasksds_lift} shows the code for the implementation of \cleaninline{lowerSds} that uses the \cleaninline{lens} function shown earlier.
 234 First, the \gls{SDS} to be lowered is extracted from the expression by bootstrapping the fixed point with a dummy value.
 235 This is safe because the expression on the right-hand side of the \cleaninline{In} is never evaluated.
 236 Then, using \cleaninline{addSdsIfNotExist}, the identifier for this particular \gls{SDS} is either retrieved from the compiler state or generated freshly.
 237 This identifier is then used to provide a reference to the \cleaninline{def} definition to evaluate the main expression.
 238
 239 % VimTeX: SynIgnore on
 240 \begin{lstClean}[label={lst:mtask_itasksds_lift},caption={The implementation for lowering \glspl{SDS} in \gls{MTASK}.}]
 241 instance lowerSds BCInterpret where
 242         lowerSds def = {main =
 243                         let (t In _) = def (abort "lowerSds: expression too strict")
 244                         in addSdsIfNotExist (Right $ lens t)
 245                                 >>= \sdsi->let (_ In e) = def (pure (Sds sdsi)) in e.main
 246                 }\end{lstClean}
 247 % VimTeX: SynIgnore off
 248
 249 \section{Conclusion}
 250 When \gls{IOT} edge devices run the \gls{MTASK} \gls{RTS}, they become little \gls{TOP} engines of their own.
 251 Using just three \gls{ITASK} functions, \gls{MTASK} devices are integrated in \gls{ITASK} seamlessly.
 252 Devices, using any supported type of connection, are integrated in \gls{ITASK} using the \cleaninline{withDevice} function.
 253 Once connected, \gls{MTASK} tasks are sent to the device for execution using \cleaninline{liftmTask}, lifting them to full-fledged \gls{ITASK} tasks.
 254 To lower the bandwidth, tasks can also be preloaded.
 255 Furthermore, the \gls{MTASK} tasks interact with \gls{ITASK} \glspl{SDS} using the \cleaninline{lowerSds} construct.
 256 All of this together allows programming all layers of an \gls{IOT} system from a single source and in a single paradigm.
 257 All details regarding interoperation are automatically taken care of.
 258 The following section contains an elaborate example using all integration functions that has deliberately been placed after the conclusion so that the code listing and description are on facing pages.
 259
 260 \begin{figure}[p]
 261         \begin{leftfullpage}
 262                 \vspace{\headsep}
 263 \section{Home automation}
 264 This section presents an interactive home automation program (\cref{lst:example_home_automation}) to illustrate the integration of the \gls{MTASK} language and the \gls{ITASK} system.
 265 It consists of a web interface for the user to control which tasks are executed on either one of two connected devices: an \gls{ARDUINO} UNO, connected via a serial port; and an ESP8266 based prototyping board called NodeMCU, connected via \gls{TCP} over \gls{WIFI}.
 266
 267 \Crefrange{lst:example:spec1}{lst:example:spec2} show the specification for the devices.
 268 The UNO is connected via serial using the unix filepath \path{/dev/ttyACM0} and the default serial port settings.
 269 The NodeMCU is connected via \gls{WIFI} and hence the \cleaninline{TCPSettings} record is used.
 270 Both types have \cleaninline{channelSync} instances.
 271
 272 The code consists of an \gls{ITASK} part and several \gls{MTASK} parts.
 273 \Crefrange{lst:example:task1}{lst:example:task2} contains the \gls{ITASK} task that coordinates the \gls{IOT} application.
 274 First the devices are connected (\crefrange{lst:example:conn1}{lst:example:conn2}) followed by launching a \cleaninline{parallel} task, visualized as a tabbed window, and a shutdown button to terminate the program (\crefrange{lst:example:par1}{lst:example:par2}).
 275 This parallel task is the controller of the tasks that run on the edge devices.
 276 It contains one task that allows adding new tasks (using \cleaninline{appendTask}) and all other tasks in the process list will be \gls{MTASK} tasks once they are added by the user.
 277 The controller task, \cleaninline{chooseTask} as shown in \crefrange{lst:example:ct1}{lst:example:ct2}, allows the user to pick a task, and sending it to the specified device.
 278 Tasks are picked by index from the \cleaninline{tasks} list (\crefrange{lst:example:tasks1}{lst:example:tasks2}) using \cleaninline{enterChoice}.
 279 The interface that is generated for this is seen in \cref{fig:example_screenshots1}.
 280 After selecting the task, a device is selected (see \cref{fig:example_screenshots2,lst:example:selectdev}).
 281 When both a task and a device are selected, an \gls{ITASK} task is added to the process list using \cleaninline{appendTask}.
 282 Using the helper function \cleaninline{mkTask}, the actual task is selected from the \cleaninline{tasks} list and executed by providing it the device argument.
 283 For example, when selecting the \cleaninline{temperature} task, the current temperature is shown to the user (\cref{fig:example_screenshots3}).
 284 This task just sends a simple temperature monitoring task to the device using \cleaninline{liftmTask} and provides a view on its task value using the \cleaninline{>\&>} \gls{ITASK} combinator.
 285 This combinator allows the observation of the left-hand side task's value through \pgls{SDS}.
 286 The light switch task at \crefrange{lst:example:ls1}{lst:example:ls2} is a task that has bidirectional interaction using the definition of \cleaninline{lightswitch} shown in \cref{lst:mtask_liftsds_ex}.
 287 Using \cleaninline{liftsds}, the status of the light switch is synchronised with the user.
 288 Finally, a task that calculates the factorial of a user-provided number is shown in the list.
 289
 290                 \vspace{4ex}
 291                 \begin{center}
 292                         \begin{subfigure}[b]{.3\linewidth}
 293                                 \includegraphics[width=\linewidth]{home_auto1}
 294                                 \caption{Select task.}%
 295                                 \label{fig:example_screenshots1}
 296                         \end{subfigure}
 297                         \begin{subfigure}[b]{.3\linewidth}
 298                                 \includegraphics[width=\linewidth]{home_auto2}
 299                                 \caption{Select device.}%
 300                                 \label{fig:example_screenshots2}
 301                         \end{subfigure}
 302                         \begin{subfigure}[b]{.3\linewidth}
 303                                 \includegraphics[width=\linewidth]{home_auto3}
 304                                 \caption{View result.}%
 305                                 \label{fig:example_screenshots3}
 306                         \end{subfigure}
 307                         \caption{Screenshots of the home automation example program in action.}%
 308                         \label{fig:example_screenshots}
 309                 \end{center}
 310         \end{leftfullpage}
 311 \end{figure}
 312
 313 \begin{figure}[p]
 314         \begin{fullpage}
 315                 \cleaninputlisting[firstline=12,lastline=50,numbers=left,belowskip=0pt]{lst/example.icl}
 316                 \begin{lstClean}[numbers=left,firstnumber=40,aboveskip=0pt,caption={An example of a home automation program.},label={lst:example_home_automation}]
 317         , ...][+\label{lst:example:tasks2}+]\end{lstClean}
 318         \end{fullpage}
 319 \end{figure}
 320
 321 \input{subfilepostamble}
 322 \end{document}