top/int.tex

   1 \documentclass[../thesis.tex]{subfiles}
   2
   3 \input{subfilepreamble}
   4
   5 \setcounter{chapter}{5}
   6
   7 \begin{document}
   8 \input{subfileprefix}
   9 \chapter{Integration of \texorpdfstring{\gls{MTASK}}{mTask} and \texorpdfstring{\gls{ITASK}}{iTask}}%
  10 \label{chp:integration_with_itask}
  11 \begin{chapterabstract}
  12         This chapter shows the integration of \gls{MTASK} with \gls{ITASK} by showing:
  13         \begin{itemize}
  14                 \item an architectural overview \gls{MTASK} applications;
  15                 \item on the interface for connecting devices;
  16                 \item the interface for lifting \gls{MTASK} tasks to \gls{ITASK} tasks;
  17                 \item a interface for lowering \gls{ITASK} \glspl{SDS} to \gls{MTASK} \glspl{SDS};
  18                 \item and finishes with non-trivial home automation example application using all integration mechanisms;
  19         \end{itemize}
  20 \end{chapterabstract}
  21
  22 The \gls{MTASK} language is a multi-view \gls{DSL}, there are multiple interpretations possible for a single \gls{MTASK} term.
  23 Using the byte code compiler (\cleaninline{BCInterpret}) \gls{DSL} interpretation, \gls{MTASK} tasks are fully integrated in \gls{ITASK}.
  24 They are executed as if they are regular \gls{ITASK} tasks and they can access \glspl{SDS} from \gls{ITASK}.
  25 Devices in the \gls{MTASK} system contain a domain-specific \gls{OS} and are little \gls{TOP} engines in their own respect, being able to execute tasks.
  26
  27 \Cref{fig:mtask_integration} shows the architectural layout of a typical \gls{IOT} system created with \gls{ITASK} and \gls{MTASK}.
  28 The entire system is written as a single \gls{CLEAN} specification where multiple tasks are executed at the same time.
  29 Tasks can access \glspl{SDS} according to many-to-many communication and multiple clients can work on the same task.
  30 The diagram contains three labelled arrows that denote the integration functions between \gls{ITASK} and \gls{MTASK}.
  31 Devices are integrated into the system using the \cleaninline{withDevice} function (see \cref{sec:withdevice}).
  32 Using \cleaninline{liftmTask}, \gls{MTASK} tasks are lifted to a device (see \cref{sec:liftmtask}).
  33 \glspl{SDS} from \gls{ITASK} are lowered to the \gls{MTASK} device using \cleaninline{lowerSds} (see \cref{sec:liftsds}).
  34
  35 \begin{figure}
  36         \centering
  37         \includestandalone{mtask_integration}
  38 \caption{An architectural view of an \imtask{} applications.}%
  39         \label{fig:mtask_integration}
  40 \end{figure}
  41
  42 \section{Connecting edge devices}\label{sec:withdevice}
  43 Edge devices in an \gls{MTASK} application are always coordinated by a server.
  44 This means that they wait for a server to connect to them and send work.
  45 The heavy lifting of connecting an \gls{MTASK} device to an \gls{ITASK} server is done with the \cleaninline{withDevice} function.
  46 This function is given a communication specification and a function producing an \gls{ITASK} task that does something with an \gls{MTASK} device, e.g.\ lift tasks.
  47 By using \gls{HOAS} like this, setting up and tearing down the connection to the device is fully controlled.
  48
  49 In the implementation of the function, all communication with a device happens through a so-called \emph{channels} \gls{SDS}.
  50 The channels contain three fields, a queue of messages that are received, a queue of messages to send and a stop flag.
  51 Every communication method that implements the \cleaninline{channelSync} class can provide the communication with an \gls{MTASK} device.
  52 As of now, serial port communication, direct \gls{TCP} communication and \gls{MQTT} over \gls{TCP} are supported as communication providers (see \cref{lst:connection_types}).
  53 Internally, the \cleaninline{withDevice} task sets up the communication, exchanges specifications with the device, executes the inner task while handling errors, and finally cleans up after closing.
  54 \Cref{lst:mtask_device} shows the types and interface to connecting devices.
  55
  56 \begin{lstClean}[label={lst:mtask_device},caption={Device communication interface in \gls{MTASK}.}]
  57 :: MTDevice //abstract
  58 :: Channels :== ([MTMessageFro], [MTMessageTo], Bool)
  59
  60 class channelSync a :: a (Shared sds Channels) -> Task () | RWShared sds
  61
  62 withDevice :: a (MTDevice -> Task b)
  63         -> Task b | iTask b & channelSync, iTask a
  64 \end{lstClean}
  65
  66 \subsection{Implementation}
  67 \Cref{lst:pseudo_withdevice} shows a pseudocode implementation of the \cleaninline{withDevice} function.
  68 The \cleaninline{MTDevice} abstract type is internally represented as three \gls{ITASK} \gls{SDS} that contain all the current information about the tasks.
  69 The first \gls{SDS} is the information about the \gls{RTS} of the device, i.e.\ metadata on the tasks that are executing, the hardware specification and capabilities, and a list of fresh task identifiers.
  70 The second \gls{SDS} is a map storing downstream \gls{SDS} updates.
  71 When a lowered \gls{SDS} is updated on the device, a message is sent to the server.
  72 This message is initially queued in the map to allow for asynchronous handling of multiple updates.
  73 Finally, the \cleaninline{MTDevices} type contains the communication channels.
  74
  75 The \cleaninline{withDevice} task itself first constructs the \glspl{SDS} using the \gls{ITASK} function \cleaninline{withShared} to create anonymous local \glspl{SDS}.
  76 Then, it performs the following four tasks in parallel to monitor the edge device.
  77 \begin{enumerate}
  78         \item It synchronises the channels using the \cleaninline{channelSync} overloaded function.
  79                 Errors that occur here are converted to the proper \gls{MTASK} exception.
  80         \item Watches the channels for the shutdown flag.
  81                 If the connection is lost with the device unexpectedly, an \gls{MTASK} exception is thrown.
  82         \item Watches the channels for messages and processes them accordingly by changing the device information \gls{SDS} or adding the lowered \gls{SDS} updates to the corresponding \gls{SDS} update queue.
  83         \item Sends a request for a specification. Once the specification is received, the device task is run.
  84                 The task value of this device task is then used as the task value of the \cleaninline{withDevice} task.
  85 \end{enumerate}
  86
  87 \begin{lstClean}[caption={Pseudocode for the \texttt{widthDevice} function},label={lst:pseudo_withdevice}]
  88 withDevice spec deviceTask =
  89         withShared newMap \sdsupdates->
  90         withShared ([], [MTTSpecRequest], False) \channels->
  91         withShared default \dev->parallel
  92                 [ channelSync spec channels
  93                 , watchForShutdown channels
  94                 , watchChannelMessages dev channels
  95                 , waitForSpecification
  96                         >>| deviceTask
  97                         >>* [ifStable: issueShutdown]
  98                 ]
  99 \end{lstClean}
 100
 101 If at any stage an unrecoverable device error occurs, an \gls{ITASK} exception is thrown on the \cleaninline{withDevice} task.
 102 This exception can be caught in order to devise fail-safe mechanisms.
 103 For example, if a device fails, the task can be sent to another device as can be seen in \cref{lst:failover}.
 104 This function executes an \gls{MTASK} task on a pool of devices connected through \gls{TCP}.
 105 If a device error occurs during execution, the next device in the pool is tried until the pool is exhausted
 106 If another error occurs, it is rethrown for a parent task to catch.
 107
 108 \begin{lstClean}[caption={An \gls{MTASK} failover combinator.},label={lst:failover}]
 109 failover :: [TCPSettings] (Main (MTask BCInterpret a)) -> Task a
 110 failover []     _     = throw "Exhausted device pool"
 111 failover [d:ds] mtask = try (withDevice d (liftmTask mtask)) except
 112 where except MTEUnexpectedDisconnect = failover ds mtask
 113           except _                       = throw e
 114 \end{lstClean}
 115
 116 \section{Lifting \texorpdfstring{\gls{MTASK}}{mTask} tasks}\label{sec:liftmtask}
 117 Once the connection with the device is established, \gls{MTASK} tasks are lifted to \gls{MTASK} tasks using the \cleaninline{liftmTask} family of functions (see \cref{lst:liftmtask}).
 118 Given an \gls{MTASK} task in the \cleaninline{BCInterpret} view and a device obtained from \cleaninline{withDevice}, an \gls{ITASK} task is returned.
 119 This \gls{ITASK} task proxies the \gls{MTASK} task that is executed on the microcontroller.
 120 Hence, when for example observing the task value, the actual task value from the microcontroller is observed.
 121
 122 \begin{lstClean}[label={lst:liftmtask},caption={The interface for lifting \gls{MTASK} tasks to \gls{ITASK} tasks.}]
 123 liftmTask :: (Main (MTask BCInterpret a)) MTDevice -> Task a | iTask a
 124 \end{lstClean}
 125
 126 \subsection{Implementation}
 127 \Cref{lst:liftmTask_pseudo} shows the pseudocode for the \cleaninline{liftmTask} implementation
 128 The first argument is the task and the second argument is the device which is just an \gls{ADT} containing the \glspl{SDS} referring to the device information, the \gls{SDS} update queue, and the channels.
 129 First a fresh identifier for the task is generated using the device state.
 130 With this identifier, the cleanup hook can be installed.
 131 This is done to assure the task is removed from the edge device if the \gls{ITASK} task coordinating it is destroyed.
 132 Tasks in \gls{ITASK} are destroyed when for example it is executed in parallel with another task and the parallel combinator terminates or when the condition to step holds in a sequential task combination.
 133 Then the \gls{MTASK} compiler is invoked, its only argument besides the task is a function doing something with the results of the compilation, i.e.\ the lowered \glspl{SDS} and the messages containing the compiled and serialised task.
 134 With the result of the compilation, the task can be executed.
 135 First the messages are put in the channels, sending them to the device.
 136 Then, in parallel:
 137 \begin{enumerate*}
 138         \item the value is watched by looking in the device state \gls{SDS}, this task also determines the task value of the whole task
 139         \item the downstream \glspl{SDS} are monitored, i.e.\ the \cleaninline{sdsupdates} \gls{SDS} is monitored and updates from the device are applied to the associated \gls{ITASK} \gls{SDS}
 140         \item the upstroam \glspl{SDS} are monitored by spawning tasks that watch these \glspl{SDS}, if one is updated, the novel value is sent to the edge device.
 141 \end{enumerate*}
 142
 143 \begin{lstClean}[label={lst:liftmTask_pseudo},caption={Pseudocode implementation for \texttt{liftmTask}.}]
 144 liftmTask task (MTDevice dev sdsupdates channels)
 145         = freshTaskId dev
 146         >>= \tid->withCleanupHook (sendmessage [MTTTaskDel tid] channels) (
 147                 compile task \mrefs msgs->
 148                         sendMessage msgs channels
 149                         >>| waitForReturnAndValue tid dev
 150                         -|| watchSharesDownstream mrefs tid sdsupdates
 151                         -|| watchSharesUpstream mrefs channels tid)
 152 \end{lstClean}
 153
 154 Sending the complete byte code to the device is not always a suitable option.
 155 For example, when the device is connected through an unstable or slow connection, sending the entire byte code may induce lots of delay.
 156 To mitigate this problem, \gls{MTASK} tasks can be preloaded on a device.
 157 Preloading means that the task is compiled and integrated into the device firmware.
 158 On receiving a \cleaninline{TaskPrep}, a hashed value of the task to be sent is included.
 159 The device then checks the preloaded task registry and uses the local version if the hash matches.
 160 Of course this only works for tasks that are not tailor made for the current work specification and not depend on run time information.
 161 The interface for task preloading can be found in \cref{lst:preload}.
 162 Given an \gls{MTASK} task, a header file is created that is placed in the source code directory of the \gls{RTS} to include it in the firmware.
 163
 164 \begin{lstClean}[label={lst:preload},caption={Preloading tasks in \gls{MTASK}.}]
 165 preloadTask :: (Main (MTask BCInterpret u)) -> Task String
 166 \end{lstClean}
 167
 168 \section{Lowering \texorpdfstring{\gls{ITASK}}{iTask} \texorpdfstring{\glsxtrlongpl{SDS}}{shared data sources}}\label{sec:liftsds}
 169 Lowering \gls{ITASK} \glspl{SDS} to \gls{MTASK} \glspl{SDS} is something that mostly happens at the compiler level using the \cleaninline{lowerSds} function (see \cref{lst:mtask_itasksds}).
 170 \Glspl{SDS} in \gls{MTASK} must always have an initial value.
 171 For regular \gls{SDS} this value is given in the source code, for lowered \gls{ITASK} \glspl{SDS} this value is obtained by reading the values once just before sending the task to the edge device.
 172 On the device itself, there is just one difference between lowered \glspl{SDS} and regular \glspl{SDS}: after changing \pgls{SDS}, a message is sent to the server containing this new value.
 173 The \cleaninline{withDevice} task (see \cref{sec:withdevice}) receives and processes this message by writing to the \gls{ITASK} \gls{SDS}.
 174 Tasks watching this \gls{SDS} get notified then through the normal notification mechanism of \gls{ITASK}.
 175
 176 \begin{lstClean}[label={lst:mtask_itasksds},caption={Lowered \gls{ITASK} \glspl{SDS} in \gls{MTASK}.}]
 177 class lowerSds v where
 178         lowerSds :: ((v (Sds t)) -> In (Shared sds t) (Main (MTask v u)))
 179                 -> Main (MTask v u) | RWShared sds
 180 \end{lstClean}
 181
 182 As an example, \cref{lst:mtask_liftsds_ex} shows a light switch function producing an \imtask{} task when given a device handle.
 183 First an \gls{ITASK} \gls{SDS} of the type boolean is created.
 184 This boolean represents the state of the light.
 185 The \gls{MTASK} task uses this \gls{SDS} to turn on or off the light.
 186 The \gls{ITASK} task that runs in parallel allows interactive updating of this state.
 187
 188 \begin{lstClean}[label={lst:mtask_liftsds_ex},caption={Interactive light switch program.}]
 189 lightswitch :: MTDevice -> Task Bool
 190 lightswitch dev  =
 191         withShared False \sh->
 192                     liftmTask (mtask sh) dev
 193                 -|| updateSharedInformation [] sh
 194                 <<@ Hint "Light switch"
 195 where
 196         mtask :: (Shared sds Bool) -> Main (MTask v Bool)
 197                 | mtask, lowerSds v & RWShared sds
 198         mtask sh =
 199                 declarePin D13 PMOutput \d13->
 200                 lowerSds \ls=sh
 201                 In fun \f=(\st->
 202                              getSds ls
 203                         >>*. [IfValue ((!=.)st) (\v->writeD d13 v)]
 204                         >>|. f (Not st))
 205                 In {main=f true}
 206 \end{lstClean}
 207
 208 \subsection{Implementation}
 209 The compilation of the code and the serialisation of the data throws away all typing information.
 210 \Glspl{SDS} are stored in the compiler state as a map from identifiers to either an initial value or an \cleaninline{MTLens}.
 211 The \cleaninline{MTLens} is a type synonym for a \gls{SDS} that represents the typeless serialised value of the underlying \gls{SDS}.
 212 This is done so that the \cleaninline{withDevice} task can write the received \gls{SDS} updates to the according \gls{SDS} independently.
 213 The \gls{ITASK} notification mechanism then takes care of the rest.
 214 Such \pgls{SDS} is created by using the \cleaninline{mapReadWriteError} which, given a pair of read and write functions with error handling, produces \pgls{SDS} with the lens embedded.
 215 The read function transforms converts the typed value to a typeless serialised value.
 216 The write function will, given the new serialised value and the old typed value, produce a new typed value.
 217 It tries to decode the serialised value, if that succeeds, it is written to the underlying \gls{SDS}, an error is thrown otherwise.
 218 \Cref{lst:mtask_itasksds_lens} provides the implementation for this:
 219
 220 % VimTeX: SynIgnore on
 221 \begin{lstClean}[label={lst:mtask_itasksds_lens},caption={Lens applied to lowered \gls{ITASK} \glspl{SDS} in \gls{MTASK}.}]
 222 lens :: (Shared sds a) -> MTLens | type a & RWShared sds
 223 lens sds = mapReadWriteError
 224         ( \r->   Ok (fromString (toByteCode{|*|} r)
 225         , \w r-> ?Just <$> iTasksDecode (toString w)
 226         ) ?None sds
 227 \end{lstClean}
 228 % VimTeX: SynIgnore off
 229
 230 \Cref{lst:mtask_itasksds_lift} shows the code for the implementation of \cleaninline{lowerSds} that uses the \cleaninline{lens} function shown earlier.
 231 First, the \gls{SDS} to be lowered is extracted from the expression by bootstrapping the fixed point with a dummy value.
 232 This is safe because the expression on the right-hand side of the \cleaninline{In} is never evaluated.
 233 Then, using \cleaninline{addSdsIfNotExist}, the identifier for this particular \gls{SDS} is either retrieved from the compiler state or generated freshly.
 234 This identifier is then used to provide a reference to the \cleaninline{def} definition to evaluate the main expression.
 235
 236 % VimTeX: SynIgnore on
 237 \begin{lstClean}[label={lst:mtask_itasksds_lift},caption={Lens applied to lowered \gls{ITASK} \glspl{SDS} in \gls{MTASK}.}]
 238 instance lowerSds BCInterpret where
 239         lowerSds def = {main =
 240                         let (t In _) = def (abort "lowerSds: expression too strict")
 241                         in addSdsIfNotExist (Right $ lens t)
 242                                 >>= \sdsi->let (_ In e) = def (pure (Sds sdsi)) in e.main
 243                 }\end{lstClean}
 244 % VimTeX: SynIgnore off
 245
 246 \section{Home automation}
 247 \todo[inline]{Staat dit hier goed of moet dit naar een andere sectie?}
 248 This section presents a interactive home automation program (\Cref{lst:example_home_automation}) to illustrate \gls{MTASK}'s integration with \gls{ITASK}.
 249 It consists of a web interface for the user to control which tasks may be executed on either of two connected devices: an \gls{ARDUINO} UNO, connected via a serial port; and an ESP8266 based prototyping board called NodeMCU, connected via \gls{TCP} over \gls{WIFI}.
 250
 251 \Crefrange{lst:example:spec1}{lst:example:spec2} show the specification for the devices.
 252 The UNO is connected via serial using the unix filepath \path{/dev/ttyACM0} and the default serial port settings.
 253 The NodeMCU is connected via \gls{WIFI} and hence the \cleaninline{TCPSettings} record is used.
 254 Both types have \cleaninline{channelSync} instances.
 255
 256 The code consists of an \gls{ITASK} part and several \gls{MTASK} parts.
 257 \Crefrange{lst:example:task1}{lst:example:task2} containing the \gls{ITASK} task that coordinates the \gls{IOT} application.
 258 It first connects the devices (\crefrange{lst:example:conn1}{lst:example:conn2}) followed by launching a \cleaninline{parallel} task, visualized as a tabbed window, and a shutdown button to terminate the program (\crefrange{lst:example:par1}{lst:example:par2}).
 259 This parallel task is the controller of the tasks that run on the edge devices.
 260 It contains one task that allows adding new tasks (using \cleaninline{appendTask}) and all other tasks in the process list will be \gls{MTASK} tasks once they are added by the user.
 261 The controller task, \cleaninline{chooseTask} as shown in \crefrange{lst:example:ct1}{lst:example:ct2}, allows the user to pick a task, sending it to the specified device.
 262 Tasks are picked by index from the \cleaninline{tasks} list (\crefrange{lst:example:tasks1}{lst:example:tasks2}) using \cleaninline{enterChoice}.
 263 The interface that is generated for this can be seen in \cref{fig:example_screenshots1}.
 264 After selecting the task, a device is selected (see \cref{fig:example_screenshots2,lst:example:selectdev}).
 265 When both a task and a device is selected, an \gls{ITASK} task is added to the process list using \cleaninline{appendTask}.
 266 Using the helper function \cleaninline{mkTask}, the actual task is selected from the \cleaninline{tasks} list and executed by providing the device argument.
 267 For example, when selecting the \cleaninline{temperature} task, the current temperature is shown to the user (\cref{fig:example_screenshots3}).
 268 This task just sends a simple temperature monitoring task to the device using \cleaninline{liftmTask} and provides a view on its task value using the \cleaninline{>\&>}\footnotemark{} \gls{ITASK} combinator.
 269 \footnotetext{\cleaninline{(>\&>) infixl 1 :: (Task a) ((SDSLens () (? a) ()) -> Task b) -> Task b \| iTask a \& iTask b}}
 270 The light switch task at \crefrange{lst:example:ls1}{lst:example:ls2} is a task that has bidirectional interaction using the definition of \cleaninline{lightswitch} shown in \cref{lst:mtask_liftsds_ex}.
 271 Using \cleaninline{liftsds}, the status of the light switch is synchronised with the user.
 272 The task on the edge device continuously monitors the value of the lowered \gls{SDS}.
 273 If it is different from the current state, the new value is written to the digital \gls{GPIO} pin 13 and the monitoring function is recursively called.
 274
 275 \begin{figure}[ht]
 276         \centering
 277         \begin{subfigure}[b]{.3\linewidth}
 278                 \includegraphics[width=\linewidth]{home_auto1}
 279                 \caption{Select task.}%
 280                 \label{fig:example_screenshots1}
 281         \end{subfigure}
 282         \begin{subfigure}[b]{.3\linewidth}
 283                 \includegraphics[width=\linewidth]{home_auto2}
 284                 \caption{Select device.}%
 285                 \label{fig:example_screenshots2}
 286         \end{subfigure}
 287         \begin{subfigure}[b]{.3\linewidth}
 288                 \includegraphics[width=\linewidth]{home_auto3}
 289                 \caption{View result.}%
 290                 \label{fig:example_screenshots3}
 291         \end{subfigure}
 292         \caption{Screenshots of the home automation example program in action.}%
 293         \label{fig:example_screenshots}
 294 \end{figure}
 295
 296 \begin{figure}
 297         \cleaninputlisting[firstline=12,lastline=50,numbers=left,belowskip=0pt]{lst/example.icl}
 298         \begin{lstClean}[numbers=left,firstnumber=40,aboveskip=0pt,caption={An example of a home automation program.},label={lst:example_home_automation}]
 299         , ...][+\label{lst:example:tasks2}+]\end{lstClean}
 300 \end{figure}
 301
 302 \section{Conclusion}
 303 When \gls{IOT} edge devices run the \gls{MTASK} \gls{RTS}, they become little \gls{TOP} engines of their own.
 304 Using just three \gls{ITASK} functions, \gls{MTASK} devices are integrated in \gls{ITASK} seamlessly.
 305 Devices, using any supported type of connection, are integrated in \gls{ITASK} using the \cleaninline{withDevice} function.
 306 Once connected, \gls{MTASK} tasks are sent to the device for execution using \cleaninline{liftmTask}, lifting them to full-fledged \gls{ITASK} tasks.
 307 To lower the bandwidth, tasks can also be preloaded.
 308 Furthermore, the \gls{MTASK} tasks interact with \gls{ITASK} \glspl{SDS} using the \cleaninline{lowerSds} construct.
 309 All of this together allows programming all layers of an \gls{IOT} system from a single source and in a single paradigm.
 310 All details regarding interoperation are automatically taken care of.
 311
 312 \input{subfilepostamble}
 313 \end{document}