X-Git-Url: https://git.martlubbers.net/?a=blobdiff_plain;f=top%2Fint.tex;h=93a161eab64a41820848add9e8f4c1eb7e03d333;hb=c1b9a0995445470a66854861c04df440b7d7ffb3;hp=a1742fc6a1019c42ae968a2c784bbd3f0575608b;hpb=d6c8d6cbc9c88d449ac784eda20a213fd6b46669;p=phd-thesis.git diff --git a/top/int.tex b/top/int.tex index a1742fc..93a161e 100644 --- a/top/int.tex +++ b/top/int.tex @@ -2,48 +2,55 @@ \input{subfilepreamble} +\setcounter{chapter}{5} + \begin{document} \input{subfileprefix} - \chapter{Integration with \texorpdfstring{\gls{ITASK}}{iTask}}% \label{chp:integration_with_itask} \begin{chapterabstract} This chapter shows the integration of \gls{MTASK} with \gls{ITASK} by showing: \begin{itemize} - \item an architectural overview of \gls{MTASK}; + \item an architectural overview \gls{MTASK} applications; \item on the interface for connecting devices; \item the interface for lifting \gls{MTASK} tasks to \gls{ITASK} tasks; - \item and interface for lifting \gls{ITASK} \glspl{SDS} to \gls{MTASK} \glspl{SDS}. + \item a interface for lowering \gls{ITASK} \glspl{SDS} to \gls{MTASK} \glspl{SDS}; + \item and finishes with non-trivial home automation example application using all integration mechanisms; \end{itemize} \end{chapterabstract} -The \gls{MTASK} language is a multi-view \gls{DSL}, i.e.\ there are multiple interpretations possible for a single \gls{MTASK} term. -Using the byte code compiler (\cleaninline{BCInterpret}) \gls{DSL} interpretation, \gls{MTASK} tasks can be fully integrated in \gls{ITASK}. -They are executed as if they are regular \gls{ITASK} tasks and they communicate may access \glspl{SDS} from \gls{ITASK} as well. -\Gls{MTASK} devices contain a domain-specific \gls{OS} (\gls{RTS}) and are little \gls{TOP} engines in their own respect, being able to execute tasks. +The \gls{MTASK} language is a multi-view \gls{DSL}, there are multiple interpretations possible for a single \gls{MTASK} term. +Using the byte code compiler (\cleaninline{BCInterpret}) \gls{DSL} interpretation, \gls{MTASK} tasks are fully integrated in \gls{ITASK}. +They are executed as if they are regular \gls{ITASK} tasks and they can access \glspl{SDS} from \gls{ITASK}. +Devices in the \gls{MTASK} system contain a domain-specific \gls{OS} and are little \gls{TOP} engines in their own respect, being able to execute tasks. + \Cref{fig:mtask_integration} shows the architectural layout of a typical \gls{IOT} system created with \gls{ITASK} and \gls{MTASK}. The entire system is written as a single \gls{CLEAN} specification where multiple tasks are executed at the same time. Tasks can access \glspl{SDS} according to many-to-many communication and multiple clients can work on the same task. +The diagram contains three labelled arrows that denote the integration functions between \gls{ITASK} and \gls{MTASK}. Devices are integrated into the system using the \cleaninline{withDevice} function (see \cref{sec:withdevice}). Using \cleaninline{liftmTask}, \gls{MTASK} tasks are lifted to a device (see \cref{sec:liftmtask}). -\Gls{ITASK} \glspl{SDS} are lifted to the \gls{MTASK} device using \cleaninline{liftsds} (see \cref{sec:liftmtask}). +\glspl{SDS} from \gls{ITASK} are lowered to the \gls{MTASK} device using \cleaninline{lowerSds} (see \cref{sec:liftsds}). -\begin{figure}[ht] +\begin{figure} \centering \includestandalone{mtask_integration} - \caption{\Gls{MTASK}'s integration with \gls{ITASK}.}% +\caption{An architectural view of an \imtask{} applications.}% \label{fig:mtask_integration} \end{figure} -\section{Devices}\label{sec:withdevice} -When interpreted by the byte code compiler view, an \gls{MTASK} task produces a compiler. -This compiler is exceuted at run time so that the resulting byte code can be sent to an edge device. -All communication with this device happens through a so-called \emph{channels} \gls{SDS}. +\section{Connecting edge devices}\label{sec:withdevice} +Edge devices in an \gls{MTASK} application are always coordinated by a server. +This means that they wait for a server to connect to them and send work. +The heavy lifting of connecting an \gls{MTASK} device to an \gls{ITASK} server is done with the \cleaninline{withDevice} function. +This function is given a communication specification and a function producing an \gls{ITASK} task that does something with an \gls{MTASK} device, e.g.\ lift tasks. +By using \gls{HOAS} like this, setting up and tearing down the connection to the device is fully controlled. + +In the implementation of the function, all communication with a device happens through a so-called \emph{channels} \gls{SDS}. The channels contain three fields, a queue of messages that are received, a queue of messages to send and a stop flag. Every communication method that implements the \cleaninline{channelSync} class can provide the communication with an \gls{MTASK} device. -As of now, serial port communication, direct \gls{TCP} communication and \gls{MQTT} over \gls{TCP} are supported as communication providers. -The \cleaninline{withDevice} function transforms a communication provider and a task that does something with this device to an \gls{ITASK} task. -Internally, the task sets up the communication, exchanges specifications, executes the inner task while handling errors, and finally cleans up after closing. +As of now, serial port communication, direct \gls{TCP} communication and \gls{MQTT} over \gls{TCP} are supported as communication providers (see \cref{lst:connection_types}). +Internally, the \cleaninline{withDevice} task sets up the communication, exchanges specifications with the device, executes the inner task while handling errors, and finally cleans up after closing. \Cref{lst:mtask_device} shows the types and interface to connecting devices. \begin{lstClean}[label={lst:mtask_device},caption={Device communication interface in \gls{MTASK}.}] @@ -52,33 +59,255 @@ Internally, the task sets up the communication, exchanges specifications, execut class channelSync a :: a (Shared sds Channels) -> Task () | RWShared sds -withDevice :: a (MTDevice -> Task b) -> Task b | iTask b & channelSync, iTask a +withDevice :: a (MTDevice -> Task b) + -> Task b | iTask b & channelSync, iTask a +\end{lstClean} + +\subsection{Implementation} +\Cref{lst:pseudo_withdevice} shows a pseudocode implementation of the \cleaninline{withDevice} function. +The \cleaninline{MTDevice} abstract type is internally represented as three \gls{ITASK} \gls{SDS} that contain all the current information about the tasks. +The first \gls{SDS} is the information about the \gls{RTS} of the device, i.e.\ metadata on the tasks that are executing, the hardware specification and capabilities, and a list of fresh task identifiers. +The second \gls{SDS} is a map storing downstream \gls{SDS} updates. +When a lowered \gls{SDS} is updated on the device, a message is sent to the server. +This message is initially queued in the map to allow for asynchronous handling of multiple updates. +Finally, the \cleaninline{MTDevices} type contains the communication channels. + +The \cleaninline{withDevice} task itself first constructs the \glspl{SDS} using the \gls{ITASK} function \cleaninline{withShared} to create anonymous local \glspl{SDS}. +Then, it performs the following four tasks in parallel to monitor the edge device. +\begin{enumerate} + \item It synchronises the channels using the \cleaninline{channelSync} overloaded function. + Errors that occur here are converted to the proper \gls{MTASK} exception. + \item Watches the channels for the shutdown flag. + If the connection is lost with the device unexpectedly, an \gls{MTASK} exception is thrown. + \item Watches the channels for messages and processes them accordingly by changing the device information \gls{SDS} or adding the lowered \gls{SDS} updates to the corresponding \gls{SDS} update queue. + \item Sends a request for a specification. Once the specification is received, the device task is run. + The task value of this device task is then used as the task value of the \cleaninline{withDevice} task. +\end{enumerate} + +\begin{lstClean}[caption={Pseudocode for the \texttt{widthDevice} function},label={lst:pseudo_withdevice}] +withDevice spec deviceTask = + withShared newMap \sdsupdates-> + withShared ([], [MTTSpecRequest], False) \channels-> + withShared default \dev->parallel + [ channelSync spec channels + , watchForShutdown channels + , watchChannelMessages dev channels + , waitForSpecification + >>| deviceTask + >>* [ifStable: issueShutdown] + ] \end{lstClean} -\section{Lifting tasks}\label{sec:liftmtask} -Once the connection with the device is established, \gls{MTASK} tasks can be lifted to \gls{MTASK} tasks using the \cleaninline{liftmTask} family of functions (see \cref{lst:liftmtask}). +If at any stage an unrecoverable device error occurs, an \gls{ITASK} exception is thrown on the \cleaninline{withDevice} task. +This exception can be caught in order to devise fail-safe mechanisms. +For example, if a device fails, the task can be sent to another device as can be seen in \cref{lst:failover}. +This function executes an \gls{MTASK} task on a pool of devices connected through \gls{TCP}. +If a device error occurs during execution, the next device in the pool is tried until the pool is exhausted +If another error occurs, it is rethrown for a parent task to catch. + +\begin{lstClean}[caption={An \gls{MTASK} failover combinator.},label={lst:failover}] +failover :: [TCPSettings] (Main (MTask BCInterpret a)) -> Task a +failover [] _ = throw "Exhausted device pool" +failover [d:ds] mtask = try (withDevice d (liftmTask mtask)) except +where except MTEUnexpectedDisconnect = failover ds mtask + except _ = throw e +\end{lstClean} + +\section{Lifting \texorpdfstring{\gls{MTASK}}{mTask} tasks}\label{sec:liftmtask} +Once the connection with the device is established, \gls{MTASK} tasks are lifted to \gls{MTASK} tasks using the \cleaninline{liftmTask} family of functions (see \cref{lst:liftmtask}). Given an \gls{MTASK} task in the \cleaninline{BCInterpret} view and a device obtained from \cleaninline{withDevice}, an \gls{ITASK} task is returned. -This \gls{ITASK} task tethers the \gls{MTASK} task that is executed on the microcontroller. +This \gls{ITASK} task proxies the \gls{MTASK} task that is executed on the microcontroller. Hence, when for example observing the task value, the actual task value from the microcontroller is observed. \begin{lstClean}[label={lst:liftmtask},caption={The interface for lifting \gls{MTASK} tasks to \gls{ITASK} tasks.}] -liftmTask :: (Main (MTask BCInterpret u)) MTDevice -> Task u | iTask u +liftmTask :: (Main (MTask BCInterpret a)) MTDevice -> Task a | iTask a +\end{lstClean} + +\subsection{Implementation} +\Cref{lst:liftmTask_pseudo} shows the pseudocode for the \cleaninline{liftmTask} implementation +The first argument is the task and the second argument is the device which is just an \gls{ADT} containing the \glspl{SDS} referring to the device information, the \gls{SDS} update queue, and the channels. +First a fresh identifier for the task is generated using the device state. +With this identifier, the cleanup hook can be installed. +This is done to assure the task is removed from the edge device if the \gls{ITASK} task coordinating it is destroyed. +Tasks in \gls{ITASK} are destroyed when for example it is executed in parallel with another task and the parallel combinator terminates or when the condition to step holds in a sequential task combination. +Then the \gls{MTASK} compiler is invoked, its only argument besides the task is a function doing something with the results of the compilation, i.e.\ the lowered \glspl{SDS} and the messages containing the compiled and serialised task. +With the result of the compilation, the task can be executed. +First the messages are put in the channels, sending them to the device. +Then, in parallel: +\begin{enumerate*} + \item the value is watched by looking in the device state \gls{SDS}, this task also determines the task value of the whole task + \item the downstream \glspl{SDS} are monitored, i.e.\ the \cleaninline{sdsupdates} \gls{SDS} is monitored and updates from the device are applied to the associated \gls{ITASK} \gls{SDS} + \item the upstroam \glspl{SDS} are monitored by spawning tasks that watch these \glspl{SDS}, if one is updated, the novel value is sent to the edge device. +\end{enumerate*} + +\begin{lstClean}[label={lst:liftmTask_pseudo},caption={Pseudocode implementation for \texttt{liftmTask}.}] +liftmTask task (MTDevice dev sdsupdates channels) + = freshTaskId dev + >>= \tid->withCleanupHook (sendmessage [MTTTaskDel tid] channels) ( + compile task \mrefs msgs-> + sendMessage msgs channels + >>| waitForReturnAndValue tid dev + -|| watchSharesDownstream mrefs tid sdsupdates + -|| watchSharesUpstream mrefs channels tid) +\end{lstClean} + +Sending the complete byte code to the device is not always a suitable option. +For example, when the device is connected through an unstable or slow connection, sending the entire byte code may induce lots of delay. +To mitigate this problem, \gls{MTASK} tasks can be preloaded on a device. +Preloading means that the task is compiled and integrated into the device firmware. +On receiving a \cleaninline{TaskPrep}, a hashed value of the task to be sent is included. +The device then checks the preloaded task registry and uses the local version if the hash matches. +Of course this only works for tasks that are not tailor made for the current work specification and not depend on run time information. +The interface for task preloading can be found in \cref{lst:preload}. +Given an \gls{MTASK} task, a header file is created that is placed in the source code directory of the \gls{RTS} to include it in the firmware. + +\begin{lstClean}[label={lst:preload},caption={Preloading tasks in \gls{MTASK}.}] +preloadTask :: (Main (MTask BCInterpret u)) -> Task String \end{lstClean} -Under the hood, \cleaninline{liftmTask} compiler the \gls{MTASK} task to byte code, gathers the initial values for the lifted \glspl{SDS} and sends the data to the device. -Once acknowledged, the \gls{MTASK} task value and the \glspl{SDS} are monitored. -If the \gls{ITASK} server updates the value of a lifted \gls{SDS}, the appropriate message is sent to the \gls{MTASK} to notify it of the change. -Furthermore, the \gls{MTASK} device may update the \gls{SDS}, and in that case the message is picked up by the \cleaninline{liftmTask} task and the \gls{SDS} on the \gls{ITASK} server updated accordingly. +\section{Lowering \texorpdfstring{\gls{ITASK}}{iTask} \texorpdfstring{\glsxtrlongpl{SDS}}{shared data sources}}\label{sec:liftsds} +Lowering \gls{ITASK} \glspl{SDS} to \gls{MTASK} \glspl{SDS} is something that mostly happens at the compiler level using the \cleaninline{lowerSds} function (see \cref{lst:mtask_itasksds}). +\Glspl{SDS} in \gls{MTASK} must always have an initial value. +For regular \gls{SDS} this value is given in the source code, for lowered \gls{ITASK} \glspl{SDS} this value is obtained by reading the values once just before sending the task to the edge device. +On the device itself, there is just one difference between lowered \glspl{SDS} and regular \glspl{SDS}: after changing \pgls{SDS}, a message is sent to the server containing this new value. +The \cleaninline{withDevice} task (see \cref{sec:withdevice}) receives and processes this message by writing to the \gls{ITASK} \gls{SDS}. +Tasks watching this \gls{SDS} get notified then through the normal notification mechanism of \gls{ITASK}. -\section{Lifting \texorpdfstring{\glsxtrlongpl{SDS}}{shared data sources}}\label{sec:liftsds} -\begin{lstClean}[label={lst:mtask_itasksds},caption={Lifted \gls{ITASK} \glspl{SDS} in \gls{MTASK}.}] -class liftsds v where - liftsds :: ((v (Sds t)) -> In (Shared sds t) (Main (MTask v u))) +\begin{lstClean}[label={lst:mtask_itasksds},caption={Lowered \gls{ITASK} \glspl{SDS} in \gls{MTASK}.}] +class lowerSds v where + lowerSds :: ((v (Sds t)) -> In (Shared sds t) (Main (MTask v u))) -> Main (MTask v u) | RWShared sds \end{lstClean} -\section{Conclusion} +As an example, \cref{lst:mtask_liftsds_ex} shows a light switch function producing an \imtask{} task when given a device handle. +First an \gls{ITASK} \gls{SDS} of the type boolean is created. +This boolean represents the state of the light. +The \gls{MTASK} task uses this \gls{SDS} to turn on or off the light. +The \gls{ITASK} task that runs in parallel allows interactive updating of this state. + +\begin{lstClean}[label={lst:mtask_liftsds_ex},caption={Interactive light switch program.}] +lightswitch :: MTDevice -> Task Bool +lightswitch dev = + withShared False \sh-> + liftmTask (mtask sh) dev + -|| updateSharedInformation [] sh + <<@ Hint "Light switch" +where + mtask :: (Shared sds Bool) -> Main (MTask v Bool) + | mtask, lowerSds v & RWShared sds + mtask sh = + declarePin D13 PMOutput \d13-> + lowerSds \ls=sh + In fun \f=(\st-> + getSds ls + >>*. [IfValue ((!=.)st) (\v->writeD d13 v)] + >>|. f (Not st)) + In {main=f true} +\end{lstClean} + +\subsection{Implementation} +The compilation of the code and the serialisation of the data throws away all typing information. +\Glspl{SDS} are stored in the compiler state as a map from identifiers to either an initial value or an \cleaninline{MTLens}. +The \cleaninline{MTLens} is a type synonym for a \gls{SDS} that represents the typeless serialised value of the underlying \gls{SDS}. +This is done so that the \cleaninline{withDevice} task can write the received \gls{SDS} updates to the according \gls{SDS} independently. +The \gls{ITASK} notification mechanism then takes care of the rest. +Such \pgls{SDS} is created by using the \cleaninline{mapReadWriteError} which, given a pair of read and write functions with error handling, produces \pgls{SDS} with the lens embedded. +The read function transforms converts the typed value to a typeless serialised value. +The write function will, given the new serialised value and the old typed value, produce a new typed value. +It tries to decode the serialised value, if that succeeds, it is written to the underlying \gls{SDS}, an error is thrown otherwise. +\Cref{lst:mtask_itasksds_lens} provides the implementation for this: + +% VimTeX: SynIgnore on +\begin{lstClean}[label={lst:mtask_itasksds_lens},caption={Lens applied to lowered \gls{ITASK} \glspl{SDS} in \gls{MTASK}.}] +lens :: (Shared sds a) -> MTLens | type a & RWShared sds +lens sds = mapReadWriteError + ( \r-> Ok (fromString (toByteCode{|*|} r) + , \w r-> ?Just <$> iTasksDecode (toString w) + ) ?None sds +\end{lstClean} +% VimTeX: SynIgnore off + +\Cref{lst:mtask_itasksds_lift} shows the code for the implementation of \cleaninline{lowerSds} that uses the \cleaninline{lens} function shown earlier. +First, the \gls{SDS} to be lowered is extracted from the expression by bootstrapping the fixed point with a dummy value. +This is safe because the expression on the right-hand side of the \cleaninline{In} is never evaluated. +Then, using \cleaninline{addSdsIfNotExist}, the identifier for this particular \gls{SDS} is either retrieved from the compiler state or generated freshly. +This identifier is then used to provide a reference to the \cleaninline{def} definition to evaluate the main expression. + +% VimTeX: SynIgnore on +\begin{lstClean}[label={lst:mtask_itasksds_lift},caption={Lens applied to lowered \gls{ITASK} \glspl{SDS} in \gls{MTASK}.}] +instance lowerSds BCInterpret where + lowerSds def = {main = + let (t In _) = def (abort "lowerSds: expression too strict") + in addSdsIfNotExist (Right $ lens t) + >>= \sdsi->let (_ In e) = def (pure (Sds sdsi)) in e.main + }\end{lstClean} +% VimTeX: SynIgnore off + +\section{Home automation} +\todo[inline]{Staat dit hier goed of moet dit naar een andere sectie?} +This section presents a interactive home automation program (\Cref{lst:example_home_automation}) to illustrate \gls{MTASK}'s integration with \gls{ITASK}. +It consists of a web interface for the user to control which tasks may be executed on either of two connected devices: an \gls{ARDUINO} UNO, connected via a serial port; and an ESP8266 based prototyping board called NodeMCU, connected via \gls{TCP} over \gls{WIFI}. +\Crefrange{lst:example:spec1}{lst:example:spec2} show the specification for the devices. +The UNO is connected via serial using the unix filepath \path{/dev/ttyACM0} and the default serial port settings. +The NodeMCU is connected via \gls{WIFI} and hence the \cleaninline{TCPSettings} record is used. +Both types have \cleaninline{channelSync} instances. + +The code consists of an \gls{ITASK} part and several \gls{MTASK} parts. +\Crefrange{lst:example:task1}{lst:example:task2} containing the \gls{ITASK} task that coordinates the \gls{IOT} application. +It first connects the devices (\crefrange{lst:example:conn1}{lst:example:conn2}) followed by launching a \cleaninline{parallel} task, visualized as a tabbed window, and a shutdown button to terminate the program (\crefrange{lst:example:par1}{lst:example:par2}). +This parallel task is the controller of the tasks that run on the edge devices. +It contains one task that allows adding new tasks (using \cleaninline{appendTask}) and all other tasks in the process list will be \gls{MTASK} tasks once they are added by the user. +The controller task, \cleaninline{chooseTask} as shown in \crefrange{lst:example:ct1}{lst:example:ct2}, allows the user to pick a task, sending it to the specified device. +Tasks are picked by index from the \cleaninline{tasks} list (\crefrange{lst:example:tasks1}{lst:example:tasks2}) using \cleaninline{enterChoice}. +The interface that is generated for this can be seen in \cref{fig:example_screenshots1}. +After selecting the task, a device is selected (see \cref{fig:example_screenshots2,lst:example:selectdev}). +When both a task and a device is selected, an \gls{ITASK} task is added to the process list using \cleaninline{appendTask}. +Using the helper function \cleaninline{mkTask}, the actual task is selected from the \cleaninline{tasks} list and executed by providing the device argument. +For example, when selecting the \cleaninline{temperature} task, the current temperature is shown to the user (\cref{fig:example_screenshots3}). +This task just sends a simple temperature monitoring task to the device using \cleaninline{liftmTask} and provides a view on its task value using the \cleaninline{>\&>}\footnotemark{} \gls{ITASK} combinator. +\footnotetext{\cleaninline{(>\&>) infixl 1 :: (Task a) ((SDSLens () (? a) ()) -> Task b) -> Task b \| iTask a \& iTask b}} +The light switch task at \crefrange{lst:example:ls1}{lst:example:ls2} is a task that has bidirectional interaction using the definition of \cleaninline{lightswitch} shown in \cref{lst:mtask_liftsds_ex}. +Using \cleaninline{liftsds}, the status of the light switch is synchronised with the user. +The task on the edge device continuously monitors the value of the lowered \gls{SDS}. +If it is different from the current state, the new value is written to the digital \gls{GPIO} pin 13 and the monitoring function is recursively called. + +\begin{figure}[ht] + \centering + \begin{subfigure}[b]{.3\linewidth} + \includegraphics[width=\linewidth]{home_auto1} + \caption{Select task.}% + \label{fig:example_screenshots1} + \end{subfigure} + \begin{subfigure}[b]{.3\linewidth} + \includegraphics[width=\linewidth]{home_auto2} + \caption{Select device.}% + \label{fig:example_screenshots2} + \end{subfigure} + \begin{subfigure}[b]{.3\linewidth} + \includegraphics[width=\linewidth]{home_auto3} + \caption{View result.}% + \label{fig:example_screenshots3} + \end{subfigure} + \caption{Screenshots of the home automation example program in action.}% + \label{fig:example_screenshots} +\end{figure} + +\begin{figure} + \cleaninputlisting[firstline=12,lastline=50,numbers=left,belowskip=0pt]{lst/example.icl} + \begin{lstClean}[numbers=left,firstnumber=40,aboveskip=0pt,caption={An example of a home automation program.},label={lst:example_home_automation}] + , ...][+\label{lst:example:tasks2}+]\end{lstClean} +\end{figure} + +\section{Conclusion} +When \gls{IOT} edge devices run the \gls{MTASK} \gls{RTS}, they become little \gls{TOP} engines of their own. +Using just three \gls{ITASK} functions, \gls{MTASK} devices are integrated in \gls{ITASK} seamlessly. +Devices, using any supported type of connection, are integrated in \gls{ITASK} using the \cleaninline{withDevice} function. +Once connected, \gls{MTASK} tasks are sent to the device for execution using \cleaninline{liftmTask}, lifting them to full-fledged \gls{ITASK} tasks. +To lower the bandwidth, tasks can also be preloaded. +Furthermore, the \gls{MTASK} tasks interact with \gls{ITASK} \glspl{SDS} using the \cleaninline{lowerSds} construct. +All of this together allows programming all layers of an \gls{IOT} system from a single source and in a single paradigm. +All details regarding interoperation are automatically taken care of. \input{subfilepostamble} \end{document}