results.arch.tex

   1 \section{Overview \& Terminology}
   2 The goal of the architecture is to facilitate an ecosystem in which an
   3 \gls{iTasks}-system can add, change and remove devices at runtime. Moreover,
   4 the \gls{iTasks}-system can send \glspl{mTask}, compiled at runtime to
   5 bytecode, to the device. The device runs an interpreter which can execute the
   6 \gls{Task}'s bytecode. Devices are persistent during reboots of the
   7 \gls{iTasks}-system. The methods of interacting with \glspl{mTask} is analogous
   8 to interacting with \gls{iTasks}-\glspl{Task} and programmers can access the
   9 \glspl{SDS} made for a device in the same way as a regular \glspl{SDS}. The
  10 following terms will be used throughout the architecture description.
  11
  12 \begin{itemize}
  13         \item Device, Client
  14
  15                 This is the actual device connected to the system. This can be a real
  16                 device such as a microcontroller but also just a program on the same
  17                 machine as the server.
  18         \item Server, \gls{iTasks}-System
  19
  20                 The actual executable serving the \gls{iTasks} interfaces. The system
  21                 will contain \glspl{Task} taking care of the communication with the
  22                 clients.
  23         \item System
  24
  25                 The complete ecosystem, thus containing both the server and client
  26                 programs.
  27         \item Engine
  28
  29                 The runtime system of the client. This system handles the communication
  30                 with the server and interprets the \glspl{Task}.
  31 \end{itemize}
  32
  33 \section{Devices}
  34 The engine for the devices is compiled from one codebase. For a device to
  35 be eligible for \glspl{mTask}, it must be able to compile the shared codebase
  36 and implement (part of) the device specific interface. The shared codebase only
  37 uses standard \gls{C} and no special libraries or tricks are used. Therefore
  38 the code is compilable for almost any device or system. Note that it is not
  39 needed to implement a full interface. The full interface --- excluding the
  40 device specific settings --- is listed in Appendix~\ref{app:device-interface}.
  41 The interface works in a similar fashion as the \gls{EDSL}. Devices do not have
  42 to implement all functionality, this is analogous to the fact that views do not
  43 have to implement all type classes in the \gls{EDSL}.  When the device connects
  44 for the first time with a server the specifications of what is implemented is
  45 communicated.
  46
  47 At the time of writing the following device families are supported and can run
  48 the device software.
  49 \begin{itemize}
  50         \item \texttt{POSIX} compatible systems
  51
  52                 This includes systems running \emph{Linux} and \emph{MacOS}.
  53         \item \texttt{STM32} family microcontrollers supported by \texttt{ChibiOS}.
  54
  55                 This is tested in particular on the \texttt{STM32f7x} series \gls{ARM}
  56                 development board.
  57         \item Microcontrollers programmable by the \gls{Arduino} \gls{IDE}.\\
  58
  59                 This does not only include \gls{Arduino} compatible boards but also
  60                 other boards capable of running \gls{Arduino} code. The code
  61                 has been found working on the \texttt{ESP8266} powered \emph{NodeMCU}.
  62                 It is tested on devices as small as the regular \gls{Arduino}
  63                 \emph{UNO} board that only boasts a meager \emph{2K} of \emph{RAM}.
  64 \end{itemize}
  65
  66 \section{Specification}
  67 The server stores a description for every device available in a record type
  68 which are stored in a \gls{SDS}. From the macro settings in
  69 the interface file, a profile is created for the device that describes the
  70 specification. When a connection between the server and a client is established
  71 the server will send a request for specification. The client will serialize his
  72 specification and send it to the server so that the server knows what the
  73 client is capable of. The exact specification is shown in
  74 Listing~\ref{lst:devicespec} and stores the peripheral availability, the memory
  75 available for storing \glspl{Task} and \glspl{SDS}.
  76
  77 \begin{lstlisting}[label={lst:devicespec},
  78         caption={Device specification for \glspl{mTask}}]
  79 :: MTaskDeviceSpec =
  80         { haveLed     :: Bool
  81         , haveAio     :: Bool
  82         , haveDio     :: Bool
  83         , aPins       :: Int
  84         , dPins       :: Int
  85         , bytesMemory :: Int
  86         }
  87 \end{lstlisting}
  88
  89 \section{Device Storage}
  90 All devices available in the system are stored in a big \gls{SDS} that contains
  91 a list of \CI{MTaskDevice}s. The exact specification is defined as in
  92 Listing~\ref{lst:mtaskdevice} with the accompanying classes and types.
  93
  94 The \CI{deviceResource} component of the record must implement the
  95 \CI{MTaskDuplex} interface that provides a function that launches a \gls{Task}
  96 used for synchronizing the channels.  The \CI{deviceTask} stores the
  97 \gls{Task}-id for this \gls{Task} when active so that it can be checked upon.
  98 This top-level task has the duty to report exceptions and errors as they are
  99 thrown by setting the \CI{deviceError} field. All communication goes via these
 100 channels. If the system wants to send a message to the device, it just puts it
 101 in the channels.  Messages sent from the client to the server are also placed
 102 in there. In the case of the \gls{TCP} device type, the \gls{Task} is just a
 103 simple wrapper around the existing \CI{tcpconnect} function in \gls{iTasks}. In
 104 case of a device connected by a serial connection, it uses the newly developed
 105 serial port library of \gls{Clean}\footnote{\url{%
 106 https://gitlab.science.ru.nl/mlubbers/CleanSerial}}.
 107
 108 Besides all the communication information, the record also keeps track of the
 109 \glspl{Task} currently on the device, the compiler state (see
 110 Section~\ref{sec:compiler}) and the according \glspl{SDS}. Finally, it stores
 111 the specification of the device that is received when connecting. All of this
 112 is given in Listing~\ref{lst:mtaskdevice}. The definitions of the message
 113 format are explained in the following section.
 114
 115 \begin{lstlisting}[caption={Device type},label={lst:mtaskdevice}]
 116 deviceStoreNP :: Shared [MTaskDevice]
 117
 118 :: Channels :== ([MTaskMSGRecv], [MTaskMSGSend], Bool)
 119 :: BCState = ... // Compiler state, explained in later sections
 120 :: MTaskResource
 121         = TCPDevice TCPSettings
 122         | SerialDevice TTYSettings
 123 :: MTaskDevice =
 124                 { deviceTask :: Maybe TaskId
 125                 , deviceError :: Maybe String
 126                 , deviceChannels :: String
 127                 , deviceName :: String
 128                 , deviceState :: BCState
 129                 , deviceTasks :: [MTaskTask]
 130                 , deviceData :: MTaskResource
 131                 , deviceSpec :: Maybe MTaskDeviceSpec
 132                 , deviceShares :: [MTaskShare]
 133         }
 134
 135 channels :: MTaskDevice -> Shared Channels
 136
 137 class MTaskDuplex a where
 138         synFun :: a (Shared Channels) -> Task ()
 139 \end{lstlisting}
 140
 141 \section{Communication}
 142 The communication from the server to the client and vice versa is just a
 143 character stream containing encoded \gls{mTask} messages. The specific encoding
 144 is visible in Appendix~\ref{app:communication-protocol}. The type holding the
 145 messages in Listing~\ref{lst:avmsg}. Detailed explaination about the message
 146 types will be given in the following subsections.
 147
 148 \begin{lstlisting}[label={lst:avmsg},caption={Available messages}]
 149 :: MTaskId :== Int
 150 :: MSDSId :== Int
 151 :: MTaskFreeBytes :== Int
 152 :: MTaskMSGRecv
 153         = MTTaskAck MTaskId MTaskFreeBytes | MTTaskDelAck MTaskId
 154         | MTSDSAck MSDSId                  | MTSDSDelAck MSDSId
 155         | MTPub MSDSId BCValue             | MTMessage String
 156         | MTDevSpec MTaskDeviceSpec        | MTEmpty
 157
 158 :: MTaskMSGSend
 159         = MTTask MTaskInterval String | MTTaskDel MTaskId
 160         | MTShutdown                  | MTSds MSDSId BCValue
 161         | MTUpd MSDSId BCValue        | MTSpec
 162
 163 :: MTaskInterval = OneShot | OnInterval Int | OnInterrupt Int
 164 \end{lstlisting}
 165
 166 \subsection{Add a device}
 167 A device can be added by filling in the \CI{MTaskDevice} record as much as
 168 possible and running the \CI{connectDevice} function. This function grabs the
 169 channels, starts the synchronization \gls{Task}, makes sure the errors are
 170 handled when needed and runs a processing function in parallel to react on the
 171 incoming messages. Moreover, it sends a specification request to the device in
 172 question to determine the details of the device and updates the record to
 173 contain the top-level \gls{Task}-id. All the device functionality heavily
 174 depends on the \CI{withDevices} function that applies a function a device in
 175 the \gls{SDS} when they are equal. Device equality is defined as equality on
 176 their channels. This allows you to give an old device record to the function
 177 and still update the latest instance. Listing~\ref{lst:connectDevice} shows the
 178 connection function.
 179
 180 \begin{lstlisting}[label={lst:connectDevice},%
 181         caption={Connect a device}]
 182 withDevices :: MTaskDevice (MTaskDevice -> MTaskDevice) -> Task [MTaskDevice]
 183
 184 connectDevice :: (MTaskDevice (Shared Channels) -> Task ()) MTaskDevice -> Task Channels
 185 connectDevice procFun device = let ch = channels device
 186         in appendTopLevelTask 'DM'.newMap True
 187                 (procFun device ch -||- catchAll (getSynFun d.deviceData ch) errHdl)
 188                 >>= \tid->withDevices device (\d->{d&deviceTask=Just tid,deviceError=Nothing})
 189                 >>| upd (\(r,s,ss)->(r,s++[MTSpec],ss)) ch
 190         where
 191                 errHdl e = withDevices device (\d->{d & deviceTask=Nothing, deviceError=Just e}) @! ()
 192 \end{lstlisting}
 193
 194 Figure~\ref{fig:handshake} shows the connection diagram. The client responds to
 195 the server with their device specification. This is detected by the processing
 196 function and the record is updated accordingly.
 197
 198 \begin{figure}[H]
 199         \centering
 200         \begin{sequencediagram}
 201                 \newthread{s}{Server}
 202                 \newinst[4]{c}{Client}
 203                 \begin{call}{s}{MTSpec}{c}{MTDevSpec}
 204                 \end{call}
 205         \end{sequencediagram}
 206         \caption{Connect a device}\label{fig:handshake}
 207 \end{figure}
 208
 209 \subsection{\glspl{Task} \& \glspl{SDS}}
 210 When a \gls{Task} is sent to the device it is added to the device record
 211 without an identifier. The actual identifier is added to the record when the
 212 acknowledgement of the \gls{Task} by the device is received. The connection
 213 diagram is shown in Figure~\ref{fig:tasksend}.
 214
 215 \begin{figure}[H]
 216         \centering
 217         \begin{sequencediagram}
 218                 \newthread{s}{Server}
 219                 \newinst[4]{c}{Client}
 220                 \begin{call}{s}{MTSDS}{c}{MTSDSAck}
 221                 \end{call}
 222                 \begin{call}{s}{MTTask}{c}{MTTaskAck}
 223                 \end{call}
 224         \end{sequencediagram}
 225         \caption{Sending a \gls{Task} to a device}\label{fig:tasksend}
 226 \end{figure}
 227
 228 The function for sending a \gls{Task} to the device is shown in
 229 Listing~\ref{lst:sendtask}. First the \gls{Task} is compiled into messages. The
 230 details of the compilation process are given in Section~\ref{sec:compiler}.
 231 The new \glspl{SDS} that were made during compilation are added to the
 232 deviceshares that were made during the compilation are merged with the existing
 233 shares on the device. Furthermore the messages are placed in the channel share
 234 of the device. This will result in sending the actual \gls{SDS} specification
 235 and \gls{Task} specifications to the device. A \gls{Task} record is created
 236 with the identifier $-1$ to denote a \gls{Task} not yet acknowledged. Finally
 237 the device itself is updated with the new state and with the new \gls{Task}.
 238 When the device returns an acknowledgement the \gls{Task} is updated
 239 accordingly.
 240
 241 \begin{lstlisting}[label={lst:sendtask},%
 242         caption={Sending a \gls{Task} to a device}]
 243 makeTask :: String Int -> Task MTaskTask
 244 makeTask name ident = get currentDateTime @ \dt->{MTaskTask | name=name, ident=ident, dateAdded=dt}
 245
 246 makeShare :: String Int BCValue -> MTaskShare
 247 makeShare withTask identifier value = {MTaskShare | withTask=[withTask], identifier=identifier, value=value}
 248
 249 sendTaskToDevice :: String (Main (ByteCode a Stmt)) (MTaskDevice, MTaskInterval) -> Task [MTaskDevice]
 250 sendTaskToDevice wta mTask (device, timeout)
 251 # (msgs, newState) = toMessages timeout mTask device.deviceState
 252 # shares = [makeShare wta sdsi sdsval\\{sdsi,sdsval}<-newState.sdss, (MTSds sdsi` _)<-msgs | sdsi == sdsi`]
 253 = updateShares device ((++) shares)
 254         >>| sendMessages msgs device
 255         >>| makeTask wta -1
 256         >>= withDevices device o addTaskUpState newState
 257         where
 258                 addTaskUpState :: BCState MTaskTask MTaskDevice -> MTaskDevice
 259                 addTaskUpState st task device = {MTaskDevice | device &
 260                         deviceState=st, deviceTasks=[task:device.deviceTasks]}
 261 \end{lstlisting}
 262
 263 \subsection{Miscellaneous Messages}
 264 There exists one special type of message that is sent to the device only when
 265 it needs to reboot. When the server wants to stop the bond with the device it
 266 sends the \CI{MTShutdown} message. The device will then clear his memory, thus
 267 losing all the \glspl{SDS} and \glspl{Task} that were stored and reset itself.
 268 Shortly after the shutdown message a new server can connect to the device
 269 because the device is back in listening mode.