results.arch.tex

   1 \section{Overview \& Terminology}
   2 The goal of the architecture is to facilitate an ecosystem in which an
   3 \gls{iTasks}-system can add, change and remove devices at runtime. Moreover,
   4 the \gls{iTasks}-system can send \glspl{mTask} --- compiled at runtime to
   5 bytecode --- to the device. The device runs an interpreter which can execute
   6 the \gls{Task}'s bytecode. Devices are persistent during reboots of the
   7 \gls{iTasks}-system. The methods of interacting with \glspl{mTask} is analogous
   8 to interacting with \gls{iTasks}-\glspl{Task} and programmers can access the
   9 \glspl{SDS} made for a device in the same way as a regular \glspl{SDS}. The
  10 following terms will be used throughout the architecture description.
  11
  12 \begin{itemize}
  13         \item Device, Client
  14
  15                 This is the actual device connected to the system. This can be a real
  16                 device such as a microcontroller but also just a program on the same
  17                 machine as the server.
  18         \item Server, \gls{iTasks}-System
  19
  20                 The actual executable serving the \gls{iTasks} interfaces. The system
  21                 will contain \glspl{Task} taking care of the communication with the
  22                 clients.
  23         \item System
  24
  25                 The complete ecosystem, thus containing both the server and client
  26                 programs.
  27         \item Engine
  28
  29                 The runtime system of the client. This system handles the communication
  30                 with the server and interprets the \glspl{Task}.
  31 \end{itemize}
  32
  33 \section{Devices}
  34 The engine for the devices is compiled from one codebase. For a device to
  35 be eligible for \glspl{mTask}, it must be able to compile the shared codebase
  36 and implement (part of) the device specific interface. The shared codebase only
  37 uses standard \gls{C} and no special libraries or tricks are used. Therefore
  38 the code is compilable for almost any device or system. Note that it is not
  39 needed to implement a full interface. The full interface --- excluding the
  40 device specific settings --- is listed in Appendix~\ref{app:device-interface}.
  41 The interface works in a similar fashion as the \gls{EDSL}. Devices do not have
  42 to implement all functionality, this is analogous to the fact that views do not
  43 have to implement all type classes in the \gls{EDSL}.  When the device connects
  44 for the first time with a server the specifications of what is implemented is
  45 communicated.
  46
  47 At the time of writing the following device families are supported and can run
  48 the device software.
  49 \begin{itemize}
  50         \item \texttt{POSIX} compatible systems
  51
  52                 This includes systems running \emph{Linux} and \emph{MacOS}.
  53         \item \texttt{STM32} family microcontrollers supported by \texttt{ChibiOS}.
  54
  55                 This is tested in particular on the \texttt{STM32f7x} series \gls{ARM}
  56                 development board.
  57         \item Microcontrollers programmable by the \gls{Arduino} \gls{IDE}.\\
  58
  59                 This does not only include \gls{Arduino} compatible boards but also
  60                 other boards capable of running \gls{Arduino} code. The code
  61                 has been found working on the \texttt{ESP8266} powered \emph{NodeMCU}.
  62                 It is tested on devices as small as the regular \gls{Arduino}
  63                 \emph{UNO} board that only boasts a meager \emph{2K} of \emph{RAM}.
  64 \end{itemize}
  65
  66 \section{Specification}
  67 The server stores a description for every device available in a record type
  68 which are stored in a \gls{SDS}. From the macro settings in
  69 the interface file, a profile is created for the device that describes the
  70 specification. When a connection between the server and a client is established
  71 the server will send a request for specification. The client will serialize his
  72 specification and send it to the server so that the server knows what the
  73 client is capable of. The exact specification is shown in
  74 Listing~\ref{lst:devicespec} and stores the peripheral availability, the memory
  75 available for storing \glspl{Task} and \glspl{SDS} and the size of the stack.
  76
  77 \begin{lstlisting}[label={lst:devicespec},
  78         caption={Device specification for \glspl{mTask}}]
  79 :: MTaskDeviceSpec =
  80         { haveLed     :: Bool
  81         , haveAio     :: Bool
  82         , haveDio     :: Bool
  83         , bytesMemory :: Int
  84         , stackSize   :: Int
  85         , aPins       :: Int
  86         , dPins       :: Int
  87         }
  88 \end{lstlisting}
  89
  90 \section{Device Storage}
  91 All devices available in the system are stored in a big \gls{SDS} that contains
  92 a list of \CI{MTaskDevice}s. The exact specification is defined as in
  93 Listing~\ref{lst:mtaskdevice} accompanied with the used classes and types.
  94
  95 The \CI{deviceResource} component of the record must implement the
  96 \CI{MTaskDuplex} interface that provides a function that launches a \gls{Task}
  97 used for synchronizing the channels.  The \CI{deviceTask} stores the
  98 \gls{Task}-id for this \gls{Task} when active so that it can be checked upon.
  99 This top-level task has the duty to report exceptions and errors as they are
 100 thrown by setting the \CI{deviceError} field. All communication goes via these
 101 channels. If the system wants to send a message to the device, it just puts it
 102 in the channels.  Messages sent from the client to the server are also placed
 103 in there. In the case of the \gls{TCP} device type, the \gls{Task} is just a
 104 simple wrapper around the existing \CI{tcpconnect} function in \gls{iTasks}. In
 105 case of a device connected by a serial connection, it uses the newly developed
 106 serial port library of \gls{Clean}\footnote{\url{%
 107 https://gitlab.science.ru.nl/mlubbers/CleanSerial}}.
 108
 109 Besides all the communication information, the record also keeps track of the
 110 \glspl{Task} currently on the device, the compiler state (see
 111 Section~\ref{sec:compiler}) and the according \glspl{SDS}. Finally, it stores
 112 the specification of the device that is received when connecting. All of this
 113 is given in Listing~\ref{lst:mtaskdevice}. The definitions of the message
 114 format are explained in the following section.
 115
 116 \begin{lstlisting}[caption={Device type},label={lst:mtaskdevice}]
 117 deviceStoreNP :: Shared [MTaskDevice]
 118
 119 :: Channels :== ([MTaskMSGRecv], [MTaskMSGSend], Bool)
 120 :: BCState = ... // Compiler state, explained in later sections
 121 :: MTaskResource
 122         = TCPDevice TCPSettings
 123         | SerialDevice TTYSettings
 124 :: MTaskDevice =
 125                 { deviceTask :: Maybe TaskId
 126                 , deviceError :: Maybe String
 127                 , deviceChannels :: String
 128                 , deviceName :: String
 129                 , deviceState :: BCState
 130                 , deviceTasks :: [MTaskTask]
 131                 , deviceData :: MTaskResource
 132                 , deviceSpec :: Maybe MTaskDeviceSpec
 133                 , deviceShares :: [MTaskShare]
 134         }
 135
 136 channels :: MTaskDevice -> Shared Channels
 137
 138 class MTaskDuplex a where
 139         synFun :: a (Shared Channels) -> Task ()
 140 \end{lstlisting}
 141
 142 \section{Communication}
 143 The communication from the server to the client and vice versa is just a
 144 character stream containing encoded \gls{mTask} messages. The specific encoding
 145 is visible in Appendix~\ref{app:communication-protocol}. The type holding the
 146 messages in Listing~\ref{lst:avmsg}. Detailed explanation about the message
 147 types will be given in the following subsections.
 148
 149 \begin{lstlisting}[label={lst:avmsg},caption={Available messages}]
 150 :: MTaskId :== Int
 151 :: MSDSId :== Int
 152 :: MTaskFreeBytes :== Int
 153 :: MTaskMSGRecv
 154         = MTTaskAck MTaskId MTaskFreeBytes | MTTaskDelAck MTaskId
 155         | MTSDSAck MSDSId                  | MTSDSDelAck MSDSId
 156         | MTPub MSDSId BCValue             | MTMessage String
 157         | MTDevSpec MTaskDeviceSpec        | MTEmpty
 158
 159 :: MTaskMSGSend
 160         = MTTask MTaskInterval String | MTTaskDel MTaskId
 161         | MTShutdown                  | MTSds MSDSId BCValue
 162         | MTUpd MSDSId BCValue        | MTSpec
 163
 164 :: MTaskInterval = OneShot | OnInterval Int | OnInterrupt Int
 165 \end{lstlisting}
 166
 167 \subsection{Add a device}
 168 A device can be added by filling in the \CI{MTaskDevice} record as much as
 169 possible and running the \CI{connectDevice} function. This function grabs the
 170 channels, starts the synchronization \gls{Task}, makes sure the errors are
 171 handled when needed and runs a processing function in parallel to react on the
 172 incoming messages. Moreover, it sends a specification request to the device in
 173 question to determine the details of the device and updates the record to
 174 contain the top-level \gls{Task}-id. All the device functionality heavily
 175 depends on the \CI{withDevices} function that applies a function a device in
 176 the \gls{SDS} when they are equal. Device equality is defined as equality on
 177 their channels. This allows you to give an old device record to the function
 178 and still update the latest instance. Listing~\ref{lst:connectDevice} shows the
 179 connection function.
 180
 181 \begin{lstlisting}[label={lst:connectDevice},%
 182         caption={Connect a device}]
 183 withDevices :: MTaskDevice (MTaskDevice -> MTaskDevice) -> Task [MTaskDevice]
 184
 185 connectDevice :: (MTaskDevice (Shared Channels) -> Task ()) MTaskDevice -> Task Channels
 186 connectDevice procFun device = let ch = channels device
 187         in appendTopLevelTask 'DM'.newMap True
 188                 (procFun device ch -||- catchAll (getSynFun d.deviceData ch) errHdl)
 189                 >>= \tid->withDevices device (\d->{d&deviceTask=Just tid,deviceError=Nothing})
 190                 >>| upd (\(r,s,ss)->(r,s++[MTSpec],ss)) ch
 191         where
 192                 errHdl e = withDevices device (\d->{d & deviceTask=Nothing, deviceError=Just e}) @! ()
 193 \end{lstlisting}
 194
 195 Figure~\ref{fig:handshake} shows the connection diagram. The client responds to
 196 the server with their device specification. This is detected by the processing
 197 function and the record is updated accordingly.
 198
 199 \begin{figure}[H]
 200         \centering
 201         \begin{sequencediagram}
 202                 \newthread{s}{Server}
 203                 \newinst[4]{c}{Client}
 204                 \begin{call}{s}{MTSpec}{c}{MTDevSpec}
 205                 \end{call}
 206         \end{sequencediagram}
 207         \caption{Connect a device}\label{fig:handshake}
 208 \end{figure}
 209
 210 \subsection{\glspl{Task} \& \glspl{SDS}}
 211 When a \gls{Task} is sent to the device it is added to the device record
 212 without an identifier. The actual identifier is added to the record when the
 213 acknowledgement of the \gls{Task} by the device is received. The connection
 214 diagram is shown in Figure~\ref{fig:tasksend}.
 215
 216 \begin{figure}[H]
 217         \centering
 218         \begin{sequencediagram}
 219                 \newthread{s}{Server}
 220                 \newinst[4]{c}{Client}
 221                 \begin{call}{s}{MTSDS}{c}{MTSDSAck}
 222                 \end{call}
 223                 \begin{call}{s}{MTTask}{c}{MTTaskAck}
 224                 \end{call}
 225         \end{sequencediagram}
 226         \caption{Sending a \gls{Task} to a device}\label{fig:tasksend}
 227 \end{figure}
 228
 229 The function for sending a \gls{Task} to the device is shown in
 230 Listing~\ref{lst:sendtask}. First the \gls{Task} is compiled into messages. The
 231 details of the compilation process are given in Section~\ref{sec:compiler}.
 232 The new \glspl{SDS} that were made during compilation are added to the
 233 deviceshares that were made during the compilation are merged with the existing
 234 shares on the device. Furthermore the messages are placed in the channel share
 235 of the device. This will result in sending the actual \gls{SDS} specification
 236 and \gls{Task} specifications to the device. A \gls{Task} record is created
 237 with the identifier $-1$ to denote a \gls{Task} not yet acknowledged. Finally
 238 the device itself is updated with the new state and with the new \gls{Task}.
 239 When the device returns an acknowledgement the \gls{Task} is updated
 240 accordingly.
 241
 242 \begin{lstlisting}[label={lst:sendtask},%
 243         caption={Sending a \gls{Task} to a device}]
 244 makeTask :: String Int -> Task MTaskTask
 245 makeTask name ident = get currentDateTime @ \dt->{MTaskTask | name=name, ident=ident, dateAdded=dt}
 246
 247 makeShare :: String Int BCValue -> MTaskShare
 248 makeShare withTask identifier value = {MTaskShare | withTask=[withTask], identifier=identifier, value=value}
 249
 250 sendTaskToDevice :: String (Main (ByteCode a Stmt)) (MTaskDevice, MTaskInterval) -> Task [MTaskDevice]
 251 sendTaskToDevice wta mTask (device, timeout)
 252 # (msgs, newState) = toMessages timeout mTask device.deviceState
 253 # shares = [makeShare wta sdsi sdsval\\{sdsi,sdsval}<-newState.sdss, (MTSds sdsi` _)<-msgs | sdsi == sdsi`]
 254 = updateShares device ((++) shares)
 255         >>| sendMessages msgs device
 256         >>| makeTask wta -1
 257         >>= withDevices device o addTaskUpState newState
 258         where
 259                 addTaskUpState :: BCState MTaskTask MTaskDevice -> MTaskDevice
 260                 addTaskUpState st task device = {MTaskDevice | device &
 261                         deviceState=st, deviceTasks=[task:device.deviceTasks]}
 262 \end{lstlisting}
 263
 264 \subsection{Miscellaneous Messages}
 265 There exists one special type of message that is sent to the device only when
 266 it needs to reboot. When the server wants to stop the bond with the device it
 267 sends the \CI{MTShutdown} message. The device will then clear his memory, thus
 268 losing all the \glspl{SDS} and \glspl{Task} that were stored and reset itself.
 269 Shortly after the shutdown message a new server can connect to the device
 270 because the device is back in listening mode.