X-Git-Url: https://git.martlubbers.net/?a=blobdiff_plain;f=results.arch.tex;h=dda1247640a4fe6ec07d400417903e3a7df26288;hb=a182f0a12fdb361d341337ce8240cbcd6ca96ebc;hp=626f0a459636ddcb5011d7aae9f36ed0d01e70a6;hpb=e7aa975e1af0007783d9a100ca83ddd5c25d284b;p=msc-thesis1617.git diff --git a/results.arch.tex b/results.arch.tex index 626f0a4..dda1247 100644 --- a/results.arch.tex +++ b/results.arch.tex @@ -1,5 +1,37 @@ +\section{Overview \& Terminology} +The goal of the architecture is to facilitate an ecosystem in which an +\gls{iTasks}-system can add, change and remove devices at runtime. Moreover, +the \gls{iTasks}-system can send \glspl{mTask} --- compiled at runtime to +bytecode --- to the device. The device runs an interpreter which can execute +the \gls{Task}'s bytecode. Devices are persistent during reboots of the +\gls{iTasks}-system. The methods of interacting with \glspl{mTask} is analogous +to interacting with \gls{iTasks}-\glspl{Task} and programmers can access the +\glspl{SDS} made for a device in the same way as a regular \glspl{SDS}. The +following terms will be used throughout the architecture description. + +\begin{itemize} + \item Device, Client + + This is the actual device connected to the system. This can be a real + device such as a microcontroller but it can also just be a program on + the same machine as the server. + \item Server, \gls{iTasks}-System + + The actual executable serving the \gls{iTasks} application. The system + will contain \glspl{Task} taking care of the communication with the + clients. + \item System + + The complete ecosystem, thus containing both the server and client + programs. + \item Engine + + The runtime system of the client. This program handles the + communication with the server and interprets the \glspl{Task}. +\end{itemize} + \section{Devices} -The client code for the devices is compiled from one codebase. For a device to +The engine for the devices is compiled from one codebase. For a device to be eligible for \glspl{mTask}, it must be able to compile the shared codebase and implement (part of) the device specific interface. The shared codebase only uses standard \gls{C} and no special libraries or tricks are used. Therefore @@ -16,14 +48,14 @@ At the time of writing the following device families are supported and can run the device software. \begin{itemize} \item \texttt{POSIX} compatible systems - + This includes systems running \emph{Linux} and \emph{MacOS}. \item \texttt{STM32} family microcontrollers supported by \texttt{ChibiOS}. This is tested in particular on the \texttt{STM32f7x} series \gls{ARM} development board. - \item Microcontrollers programmable by the \gls{Arduino} \gls{IDE}.\\ - + \item Microcontrollers who are programmable in the \gls{Arduino} \gls{IDE}.\\ + This does not only include \gls{Arduino} compatible boards but also other boards capable of running \gls{Arduino} code. The code has been found working on the \texttt{ESP8266} powered \emph{NodeMCU}. @@ -31,50 +63,113 @@ the device software. \emph{UNO} board that only boasts a meager \emph{2K} of \emph{RAM}. \end{itemize} -\section{Specification} -Devices are stored in a record type and all devices in the system are stored in -a \gls{SDS} containing all devices. From the macro settings in the interface -file a profile is created for the device that describes the specification. When -a connection between the server and a client is established the server will -send a request for specification. The client will serialize his specification -and send it to the server so that the server knows what the client is capable -of. The exact specification is listed in Listing~\ref{lst:devicespec} +\subsection{Interpreter} +\todo{Structuur} +The client contains an interpreter to execute a \gls{Task}'s bytecode. + +Before execution some preparatory work is done. The stack will be initialized +and the program counter and stack pointer are set to zero and the bottom +respectively. Then, the interpreter executes one step at the time while the +program counter is smaller than the program length. The code for this is listed +in Listing~\ref{lst:interpr}. One execution step is basically a big switch +statement going over all possible bytecode instructions. Some instructions are +detailed upon in the listing. The \CI{BCPush} instruction is a little more +complicated in real life because some decoding will take place as not all +\CI{BCValue}'s are of the same length. + +\begin{lstlisting}[language=C,label={lst:interpr},% + caption={Rough code outline for interpretation}] +#define f16(p) program[pc]*265+program[pc+1] + +void run_task(struct task *t){ + uint8_t *program = t->bc; + int plen = t->tasklength; + int pc = 0; + int sp = 0; + while(pc < plen){ + switch(program[pc++]){ + case BCNOP: + break; + case BCPUSH: + stack[sp++] = pc++ //Simplified + break; + case BCPOP: + sp--; + break; + case BCSDSSTORE: + sds_store(f16(pc), stack[--sp]); + pc+=2; + break; + // ... + case BCADD: trace("add"); + stack[sp-2] = stack[sp-2] + stack[sp-1]; + sp -= 1; + break; + // ... + case BCJMPT: trace("jmpt to %d", program[pc]); + pc = stack[--sp] ? program[pc]-1 : pc+1; + break; +} +\end{lstlisting} + +\subsection{Specification} +The server stores a description for every device available in a record type +which are stored in a \gls{SDS}. From the macro settings in +the interface file, a profile is created for the device that describes the +specification. When a connection between the server and a client is established +the server will send a request for specification. The client will serialize his +specification and send it to the server so that the server knows what the +client is capable of. The exact specification is shown in +Listing~\ref{lst:devicespec} and stores the peripheral availability, the memory +available for storing \glspl{Task} and \glspl{SDS} and the size of the stack. -\begin{lstlisting}[language=Clean,label={lst:devicespec}, +\begin{lstlisting}[label={lst:devicespec}, caption={Device specification for \glspl{mTask}}] :: MTaskDeviceSpec = - {haveLed :: Bool - ,haveAio :: Bool - ,haveDio :: Bool - ,bytesMemory :: Int + { haveLed :: Bool + , haveLcd :: Bool + , have... + , bytesMemory :: Int + , stackSize :: Int + , aPins :: Int + , dPins :: Int } \end{lstlisting} -\section{Device Storage} +\subsection{Device Storage} All devices available in the system are stored in a big \gls{SDS} that contains -a list of \CI{MTaskDevice}s. The exact specification is listed in -Listing~\ref{lst:mtaskdevice} with the accompanying classes and types. +a list of \CI{MTaskDevice}s. The exact specification is defined as in +Listing~\ref{lst:mtaskdevice} accompanied with the used classes and types. The \CI{deviceResource} component of the record must implement the -\CI{MTaskDuplex} interface that provides a function that launches a task used -for synchronizing the channels. The \CI{deviceTask} stores the \gls{Task}-id -for this \gls{Task} when active so that it can be checked upon. This top-level -task has the duty to report set the \CI{deviceError} field whenever an error -occurs. All communication goes via these channels. If the system wants to send -a message to the device it just puts it in the channels. Messages sent from the -client to the server are also placed in there. In the case of the \gls{TCP} -device type the \gls{Task} is just a simple wrapper around the existing -\CI{tcpconnect} function in \gls{iTasks}. In case of the serial device type -it uses the newly developed serial port library of \gls{Clean}\footnote{\url{% +\CI{MTaskDuplex} interface that provides a function that launches a \gls{Task} +used for synchronizing the channels. The \CI{deviceTask} stores the +\gls{Task}-id for this \gls{Task} when active so that it can be checked upon. +This top-level task has the duty to report exceptions and errors as they are +thrown by setting the \CI{deviceError} field. All communication goes via these +channels. If the system wants to send a message to the device, it just puts it +in the channels. Messages sent from the client to the server are also placed +in there. In the case of the \gls{TCP} device type, the \gls{Task} is just a +simple wrapper around the existing \CI{tcpconnect} function in \gls{iTasks}. In +case of a device connected by a serial connection, it uses the newly developed +serial port library of \gls{Clean}\footnote{\url{% https://gitlab.science.ru.nl/mlubbers/CleanSerial}}. -Besides all the communication information the record also keeps track of the -\glspl{Task} currently on the device and the according \glspl{SDS}. Finally it -stores the specification of the device that is received when connecting. -All of this is listed in Listing~\ref{lst:mtaskdevice}. +Besides all the communication information, the record also keeps track of the +\glspl{Task} currently on the device, the compiler state (see +Section~\ref{sec:compiler}) and the according \glspl{SDS}. Finally, it stores +the specification of the device that is received when connecting. All of this +is given in Listing~\ref{lst:mtaskdevice}. The definitions of the message +format are explained in the following section. Specialized shares are available +per device. The internal mechanism for this is given in +Chapter~\ref{chp:itasksint}. + +\begin{lstlisting}[caption={Device type},label={lst:mtaskdevice}] +deviceStoreNP :: Shared [MTaskDevice] +deviceShare :: MTaskDevice -> Shared MTaskDevice -\begin{lstlisting}[language=Clean,caption={Device type},label={lst:mtaskdevice}] :: Channels :== ([MTaskMSGRecv], [MTaskMSGSend], Bool) +:: BCState = ... // Compiler state, explained in later sections :: MTaskResource = TCPDevice TCPSettings | SerialDevice TTYSettings @@ -83,6 +178,7 @@ All of this is listed in Listing~\ref{lst:mtaskdevice}. , deviceError :: Maybe String , deviceChannels :: String , deviceName :: String + , deviceState :: BCState , deviceTasks :: [MTaskTask] , deviceData :: MTaskResource , deviceSpec :: Maybe MTaskDeviceSpec @@ -95,5 +191,245 @@ class MTaskDuplex a where synFun :: a (Shared Channels) -> Task () \end{lstlisting} +\section{iTasks} +The server part of the system is written in \gls{iTasks}. Functions for +managing devices, \glspl{Task} and \glspl{SDS} have been created to support the +functionality. An interactive application has been created that allows an +interactive management console for the \gls{mTask} system. This interface +provides functionality to list \glspl{SDS}, add \glspl{Task}, remove +\glspl{Task}, administrate devices and view the state of the system. + +\subsection{Integration} +When the system starts up the devices from the previous execution still +residing in the \gls{SDS} must be cleaned up. It might be the case that they +contain \glspl{Task}, \glspl{SDS} or errors that are no longer applicable in +this run. A user or programmer can later choose to reconnect to some devices. + +\begin{lstlisting}[caption={Starting up the devices},% + label={lst:startupdevs}] +startupDevices :: Task [MTaskDevice] +startupDevices = upd (map reset) deviceStoreNP + where reset d = {d & deviceTask=Nothing, deviceTasks=[], deviceError=Nothing} +\end{lstlisting} + +An image of the management interface is shown in Figure~\ref{lst:manage}. +The system management is done by a single \gls{Task} called \CI{mTaskManager}. +To manage the system, a couple of different functionalities are needed and +are launched. The left sidebar of the interface shows the list of example +\glspl{Task} that are present in the system. When clicking a \gls{Task}, a +dialog opens in which you can select the device to send the \gls{Task} to. The +dialog might contain user specified variables. All example \glspl{mTask} are of +the type \CI{Task (Main (ByteCode () Stmt))} and can thus ask for user input +first. + +The bottom panel shows the device information. In this panel, the devices can +be created and modified. Moreover, this panel allows the user to reconnect with +a device after a restart of the server application. + +\begin{figure}[H] + \centering + \includegraphics[width=\linewidth]{manage} + \caption{The device management interface}\label{lst:manage} +\end{figure} + +\subsection{Shares} +The architecture of the system needs to keep track of the \glspl{SDS} stored on +the client. \glspl{SDS} can be stored on only one device at the same time. +that also stores the of devices. This means that if a \gls{SDS} updates, +everyone watching it will be notified. This would result in to a lot of +notifications that are not ment to be for the listener. Moreover, when a client +updates the \gls{SDS} this is processed by the connection handler and results +in an update of the real \gls{SDS}. +Finally, the \gls{SDS} of a client must be synchronized with the actual device. +There are several ways of tackling this problem each with their own pros and +cons and their own level of abstraction. + +\begin{itemize} + \item Instantiate an actual \gls{iTasks}-\gls{SDS} for every \gls{SDS} used + in a client. + + \item Instantiate a \gls{iTasks}-\gls{SDS} for every device that stores all + their \glspl{SDS}. + + \item Use only one \gls{iTasks}-\gls{SDS} for all devices. +\end{itemize} + +\begin{lstlisting}[label={lst:actualdev},% + caption={Real types for the device \gls{SDS}}] +deviceStoreNP :: Shared [MTaskDevice] +deviceStore :: RWShared (Maybe (MTaskDevice, Int)) [MTaskDevice] [MTaskDevice] +\end{lstlisting} + +\subsection{Parametric Lenses} +The type of the parametric lens is \CI{Maybe (MTaskDevice, Int)}. The \gls{SDS} +can be responsible for the entire list of devices, from now on global. +Moreover, the \gls{SDS} can focus on a single device, from now on local. A +local \gls{SDS} can also specifically focus on a single \gls{SDS} on a single +device, from now on called local-share. + +\paragraph{Global \glspl{SDS}: } +Accessing the global \gls{SDS} is just a matter of focussing the +\CI{deviceStore} with the \CI{Nothing} parameter. The signature for +\CI{deviceStore} was given in Chapter~\ref{chp:arch}. The actual implementation +is as in Listing~\ref{lst:global} + +\begin{lstlisting}[label={lst:shareimpl},% + caption={Base share implementation}] +deviceStoreNP :: RWShared (Maybe (MTaskDevice, Int)) [MTaskDevice] [MTaskDevice] +deviceStoreNP = sdsFocus Nothing deviceStore +\end{lstlisting} + + + + +\paragraph{Local \glspl{SDS}: } +\paragraph{Local-share specific \glspl{SDS}: } + +The implementation for the share is shown in Listing~\ref{lst:shareimpl}. The +\CI{realDeviceStore} \gls{SDS} is not exported through the header files. This +\gls{SDS} contains the actual \gls{SDS} that writes to disk or memory. +\CI{Int} is the identifier of the \gls{SDS}. The \gls{iTasks} way of applying +lenses is through the \CI{sdsFocus} function and through the \CI{sdsLens} +functions. \CI{sdsFocus} allows the programmer to fix the parameter. +\CI{sdsLens} is basically a \CI{mapReadWrite} that has access to the parameter. +This allows the programmer to create filters and lenses. Both of the methods +are not good enough for the device \gls{SDS} because they do not achieve the +writing to the actual device. Writing to a device requires being able to write +to \glspl{SDS}. To solve this problem, a real base \gls{SDS} is created. All +the details are visible in Listing~\ref{lst:shareimpl}. + \section{Communication} -\todo{Connectie, hoe gaat dat in zijn werk} +The communication from the server to the client and vice versa is just a +character stream containing encoded \gls{mTask} messages. The specific encoding +is visible in Appendix~\ref{app:communication-protocol}. The type holding the +messages in Listing~\ref{lst:avmsg}. Detailed explanation about the message +types will be given in the following subsections. + +\begin{lstlisting}[label={lst:avmsg},caption={Available messages}] +:: MTaskId :== Int +:: MSDSId :== Int +:: MTaskFreeBytes :== Int +:: MTaskMSGRecv + = MTTaskAck MTaskId MTaskFreeBytes | MTTaskDelAck MTaskId + | MTSDSAck MSDSId | MTSDSDelAck MSDSId + | MTPub MSDSId BCValue | MTMessage String + | MTDevSpec MTaskDeviceSpec | MTEmpty + +:: MTaskMSGSend + = MTTask MTaskInterval String | MTTaskDel MTaskId + | MTShutdown | MTSds MSDSId BCValue + | MTUpd MSDSId BCValue | MTSpec + +:: MTaskInterval = OneShot | OnInterval Int | OnInterrupt Int +\end{lstlisting} + +\subsection{Add a device} +A device can be added by filling in the \CI{MTaskDevice} record as much as +possible and running the \CI{connectDevice} function. This function grabs the +channels, starts the synchronization \gls{Task}, makes sure the errors are +handled when needed and runs a processing function in parallel to react on the +incoming messages. Moreover, it sends a specification request to the device in +question to determine the details of the device and updates the record to +contain the top-level \gls{Task}-id. All the device functionality heavily +depends on the specific \CI{deviceShare} function that applies a function a device in +the \gls{SDS} when they are equal. Device equality is defined as equality on +their channels. This allows you to give an old device record to the function +and still update the latest instance. Listing~\ref{lst:connectDevice} shows the +connection function. + +\begin{lstlisting}[label={lst:connectDevice},% + caption={Connect a device}] +withDevices :: MTaskDevice (MTaskDevice -> MTaskDevice) -> Task [MTaskDevice] + +connectDevice :: (MTaskDevice (Shared Channels) -> Task ()) MTaskDevice -> Task Channels +connectDevice procFun device = let ch = channels device + in appendTopLevelTask 'DM'.newMap True + (procFun device ch -||- catchAll (getSynFun d.deviceData ch) errHdl) + >>= \tid->withDevices device (\d->{d&deviceTask=Just tid,deviceError=Nothing}) + >>| upd (\(r,s,ss)->(r,s++[MTSpec],ss)) ch + where + errHdl e = withDevices device (\d->{d & deviceTask=Nothing, deviceError=Just e}) @! () +\end{lstlisting} + +Figure~\ref{fig:handshake} shows the connection diagram. The client responds to +the server with their device specification. This is detected by the processing +function and the record is updated accordingly. + +\begin{figure}[H] + \centering + \begin{sequencediagram} + \newthread{s}{Server} + \newinst[4]{c}{Client} + \begin{call}{s}{MTSpec}{c}{MTDevSpec} + \end{call} + \end{sequencediagram} + \caption{Connect a device}\label{fig:handshake} +\end{figure} + +\subsection{\glspl{Task} \& \glspl{SDS}} +When a \gls{Task} is sent to the device it is added to the device record +without an identifier. The actual identifier is added to the record when the +acknowledgement of the \gls{Task} by the device is received. The connection +diagram is shown in Figure~\ref{fig:tasksend}. + +\begin{figure}[H] + \centering + \begin{sequencediagram} + \newthread{s}{Server} + \newinst[4]{c}{Client} + \begin{call}{s}{MTSDS}{c}{MTSDSAck} + \end{call} + \begin{call}{s}{MTTask}{c}{MTTaskAck} + \end{call} + \end{sequencediagram} + \caption{Sending a \gls{Task} to a device}\label{fig:tasksend} +\end{figure} + +The function for sending a \gls{Task} to the device is shown in +Listing~\ref{lst:sendtask}. First the \gls{Task} is compiled into messages. The +details of the compilation process are given in Section~\ref{sec:compiler}. +The new \glspl{SDS} that were made during compilation are added to the +deviceshares that were made during the compilation are merged with the existing +shares on the device. Furthermore the messages are placed in the channel share +of the device. This will result in sending the actual \gls{SDS} specification +and \gls{Task} specifications to the device. A \gls{Task} record is created +with the identifier $-1$ to denote a \gls{Task} not yet acknowledged. Finally +the device itself is updated with the new state and with the new \gls{Task}. +When the device returns an acknowledgement the \gls{Task} is updated +accordingly. + +\begin{lstlisting}[label={lst:sendtask},% + caption={Sending a \gls{Task} to a device}] +makeTask :: String Int -> Task MTaskTask +makeTask name ident = get currentDateTime @ \dt->{MTaskTask | name=name, ident=ident, dateAdded=dt} + +makeShare :: String Int BCValue -> MTaskShare +makeShare withTask identifier value = {MTaskShare | withTask=[withTask], identifier=identifier, value=value} + +sendTaskToDevice :: String (Main (ByteCode a Stmt)) (MTaskDevice, MTaskInterval) -> Task [MTaskDevice] +sendTaskToDevice wta mTask (device, timeout) +# (msgs, newState) = toMessages timeout mTask device.deviceState +# shares = [makeShare wta sdsi sdsval\\{sdsi,sdsval}<-newState.sdss, (MTSds sdsi` _)<-msgs | sdsi == sdsi`] += updateShares device ((++) shares) + >>| sendMessages msgs device + >>| makeTask wta -1 + >>= withDevices device o addTaskUpState newState + where + addTaskUpState :: BCState MTaskTask MTaskDevice -> MTaskDevice + addTaskUpState st task device = {MTaskDevice | device & + deviceState=st, deviceTasks=[task:device.deviceTasks]} +\end{lstlisting} + +\subsection{Miscellaneous Messages} +There exists one special type of message that is sent to the device only when +it needs to reboot. When the server wants to stop the bond with the device it +sends the \CI{MTShutdown} message. The device will then clear his memory, thus +losing all the \glspl{SDS} and \glspl{Task} that were stored and reset itself. +Shortly after the shutdown message a new server can connect to the device +because the device is back in listening mode. + +\section{Lifting mTask to iTasks} +\todo{task lifting} + +\section{Examples} +\todo{example program (demo)}