\input{subfilepreamble}
-\setcounter{chapter}{4}
+\setcounter{chapter}{6}
\begin{document}
\input{subfileprefix}
This chapter shows the implementation of the \gls{MTASK} system by:
\begin{itemize}
\item showing the compilation and execution toolchain;
- \item elaborating on the implementation and architecture of the \gls{RTS} of \gls{MTASK};
\item showing the implementation of the byte code compiler for the \gls{MTASK} language;
- \item and explaining the machinery used to automatically serialise and deserialise data to-and-fro the device.
+ \item elaborating on the implementation and architecture of the \gls{RTS} of \gls{MTASK};
+ \item and explaining the machinery used to automatically serialise and deserialise data to and fro the device.
\end{itemize}
\end{chapterabstract}
The microcontrollers usually have flash-based program memory which wears out fairly quickly.
For example, the flash memory of the popular atmega328p powering the \gls{ARDUINO} UNO is rated for \num{10000} write cycles.
While this sounds like a lot, if new tasks are sent to the device every minute or so, a lifetime of only seven days is guaranteed.
-Hence, for dynamic applications, storing the program in the \gls{RAM} of the device and interpreting this code is necessary in order to save precious write cycles of the program memory.
+Hence, for dynamic applications, storing the program in the \gls{RAM} of the device and thus interpreting this code is necessary in order to save precious write cycles of the program memory.
In the \gls{MTASK} system, the \gls{MTASK} \gls{RTS}, a domain-specific \gls{OS}, is responsible for interpreting the programs.
Programs in \gls{MTASK} are \gls{DSL} terms constructed at run time in an \gls{ITASK} system.
\Cref{fig:toolchain} shows the compilation and execution toolchain of such programs.
First, the source code is compiled to a byte code specification, this specification contains the compiled main expression, the functions, and the \gls{SDS} and peripheral configuration.
How an \gls{MTASK} task is compiled to this specification is shown in \cref{sec:compiler_imp}.
-This package is then sent to the \gls{RTS} of the device for execution, shown in \cref{sec:compiler_rts}.
+This package is then sent to the \gls{RTS} of the device for execution.
+In order to execute a task, first the main expression is evaluated in the interpreter, resulting in a task tree.
+Then, using small-step reduction, the task tree is continuously rewritten by the rewrite engine of the \gls{RTS}.
+At times, the reduction requires the evaluation of expressions, using the interpreter.
+During every rewrite step, a task value is produced.
On the device, the \gls{RTS} may have multiple tasks at the same time active.
+By interleaving the rewrite steps, parallel operation is achieved.
+The design, architecture and implementation of the \gls{RTS} is shown in \cref{sec:compiler_rts}.
\begin{figure}
\centering
\label{fig:toolchain}
\end{figure}
-\section{Run-time system}\label{sec:compiler_rts}
-The \gls{RTS} is a customisable domain-specific \gls{OS} that takes care of the execution of tasks.
-Furthermore, it also takes care of low-level mechanisms such as the communication, multitasking, and memory management.
-Once a device is programmed with the \gls{MTASK} \gls{RTS}, it can continuously receive new tasks without the need for reprogramming.
-The \gls{OS} is written in portable \ccpp{} and only contains a small device-specific portion.
-In order to keep the abstraction level high and the hardware requirements low, much of the high-level functionality of the \gls{MTASK} language is implemented not in terms of lower-level constructs from \gls{MTASK} language but in terms of \ccpp{} code.
-
-Most microcontrollers software consists of a cyclic executive instead of an \gls{OS}, this one loop function is continuously executed and all work is performed there.
-In the \gls{RTS} of the \gls{MTASK} system, there is also such an event loop function.
-It is a function with a relatively short execution time that gets called repeatedly.
-The event loop consists of three distinct phases.
-After doing the three phases, the devices goes to sleep for as long as possible (see \cref{chp:green_computing_mtask} for more details on task scheduling).
-
-\subsection{Communication phase}
-In the first phase, the communication channels are processed.
-The exact communication method is a customisable device-specific option baked into the \gls{RTS}.
-The interface is kept deliberately simple and consists of two layers: a link interface and a communication interface.
-Besides opening, closing and cleaning up, the link interface has three functions that are shown in \cref{lst:link_interface}.
-Consequently, implementing this link interface is very simple but it is still possible to implement more advanced link features such as buffering.
-There are implementations for this interface for serial or \gls{WIFI} connections using \gls{ARDUINO}, and \gls{TCP} connections for Linux.
-
-\begin{lstArduino}[caption={Link interface of the \gls{MTASK} \gls{RTS}.},label={lst:link_interface}]
-bool link_input_available(void);
-uint8_t link_read_byte(void);
-void link_write_byte(uint8_t b);
-\end{lstArduino}
-
-The communication interface abstracts away from this link interface and is typed instead.
-It contains only two functions as seen in \cref{lst:comm_interface}.
-There are implementations for direct communication, or communication using an \gls{MQTT} broker.
-Both use the automatic serialisation and deserialisation shown in \cref{sec:ccodegen}.
-
-\begin{lstArduino}[caption={Communication interface of the \gls{MTASK} \gls{RTS}.},label={lst:comm_interface}]
-struct MTMessageTo receive_message(void);
-void send_message(struct MTMessageFro msg);
-\end{lstArduino}
-
-Processing the received messages from the communication channels happens synchronously and the channels are exhausted completely before moving on to the next phase.
-There are several possible messages that can be received from the server:
-
-\begin{description}
- \item[SpecRequest]
- is a message instructing the device to send its specification and it is received immediately after connecting.
- The \gls{RTS} responds with a \texttt{Spec} answer containing the specification.
- \item[TaskPrep]
- tells the device a task is on its way.
- Especially on faster connections, it may be the case that the communication buffers overflow because a big message is sent while the \gls{RTS} is busy executing tasks.
- This message allows the \gls{RTS} to postpone execution for a while, until the larger task has been received.
- The server sends the task only after the device acknowledged the preparation by by sending a \texttt{TaskPrepAck} message.
- \item[Task]
- contains a new task, its peripheral configuration, the \glspl{SDS}, and the byte code.
- The new task is immediately copied to the task storage but is only initialised during the next phase.
- The device acknowledges the task by sending a \texttt{TaskAck} message.
- \item[SdsUpdate]
- notifies the device of the new value for a lowered \gls{SDS}.
- The old value of the lowered \gls{SDS} is immediately replaced with the new one.
- There is no acknowledgement required.
- \item[TaskDel]
- instructs the device to delete a running task.
- Tasks are automatically deleted when they become stable.
- However, a task may also be deleted when the surrounding task on the server is deleted, for example when the task is on the left-hand side of a step combinator and the condition to step holds.
- The device acknowledges the deletion by sending a \texttt{TaskDelAck}.
- \item[Shutdown]
- tells the device to reset.
-\end{description}
-
-\subsection{Execution phase}
-The second phase performs one execution step for all tasks that wish for it.
-Tasks are ordered in a priority queue ordered by the time a task needs to execute, the \gls{RTS} selects all tasks that can be scheduled, see \cref{sec:scheduling} for more details.
-Execution of a task is always an interplay between the interpreter and the rewriter.
-
-When a new task is received, the main expression is evaluated to produce a task tree.
-A task tree is a tree structure in which each node represents a task combinator and the leaves are basic tasks.
-If a task is not initialized yet, i.e.\ the pointer to the current task tree is still null, the byte code of the main function is interpreted.
-The main expression always produces a task tree.
-Execution of a task consists of continuously rewriting the task until its value is stable.
-
-Rewriting is a destructive process, i.e.\ the rewriting is done in place.
-The rewriting engine uses the interpreter when needed, e.g.\ to calculate the step continuations.
-The rewriter and the interpreter use the same stack to store intermediate values.
-Rewriting steps are small so that interleaving results in seemingly parallel execution.
-In this phase new task tree nodes may be allocated.
-Both rewriting and initialization are atomic operations in the sense that no processing on \glspl{SDS} is done other than \gls{SDS} operations from the task itself.
-The host is notified if a task value is changed after a rewrite step by sending a \texttt{TaskReturn} message.
-
-Take for example a blink task for which the code is shown in \cref{lst:blink_code}.
-
-\begin{lstClean}[caption={Code for a blink program.},label={lst:blink_code}]
-fun \blink=(\st->delay (lit 500) >>|. writeD d3 st >>=. blink o Not)
-In {main = blink true}
-\end{lstClean}
-
-On receiving this task, the task tree is still null and the initial expression \cleaninline{blink true} is evaluated by the interpreter.
-This results in the task tree shown in \cref{fig:blink_tree}.
-Rewriting always starts at the top of the tree and traverses to the leaves, the basic tasks that do the actual work.
-The first basic task encountered is the \cleaninline{delay} task, that yields no value until the time, \qty{500}{\ms} in this case, has passed.
-When the \cleaninline{delay} task yielded a stable value after a number of rewrites, the task continues with the right-hand side of the \cleaninline{>>\|.} combinator.
-This combinator has a \cleaninline{writeD} task at the left-hand side that becomes stable after one rewrite step in which it writes the value to the given pin.
-When \cleaninline{writeD} becomes stable, the written value is the task value that is observed by the right-hand side of the \cleaninline{>>=.} combinator.
-This will call the interpreter to evaluate the expression, now that the argument of the function is known.
-The result of the function is again a task tree, but now with different arguments to the tasks, e.g.\ the state in \cleaninline{writeD} is inversed.
-
-\begin{figure}
- \centering
- \includestandalone{blinktree}
- \caption{The task tree for a blink task in \cref{lst:blink_code} in \gls{MTASK}.}%
- \label{fig:blink_tree}
-\end{figure}
-
-\subsection{Memory management}
-The third and final phase is memory management.
-The \gls{MTASK} \gls{RTS} is designed to run on systems with as little as \qty{2}{\kibi\byte} of \gls{RAM}.
-Aggressive memory management is therefore vital.
-Not all firmwares for microprocessors support heaps and---when they do---allocation often leaves holes when not used in a \emph{last in first out} strategy.
-The \gls{RTS} uses a chunk of memory in the global data segment with its own memory manager tailored to the needs of \gls{MTASK}.
-The size of this block can be changed in the configuration of the \gls{RTS} if necessary.
-On an \gls{ARDUINO} UNO---equipped with \qty{2}{\kibi\byte} of \gls{RAM}---the maximum viable size is about \qty{1500}{\byte}.
-The self-managed memory uses a similar layout as the memory layout for \gls{C} programs only the heap and the stack are switched (see \cref{fig:memory_layout}).
-
-\begin{figure}
- \centering
- \includestandalone{memorylayout}
- \caption{Memory layout in the \gls{MTASK} \gls{RTS}.}\label{fig:memory_layout}
-\end{figure}
-
-A task is stored below the stack and its complete state is a \gls{CLEAN} record contain most importantly the task id, a pointer to the task tree in the heap (null if not initialised yet), the current task value, the configuration of \glspl{SDS}, the configuration of peripherals, the byte code and some scheduling information.
-
-In memory, task data grows from the bottom up and the interpreter stack is located directly on top of it growing in the same direction.
-As a consequence, the stack moves when a new task is received.
-This never happens within execution because communication is always processed before execution.
-Values in the interpreter are always stored on the stack.
-Compound data types are stored unboxed and flattened.
-Task trees grow from the top down as in a heap.
-This approach allows for flexible ratios, i.e.\ many tasks and small trees or few tasks and big trees.
-
-Stable tasks, and unreachable task tree nodes are removed.
-If a task is to be removed, tasks with higher memory addresses are moved down.
-For task trees---stored in the heap---the \gls{RTS} already marks tasks and task trees as trash during rewriting so the heap can be compacted in a single pass.
-This is possible because there is no sharing or cycles in task trees and nodes contain pointers pointers to their parent.
-
\section{Compiler}\label{sec:compiler_imp}
+The byte code compiler for \gls{MTASK} is an interpretation of the \gls{MTASK} language.
+In order to compile terms, instances for all \gls{MTASK} type classes are required for the \cleaninline{:: BCInterpret a} type.
+Terms in \gls{MTASK} are constructed and compiled at run time, but type checked at compile time in the host language \gls{CLEAN}.
+The compiled tasks are sent to the device for interpretation.
+The result of the compilation is the byte code and some metadata regarding the used peripherals and \glspl{SDS}.
+The compilation target is the interpreter of the \gls{MTASK} \gls{RTS}.
+In order to keep the hardware requirements down, all expressions are evaluated on a stack.
+Rewriting of tasks uses the same stack and also a heap.
+The heap usage is minimised by applying aggressive memory management.
+A detailed overview of the \gls{RTS} including the interpreter and rewriter is found in \cref{sec:compiler_rts}.
+
\subsection{Compiler infrastructure}
-The bytecode compiler interpretation for the \gls{MTASK} language is implemented as a monad stack containing a writer monad and a state monad.
+The byte code compiler interpretation for the \gls{MTASK} language is implemented as a monad stack containing a writer monad and a state monad.
The writer monad is used to generate code snippets locally without having to store them in the monadic values.
The state monad accumulates the code, and stores the state the compiler requires.
\Cref{lst:compiler_state} shows the data type for the state, storing:
After compilation, several post-processing steps are applied to make the code suitable for the microprocessor.
First, in all tail call \cleaninline{BCReturn} instructions are replaced by \cleaninline{BCTailCall} instructions to optimise the tail calls.
Furthermore, all byte code is concatenated, resulting in one big program.
-Many instructions have commonly used arguments so shorthands are introduced to reduce the program size.
-For example, the \cleaninline{BCArg} instruction is often called with argument \numrange{0}{2} and can be replaced by the \numrange[parse-numbers=false]{\cleaninline{BCArg0}}{\cleaninline{BCArg2}} shorthands.
+Many instructions have commonly used arguments, so shorthands are introduced to reduce the program size.
+For example, the \cleaninline{BCArg} instruction is often called with argument \numrange{0}{2} and can be replaced by the \cleaninline{BCArg0} to \cleaninline{BCArg2} shorthands.
Furthermore, redundant instructions such as pop directly after push are removed as well in order not to burden the code generation with these intricacies.
-Finally the labels are resolved to represent actual program addresses instead of the freshly generated identifiers.
+Finally, the labels are resolved to represent actual program addresses instead of the freshly generated identifiers.
After the byte code is ready, the lowered \glspl{SDS} are resolved to provide an initial value for them.
-The byte code, \gls{SDS} specification and perpipheral specifications are the result of the process, ready to be sent to the device.
+The byte code, \gls{SDS} specification and peripheral specifications are the result of the process, ready to be sent to the device.
\subsection{Instruction set}
The instruction set is a fairly standard stack machine instruction set extended with special \gls{TOP} instructions for creating task tree nodes.
\begin{table}
\centering
- \caption{An overview of the compilation schemes.}
+ \caption{An overview of the compilation rules.}
\begin{tabularx}{\linewidth}{l X}
\toprule
Scheme & Description\\
\midrule
- $\cschemeE{e}{r}$ & Produces the value of expression $e$ given the context $r$ and pushes it on the stack.
- The result can be a basic value or a pointer to a task.\\
- $\cschemeF{e}$ & Generates the bytecode for functions.\\
- $\cschemeS{e}{r}{w} $ & Generates the function for the step continuation given the context $r$ and the width $w$ of the left-hand side task value.\\
+ $\cschemeE{e}{r}$ & Generates code for expressions given the context $r$\\
+ $\cschemeF{e}$ & Generates the code for functions.\\
+ $\cschemeS{e}{r}{w} $ & Generates the code for the step continuations given the context $r$ and the width $w$ of the left-hand side task value.\\
\bottomrule
\end{tabularx}
\end{table}
Values are always placed on the stack; tuples and other compound data types are unpacked.
Function calls, function arguments and tasks are also compiled using $\mathcal{E}$ but their compilations is explained later.
+\begingroup
+\allowdisplaybreaks%
\begin{align*}
\cschemeE{\text{\cleaninline{lit}}~e}{r} & = \text{\cleaninline{BCPush (bytecode e)}};\\
\cschemeE{e_1\mathbin{\text{\cleaninline{+.}}}e_2}{r} & = \cschemeE{e_1}{r};
{} & \text{\emph{Where $w_l$ is the width of the left and, $w_r$ of the right value}}\\
{} & \text{\emph{similar for other unboxed compound data types}}\\
\end{align*}
+\endgroup
Translating $\mathcal{E}$ to \gls{CLEAN} code is very straightforward, it basically means writing the instructions to the writer monad.
Almost always, the type of the interpretation is not used, i.e.\ it is a phantom type.
To still have the functions return the correct type, the \cleaninline{tell`}\footnote{\cleaninline{tell` :: [BCInstr] -> BCInterpret a}} helper is used.
-This function is similar to the writer monad's \cleaninline{tell} function but is casted to the correct type.
+This function is similar to the writer monad's \cleaninline{tell} function but is cast to the correct type.
\Cref{lst:imp_arith} shows the implementation for the arithmetic and conditional expressions.
Note that $r$, the context, is not an explicit argument here but stored in the state.
\end{align*}
A function call starts by pushing the stack and frame pointer, and making space for the program counter (\cref{lst:funcall_pushptrs}) followed by evaluating the arguments in reverse order (\cref{lst:funcall_args}).
-On executing \cleaninline{BCJumpSR}, the program counter is set and the interpreter jumps to the function (\cref{lst:funcall_jumpsr}).
+On executing \cleaninline{BCJumpSR}, the program counter is set, and the interpreter jumps to the function (\cref{lst:funcall_jumpsr}).
When the function returns, the return value overwrites the old pointers and the arguments.
This occurs right after a \cleaninline{BCReturn} (\cref{lst:funcall_ret}).
-Putting the arguments on top of pointers and not reserving space for the return value uses little space and facilitates tail call optimization.
+Putting the arguments on top of pointers and not reserving space for the return value uses little space and facilitates tail call optimisation.
\begin{figure}
\begin{subfigure}{.24\linewidth}
\end{figure}
Calling a function and referencing function arguments are an extension to $\mathcal{E}$ as shown below.
-Arguments may be at different places on the stack at different times (see \cref{ssec:step}) and therefore the exact location always is be determined from the context using \cleaninline{findarg}\footnote{\cleaninline{findarg [l`:r] l = if (l == l`) 0 (1 + findarg r l)}}.
+Arguments may be at different places on the stack at different times (see \cref{ssec:step}) and therefore the exact location is always is determined from the context using \cleaninline{findarg}\footnote{\cleaninline{findarg [l`:r] l = if (l == l`) 0 (1 + findarg r l)}}.
Compiling argument $a_{f^i}$, the $i$th argument in function $f$, consists of traversing all positions in the current context.
Arguments wider than one stack cell are fetched in reverse to reconstruct the original order.
It pops arguments from the stack according to the given task type.
The following extension of $\mathcal{E}$ shows this compilation scheme (except for the step combinator, explained in \cref{ssec:step}).
+\begingroup
+\allowdisplaybreaks%
\begin{align*}
\cschemeE{\text{\cleaninline{rtrn}}~e}{r} & =
\cschemeE{e}{r};
\cschemeE{e_2}{r};
\text{\cleaninline{BCMkTask BCAnd}};\\
\end{align*}
+\endgroup%
-This translates to Clean code by writing the correct \cleaninline{BCMkTask} instruction as exemplified in \cref{lst:imp_ret}.
+This compilation scheme translates to Clean code by first writing the arguments and then the correct \cleaninline{BCMkTask} instruction.
+This is shown for the \cleaninline{.&&.} task in \cref{lst:imp_ret}.
\begin{lstClean}[caption={The byte code interpretation implementation for \cleaninline{rtrn}.},label={lst:imp_ret}]
-instance rtrn BCInterpret
-where
- rtrn m = m >>| tell` [BCMkTask (bcstable m)]
+instance .&&. BCInterpret where
+ (.&&.) l r = l >>| r >>| tell` [BCMkTask BCTAnd]
\end{lstClean}
\subsection{Sequential combinator}\label{ssec:step}
Furthermore, it is a lazy node in the task tree: the right-hand side may yield a new task tree after several rewrite steps, i.e.\ it is allowed to create infinite task trees using step combinators.
The function is generated using the $\mathcal{S}$ scheme that requires two arguments: the context $r$ and the width of the left-hand side so that it can determine the position of the stability which is added as an argument to the function.
The resulting function is basically a list of if-then-else constructions to check all predicates one by one.
-Some optimization is possible here but has currently not been implemented.
+Some optimisation is possible here but has currently not been implemented.
\begin{align*}
- \cschemeE{t_1\text{\cleaninline{>>*.}}t_2}{r} & =
+ \cschemeE{t_1\text{\cleaninline{>>*.}}e_2}{r} & =
\cschemeE{a_{f^i}}{r}, \langle f, i\rangle\in r;
\text{\cleaninline{BCMkTask}}~\text{\cleaninline{BCStable}}_{\stacksize{r}}; \cschemeE{t_1}{r};\\
- {} & \mathbin{\phantom{=}} \text{\cleaninline{BCMkTask}}~\text{\cleaninline{BCAnd}}; \text{\cleaninline{BCMkTask}}~(\text{\cleaninline{BCStep}}~(\cschemeS{t_2}{(r + [\langle l_s, i\rangle])}{\stacksize{t_1}}));\\
+ {} & \mathbin{\phantom{=}} \text{\cleaninline{BCMkTask}}~\text{\cleaninline{BCAnd}}; \text{\cleaninline{BCMkTask}}~(\text{\cleaninline{BCStep}}~(\cschemeS{e_2}{(r + [\langle l_s, i\rangle])}{\stacksize{t_1}}));\\
\end{align*}
\begin{align*}
{} & \text{\emph{Similar for \cleaninline{IfStable} and \cleaninline{IfUnstable}}}\\
\end{align*}
-First the context is evaluated.
+The step combinator has a task as the left-hand side and a list of continuations at the right-hand side.
+First the context is evaluated ($\cschemeE{a_{f^i}}{r}$).
The context contains arguments from functions and steps that need to be preserved after rewriting.
-The evaluated context is combined with the left-hand side task value by means of a \cleaninline{.&&.} combinator to store it in the task tree so that it is available after a rewrite.
-This means that the task tree is be transformed as seen in \cref{lst:context_tree}.
+The evaluated context is combined with the left-hand side task value by means of a \cleaninline{.&&.} combinator to store it in the task tree so that it is available after a rewrite step.
+This means that the task tree is transformed as seen in \cref{lst:context_tree}.
+In this figure, the expression \cleaninline{t1 >>=. \\v1->t2 >>=. \\v2->...} is shown\footnote{%
+ \cleaninline{t >>=. e} is a shorthand combinator for \cleaninline{t >>* [OnStable (\\_->true) e].}}.
+Then, the right-hand side list of continuations is converted to an expression using $\mathcal{S}$.
\begin{figure}
\begin{subfigure}{.5\textwidth}
\includestandalone{contexttree2}
\caption{With the embedded context.}
\end{subfigure}
- \caption{Context embedded in a task tree.}%
+ \caption{Context embedded in a virtual task tree.}%
\label{lst:context_tree}
\end{figure}
//Return width (always 1, a task pointer)
(Just one)
>>| modify (\s->{s & bcs_context=ctx})
- >>| tell` [BCMkTask (instr rhswidth funlab)]
-
+ >>| tell` [BCMkTask (instr rhswidth funlab)][+\pagebreak+]
toContFun :: JumpLabel UInt8 -> BCInterpret a
toContFun steplabel contextwidth
= foldr tcf (tell` [BCPush fail]) cont
\end{lstClean}
\subsection{Shared data sources}\label{lst:imp_sds}
-The compilation scheme for \gls{SDS} definitions is a trivial extension to $\mathcal{F}$ since there is no code generated as seen below.
-
-\begin{align*}
- \cschemeF{\text{\cleaninline{sds}}~x=i~\text{\cleaninline{In}}~m} & =
- \cschemeF{m};\\
-\end{align*}
-
+The compilation scheme for \gls{SDS} definitions is a trivial extension to $\mathcal{F}$.
+While there is no code generated in the definition, the byte code compiler is storing all \gls{SDS} data in the \cleaninline{bcs_sdses} field in the compilation state.
+Regular \glspl{SDS} are stored as \cleaninline{Right String255} values.
+The \glspl{SDS} are typed as functions in the host language, so an argument for this function must be created that represents the \gls{SDS} on evaluation.
+For this, an \cleaninline{BCInterpret} is created that emits this identifier.
+When passing it to the function, the initial value of the \gls{SDS} is returned.
+In the case of a local \gls{SDS}, this initial value is stored as a byte code encoded value in the state and the compiler continues with the rest of the program.
The \gls{SDS} access tasks have a compilation scheme similar to other tasks (see \cref{ssec:scheme_tasks}).
The \cleaninline{getSds} task just pushes a task tree node with the \gls{SDS} identifier embedded.
The \cleaninline{setSds} task evaluates the value, lifts that value to a task tree node and creates \pgls{SDS} set node.
\begin{align*}
+ \cschemeF{\text{\cleaninline{sds}}~x=i~\text{\cleaninline{In}}~m} & =
+ \cschemeF{m};\\
+ \\
\cschemeE{\text{\cleaninline{getSds}}~s}{r} & =
- \text{\cleaninline{BCMkTask}} (\text{\cleaninline{BCSdsGet}} s);\\
+ \text{\cleaninline{BCMkTask}}~(\text{\cleaninline{BCSdsGet}}~s);\\
\cschemeE{\text{\cleaninline{setSds}}~s~e}{r} & =
\cschemeE{e}{r};
\text{\cleaninline{BCMkTask BCStable}}_{\stacksize{e}};\\
- {} & \mathbin{\phantom{=}} \text{\cleaninline{BCMkTask}} (\text{\cleaninline{BCSdsSet}} s);\\
+ {} & \mathbin{\phantom{=}} \text{\cleaninline{BCMkTask}}~(\text{\cleaninline{BCSdsSet}}~s);\\
\end{align*}
-While there is no code generated in the definition, the byte code compiler is storing all \gls{SDS} data in the \cleaninline{bcs_sdses} field in the compilation state.
-Regular \glspl{SDS} are stored as \cleaninline{Right String255} values.
-The \glspl{SDS} are typed as functions in the host language so an argument for this function must be created that represents the \gls{SDS} on evaluation.
-For this, an \cleaninline{BCInterpret} is created that emits this identifier.
-When passing it to the function, the initial value of the \gls{SDS} is returned.
-In the case of a local \gls{SDS}, this initial value is stored as a byte code encoded value in the state and the compiler continues with the rest of the program.
-
\Cref{lst:comp_sds} shows the implementation of the \cleaninline{sds} type class.
First, the initial \gls{SDS} value is extracted from the expression by bootstrapping the fixed point with a dummy value.
This is safe because the expression on the right-hand side of the \cleaninline{In} is never evaluated.
Then, using \cleaninline{addSdsIfNotExist}, the identifier for this particular \gls{SDS} is either retrieved from the compiler state or generated freshly.
This identifier is then used to provide a reference to the \cleaninline{def} definition to evaluate the main expression.
Compiling \cleaninline{getSds} is a matter of executing the \cleaninline{BCInterpret} representing the \gls{SDS}, which yields the identifier that can be embedded in the instruction.
-Setting the \gls{SDS} is similar: the identifier is retrieved and the value is written to put in a task tree so that the resulting task can remember the value it has written.
+Setting the \gls{SDS} is similar: the identifier is retrieved, and the value is written to put in a task tree so that the resulting task can remember the value it has written.
% VimTeX: SynIgnore on
-\begin{lstClean}[caption={Backend implementation for the SDS classes.},label={lst:comp_sds}]
+\begin{lstClean}[caption={Backend implementation for the \gls{SDS} classes.},label={lst:comp_sds}]
:: Sds a = Sds Int
instance sds BCInterpret where
sds def = {main =
This is done so that the \cleaninline{withDevice} task can write the received \gls{SDS} updates to the according \gls{SDS} while the \gls{SDS} is not in scope.
The \gls{ITASK} notification mechanism then takes care of the rest.
Such \pgls{SDS} is created by using the \cleaninline{mapReadWriteError} which, given a pair of read and write functions with error handling, produces \pgls{SDS} with the lens embedded.
-The read function transforms converts the typed value to a typeless serialised value.
+The read function transforms the typed value to a typeless serialised value.
The write function will, given a new serialised value and the old typed value, produce a new typed value.
-It tries to decode the serialised value, if that succeeds, it is written to the underlying \gls{SDS}, an error is thrown otherwise.
+It tries to decode the serialised value, if that succeeds, it is written to the underlying \gls{SDS}, otherwise, an error is thrown otherwise.
\Cref{lst:mtask_itasksds_lens} shows the implementation for this.
% VimTeX: SynIgnore on
}\end{lstClean}
% VimTeX: SynIgnore off
-\section{C code generation}\label{sec:ccodegen}
-All communication between the \gls{ITASK} server and the \gls{MTASK} server is type parametrised.
+\section{Run-time system}\label{sec:compiler_rts}
+The \gls{RTS} is a customisable domain-specific \gls{OS} that takes care of the execution of tasks.
+Furthermore, it also takes care of low-level mechanisms such as the communication, multitasking, and memory management.
+Once a device is programmed with the \gls{MTASK} \gls{RTS}, it can continuously receive new tasks without the need for reprogramming.
+The \gls{OS} is written in portable \ccpp{} and only contains a small device-specific portion.
+In order to keep the abstraction level high and the hardware requirements low, much of the high-level functionality of the \gls{MTASK} language is implemented not in terms of lower-level constructs from \gls{MTASK} language but in terms of \ccpp{} code.
+
+Most microcontroller software consists of a cyclic executive instead of an \gls{OS}.
+This one loop function is continuously executed and all work is performed there.
+In the \gls{RTS} of the \gls{MTASK} system, there is also such an event loop function.
+It is a function with a relatively short execution time that gets called repeatedly.
+The event loop consists of three distinct phases.
+After doing the three phases, the devices goes to sleep for as long as possible (see \cref{chp:green_computing_mtask} for more details on task scheduling).
+
+\subsection{Communication phase}
+In the first phase, the communication channels are processed.
+The exact communication method is a customisable device-specific option baked into the \gls{RTS}.
+The interface is kept deliberately simple and consists of two layers: a link interface and a communication interface.
+Besides opening, closing and cleaning up, the link interface has three functions that are shown in \cref{lst:link_interface}.
+Consequently, implementing this link interface is very simple, but it is still possible to implement more advanced link features such as buffering.
+There are implementations for this interface for serial or \gls{WIFI} connections using \gls{ARDUINO}, and \gls{TCP} connections for Linux.
+
+\begin{lstArduino}[caption={Link interface of the \gls{MTASK} \gls{RTS}.},label={lst:link_interface}]
+bool link_input_available(void);
+uint8_t link_read_byte(void);
+void link_write_byte(uint8_t b);
+\end{lstArduino}
+
+The communication interface abstracts away from this link interface and is typed instead.
+It contains only two functions as seen in \cref{lst:comm_interface}.
+There are implementations for direct communication, or communication using an \gls{MQTT} broker.
+Both use the automatic serialisation and deserialisation shown in \cref{sec:ccodegen}.
+
+\begin{lstArduino}[caption={Communication interface of the \gls{MTASK} \gls{RTS}.},label={lst:comm_interface}]
+struct MTMessageTo receive_message(void);
+void send_message(struct MTMessageFro msg);
+\end{lstArduino}
+
+Processing the received messages from the communication channels happens synchronously and the channels are exhausted completely before moving on to the next phase.
+There are several possible messages that can be received from the server:
+
+\begin{description}
+ \item[SpecRequest]
+ is a message instructing the device to send its specification.
+ It is received immediately after connecting.
+ The \gls{RTS} responds with a \texttt{Spec} answer containing the specification.
+ \item[TaskPrep]
+ tells the device a task is on its way.
+ Especially on faster connections, it may be the case that the communication buffers overflow because a big message is sent while the \gls{RTS} is busy executing tasks.
+ This message allows the \gls{RTS} to postpone execution for a while, until the larger task has been received.
+ The server sends the task only after the device acknowledged the preparation by sending a \texttt{TaskPrepAck} message.
+ \item[Task]
+ contains a new task, its peripheral configuration, the \glspl{SDS}, and the byte code.
+ The new task is immediately copied to the task storage but is only initialised during the next phase.
+ The device acknowledges the task by sending a \texttt{TaskAck} message.
+ \item[SdsUpdate]
+ notifies the device of the new value for a lowered \gls{SDS}.
+ The old value of the lowered \gls{SDS} is immediately replaced with the new one.
+ There is no acknowledgement required.
+ \item[TaskDel]
+ instructs the device to delete a running task.
+ Tasks are automatically deleted when they become stable.
+ However, a task may also be deleted when the surrounding task on the server is deleted, for example when the task is on the left-hand side of a step combinator and the condition to step holds.
+ The device acknowledges the deletion by sending a \texttt{TaskDelAck}.
+ \item[Shutdown]
+ tells the device to reset.
+\end{description}
+
+\subsection{Execution phase}
+The second phase performs one execution step for all tasks that wish for it.
+Tasks are placed in a priority queue orderd by the time a task needs to execute.
+The \gls{RTS} selects all tasks that can be scheduled, see \cref{sec:scheduling} for more details.
+Execution of a task is always an interplay between the interpreter and the rewriter.
+The rewriter scans the current task tree and tries to rewrite it using small-step reduction.
+Expressions in the tree are always strictly evaluated by the interpreter.
+
+When a new task is received, the main expression is evaluated to produce a task tree.
+A task tree is a tree structure in which each node represents a task combinator and the leaves are basic tasks.
+If a task is not initialised yet, i.e.\ the pointer to the current task tree is still null, the byte code of the main function is interpreted.
+The main expression of \gls{MTASK} programs sent to the device before execution always produces a task tree.
+Execution of a task consists of continuously rewriting the task until its value is stable.
+
+Rewriting is a destructive process, i.e.\ the rewriting is done in place.
+The rewriting engine uses the interpreter when needed, e.g.\ to calculate the step continuations.
+The rewriter and the interpreter use the same stack to store intermediate values.
+Rewriting steps are small so that interleaving results in seemingly parallel execution.
+In this phase new task tree nodes may be allocated.
+Both rewriting and initialization are atomic operations in the sense that no processing on \glspl{SDS} is done other than \gls{SDS} operations from the task itself.
+The host is notified if a task value is changed after a rewrite step by sending a \texttt{TaskReturn} message.
+
+Take for example a blink task for which the code is shown in \cref{lst:blink_code}.
+
+\begin{lstClean}[caption={Code for a blink program.},label={lst:blink_code}]
+declarePin D13 PMOutput \ledPin->
+fun \blink=(\st->delay (lit 500) >>|. writeD ledPin st >>=. blink o Not)
+In {main = blink true}
+\end{lstClean}
+
+On receiving this task, the task tree is still null and the initial expression \cleaninline{blink true} is evaluated by the interpreter.
+This results in the task tree shown in \cref{fig:blink_tree1}.
+Rewriting always starts at the top of the tree and traverses to the leaves, the basic tasks that do the actual work.
+The first basic task encountered is the \cleaninline{delay} task, that yields no value until the time, \qty{500}{\ms} in this case, has passed.
+When the \cleaninline{delay} task yielded a stable value after a number of rewrites, the task continues with the right-hand side of the \cleaninline{>>\|.} combinator by evaluating the expression (see \cref{fig:blink_tree2})\footnotemark.
+\footnotetext{\cleaninline{t1 >>\|. t2} is a shorthand for \cleaninline{t1 >>*. [IfStable id \\_->t2]}.}
+This combinator has a \cleaninline{writeD} task at the left-hand side that becomes stable after one rewrite step in which it writes the value to the given pin.
+When \cleaninline{writeD} becomes stable, the written value is the task value that is observed by the right-hand side of the \cleaninline{>>=.} combinator.
+Then the interpreter is used again to evaluate the expression, now that the argument of the function is known.
+The result of the call to the function is again a task tree, but now with different arguments to the tasks, e.g.\ the state in \cleaninline{writeD} is inversed.
+
+\begin{figure}
+ \centering
+ \begin{subfigure}[t]{.5\textwidth}
+ \includestandalone{blinktree1}
+ \caption{Initial task tree}%
+ \label{fig:blink_tree1}
+ \end{subfigure}%
+ \begin{subfigure}[t]{.5\textwidth}
+ \includestandalone{blinktree2}
+ \caption{Task tree after the delay passed}%
+ \label{fig:blink_tree2}
+ \end{subfigure}
+ \caption{The task trees during reduction for a blink task in \gls{MTASK}.}%
+ \label{fig:blink_tree}
+\end{figure}
+
+\subsection{Memory management}
+The third and final phase is memory management.
+The \gls{MTASK} \gls{RTS} is designed to run on systems with as little as \qty{2}{\kibi\byte} of \gls{RAM}.
+Aggressive memory management is therefore vital.
+Not all firmwares for microprocessors support heaps and---when they do---allocation often leaves holes when not used in a \emph{last in first out} strategy.
+The \gls{RTS} uses a chunk of memory in the global data segment with its own memory manager tailored to the needs of \gls{MTASK}.
+The size of this block can be changed in the configuration of the \gls{RTS} if necessary.
+On an \gls{ARDUINO} UNO---equipped with \qty{2}{\kibi\byte} of \gls{RAM}---the maximum viable size is about \qty{1500}{\byte}.
+The self-managed memory uses a similar layout as the memory layout for \gls{C} programs only the heap and the stack are switched (see \cref{fig:memory_layout}).
+
+\begin{figure}
+ \centering
+ \includestandalone{memorylayout}
+ \caption{Memory layout in the \gls{MTASK} \gls{RTS}.}\label{fig:memory_layout}
+\end{figure}
+
+A task is stored below the stack and it consists of the task id, a pointer to the task tree in the heap (null if not initialised yet), the current task value, the configuration of \glspl{SDS}, the configuration of peripherals, the byte code and some scheduling information.
+
+In memory, task data grows from the bottom up and the interpreter stack is located directly on top of it growing in the same direction.
+As a consequence, the stack moves when a new task is received.
+This never happens within execution because communication is always processed before execution.
+Values in the interpreter are always stored on the stack.
+Compound data types are stored unboxed and flattened.
+Task trees grow from the top down as in a heap.
+This approach allows for flexible ratios, i.e.\ many tasks and small trees or few tasks and big trees.
+
+Stable tasks, and unreachable task tree nodes are removed.
+If a task is to be removed, tasks with higher memory addresses are moved down.
+For task trees---stored in the heap---the \gls{RTS} already marks tasks and task trees as trash during rewriting, so the heap can be compacted in a single pass.
+This is possible because there is no sharing or cycles in task trees and nodes contain pointers to their parent.
+
+
+\section{C code generation for communication}\label{sec:ccodegen}
+All communication between the \gls{ITASK} server and the \gls{MTASK} server is type parametrised and automated.
From the structural representation of the type, a \gls{CLEAN} parser and printer is constructed using generic programming.
Furthermore, a \ccpp{} parser and printer is generated for use on the \gls{MTASK} device.
The technique for generating the \ccpp{} parser and printer is very similar to template metaprogramming and requires a rich generic programming library or compiler support that includes a lot of metadata in the record and constructor nodes.
-Using generic programming in the \gls{MTASK} system, both serialisation and deserialisation on the microcontroller and and the server is automatically generated.
+Using generic programming in the \gls{MTASK} system, both serialisation and deserialisation on the microcontroller and the server is automatically generated.
\subsection{Server}
On the server, off-the-shelve generic programming techniques are used to make the serialisation and deserialisation functions (see \cref{lst:ser_deser_server}).
Serialisation is a simple conversion from a value of the type to a string.
-Deserialisation is a little bit different in order to support streaming\footnotemark.
+Deserialisation is a bit different in order to support streaming\footnotemark.
\footnotetext{%
Here the \cleaninline{*!} variant of the generic interface is chosen that has less uniqueness constraints for the compiler-generated adaptors \citep{alimarine_generic_2005,hinze_derivable_2001}.%
}
\end{lstArduino}
\section{Conclusion}
+This chapter showed the implementation of the \gls{MTASK} byte code compiler, the \gls{RTS}, and the internals of their communication.
It is not straightforward to execute \gls{MTASK} tasks on resources-constrained \gls{IOT} edge devices.
-To achieve this, the terms in the \gls{DSL} are compiled to domain-specific byte code.
+To achieve this, the terms in the \gls{DSL} are compiled to compact domain-specific byte code.
This byte code is sent for interpretation to the light-weight \gls{RTS} of the edge device.
-The \gls{RTS} first evaluates the main expression.
+The \gls{RTS} first evaluates the main expression in the interpreter.
The result of this evaluation, a run time representation of the task, is a task tree.
-This task tree is rewritten according to rewrite rules until a stable value is observed.
-Rewriting multiple tasks at the same time is achieved by interleaving the rewrite steps, resulting in seamingly parallel execution of the tasks.
-Furthermore, the \gls{RTS} automates communication and coordinates multi tasking.
+This task tree is rewritten according to small-step reduction rules until a stable value is observed.
+Rewriting multiple tasks at the same time is achieved by interleaving the rewrite steps, resulting in seemingly parallel execution of the tasks.
+All communication, including the serialisation and deserialisation, between the server and the \gls{RTS} is automated.
+From the structural representation of the types, printers and parsers are generated for the server and the client.
\input{subfilepostamble}
\end{document}
repositories / phd-thesis.git / blobdiff