\myacronym{TCP}{TCP}{transmission control protocol}
\myacronym{TOP}{TOP}{task-oriented programming}
\myacronym{TOSD}{TOSD}{task-oriented software development}
+\myacronym{LSOC}{LSOC}{layered separation of concerns}
\myacronym{TRS}{TRS}{term rewriting system}
\myacronym{TTH}{TTH}{typed \glsxtrlong{TH}}
\myacronym{TVOC}{TVOC}{total volatile organic compounds}
}
\newglossaryentry{ARDUINO}{%
name=Arduino,
- description={is a widely used framework for programming microprocessors}
+ description={is a widely used framework for programming microcontrollers}
}
\newglossaryentry{MBED}{%
name=mbed,
- description={is a widely used framework for programming microprocessors designed for ARM cortex-M}
+ description={is a widely used framework for programming microcontrollers designed for ARM cortex-M}
}
\newglossaryentry{CPP}{
name=C\texttt{++},
}
\newglossaryentry{I2C}{
name=I\textsuperscript{2}C,
- description={is a simple serial communication protocol often used to connect sensors to microprocessors}
+ description={is a simple serial communication protocol often used to connect sensors to microcontrollers}
}
\newglossaryentry{SPI}{
name=SPI,
- description={is a synchronous serial communication protocol often used to connect sensors to microprocessors}
+ description={is a synchronous serial communication protocol often used to connect sensors to microcontrollers}
}
\newglossaryentry{TINYML}{
name=TinyML,
- description={is a deep learning framework for microprocessors}
+ description={is a deep learning framework for microcontrollers}
}
\newglossaryentry{PYTHON}{
name=Python,
}
\newglossaryentry{MICROPYTHON}{
name=MicroPython,
- description={is a \gls{PYTHON} implementation tailored for microprocessors}
+ description={is a \gls{PYTHON} implementation tailored for microcontrollers}
}
\newglossaryentry{FREERTOS}{
name=FreeRTOS,
- description={is an open-source \gls{RTOS} for microprocessors}
+ description={is an open-source \gls{RTOS} for microcontrollers}
}
\newglossaryentry{ONEWIRE}{
name=1-wire,
- description={is simple single wire communication protocol often used to connect sensors to microprocessors}
+ description={is simple single wire communication protocol often used to connect sensors to microcontrollers}
}
\newglossaryentry{JSON}{
name=JSON,
}
\newglossaryentry{WEMOS}{
name=WEMOS,
- description={is a popular ESP8266 microprocessor based prototyping platform supporting \gls{ARDUINO}.}
+ description={is a popular ESP8266 microcontroller based prototyping platform supporting \gls{ARDUINO}.}
}
% Never expand
\begin{chapterabstract}
This chapter introduces the contents of the thesis and a reading guide.
Furthermore, it provides background material on \glsxtrlong{IOT}, \glsxtrlongpl{DSL}, and \glsxtrlong{TOP}; and a detailed overview of the contributions.
+ It also gives a brief introduction to two \gls{TOP} languages: \gls{ITASK} and \gls{MTASK}.
\end{chapterabstract}
-There are at least 13.4 billion computers connected to the internet at the time of writing\footnote{\url{https://transformainsights.com/research/tam/market}, accessed on: \formatdate{13}{10}{2022}}.
+There are at least 13.4 billion devices connected to the internet at the time of writing\footnote{\url{https://transformainsights.com/research/tam/market}, accessed on: \formatdate{13}{10}{2022}}.
These devices sense, act, or otherwise interact with people, other computers, and the world surrounding us.
Notwithstanding the substantial variety among these devices, they have one thing in common: they are all require software to operate.
An increasing amount of these connected devices are so-called \emph{edge devices} that operate in the \gls{IOT}.
-Typically these edge devices are small microprocessors containing sensors and actuators to interact with the physical world.
-The variety within edge devices, and the fact that they differ substantially from other types of devices, makes it complex to program \gls{IOT} systems.\todo{make sentence logical}
+Typically these edge devices are small microcontrollers containing sensors and actuators to interact with the physical world.
+Microcontrollers are integrated circuits containing a microprocessor designed for use in embedded applications.
+The edge devices come in many different types and they differ substantially from the other devices in the system.
+Consequently, programming \gls{IOT} systems is very complex and error prone.
Hence, an \gls{IOT} programmer has to program each device and their interoperation using different programming paradigms, programming languages, and abstraction levels resulting in semantic friction.
This thesis introduces research on the many aspects of orchestrating \gls{IOT} systems using \gls{TOP}.
-\Gls{TOP} is a novel programming paradigm for programming multi-tier workflow systems using a single declarative specification of the work that needs to be done.
-Using advanced compiler technologies, much of the internals and communication of multi-tier applications are automatically generated and the result of compilation is a ready-for-work application.
+\Gls{TOP} is a innovative tierless programming paradigm for programming multi-tier interactive systems using a single declarative specification of the work that needs to be done.
+Using advanced compiler technologies, much of the internals and communication of multi-tier applications is automatically generated and the result of compilation is a ready-for-work application.
Unfortunately, because the abstraction level is so high, the hardware requirements are too excessive to be suitable for the average edge device.
This is where \glspl{DSL} come into play.
The chapters are readable independently.
\Cref{prt:top} is a monograph showing \gls{MTASK}, a \gls{TOP} \gls{DSL} for the \gls{IOT}.
Hence, the chapters are best read in order.
-\Cref{prt:tvt} is a journal article in which quantitative and qualitatively compares traditional tiered \gls{IOT} programming to tierless programming.
+\Cref{prt:tvt} is a journal article in which traditional tiered \gls{IOT} programming is qualitatively and quantitatively compared to tierless programming using a real-world application.
The chapter is readable independently.
The following sections provide background material on the \gls{IOT}, \glspl{DSL}, and \gls{TOP} after which a detailed overview of the contributions is presented.
The application layer provides the \glspl{API}, data interfaces, and data storage of the \gls{IOT} system.
A cloud server or local server provides this layer in a typical home automation application.
-All layers are connected using the network layer.
-In many applications this is implemented using conventional networking techniques such as WiFi or Ethernet.
-However, networks or layers on top of it---tailored to the needs of \gls{IOT} systems---have been increasingly popular such as \gls{BLE}, LoRa, ZigBee, LTE-M, or \gls{MQTT}.
-
The perception layer---also called edge layer---collects the data and interacts with the environment.
-It consists of edge devices such as microprocessors equipped with various sensors and actuators.
+It consists of edge devices such as microcontrollers equipped with various sensors and actuators.
In home automation this layer consists of all the devices hosting the sensors and actuators such as a smart lightbulb, an actuator to open a door or a temperature and humidity sensor.
+All layers are connected using the network layer.
+In many applications this is implemented using conventional networking techniques such as WiFi or Ethernet.
+However, networks or layers on top of it---tailored to the needs of the specific interconnection between layers---have been increasingly popular.
+Examples of this are \gls{BLE}, LoRa, ZigBee, LTE-M, or \gls{MQTT} for connecting the perception layer to the application layer and techniques such as HTTP, AJAX, and WebSocket for connecting the presentation layer to the application layer.
+
Across the layers, the devices are a large heterogeneous collection of different platforms, protocols, paradigms, and programming languages often resulting in impedance problems or semantic friction between layers when programming \citep{ireland_classification_2009}.
-Even more so, perception layer itself often is a heterogeneous collections of microprocessors in itself as well, each having their own peculiarities, language of choice, and hardware interfaces.
-As the edge hardware needs to be cheap, small-scale, and energy efficient, the microprocessors used to power these devices do not have a lot of computational power, only a soup\c{c}on of memory, and little communication bandwidth.
-Typically the devices do not run a full fledged \gls{OS} but a compiled firmware that written in an imperative language.
+Even more so, perception layer itself often is a heterogeneous collections of microcontrollers in itself as well, each having their own peculiarities, language of choice, and hardware interfaces.
+As the edge hardware needs to be cheap, small-scale, and energy efficient, the microcontrollers used to power these devices do not have a lot of computational power, only a soup\c{c}on of memory, and little communication bandwidth.
+Typically the devices are unable to run a full-fledged general-purpose \gls{OS} but use compiled firmwares that are written in an imperative language.
While devices are getting a bit faster, smaller, and cheaper, they keep these properties to an extent, greatly reducing the flexibility for dynamic systems where tasks are created on the fly, executed on demand, or require parallel execution.
As program memory is mostly flash based and only lasts a couple of thousand writes before it wears out, it is not suitable for rapid reconfiguring and reprogramming.
-These problems can be mitigated by dynamically sending code to be interpreted to the microprocessor.
+These problems can be mitigated by dynamically sending code to be interpreted to the microcontroller.
With interpretation, a specialized interpreter is flashed in the program memory once that receives the program code to execute at runtime.
Interpretation always comes with an overhead, making it challenging to create them for small edge devices.
However, the hardware requirements can be reduced by embedding domain-specific data into the programming language to be interpreted, so called \glspl{DSL}.
\section{\texorpdfstring{\Glsxtrlong{TOP}}{Task-oriented programming}}\label{sec:back_top}
\Gls{TOP} is a declarative programming paradigm designed to model interactive systems \citep{plasmeijer_task-oriented_2012}.
\Citet{steenvoorden_tophat_2022} defines two instruments for \gls{TOP}: \gls{TOP} languages and \gls{TOP} engines.
-A \gls{TOP} language is a language to specify workflows.\todo{improve sentence}
-A \gls{TOP} engine executes such a specification as a ready-for-work application.
-Instead of dividing problems into layers or tiers, as is done in \gls{IOT} architectures, it deals with separation of concerns in a novel way.
+A \gls{TOP} language is the language to specify interactive systems.
+A \gls{TOP} engine is software or hardware that executes such a specification as a ready-for-work application.
+Instead of dividing problems into \gls{LSOC} it deals with separation of concerns in a novel way.
From the data types, utilising various \emph{type-parametrised} concepts, all other aspects are handled automatically (see \cref{fig:tosd}).
This approach to software development is called \gls{TOSD} \citep{wang_maintaining_2018}.
\begin{subfigure}[t]{.5\textwidth}
\centering
\includestandalone{traditional}
- \caption{Traditional layered approach.}
+ \caption{\Gls{LSOC} approach.}
\end{subfigure}%
\begin{subfigure}[t]{.5\textwidth}
\centering
\label{fig:tosd}
\end{figure}
-\todo{describe relation with \gls{IOT} architecture}
\begin{description}
\item[\Glsxtrshort{UI} (presentation layer):]
The \gls{UI} of the system is automatically generated from the representation of the type.
The \gls{UOD} from the business layer is explicitly and separately modelled by the relations that exist in the functions of the host language.
\end{description}
+There are two ways of looking at this model when also incorporating edge devices for \gls{IOT} systems.
+Firstly, edge devices can be seen as simple resources, thus accessed through the resource access layer.
+Secondly, edge devices are miniature \gls{LSOC} systems in itself as well.
+In \gls{TOSD} the same can be applied.
+The individual components in the miniature systems, the tasks, the \glspl{SDS}, are connected to the other systems.
+
\subsection{\texorpdfstring{\Gls{ITASK}}{ITask}}
-The concept of \gls{TOP} originated from the \gls{ITASK} framework, a declarative workflow language and \gls{TOP} engine for defining multi-user distributed web applications implemented as an \gls{EDSL} in the lazy pure \gls{FP} language \gls{CLEAN} \citep{plasmeijer_itasks:_2007,plasmeijer_task-oriented_2012}.
+The concept of \gls{TOP} originated from the \gls{ITASK} framework, a declarative interactive systems language and \gls{TOP} engine for defining multi-user distributed web applications implemented as an \gls{EDSL} in the lazy pure \gls{FP} language \gls{CLEAN} \citep{plasmeijer_itasks:_2007,plasmeijer_task-oriented_2012}.
From the structural properties of the data types, the entire user interface is automatically generated.
As an example, \cref{lst:enter_person,fig:enter_person} show the \gls{ITASK} code and the corresponding \gls{UI} for a simple task for entering a person.
\end{lstClean}
\subsection{\texorpdfstring{\Gls{MTASK}}{MTask}}
-This thesis uses a novel \gls{TOP} language designed for defining workflows for \gls{IOT} edge devices \citep{koopman_task-based_2018}.
+This thesis uses \gls{ITASK} in conjunction with an innovative \gls{TOP} language designed for defining interactive systems for \gls{IOT} edge devices called \gls{MTASK} \citep{koopman_task-based_2018}.
It is written in \gls{CLEAN} as a multi-view \gls{EDSL} and hence there are multiple interpretations of the language of which the byte code compiler is the most relevant for this thesis.
From the terms in the \gls{TOP} language, a very compact binary representation of the work that needs to be done is compiled.
This specification is then sent to a device that runs the \gls{MTASK} \gls{RTS}, a domain-specific \gls{TOP} engine implemented as a feather-light domain-specific \gls{OS}.
\Gls{MTASK} is seamlessly integrated with \gls{ITASK}, it allows the programmer to define all layers of an \gls{IOT} system from a single declarative specification.
-\todo[inline]{Example application here, e.g.\ blink?}
-
+\todo[inline]{Is this example useful? Add more detailed explanation with line numbers?}
+\Cref{lst:intro_blink} shows an \gls{MTASK}\slash{}\gls{ITASK} application for an interactive application where the \gls{LED} on the microcontroller blinks every user-specified interval.
+Using a \glspl{SDS} defined in \gls{ITASK}, the blinking frequency of an \gls{LED} connected to \gls{GPIO} pin 13 can be changed on the fly.
+
+\begin{lstClean}[numbers=left,caption={\Gls{MTASK}\slash{}\gls{ITASK} interactive blinking.},label={lst:intro_blink}]
+interactiveBlink :: Task Int
+interactiveBlink =
+ withShared 500 \iInterval->
+ withDevice {TCPSettings | host = ..., port = ...} \dev->
+ liftmTask (intBlink iInterval) dev
+ -|| Hint "Interval (ms)" @>> updateSharedInformation [] iInterval
+
+intBlink :: Shared sds Int -> MTask v Int | mtask, liftsds v & RWShared sds
+intBlink iInterval =
+ declarePin D13 PMOutput \d13->
+ liftsds \mInterval=iInterval
+ In fun \blink=(\st->
+ writeD d13 st
+ >>|. getSds mInterval
+ >>=. \i->delay i
+ >>|. blink (Not st))
+ In {main = blink true}
+\end{lstClean}
\subsection{Other \texorpdfstring{\glsxtrshort{TOP}}{TOP} languages}
While \gls{ITASK} conceived \gls{TOP}, it is not the only \gls{TOP} language and engine.
\section{Contributions}\label{sec:contributions}
This section provides a thorough overview of the relation to publications and the scientific contributions of the episodes and chapters.
-\subsection{\nameref{prt:dsl}}
+\subsection{\Fullref{prt:dsl}}
The \gls{MTASK} system is a heterogeneous \gls{EDSL} and during the development of it, several novel basal techniques for embedding \glspl{DSL} in \gls{FP} languages have been found.
-This first episode is a cumulative---otherwise known as paper-based---episode consisting of two papers published on novel embedding techniques.
-Both papers are readable independently.
-
-\subsubsection*{\Fullref{chp:classy_deep_embedding}}
-This chapter is based on the paper: \citeentry{lubbers_deep_2022}\todo{change in-press when published}.
+This episode is a paper based episodes on these techniques.
+\Cref{chp:classy_deep_embedding} is based on the paper \emph{Deep Embedding with Class} \citep{lubbers_deep_2022}.
While supervising \citeauthor{amazonas_cabral_de_andrade_developing_2018}'s \citeyear{amazonas_cabral_de_andrade_developing_2018} Master's thesis, focussing on an early version of \gls{MTASK}, a seed was planted for a novel deep embedding technique for \glspl{DSL} where the resulting language is extendible both in constructs and in interpretation using type classes and existential data types.
Slowly the ideas organically grew to form the technique shown in the paper.
The related work section is updated with the research found only after publication.
\Cref{sec:classy_reprise} was added after publication and contains a (yet) unpublished extension of the embedding technique for reducing the required boilerplate.
-The research from this paper and writing the paper was solely performed by me.
-
-\subsubsection*{\Fullref{chp:first-class_datatypes}}
-This chapter is based on the paper: \citeentry{lubbers_first-class_2022}\todo{change when accepted}.
+\Cref{chp:first-class_datatypes} is based on the paper \emph{First-Class Data Types in Shallow Embedded Domain-Specific Languages} \citep{lubbers_first-class_2022}.
It shows how to inherit data types from the host language in \glspl{EDSL} using metaprogramming.
It does so by providing a proof-of-concept implementation using \gls{HASKELL}'s metaprogramming system: \glsxtrlong{TH}.
Besides showing the result, the paper also serves as a gentle introduction to using \glsxtrlong{TH} and contains a thorough literature study on research that uses \glsxtrlong{TH}.
-The research in this paper and writing the paper was performed by me, though there were weekly meetings with Pieter Koopman and Rinus Plasmeijer in which we discussed and refined the ideas.
+%The research in this paper and writing the paper was performed by me, though there were weekly meetings with Pieter Koopman and Rinus Plasmeijer in which we discussed and refined the ideas.
\subsection{\nameref{prt:top}}
-This is a monograph compiled from several papers and revised lecture notes on \gls{MTASK}, the \gls{TOP} system used to orchestrate the \gls{IOT}.
+This is a monograph compiled from the following papers and revised lecture notes on \gls{MTASK}, the \gls{TOP} system used to orchestrate the \gls{IOT}.
It provides a gentle introduction to the \gls{MTASK} system elaborates on \gls{TOP} for the \gls{IOT}.
-\todo[inline]{outline the chapters}
\begin{itemize}
- \item \citeentry{koopman_task-based_2018}
+ \item \emph{A Task-Based \glsxtrshort{DSL} for Microcomputers} \citep{koopman_task-based_2018}.
This is the initial \gls{TOP}/\gls{MTASK} paper.
- It provides an overview of the initial \gls{MTASK} language and shows first versions of a pretty printer, an \gls{ITASK} simulation and a \gls{C} code generation view.
- \paragraph{Contribution}
- Pieter Koopman wrote it, I helped with the software and research.
- \item \citeentry{lubbers_task_2018}
+ It provides an overview of the initial \gls{TOP} \gls{MTASK} language and shows first versions of a pretty printer, an \gls{ITASK} simulation and a \gls{C} code generation view.
+ \item \emph{Task Oriented Programming for the \glsxtrlong{IOT}} \citep{lubbers_task_2018}.
This paper was an extension of my Master's thesis \citep{lubbers_task_2017}.
It shows how a simple imperative variant of \gls{MTASK} was integrated with \gls{ITASK}.
While the language was a lot different than later versions, the integration mechanism is still used in \gls{MTASK} today.
- \paragraph{Contribution}
- The research in this paper and writing the paper was performed by me, though there were weekly meetings with Pieter Koopman and Rinus Plasmeijer in which we discussed and refined the ideas.
- \item \citeentry{lubbers_multitasking_2019}\footnote{%
+% \paragraph{Contribution}
+% The research in this paper and writing the paper was performed by me, though there were weekly meetings with Pieter Koopman and Rinus Plasmeijer in which we discussed and refined the ideas.
+ \item \emph{Multitasking on Microcontrollers using Task Oriented Programming} \citep{lubbers_multitasking_2019}\footnote{%
This work acknowledges the support of the ERASMUS+ project ``Focusing Education on Composability, Comprehensibility and Correctness of Working Software'', no. 2017--1--SK01--KA203--035402
- }
+ }.
This paper was a short paper on the multitasking capabilities of \gls{MTASK} in contrast to traditional multitasking methods for \gls{ARDUINO}.
- \paragraph{Contribution}
- The research in this paper and writing the paper was performed by me, though there were weekly meetings with Pieter Koopman and Rinus Plasmeijer.
- \item \citeentry{koopman_simulation_2018}\footnotemark[\value{footnote}]\todo{change when published}
+% \paragraph{Contribution}
+% The research in this paper and writing the paper was performed by me, though there were weekly meetings with Pieter Koopman and Rinus Plasmeijer.
+ \item \emph{Simulation of a Task-Based Embedded Domain Specific Language for the Internet of Things} \citep{koopman_simulation_2018}\footnotemark[\value{footnote}].
These revised lecture notes are from a course on the \gls{MTASK} simulator was provided at the 2018 \gls{CEFP}/\gls{3COWS} winter school in Ko\v{s}ice, Slovakia.
- \paragraph{Contribution}
- Pieter Koopman wrote and taught it, I helped with the software and research.
- \item \citeentry{lubbers_writing_2019}\footnotemark[\value{footnote}]\todo{change when published}
+% \paragraph{Contribution}
+% Pieter Koopman wrote and taught it, I helped with the software and research.
+ \item \emph{Writing Internet of Things Applications with Task Oriented Programming} \citep{lubbers_writing_2019}\footnotemark[\value{footnote}].
These revised lecture notes are from a course on programming in \gls{MTASK} provided at the 2019 \gls{CEFP}/\gls{3COWS} summer school in Budapest, Hungary.
- \paragraph{Contribution}
- Pieter Koopman prepared and taught half of the lecture and supervised the practical session.
- I taught the other half of the lecture, wrote the lecture notes, made the assignments and supervised the practical session.
- \item \citeentry{lubbers_interpreting_2019}
+% \paragraph{Contribution}
+% Pieter Koopman prepared and taught half of the lecture and supervised the practical session.
+% I taught the other half of the lecture, wrote the lecture notes, made the assignments and supervised the practical session.
+ \item \emph{Interpreting Task Oriented Programs on Tiny Computers} \citep{lubbers_interpreting_2019}.
This paper shows an implementation for \gls{MTASK} for microcontrollers in the form of a compilation scheme and informal semantics description.
- \paragraph{Contribution}
- The research in this paper and writing the paper was performed by me, though there were weekly meetings with Pieter Koopman and Rinus Plasmeijer.
- \item \citeentry{crooijmans_reducing_2022}\todo{change when published}
+% \paragraph{Contribution}
+% The research in this paper and writing the paper was performed by me, though there were weekly meetings with Pieter Koopman and Rinus Plasmeijer.
+ \item \emph{Reducing the Power Consumption of IoT with Task-Oriented Programming} \citep{crooijmans_reducing_2022}.
This paper shows how to create a scheduler so that devices running \gls{MTASK} tasks can go to sleep more automatically.
- \paragraph{Contribution}
- The research was carried out by \citet{crooijmans_reducing_2021} during his Master's thesis.
- I did the daily supervision and helped with the research, Pieter Koopman was the formal supervisor and wrote most of the paper.
+% \paragraph{Contribution}
+% The research was carried out by \citet{crooijmans_reducing_2021} during his Master's thesis.
+% I did the daily supervision and helped with the research, Pieter Koopman was the formal supervisor and wrote most of the paper.
\item \emph{Green Computing for the Internet of Things}\footnote{
- This work acknowledges the support of the Erasmus+ project ``SusTrainable---Promoting Sustainability as a Fundamental Driver in Software Development Training and Education'', no. 2020--1--PT01--KA203--078646}\todo{change when published}
+ This work acknowledges the support of the Erasmus+ project ``SusTrainable---Promoting Sustainability as a Fundamental Driver in Software Development Training and Education'', no. 2020--1--PT01--KA203--078646}.
- \paragraph{Contribution}
- These revised lecture notes are from a course on sustainable programming using \gls{MTASK} provided at the 2022 SusTrainable summer school in Rijeka, Croatia.
- Pieter prepared and taught a quarter of the lecture and supervised the practical session.
- I prepared and taught the other three quarters of the lecture, made the assignments and supervised the practical session\todo{writing contribution}.
+% \paragraph{Contribution}
+% These revised lecture notes are from a course on sustainable programming using \gls{MTASK} provided at the 2022 SusTrainable summer school in Rijeka, Croatia.
+% Pieter prepared and taught a quarter of the lecture and supervised the practical session.
+% I prepared and taught the other three quarters of the lecture, made the assignments and supervised the practical session
\end{itemize}
+\paragraph{Contribution:}
+The original imperative predecessors the \gls{MTASK} language and their initial interpretations were developed by Pieter Koopman and Rinus Plasmeijer.
+I continued with the language; developed the byte code interpreter, the precursor to the \gls{C} code generation interpretation; the integration with \gls{ITASK}; and the \gls{RTS}.
+The paper of which I am first author are written by me.
+
\subsection{\nameref{prt:tvt}}
\Cref{prt:tvt} is based on a journal paper that quantitatively and qualitatively compares traditional \gls{IOT} architectures with \gls{IOT} systems using \gls{TOP} and contains a single chapter.
-This chapter is based on the journal paper: \citeentry{lubbers_could_2022}\todo{change when published}\footnote{This work is an extension of the conference article: \citeentry{lubbers_tiered_2020}\footnotemark{}}.
+This chapter is based on the journal paper: \emph{Could Tierless Programming Reduce IoT Development Grief?} \citep{lubbers_could_2022}\footnote{This work is an extension of the conference article: \emph{Tiered versus Tierless IoT Stacks: Comparing Smart Campus Software Architectures} \citep{lubbers_tiered_2020}\footnotemark{}}.
\footnotetext{This paper was partly funded by the Radboud-Glasgow Collaboration Fund.}
It compares programming traditional tiered architectures to tierless architectures by showing a qualitative and a quantitative four-way comparison of a smart-campus application.
-\paragraph{Contribution}
+\paragraph{Contribution:}
Writing the paper was performed by all authors.
I created the server application, the \gls{CLEAN}/\gls{ITASK}/\gls{MTASK} implementation (\glsxtrshort{CWS}) and the \gls{CLEAN}/\gls{ITASK} implementation (\glsxtrshort{CRS})
Adrian Ramsingh created the \gls{MICROPYTHON} implementation (\glsxtrshort{PWS}), the original \gls{PYTHON} implementation (\glsxtrshort{PRS}) and the server application were created by \citet{hentschel_supersensors:_2016}.
\label{chp:top4iot}
\todo{betere chapter naam}
\begin{chapterabstract}
- This chapter introduces \gls{MTASK} and puts it into perspective compared to traditional microprocessor programming.
- It does so by showing how to program microprocessors using \gls{ARDUINO}, a popular microprocessor framework, and the equivalent \gls{MTASK} programs.
+ This chapter introduces \gls{MTASK} and puts it into perspective compared to traditional microcontroller programming.
+ It does so by showing how to program microcontrollers using \gls{ARDUINO}, a popular microcontroller framework, and the equivalent \gls{MTASK} programs.
\end{chapterabstract}
-The edge layer of \gls{IOT} system mostly consists of microprocessors that require a different method of programming.
-Usually, programming microprocessors requires an elaborate multi-step toolchain of compilation, linkage, binary image creation, and burning this image onto the flash memory of the microprocessor in order to compile and run a program.
+The edge layer of \gls{IOT} system mostly consists of microcontrollers that require a different method of programming.
+Usually, programming microcontrollers requires an elaborate multi-step toolchain of compilation, linkage, binary image creation, and burning this image onto the flash memory of the microcontroller in order to compile and run a program.
The programs are usually cyclic executives instead of tasks running in an operating system, i.e.\ there is only a single task that continuously runs on the bare metal.
-Each type of microprocessors comes with vendor-provided drivers, compilers and \glspl{RTS} but there are many platform that abstract away from this such as \gls{MBED} and \gls{ARDUINO} of which \gls{ARDUINO} is specifically designed for education and prototyping and hence used here.
-The popular \gls{ARDUINO} \gls{C}\slash\gls{CPP} dialect and accompanying libraries provide an abstraction layer for common microprocessor behaviour allowing the programmer to program multiple types of microprocessors using a single language.
+Each type of microcontrollers comes with vendor-provided drivers, compilers and \glspl{RTS} but there are many platform that abstract away from this such as \gls{MBED} and \gls{ARDUINO} of which \gls{ARDUINO} is specifically designed for education and prototyping and hence used here.
+The popular \gls{ARDUINO} \gls{C}\slash\gls{CPP} dialect and accompanying libraries provide an abstraction layer for common microcontroller behaviour allowing the programmer to program multiple types of microcontrollers using a single language.
Originally it was designed for the in-house developed open-source hardware with the same name but the setup allows porting to many architectures.
It provides an \gls{IDE} and toolchain automation to perform all steps of the toolchain with a single command.
\section{Hello world!}
Traditionally, the first program that one writes when trying a new language is the so called \emph{Hello World!} program.
This program has the single task of printing the text \emph{Hello World!} to the screen and exiting again, useful to become familiarised with the syntax and verify that the toolchain and runtime environment is working.
-On microprocessors, there usually is no screen for displaying text.
+On microcontrollers, there usually is no screen for displaying text.
Nevertheless, almost always there is a built-in monochrome $1\times1$ pixel screen, namely an \gls{LED}.
-The \emph{Hello World!} equivalent on microprocessors blinks this \gls{LED}.
+The \emph{Hello World!} equivalent on microcontrollers blinks this \gls{LED}.
\Cref{lst:arduinoBlink} shows how the logic of a blink program might look when using \gls{ARDUINO}'s \gls{C}\slash\gls{CPP} dialect.
Every \gls{ARDUINO} program contains a \arduinoinline{setup} and a \arduinoinline{loop} function.
Listing~\ref{lst:blinkthread} shows how three different blinking patterns might be achieved in \gls{ARDUINO} using the slicing method.
If we want the blink function to be a separate parametrizable function we need to explicitly provide all references to the required state.
Furthermore, the \arduinoinline{delay} function can not be used and polling \arduinoinline{millis} is required.
-The \arduinoinline{millis} function returns the number of milliseconds that have passed since the boot of the microprocessor.
+The \arduinoinline{millis} function returns the number of milliseconds that have passed since the boot of the microcontroller.
Some devices use very little energy when in \arduinoinline{delay} or sleep state.
Resulting in \arduinoinline{millis} potentially affects power consumption since the processor is basically busy looping all the time.
In the simple case of blinking three \glspl{LED} on fixed intervals, it might be possible to calculate the delays in advance using static analysis and generate the appropriate \arduinoinline{delay} code.
\section{\texorpdfstring{\Gls{MTASK}}{MTask} history}
\subsection{Generating \texorpdfstring{\gls{C}/\gls{CPP}}{C/C++} code}
-A first throw at a class-based shallowly \gls{EDSL} for microprocessors was made by \citet{plasmeijer_shallow_2016}.
+A first throw at a class-based shallowly \gls{EDSL} for microcontrollers was made by \citet{plasmeijer_shallow_2016}.
The language was called \gls{ARDSL} and offered a type safe interface to \gls{ARDUINO} \gls{CPP} dialect.
A \gls{CPP} code generation backend was available together with an \gls{ITASK} simulation backend.
There was no support for tasks or even functions.
The \gls{MTASK} language as it is now was introduced in 2018 \citep{koopman_task-based_2018}.
This paper updated the language to support functions, tasks and \glspl{SDS} but still compiled to \gls{CPP} \gls{ARDUINO} code.
Later the bytecode compiler and \gls{ITASK} integration was added to the language \citep{lubbers_interpreting_2019}.
-Moreover, it was shown that it is very intuitive to write microprocessor applications in a \gls{TOP} language \citep{lubbers_multitasking_2019}.
+Moreover, it was shown that it is very intuitive to write microcontroller applications in a \gls{TOP} language \citep{lubbers_multitasking_2019}.
One reason for this is that a lot of design patterns that are difficult using standard means are for free in \gls{TOP} (e.g.\ multithreading).
In 2019, the \gls{CEFP} summer school in Budapest, Hungary hosted a course on developing \gls{IOT} applications with \gls{MTASK} as well \citep{lubbers_writing_2019}.
This chapter serves as a complete guide to the \gls{MTASK} language, from an \gls{MTASK} programmer's perspective.
\end{chapterabstract}
-The \gls{MTASK} system is a complete \gls{TOP} programming environment for programming microprocessors.
+The \gls{MTASK} system is a complete \gls{TOP} programming environment for programming microcontrollers.
It is implemented as an \gls{EDSL} in \gls{CLEAN} using class-based---or tagless-final---embedding (see \cref{sec:tagless-final_embedding}).
Due to the nature of the embedding technique, it is possible to have multiple views on-programs written in the \gls{MTASK} language.
\item[Byte code compiler]
The compiler compiles the \gls{MTASK} program at runtime to a specialised byte code.
- Using a handful of integration functions and tasks, \gls{MTASK} tasks can be executed on microprocessors and integrated in \gls{ITASK} as if they were regular \gls{ITASK} tasks.
+ Using a handful of integration functions and tasks, \gls{MTASK} tasks can be executed on microcontrollers and integrated in \gls{ITASK} as if they were regular \gls{ITASK} tasks.
Furthermore, with special language constructs, \glspl{SDS} can be shared between \gls{MTASK} and \gls{ITASK} programs.
\end{description}
When using the compiler interpretation in conjunction with the \gls{ITASK} integration, \gls{MTASK} is a heterogeneous \gls{DSL}.
-I.e.\ some components---e.g.\ the \gls{RTS} on the microprocessor---is largely unaware of the other components in the system, and it is executed on a completely different architecture.
+I.e.\ some components---e.g.\ the \gls{RTS} on the microcontroller---is largely unaware of the other components in the system, and it is executed on a completely different architecture.
The \gls{MTASK} language is an enriched simply-typed $\lambda$-calculus with support for some basic types, arithmetic operations, and function definition; and a task language (see \cref{sec:top}).
\section{Types}
To leverage the type checker of the host language, types in the \gls{MTASK} language are expressed as types in the host language, to make the language type safe.
-However, not all types in the host language are suitable for microprocessors that may only have \qty{2}{\kibi\byte} of \gls{RAM} so class constraints are therefore added to the \gls{DSL} functions.
+However, not all types in the host language are suitable for microcontrollers that may only have \qty{2}{\kibi\byte} of \gls{RAM} so class constraints are therefore added to the \gls{DSL} functions.
The most used class constraint is the \cleaninline{type} class collection containing functions for serialization, printing, \gls{ITASK} constraints \etc.
Many of these functions can be derived using generic programming.
An even stronger restriction on types is defined for types that have a stack representation.
\Gls{MTASK}'s task language can be divided into three categories, namely
\begin{enumerate*}
\item Basic tasks, in most \gls{TOP} systems, the basic tasks are called editors, modelling the interactivity with the user.
- In \gls{MTASK}, there are no \emph{editors} in that sense but there is interaction with the outside world through microprocessor peripherals such as sensors and actuators.
+ In \gls{MTASK}, there are no \emph{editors} in that sense but there is interaction with the outside world through microcontroller peripherals such as sensors and actuators.
\item Task combinators provide a way of describing the workflow.
They combine one or more tasks into a compound task.
\item \glspl{SDS} in \gls{MTASK} can be seen as references to data that can be shared using many-to-many communication and are only accessible from within the task language to ensure atomicity.
\Gls{GPIO} access is divided into three classes: analog, digital and pin modes.
For all pins and pin modes an \gls{ADT} is available that enumerates the pins.
-The analog \gls{GPIO} pins of a microprocessor are connected to an \gls{ADC} that translates the voltage to an integer.
+The analog \gls{GPIO} pins of a microcontroller are connected to an \gls{ADC} that translates the voltage to an integer.
Analog \gls{GPIO} pins can be either read or written to.
Digital \gls{GPIO} pins only report a high or a low value.
The type class definition is a bit more complex since analog \gls{GPIO} pins can be used as digital \gls{GPIO} pins as well.
\label{chp:implementation}
\begin{chapterabstract}
This chapter shows the implementation of the \gls{MTASK} system.
- It is threefold: first it shows the implementation of the byte code compiler for \gls{MTASK}'s \gls{TOP} language, then is details of the implementation of \gls{MTASK}'s \gls{TOP} engine that executes the \gls{MTASK} tasks on the microprocessor, and finally it shows how the integration of \gls{MTASK} tasks and \glspl{SDS} is implemented both on the server and on the device.
+ It is threefold: first it shows the implementation of the byte code compiler for \gls{MTASK}'s \gls{TOP} language, then is details of the implementation of \gls{MTASK}'s \gls{TOP} engine that executes the \gls{MTASK} tasks on the microcontroller, and finally it shows how the integration of \gls{MTASK} tasks and \glspl{SDS} is implemented both on the server and on the device.
\end{chapterabstract}
IFL19 paper, bytecode instructieset~\cref{chp:bytecode_instruction_set}
We report a systematic comparative evaluation of two tierless language technologies for \gls{IOT} stacks: one for resource-rich sensor nodes (\gls{CLEAN} with \gls{ITASK}), and one for resource-constrained sensor nodes (\gls{CLEAN} with \gls{ITASK} and \gls{MTASK}).
The evaluation is based on four implementations of a typical smart campus application: two tierless and two \gls{PYTHON}-based tiered.
- %, with two targeting microprocessors and two targeting supersensors.
+ %, with two targeting microcontrollers and two targeting supersensors.
\begin{enumerate*}
\item We show that tierless languages have the potential to significantly reduce the development effort for \gls{IOT} systems, requiring 70\% less code than the tiered implementations. Careful analysis attributes this code reduction to reduced interoperation (e.g.\ two \glspl{EDSL} and one paradigm versus seven languages and two paradigms), automatically generated distributed communication, and powerful \gls{IOT} programming abstractions.
\item We show that tierless languages have the potential to significantly improve the reliability of \gls{IOT} systems, describing how \gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} maintains type safety, provides higher order failure management, and simplifies maintainability.
In a tierless language the developer writes the application as a single program. The code for different tiers is simultaneously checked by the compiler, and compiled to the required component languages. For example, Links compiles to HTML and JavaScript for the web client and to SQL on the server to interact with the database system. Tierless languages for \gls{IOT} stacks are more recent and less common, examples include
Potato \citep{troyer_building_2018} and \gls{CLEAN} with \gls{ITASK}\slash\gls{MTASK} \citep{lubbers_interpreting_2019}.
-\Gls{IOT} sensor nodes may be microprocessors with very limited compute resources, or supersensors: resource-rich single board computers like a Raspberry Pi. A tierless language may target either class of sensor node, and microcontrollers are the more demanding target due to the limited resources, e.g.\ small memory, executing on bare metal \etc.
+\Gls{IOT} sensor nodes may be microcontrollers with very limited compute resources, or supersensors: resource-rich single board computers like a Raspberry Pi. A tierless language may target either class of sensor node, and microcontrollers are the more demanding target due to the limited resources, e.g.\ small memory, executing on bare metal \etc.
Potentially a tierless language both reduces the development effort and improves correctness as correct interoperation and communication is automatically generated by the compiler. A tierless language may, however, introduce other problems. How expressive is the language? That is, can it readily express the required functionality? How maintainable is the software? Is the generated code efficient in terms of time, space, and power?
-This paper reports a systematic comparative evaluation of two tierless language technologies for \gls{IOT} stacks: one targeting resource-constrained microprocessors, and the other resource-rich supersensors. The basis of the comparison is four implementations of a typical smart campus \gls{IOT} stack \citep{hentschel_supersensors:_2016}. Two implementations are conventional tiered \gls{PYTHON}-based stacks: \gls{PRS} and \gls{PWS}. The other two implementations are tierless: \gls{CRS} and \gls{CWS}. Our work makes the following research contributions, and the key results are summarised, discussed, and quantified in \cref{sec_t4t:Conclusion}.
+This paper reports a systematic comparative evaluation of two tierless language technologies for \gls{IOT} stacks: one targeting resource-constrained microcontrollers, and the other resource-rich supersensors. The basis of the comparison is four implementations of a typical smart campus \gls{IOT} stack \citep{hentschel_supersensors:_2016}. Two implementations are conventional tiered \gls{PYTHON}-based stacks: \gls{PRS} and \gls{PWS}. The other two implementations are tierless: \gls{CRS} and \gls{CWS}. Our work makes the following research contributions, and the key results are summarised, discussed, and quantified in \cref{sec_t4t:Conclusion}.
\begin{description}
\item[C1] We show that \emph{tierless languages have the potential to significantly reduce the development effort for \gls{IOT} systems}.
\subsection{Tierless \texorpdfstring{\glsxtrshort{IOT}}{IoT} languages}
The use of tierless languages in \gls{IOT} applications is both more recent and less common than for web applications.
Tierless \gls{IOT} programming may extend tierless web programming by adding network and perception layers.
-The presentation layer of a tierless \gls{IOT} language, like tierless web languages, benefits from almost invariably executing in a standard browser. The perception layer faces greater challenges, often executing on one of a set of slow and resource-constrained microprocessors. Hence, tierless \gls{IOT} languages typically compile the perception layer to either \gls{C}\slash\gls{CPP} (the lingua franca of microcontrollers), or to some intermediate representation to be interpreted.
+The presentation layer of a tierless \gls{IOT} language, like tierless web languages, benefits from almost invariably executing in a standard browser. The perception layer faces greater challenges, often executing on one of a set of slow and resource-constrained microcontrollers. Hence, tierless \gls{IOT} languages typically compile the perception layer to either \gls{C}\slash\gls{CPP} (the lingua franca of microcontrollers), or to some intermediate representation to be interpreted.
-\subsubsection{\texorpdfstring{\Glsxtrlongpl{DSL}}{DSLs} for microprocessors}
-Many \glspl{DSL} provide high-level programming for microprocessors, for example providing strong typing and memory safety.
+\subsubsection{\texorpdfstring{\Glsxtrlongpl{DSL}}{DSLs} for microcontrollers}
+Many \glspl{DSL} provide high-level programming for microcontrollers, for example providing strong typing and memory safety.
For example Copilot \citep{hess_arduino-copilot_2020}
and Ivory \citep{elliott_guilt_2015} are imperative \glspl{DSL} embedded in a functional language that compile to \gls{C}\slash\gls{CPP}. In contrast to \gls{CLEAN}/\gls{ITASK}/\gls{MTASK} such \glspl{DSL} are not tierless \gls{IOT} languages as they have no automatic integration with the server, i.e.\ with the application and presentation layers.
\subsubsection{Erlang/Elixir \texorpdfstring{\glsxtrshort{IOT}}{IoT} systems}
A number of production \gls{IOT} systems are engineered in Erlang or Elixir, and many are mostly tierless.
That is the perception, network and application layers are sets of distributed Erlang processes, although the presentation layer typically uses some conventional web technology.
-A resource-rich sensor node may support many Erlang processes on an Erlang VM, or low level code (typically \gls{C}\slash\gls{CPP}) on a resource-constrained microprocessor can emulate an Erlang process.
+A resource-rich sensor node may support many Erlang processes on an Erlang VM, or low level code (typically \gls{C}\slash\gls{CPP}) on a resource-constrained microcontroller can emulate an Erlang process.
Only a small fraction of these systems are described in the academic literature, example exceptions are \citep{sivieri2012drop,shibanai_distributed_2018}, with many described only in grey literature or not at all.
\subsection{Characteristics of tierless \texorpdfstring{\glsxtrshort{IOT}}{IoT} languages}%
%\subsubsection{Placement}
%
-%In many \gls{IOT} systems the sensor nodes are microprocessors that are programmed by writing the program to flash memory. This means that without extra effort, physical access to the microcontroller is needed to change the program making updates challenging.
+%In many \gls{IOT} systems the sensor nodes are microcontrollers that are programmed by writing the program to flash memory. This means that without extra effort, physical access to the microcontroller is needed to change the program making updates challenging.
%Hence, most \gls{IOT} systems compile sensor node code directly for the target architecture or via an existing language such as \gls{C}\slash\gls{CPP}.
%
-%Techniques such as over-the-air programming and interpreters allow microprocessors to be dynamically provisioned, increasing their maintainability and resilience.
+%Techniques such as over-the-air programming and interpreters allow microcontrollers to be dynamically provisioned, increasing their maintainability and resilience.
%For example \citet{baccelli_reprogramming_2018} provide a single language \gls{IOT} system based on the RIOT \gls{OS} that allows runtime deployment of code snippets called containers.
%Both client and server are written in JavaScript. However, there is no integration between the client and the server other than that they are programmed from a single source.
%Mat\`e is an example of an early tierless sensor network framework where devices are provided with a virtual machine using TinyOS for dynamic provisioning \citep{levis_mate_2002}.
\subsubsection{Security}
-Security is a major issue and a considerable challenge for many \gls{IOT} systems \citep{10.1145/3437537}. There are potentially security issues at each layer in an \gls{IOT} application (\cref{fig_t4t:iot_arch}). The security issues and defence mechanisms at the application and presentation layers are relatively standard, e.g.\ defending against SQL injection attacks. The security issues at the network and perception layers are more challenging. Resource-rich sensor nodes can adopt some standard security measures like encrypting messages, and regularly applying software patches to the operating system. However microprocessors often lack the computational resources for encryption, and it is hard to patch their system software because the program is often stored in flash memory. In consequence there are infamous examples of \gls{IOT} systems being hijacked to create botnets \citep{203628,herwig_measurement_2019}.
+Security is a major issue and a considerable challenge for many \gls{IOT} systems \citep{10.1145/3437537}. There are potentially security issues at each layer in an \gls{IOT} application (\cref{fig_t4t:iot_arch}). The security issues and defence mechanisms at the application and presentation layers are relatively standard, e.g.\ defending against SQL injection attacks. The security issues at the network and perception layers are more challenging. Resource-rich sensor nodes can adopt some standard security measures like encrypting messages, and regularly applying software patches to the operating system. However microcontrollers often lack the computational resources for encryption, and it is hard to patch their system software because the program is often stored in flash memory. In consequence there are infamous examples of \gls{IOT} systems being hijacked to create botnets \citep{203628,herwig_measurement_2019}.
Securing the entire stack in a conventional tiered \gls{IOT} application is particularly challenging as the stack is implemented in a collection of programming languages with low level programming and communication abstractions. In such polyglot distributed systems it is hard to determine, and hence secure, the flow of data between components. In consequence a small mistake may have severe security implications.
\subsection{The \texorpdfstring{\glsentrytext{MTASK}}{mTask} \texorpdfstring{\glsxtrlong{EDSL}}{eDSL}}%
\label{sec_t4t:mtasks}
-In many \gls{IOT} systems the sensor nodes are resource constrained, e.g.\ inexpensive microprocessors. These are far cheaper, and consume far less power, than a single-board computer like a Raspberry Pi. Microprocessors also allow the programmer to easily control peripherals like sensors and actuators via the \gls{IO} pins of the processor.
+In many \gls{IOT} systems the sensor nodes are resource constrained, e.g.\ inexpensive microcontrollers. These are far cheaper, and consume far less power, than a single-board computer like a Raspberry Pi. Microcontrollers also allow the programmer to easily control peripherals like sensors and actuators via the \gls{IO} pins of the processor.
-Microprocessors have limited memory capacity, compute power and communication bandwidth, and hence typically no \gls{OS}.
+microcontrollers have limited memory capacity, compute power and communication bandwidth, and hence typically no \gls{OS}.
These limitations make it impossible to run an \gls{ITASK} server:
-there is no \gls{OS} to start the remote task, the code of the task is too big to fit in the available memory and the microprocessor processor is too slow to run it.
-The \gls{MTASK} \gls{EDSL} is designed to bridge this gap: \gls{MTASK} tasks can be communicated from the server to the sensor node, to execute within the limitations of a typical microprocessor, while providing programming abstractions that are consistent with \gls{ITASK}.
+there is no \gls{OS} to start the remote task, the code of the task is too big to fit in the available memory and the microcontroller processor is too slow to run it.
+The \gls{MTASK} \gls{EDSL} is designed to bridge this gap: \gls{MTASK} tasks can be communicated from the server to the sensor node, to execute within the limitations of a typical microcontroller, while providing programming abstractions that are consistent with \gls{ITASK}.
Like \gls{ITASK}, \gls{MTASK} is task oriented, e.g.\ there are primitive tasks that produce intermediate values, a restricted set of task combinators to compose the tasks, and (recursive) functions to construct tasks.
\Gls{MTASK} tasks communicate using task values or \glspl{SDS} that may be local or remote, and may be shared by some \gls{ITASK} tasks.
-Apart from the \gls{EDSL}, the \gls{MTASK} system contains a featherlight domain-specific \emph{operating system} running on the microprocessor.
+Apart from the \gls{EDSL}, the \gls{MTASK} system contains a featherlight domain-specific \emph{operating system} running on the microcontroller.
This \gls{OS} task scheduler receives the byte code generated from one or more \gls{MTASK} programs and interleaves the execution of those tasks. The \gls{OS} also manages \gls{SDS} updates and the passing of task results.
-The \gls{MTASK} \gls{OS} is stored in flash memory while the tasks are stored in \gls{RAM} to minimise wear on the flash memory. While sending byte code to a sensor node at runtime greatly increases the amount of communication, this can be mitigated as any tasks known at compile time can be preloaded on the microprocessor.
-In contrast, compiled programs, like \gls{C}\slash\gls{CPP}, are stored in flash memory and there can only ever be a few thousand programs uploaded during the lifetime of the microprocessor before exhausting the flash memory.
+The \gls{MTASK} \gls{OS} is stored in flash memory while the tasks are stored in \gls{RAM} to minimise wear on the flash memory. While sending byte code to a sensor node at runtime greatly increases the amount of communication, this can be mitigated as any tasks known at compile time can be preloaded on the microcontroller.
+In contrast, compiled programs, like \gls{C}\slash\gls{CPP}, are stored in flash memory and there can only ever be a few thousand programs uploaded during the lifetime of the microcontroller before exhausting the flash memory.
\subsection{Engineering tierless \texorpdfstring{\glsxtrshort{IOT}}{IoT} systems with \texorpdfstring{\glsentrytext{MTASK}}{mTask}}%
\label{sec_t4t:mtaskIOT}
-A tierless \gls{CLEAN} \gls{IOT} system with microprocessor sensor nodes integrates a set of \gls{ITASK} tasks that specify the application and presentation layers with a set of \gls{MTASK}s that specify the perception and network layers.
-We illustrate with \gls{CWTS}: a simple room temperature sensor with a web display. \Gls{CWTS} is equivalent to the \gls{ITASK} \gls{CRTS} (\cref{lst_t4t:itaskTempFull}), except that the sensor node is a \gls{WEMOS} microprocessor.
+A tierless \gls{CLEAN} \gls{IOT} system with microcontroller sensor nodes integrates a set of \gls{ITASK} tasks that specify the application and presentation layers with a set of \gls{MTASK}s that specify the perception and network layers.
+We illustrate with \gls{CWTS}: a simple room temperature sensor with a web display. \Gls{CWTS} is equivalent to the \gls{ITASK} \gls{CRTS} (\cref{lst_t4t:itaskTempFull}), except that the sensor node is a \gls{WEMOS} microcontroller.
\Cref{lst_t4t:mtasktemp} shows the complete program, and the key function is \cleaninline{devTask} with a top-level \cleaninline{Main} type (\cref{lst_t4t:mtasktemp:devTask}). In \gls{MTASK} functions, shares, and devices can only be defined at this top level.
The program uses the same shares \cleaninline{tempSDS} and~\cleaninline{latestTemp} as \gls{CRTS}, and for completeness we repeat those definitions.
%The \cleaninline{mapRead} ensures that every write to \cleaninline{localSds} is automatically time-stamped when it is written to \cleaninline{latestTemp}.
-The \cleaninline{mainTask} is a simple \gls{ITASK} task that starts the \cleaninline{devTask} \gls{MTASK} task on the device identified by \cleaninline{deviceInfo} (\cref{lst_t4t:mtasktemp:withdevice}). At runtime the \gls{MTASK} slice is compiled to byte code, shipped to the indicated device, and launched. Thereafter, \cleaninline{mainTask} reads temperature values from the \cleaninline{latestTemp} \gls{SDS} that is shared with the \gls{MTASK} device, and displays them on a web page (\cref{fig_t4t:cwtsweb}). The \gls{SDS}---shared with the device using \cleaninline{liftsds}---automatically communicates new temperature values from the microprocessor to the server.
+The \cleaninline{mainTask} is a simple \gls{ITASK} task that starts the \cleaninline{devTask} \gls{MTASK} task on the device identified by \cleaninline{deviceInfo} (\cref{lst_t4t:mtasktemp:withdevice}). At runtime the \gls{MTASK} slice is compiled to byte code, shipped to the indicated device, and launched. Thereafter, \cleaninline{mainTask} reads temperature values from the \cleaninline{latestTemp} \gls{SDS} that is shared with the \gls{MTASK} device, and displays them on a web page (\cref{fig_t4t:cwtsweb}). The \gls{SDS}---shared with the device using \cleaninline{liftsds}---automatically communicates new temperature values from the microcontroller to the server.
While this simple application makes limited use of the \gls{MTASK} \gls{EDSL}, it illustrates some powerful \gls{MTASK} program abstractions like basic tasks, task combinators, named recursive and parameterized tasks, and \glspl{SDS}.
Function composition (\cref{lst_t4t:mtasktemp:o}) and currying (\cref{lst_t4t:mtasktemp:setSds}) are inherited from the \gls{CLEAN} host language.
\section{\texorpdfstring{\glsxtrshort{UOG}}{UoG} smart campus case study}%
\label{sec_t4t:Case}
The basis for our comparison between tiered and tierless technologies are four \gls{IOT} systems that all conform to the \gls{UOG} smart campus specifications (\cref{sec_t4t:OperationalComparison}). There is a small (12 room) deployment of the conventional \gls{PYTHON}-based \gls{PRS} stack that uses Raspberry Pi supersensors, and its direct comparator is the tierless \gls{CRS} implementation: also deployed on Raspberry Pis.
-To represent the more common microprocessor sensor nodes we select ESP8266X powered \gls{WEMOS} D1 Mini microcontrollers. To evaluate tierless technologies on microcontrollers we compare the conventional \gls{PYTHON}\slash\gls{MICROPYTHON} \gls{PWS} stack with the tierless \gls{CWS} implementation.
+To represent the more common microcontroller sensor nodes we select ESP8266X powered \gls{WEMOS} D1 Mini microcontrollers. To evaluate tierless technologies on microcontrollers we compare the conventional \gls{PYTHON}\slash\gls{MICROPYTHON} \gls{PWS} stack with the tierless \gls{CWS} implementation.
%The four systems under comparison are different in architecture ---tiered or tierless--- and type of sensor node hardware ---regular or embedded--- but they share many properties as well.
-%The embedded sensor nodes are all implemented on the ESP8266X powered \gls{WEMOS} D1 Mini microprocessor.
+%The embedded sensor nodes are all implemented on the ESP8266X powered \gls{WEMOS} D1 Mini microcontroller.
A similar range of commodity sensors is connected to both the Raspberry Pi and \gls{WEMOS} sensor nodes using various low-level communication protocols such as \gls{GPIO}, \gls{I2C}, \gls{SPI} and \gls{ONEWIRE}. The sensors are as follows: Temperature \& Humidity: LOLIN SHT30; Light: LOLIN BH1750; Motion: LOLIN \gls{PIR}; Sound: SparkFun SEN-12642; \gls{ECO2}: SparkFun CCS811.
\Cref{fig_t4t:wemos_prss} shows both a prototype \gls{WEMOS}-based sensor node and sensors and a Raspberry Pi supersensor. Three different development teams developed the four implementations: \gls{CWS} and \gls{CRS} were engineered by a single developer.
relatively powerful Raspberry Pi 3 Model Bs. There is a simple object-oriented \gls{PYTHON} collector for configuring the sensors and reading their values. The collector daemon service marshals the sensor data and transmits using \gls{MQTT} to the central monitoring server at a preset frequency.
The collector caches sensor data locally when the server is unreachable.
-In contrast to \gls{PRS}, \gls{PWS}'s sensor nodes are microprocessors running \gls{MICROPYTHON}, a dialect of \gls{PYTHON} specifically designed to run on small, low powered embedded devices \citep{kodali2016low}.
+In contrast to \gls{PRS}, \gls{PWS}'s sensor nodes are microcontrollers running \gls{MICROPYTHON}, a dialect of \gls{PYTHON} specifically designed to run on small, low powered embedded devices \citep{kodali2016low}.
To enable a fair comparison between the software stacks we are careful to use the same object-oriented software architecture, e.g.\ using the same classes in \gls{PWS} and \gls{PRS}.
\Gls{PYTHON} and \gls{MICROPYTHON} are appropriate tiered comparison languages. Tiered \gls{IOT} systems are implemented in a whole range of programming languages, with \gls{PYTHON}, \gls{MICROPYTHON}, \gls{C} and \gls{CPP} being popular for some tiers in many implementations. \Gls{C}\slash\gls{CPP} implementations would probably result in more verbose programs and even less type safety.
\Gls{CRS}'s sensor nodes are Raspberry Pi 4s, and execute \gls{CLEAN}\slash\gls{ITASK} programs.
Communication from the sensor node to the server is implicit and happens via \glspl{SDS} over \gls{TCP} using platform independent execution graph serialisation \citep{oortgiese_distributed_2017}.
-\Gls{CWS}'s sensor nodes are \gls{WEMOS} microprocessors running \gls{MTASK} tasks. Communication and serialisation is, by design, very similar to \gls{ITASK}, i.e.\ via \glspl{SDS} over either a serial port connection, raw \gls{TCP}, or \gls{MQTT} over \gls{TCP}.
+\Gls{CWS}'s sensor nodes are \gls{WEMOS} microcontrollers running \gls{MTASK} tasks. Communication and serialisation is, by design, very similar to \gls{ITASK}, i.e.\ via \glspl{SDS} over either a serial port connection, raw \gls{TCP}, or \gls{MQTT} over \gls{TCP}.
\begin{figure}[ht]
\centering
%\subsubsection{Memory Consumption}%
\label{sec_t4t:MemPower}
-\paragraph{Memory} By design sensor nodes are devices with limited computational capacity, and memory is a key restriction. Even supersensors often have less than a \unit{\gibi\byte} of memory, and microprocessors often have just tens of \unit{\kibi\byte{}s}.
+\paragraph{Memory} By design sensor nodes are devices with limited computational capacity, and memory is a key restriction. Even supersensors often have less than a \unit{\gibi\byte} of memory, and microcontrollers often have just tens of \unit{\kibi\byte{}s}.
%In a tiered implementation the memory residency of the sensor node code can be minimised by carefully crafting it in a language that minimises memory residency. However, ...
As the tierless languages synthesize the code to be executed on the sensor nodes, we need to confirm that the generated code is sufficiently memory efficient.
\end{tabular}
\end{table}
-\Cref{tbl_t4t:mem} shows the maximum memory residency after garbage collection of the sensor node for all four smart campus implementations. The smart campus sensor node programs executing on the \gls{WEMOS} microprocessors have low maximum residencies: \qty{20270}{\byte} for \gls{PWS} and \qty{880}{\byte} for \gls{CWS}. In \gls{CWS} the \gls{MTASK} system generates very high level \gls{TOP} byte code that is interpreted by the \gls{MTASK} virtual machine and uses a small and predictable amount of heap memory.
+\Cref{tbl_t4t:mem} shows the maximum memory residency after garbage collection of the sensor node for all four smart campus implementations. The smart campus sensor node programs executing on the \gls{WEMOS} microcontrollers have low maximum residencies: \qty{20270}{\byte} for \gls{PWS} and \qty{880}{\byte} for \gls{CWS}. In \gls{CWS} the \gls{MTASK} system generates very high level \gls{TOP} byte code that is interpreted by the \gls{MTASK} virtual machine and uses a small and predictable amount of heap memory.
In \gls{PWS}, the hand-written \gls{MICROPYTHON} is compiled to byte code for execution on the virtual machine. Low residency is achieved with a fixed size heap and efficient memory management. For example both \gls{MICROPYTHON} and \gls{MTASK} use fixed size allocation units and mark\&sweep garbage collection to minimise memory usage at the cost of some execution time \citep{plamauer2017evaluation}.
-The smart campus sensor node programs executing on the Raspberry Pis have far higher maximum residencies than those executing on the microprocessors: \qty{3.5}{\mebi\byte} for \gls{PRS} and \qty{2.7}{\mebi\byte} for \gls{CRS}. In \gls{CRS} the sensor node code is a set of \gls{ITASK} executing on a full-fledged \gls{ITASK} server running in distributed child mode and this consumes far more memory.
+The smart campus sensor node programs executing on the Raspberry Pis have far higher maximum residencies than those executing on the microcontrollers: \qty{3.5}{\mebi\byte} for \gls{PRS} and \qty{2.7}{\mebi\byte} for \gls{CRS}. In \gls{CRS} the sensor node code is a set of \gls{ITASK} executing on a full-fledged \gls{ITASK} server running in distributed child mode and this consumes far more memory.
%The memory used is actually very similar to the memory usage of the server with a single client connected.
In \gls{PRS} the sensor node program is written in \gls{PYTHON}, a language far less focused on minimising memory usage than \gls{MICROPYTHON}. For example an object like a string is larger in \gls{PYTHON} than in \gls{MICROPYTHON} and consequently does not support all features such as \emph{f-strings}.
Furthermore, not all advanced \gls{PYTHON} feature regarding classes are available in \gls{MICROPYTHON}, i.e.\ only a subset of the \gls{PYTHON} specification is supported \citep{diffmicro}.
\paragraph{Power} Sensor nodes and sensors are designed to have low power demands, and this is particularly important if they are operating on batteries. The grey literature consensus is that with all sensors enabled a sensor node should typically have sub-\qty{1}{\watt} peak power draw.
The \gls{WEMOS} sensor nodes used in \gls{CWS} and \gls{PWS} have the low power consumption of a typical embedded device: with all sensors enabled, they consume around \qty{0.2}{\watt}.
-The Raspberry Pi supersensor node used in \gls{CRS} and \gls{PRS} use more power as they have a general purpose ARM processor and run mainstream Linux. With all sensors enabled, they consume \qtyrange{1}{2}{\watt}, depending on ambient load. So a microprocessor sensor node consumes an order of magnitude less power than a supersensor node.
+The Raspberry Pi supersensor node used in \gls{CRS} and \gls{PRS} use more power as they have a general purpose ARM processor and run mainstream Linux. With all sensors enabled, they consume \qtyrange{1}{2}{\watt}, depending on ambient load. So a microcontroller sensor node consumes an order of magnitude less power than a supersensor node.
\section[Is tierless \texorpdfstring{\glsxtrshort{IOT}}{IoT} programming easier than tiered?]{\hspace{-9pt}Is tierless \texorpdfstring{\glsxtrshort{IOT}}{IoT} programming easier than tiered?}%
\subsection{Comparing codebases for resource-rich\slash{}constrained sensor nodes}%
\label{sec_t4t:resourcerich}
-Before exploring the reasons for the smaller tierless codebase we compare the implementations for resource-rich and resource-constrained sensor nodes, again using \gls{SLOC} and code proportions. \Cref{table_t4t:multi} shows that the two tiered implementations are very similar in size: with \gls{PWS} for microprocessors requiring 562 \gls{SLOC} and \gls{PRS} for supersensors requiring 576 \gls{SLOC}.
+Before exploring the reasons for the smaller tierless codebase we compare the implementations for resource-rich and resource-constrained sensor nodes, again using \gls{SLOC} and code proportions. \Cref{table_t4t:multi} shows that the two tiered implementations are very similar in size: with \gls{PWS} for microcontrollers requiring 562 \gls{SLOC} and \gls{PRS} for supersensors requiring 576 \gls{SLOC}.
The two tierless implementations are also similar in size: \gls{CWS} requiring 166 and \gls{CRS} 155 \gls{SLOC}.
There are several main reasons for the similarity. One is that the server-side code, i.e.\ for the presentation and application layers, is identical for both resource rich/constrained implementations. The identical server code accounts for approximately 40\% of the \gls{PWS} and \gls{PRS} codebases, and approximately 85\% of the \gls{CWS} and \gls{CRS} codebases (\cref{fig_t4t:multipercentage}). For the perception and network layers on the sensor nodes, the \gls{PYTHON} and \gls{MICROPYTHON} implementations have the same structure, e.g.\ a class for each type of sensor, and use analogous libraries. Indeed, approaches like CircuitPython \citep{CircuitPython} allow the same code to execute on both resource-rich and resource-constrained sensor nodes.
{\hspace{-1.7499pt}Comparing tierless languages for resource-rich\slash{}constrained sensor nodes}%
\label{sec_t4t:ComparingTierless}
-This section compares two tierless \gls{IOT} languages: one for resource-rich, and the other for resource-constrained, sensor nodes. Key issues are the extent to which the very significant resource constraints of a microprocessor limit the language, and the benefits of executing on bare metal, i.e.\ without an \gls{OS}.
+This section compares two tierless \gls{IOT} languages: one for resource-rich, and the other for resource-constrained, sensor nodes. Key issues are the extent to which the very significant resource constraints of a microcontroller limit the language, and the benefits of executing on bare metal, i.e.\ without an \gls{OS}.
With the tierless \gls{CLEAN} technologies described here, \gls{ITASK} are always used to program the application and presentation layers of the \gls{IOT} stack. So any differences occur in the perception and network layer programming.
-If sensor nodes have the capacity to support \gls{ITASK}, a tierless \gls{IOT} system can be constructed in \gls{CLEAN} using only \gls{ITASK}, as in \gls{CRS}. Alternatively for sensor nodes with low computational power, like typical microprocessors, \gls{MTASK} is used for the perception and network layers, as in \gls{CWS}.
+If sensor nodes have the capacity to support \gls{ITASK}, a tierless \gls{IOT} system can be constructed in \gls{CLEAN} using only \gls{ITASK}, as in \gls{CRS}. Alternatively for sensor nodes with low computational power, like typical microcontrollers, \gls{MTASK} is used for the perception and network layers, as in \gls{CWS}.
This section compares the \gls{ITASK} and \gls{MTASK} \glspl{EDSL}, with reference to \gls{CRS} and \gls{CWS} as exemplars. \Cref{table_t4t:languagecomparison} summarises the differences between the \gls{CLEAN} embedded \gls{IOT} \glspl{EDSL} and their host language.
\begin{table}
User-defined datatypes & Yes & Yes & No \\
Task oriented & No & Yes & Yes \\
Higher-order tasks & {--} & Yes & No \\
- Execution Target & Commodity PC & Commodity PC & Microprocessor\\
- & & and Browser & Microprocessor\\
+ Execution Target & Commodity PC & Commodity PC & microcontroller\\
+ & & and Browser & microcontroller\\
Language Implementation & Compiled or & Compiled and & Interpreted\\
& interpreted & interpreted & Interpreted\\
\bottomrule
\subsection{Language restrictions for resource-constrained execution}
-Executing components on a resource-constrained sensor node imposes restrictions on programming abstractions available in a tierless \gls{IOT} language or \gls{DSL}. The small and fixed-size memory are key limitations. The limitations are shared by any high-level language that targets microprocessors such as BIT, PICBIT, PICOBIT, Microscheme and uLisp \citep{dube_bit:_2000,feeley_picbit:_2003,st-amour_picobit:_2009,suchocki_microscheme:_2015, johnson-davies_lisp_2020}.
-Even in low level languages some language features are disabled by default when targeting microprocessors, such as runtime type information (RTTI) in \gls{CPP}.
+Executing components on a resource-constrained sensor node imposes restrictions on programming abstractions available in a tierless \gls{IOT} language or \gls{DSL}. The small and fixed-size memory are key limitations. The limitations are shared by any high-level language that targets microcontrollers such as BIT, PICBIT, PICOBIT, Microscheme and uLisp \citep{dube_bit:_2000,feeley_picbit:_2003,st-amour_picobit:_2009,suchocki_microscheme:_2015, johnson-davies_lisp_2020}.
+Even in low level languages some language features are disabled by default when targeting microcontrollers, such as runtime type information (RTTI) in \gls{CPP}.
Here we investigate the restrictions imposed by resource-constrained sensor nodes on \gls{MTASK}, in comparison with \gls{ITASK}. While \gls{ITASK} and \gls{MTASK} are by design superficially similar languages, to execute on resource-constrained sensor nodes \gls{MTASK} tasks are more restricted, and have a different semantics.
\subsection{The benefits of a bare metal execution environment}
-Despite the language restrictions, components of a tierless language executing on a microprocessor can exploit the bare metal environment. Many of these benefits are shared by other bare metal languages like \gls{MICROPYTHON} or \gls{C}\slash\gls{CPP}. So as \gls{MTASK} executes on bare metal it has some advantages over \gls{ITASK}. Most notably \gls{MTASK} has better control of timing as on bare metal there are no other processes or threads that compete for CPU cycles.
+Despite the language restrictions, components of a tierless language executing on a microcontroller can exploit the bare metal environment. Many of these benefits are shared by other bare metal languages like \gls{MICROPYTHON} or \gls{C}\slash\gls{CPP}. So as \gls{MTASK} executes on bare metal it has some advantages over \gls{ITASK}. Most notably \gls{MTASK} has better control of timing as on bare metal there are no other processes or threads that compete for CPU cycles.
This makes the \gls{MTASK} \cleaninline{repeatEvery} (\cref{lst_t4t:mtasktemp}, \cref{lst_t4t:mtasktemp:sn2}) much more accurate than the \gls{ITASK} \cleaninline{waitForTimer} (\cref{lst_t4t:itaskTempFull}, \cref{lst_t4t:itaskTempFull:waitForTimer}).
While exact timing is not important in this example, it is significant for many other \gls{IOT} applications.
In contrast \gls{ITASK} cannot give real time guarantees. One reason is that an \gls{ITASK} server can ship an arbitrary number of \gls{ITASK} or \gls{MTASK} tasks to a device.
Intensional analysis of the declarative task description and current progress at run time allow the \gls{RTS} to schedule tasks and maximise idle time.
As the \gls{RTS} is the only program running on the device, it can enforce deep sleep and wake up without having to worry about influencing other processes.
-The \gls{MTASK} \gls{RTS} has direct control of the peripherals attached to the microprocessor, e.g.\ over \gls{GPIO} pins. There is no interaction with, or permission required from, the \gls{OS}.
-Moreover, microprocessors typically have better support for hardware interrupts, reducing the need to poll peripherals.
+The \gls{MTASK} \gls{RTS} has direct control of the peripherals attached to the microcontroller, e.g.\ over \gls{GPIO} pins. There is no interaction with, or permission required from, the \gls{OS}.
+Moreover, microcontrollers typically have better support for hardware interrupts, reducing the need to poll peripherals.
The downside of this direct control is that \gls{CWS} has to handle some exceptions that would otherwise be handled by the \gls{OS} in \gls{CRS} and hence the device management code is longer: 28 versus 20 \gls{SLOC} in \cref{table_t4t:multi}.
\subsection{Summary}
as it is in the tiered \gls{PYTHON} implementations (\cref{fig_t4t:multipercentage}). This suggests that the code for resource-constrained and resource-rich sensor nodes can be broadly similar in tierless technologies, as it is in many tiered technologies (\cref{sec_t4t:resourcerich}).
\end{enumerate*}
-We present \emph{the first comparison of two tierless \gls{IOT} languages: one designed for resource-constrained sensor nodes (\gls{CLEAN} with \gls{ITASK} and \gls{MTASK}), and the other for resource-rich sensor nodes (\gls{CLEAN} with \gls{ITASK}).} \gls{CLEAN}\slash\gls{ITASK} can implement all layers of the \gls{IOT} stack if the sensor nodes have the computational resources, as the Raspberry Pis do in \gls{CRS}. On resource constrained sensor nodes \gls{MTASK} are required to implement the perception and network layers, as on the \gls{WEMOS} minis in \gls{CWS}. We show that a bare metal execution environment allows \gls{MTASK} to have better control of peripherals, timing and energy consumption. The memory available on a microprocessor restricts the programming abstractions available in \gls{MTASK} to a fixed set of combinators, no user defined or recursive data types, strict evaluation, and makes it harder to add new abstractions. Even with these restrictions \gls{MTASK} provide a higher level of abstraction than most bare metal languages, and can readily express many \gls{IOT} applications including the \gls{CWS} \gls{UOG} smart campus application (\cref{sec_t4t:ComparingTierless}).
+We present \emph{the first comparison of two tierless \gls{IOT} languages: one designed for resource-constrained sensor nodes (\gls{CLEAN} with \gls{ITASK} and \gls{MTASK}), and the other for resource-rich sensor nodes (\gls{CLEAN} with \gls{ITASK}).} \gls{CLEAN}\slash\gls{ITASK} can implement all layers of the \gls{IOT} stack if the sensor nodes have the computational resources, as the Raspberry Pis do in \gls{CRS}. On resource constrained sensor nodes \gls{MTASK} are required to implement the perception and network layers, as on the \gls{WEMOS} minis in \gls{CWS}. We show that a bare metal execution environment allows \gls{MTASK} to have better control of peripherals, timing and energy consumption. The memory available on a microcontroller restricts the programming abstractions available in \gls{MTASK} to a fixed set of combinators, no user defined or recursive data types, strict evaluation, and makes it harder to add new abstractions. Even with these restrictions \gls{MTASK} provide a higher level of abstraction than most bare metal languages, and can readily express many \gls{IOT} applications including the \gls{CWS} \gls{UOG} smart campus application (\cref{sec_t4t:ComparingTierless}).
Our empirical results are consistent with the benefits of tierless languages listed in Section 2.1 of \citep{weisenburger2020survey}.
\subsection{Reflections}
repositories / phd-thesis.git / commitdiff