\begin{chapterabstract}
This chapter introduces the dissertation by providing:
\begin{itemize}
- \item a general introduction to the topics and research venues;
+ \item a general introduction to the topics and research directions;
\item a reading guide;
\item background material on the \glsxtrlong{IOT}, \glsxtrlongpl{DSL}, \glsxtrlong{TOP}, \gls{ITASK}, and \gls{MTASK};
\item and a detailed overview of the scientific contributions.
\end{chapterabstract}
There are at least 13.4 billion devices connected to the internet at the time of writing \citep{transformation_insights_current_2023}.
-Each of these sense, act, or otherwise, interact with people, other computers, and the environment surrounding us.
-Despite their immense diversity in characteristics, they are all computers.
-And as computers, they require software to operate.
+Each of these sense, act, or otherwise, interacts with people, computers, and the environment.
+Despite their immense diversity in characteristics, they are all computers and they they all require software to operate.
-An increasing amount of these connected devices are so-called \emph{edge devices} that operate in the \gls{IOT}.
+An increasing number of these connected devices are so-called \emph{edge devices} that operate in the \gls{IOT}.
Edge devices are the leaves of the \gls{IOT} systems.
-They perform the interaction with the physical world and are often physically embedded in the fabric, residing usually in hard-to-reach places such as light bulbs, smart electricity meters, or even farm animals.
+They perform the interaction with the physical world and are often physically embedded in the fabric itself, residing usually in hard-to-reach places such as light bulbs, clothing, smart electricity meters, buildings, or even farm animals.
+Typically, these edge devices are powered by microcontrollers.
Microcontrollers are equipped with a lot of connectivity for integrating peripherals such as sensors and actuators.
The connectivity makes them very suitable to interact with their surroundings.
-Typically, these edge devices are powered by microcontrollers.
These miniature computers contain integrated circuits that accommodate a microprocessor designed for use in embedded applications.
-Hence, microcontrollers are cheap; tiny; have little memory; and contain a slow, but energy-efficient processor.
+As a consequence, microcontrollers are cheap; tiny; have little memory; and contain a slow, but energy-efficient processor.
Unlike the conductor in an orchestra waving their baton to instruct the ensemble of instruments, in the universe of software there is room for little error.
Moreover, in dynamic \gls{IOT} applications, there is not always a coordinating conductor.
-Even though edge devices---the instruments---come and go, perform their own pieces, or are instructed to perform a certain piece, they operate without a central authority.
-In the traditional setting, an \gls{IOT} engineer has to program each device and their interoperation using different programming paradigms, programming languages, and abstraction levels.
+Edge devices---the instruments---come and go, perform their own pieces, or are sometimes instructed to perform a certain piece, they might even operate without a central authority.
+In a traditional setting, an \gls{IOT} engineer has to program each device and their interoperation using different programming paradigms, programming languages, and abstraction levels.
This results in semantic friction, which makes programming and maintaining \gls{IOT} systems a complex and error-prone process.
This dissertation describes the research carried out around orchestrating these complex \gls{IOT} systems using \gls{TOP}.
\Gls{TOP} is an innovative tierless programming paradigm for interactive multi-layered systems.
-By utilising advanced compiler technologies, much of the internals, communication, and interoperation between the tiers or layers of the applications is automatically generated.
+By utilising advanced compiler technologies, much of the internals, communication, and interoperation between the tiers or layers of the applications are automatically generated.
From a single declarative specification of the work required, the compiler makes a ready-for-work application consisting of interconnected components for all tiers.
For example, the \gls{TOP} system \gls{ITASK} is used to program all layers of multi-user distributed web applications from a single source specification.
-It is implemented in \gls{CLEAN}, executes in \gls{CLEAN}'s general-purpose run time, and therefore requires relatively powerful hardware.
-The high hardware requirements are no problem for regular computers but impractical for the average edge device.
+It is implemented in the general-purpose lazy functional programming language \gls{CLEAN}, and therefore requires relatively powerful hardware.
+The inflated hardware requirements are no problem for regular computers but impractical for the average edge device.
This is where \glspl{DSL} must be brought into play.
-Using \glspl{DSL}, hardware requirements can be drastically lowered, even with high levels of abstraction for the specified domain.
\Glspl{DSL} are programming languages created with a specific domain in mind.
-Consequently, jargon does not have to be expressed in the language itself, but they can be built-in features.
+Consequently, jargon does not have to be expressed in terms of the language itself, but they can be built-in features.
Furthermore, the \gls{DSL} can eschew language or system features that are irrelevant for the domain.
+Using \glspl{DSL}, hardware requirements can be drastically lowered, even while maintaining a high abstraction level for the specified domain.
To incorporate the plethora of edge devices in the orchestra of a \gls{TOP} system, the \gls{MTASK} system is used.
The \gls{MTASK} language is a novel programming language for programming \gls{IOT} edge devices using \gls{TOP}.
\end{quote}
This dissertation consists of three episodes.
\Cref{prt:dsl} is a paper-based---otherwise known as cumulative---episode containing chapters that provide insight in advanced \gls{DSL} embedding techniques for \gls{FP} languages.
-The chapters are readable independently.
+The chapters can be read independently from each other.
\Cref{prt:top} is a monograph showing \gls{MTASK}, a \gls{TOP} \gls{DSL} for the \gls{IOT}.
Hence, the chapters in this episode are best read in order.
-It introduces \gls{IOT} edge device programming, shows the complete \gls{MTASK} language, provides details on how \gls{MTASK} is integrated with \gls{ITASK}, shows how the byte code compiler is implemented, presents a guide for green computing with \gls{ITASK}, and ends with a conclusion and overview of future and related work.
+It introduces \gls{IOT} edge device programming, shows the complete \gls{MTASK} language, provides details on how \gls{MTASK} is integrated with \gls{ITASK}, shows how the byte code compiler is implemented, presents a guide for green computing with \gls{MTASK}, and ends with a conclusion and overview of future and related work.
\Cref{prt:tvt} is a single chapter based on a journal article in which traditional tiered \gls{IOT} programming is qualitatively and quantitatively compared to tierless programming using a real-world application.
The chapter is readable independently.
\end{quote}
Much later, CISCO states that the \gls{IOT} started when there were as many connected devices as there were people on the globe, i.e.\ around 2008 \citep{evans_internet_2011}.
-Today, \gls{IOT} is the term for a system of devices that sense the environment, act upon it, and communicate with each other and the world they live in.
+Today, \gls{IOT} is the term for a system of devices that sense the environment, act upon it, and communicate with each other and the world they operate in.
These connected devices are already in households all around us in the form of smart electricity meters, fridges, phones, watches, home automation, \etc.
When describing \gls{IOT} systems, a tiered---or layered---architecture is often used for compartmentalisation.
-The number of tiers heavily depends on the required complexity of the model.
+The number of tiers depends on the required complexity of the model.
For the intents and purposes of this thesis, the layered architecture as shown in \cref{fig:iot-layers} is used.
\begin{figure}
\end{figure}
To explain the tiers, an example \gls{IOT} application---home automation---is dissected.
-Closest to the end-user is the presentation layer, it provides the interface between the user and the \gls{IOT} systems.
+Closest to the end-user is the presentation layer, it provides the interface between the user and the \gls{IOT} system.
In home automation this may be a web interface, an app used on a phone, or wall-mounted tablet to interact with edge devices and view sensor data.
The application layer provides the \glspl{API}, data interfaces, data storage processing, and data processing of \gls{IOT} systems.
All layers are connected using the network layer.
In some applications this is implemented using conventional networking techniques such as Wi-Fi or Ethernet.
-However, network technology that is tailored to the needs of the specific interconnection between the two layers are increasingly popular.
+However, network technology that is tailored to the needs of the specific interconnection between two layers is increasingly popular.
Examples of this are BLE, LoRa, ZigBee, and LTE-M as a communication protocol for connecting the perception layer to the application layer using \gls{IOT} transport protocols such as \gls{MQTT}.
Protocols such as HTTP, AJAX, and WebSocket connecting the presentation layer to the application layer that are designed for the use in web applications.
Across the layers, the devices are a large heterogeneous collection of different platforms, protocols, paradigms, and programming languages often resulting in impedance problems or semantic friction between layers when programming \citep{ireland_classification_2009}.
-Even more so, the perception layer itself is often a heterogeneous collection of microcontrollers in itself, each having their own peculiarities, language of choice, and hardware interfaces.
-As edge hardware needs to be cheap, small scale, and energy efficient, the microcontrollers used to power them do not have a lot of computational power, only a soup\c{c}on of memory, and little communication bandwidth.
+Even more so, the perception layer itself is often a heterogeneous collection of microcontrollers in itself, each having their own peculiarities, programming language of choice, and hardware interfaces.
+As edge hardware needs to be cheap, small scale, and energy efficient, the microcontrollers used to power them do not have a lot of computational power, only a smidge of memory, and little communication bandwidth.
Typically, these devices are unable to run a full-fledged general-purpose \gls{OS}.
Rather they employ compiled firmware written in imperative languages that combines all tasks on the device in a single program.
While devices are getting a bit faster, smaller, and cheaper, they keep these properties to an extent.
% General
Programming languages can be divided up into two categories: \glspl{DSL} and \glspl{GPL} \citep{fowler_domain_2010}.
Where \glspl{GPL} are not made with a demarcated area in mind, \glspl{DSL} are tailor-made for a specific domain.
-Writing idiomatic domain-specific code in a \gls{DSL} is easier and requires less background knowledge.
+Writing idiomatic domain-specific code in a \gls{DSL} is easier and requires less \gls{GPL} knowledge for a domain expert.
This does come at the cost of the \gls{DSL} being sometimes less expressive to an extent that it may not even be Turing complete.
\Glspl{DSL} come in two main flavours: standalone and embedded (\cref{sec:standalone_embedded})\footnote{Standalone and embedded are also called external and internal respectively.}.
Standalone languages are languages for which the complete toolchain has been developed, just as for any other \gls{GPL}.
By defining the language as constructs in the host language, much of the machinery is inherited \citep{krishnamurthi_linguistic_2001}.
This greatly reduces the cost of creating embedded languages and shields the user from having to learn the host language and toolchain.
However, there are two sides to this coin.
-If the syntax of the host language is not very flexible, the syntax of the \gls{DSL} could become clumsy.
+If the syntax of the host language is not very flexible, the syntax of the \gls{DSL} can become clumsy.
Furthermore, \gls{DSL} errors shown to the programmer may be larded with host language errors, making it difficult for a non-expert of the host language to work with the \gls{DSL}.
\Gls{FP} languages are especially suitable for hosting embedded \glspl{DSL}.
-They offer tooling for building abstraction levels by a strong and versatile type systems, minimal but flexible syntax, and referential transparency.
+They offer tooling for building abstraction levels by a strong and versatile type system, minimal but flexible syntax, and referential transparency.
\subsection{Heterogeneity and homogeneity}%
\label{sec:hetero_homo}
-\Citet{tratt_domain_2008} applied a notion from metaprogramming \citep{sheard_accomplishments_2001} to \glspl{EDSL} to define homogeneity and heterogeneity of \glspl{EDSL} as follows:
+\Citet{tratt_domain_2008} applies a notion from metaprogramming \citep{sheard_accomplishments_2001} to \glspl{EDSL} to define homogeneity and heterogeneity of \glspl{EDSL} as follows:
\begin{quote}
\emph{A homogeneous system is one where all the components are specifically designed to work with each other, whereas in heterogeneous systems at least one of the components is largely, or completely, ignorant of the existence of the other parts of the system.
On the other hand, heterogeneous \glspl{EDSL} are languages that are not executed in the host language.
For example, \citet{elliott_compiling_2003} describe the language Pan, for which the final representation in the host language is a compiler that will, when executed, generate code for a completely different target platform.
-In fact, both \gls{ITASK} and \gls{MTASK} are \glspl{EDSL}.
+
+Both \gls{ITASK} and \gls{MTASK} are \glspl{EDSL}.
Programs written in \gls{ITASK} run in the host language, and it is a homogeneous \gls{DSL}.
Tasks written using \gls{MTASK} are dynamically compiled to byte code for an edge device, making it a heterogeneous \gls{DSL}.
The interpreter running on the edge device has no knowledge of the higher level task specification.
\section{\texorpdfstring{\Glsxtrlong{TOP}}{Task-oriented programming}}%
\label{sec:back_top}
-\Gls{TOP} is a recent declarative programming paradigm for modelling interactive systems \citep{plasmeijer_task-oriented_2012}.
+\Gls{TOP} is a declarative programming paradigm for modelling interactive systems \citep{plasmeijer_task-oriented_2012}.
Instead of dividing problems into layers, \gls{TOP} deals with separation of concerns in a novel way.
This approach to software development is called \gls{TOSD} \citep{wang_maintaining_2018}.
\item[Tasks:]
In \gls{TOP} languages, tasks are the basic building blocks.
A task is an abstract representation of a piece of work that needs to be done.
- It provides an intuitive abstraction over work in the real world.
+ It provides an abstraction over work in the real world.
The nature of tasks makes them observable during execution.
- It is possible to observe a---partial---typed result and act upon it, e.g.\ taking the partial result as good enough, or by starting new tasks.
+ It is possible to observe the current task value act upon it, e.g.\ taking a partial result as good enough, or by starting new tasks.
Examples of tasks are filling forms, sending emails, reading sensors or even doing physical tasks.
Just as with real-life tasks, multiple tasks can be combined in various ways such as in parallel or in sequence to form workflows.
Such combination operators are called task combinators.
\Glspl{SDS} can also represent external impure data such as the time, random numbers or sensor data.
In many \gls{TOP} languages, combinators are available to filter, combine, transform, and focus \glspl{SDS}.
\item[\Gls{UI}:]
- The \gls{UI} of the system is automatically generated from the structural representation of the type.
+ The \gls{UI} of the system is automatically generated from the structural representation of the types.
Though, practical \gls{TOP} systems allow tweaking afterwards to suit the specific needs of the application.
\item[\Gls{UOD}:]
- The \gls{UOD} is explicitly and separately modelled by the relations that exist in the functions of the host language.
+ The \gls{UOD} is explicitly and separately modelled by the data types and relations that exist in the functions of the host language.
\end{description}
\Cref{fig:tosd} differs from the presented \gls{IOT} architecture shown in \cref{fig:iot-layers} because they represent different concepts.
The \gls{IOT} architecture is an execution architecture whereas \gls{TOSD} is a software development model.
E.g.\ from a software development perspective, a task is a task, whether it is executed on a microcontroller, a server, or a client.
Only when a task is executed, the location of the execution becomes important, but this is taken care of by the system.
-Some concepts from the \gls{TOSD} model can be mapped upon the \gls{IOT} architecture though.
-Applying the concepts of \gls{TOSD} to \gls{IOT} systems can be done in two ways.
+Some concepts from the \gls{TOSD} model can be mapped upon the \gls{IOT} architecture in two ways.
Firstly, edge devices can be seen as simple resources, thus accessed through \glspl{SDS}.
The second view is that edge devices contain miniature \gls{TOP} systems in itself.
The individual components in the miniature systems, the tasks, the \glspl{SDS}, are, in the eventual execution, connected to the main system.
\subsection{The \texorpdfstring{\gls{ITASK}}{iTask} system}
The concept of \gls{TOP} originated from the \gls{ITASK} framework, a declarative language and \gls{TOP} engine for defining interactive multi-user distributed web applications.
-The \gls{ITASK} system is implemented as an \gls{EDSL} in the lazy pure \gls{FP} language \gls{CLEAN}\footnotemark{} \citep{plasmeijer_itasks:_2007,plasmeijer_task-oriented_2012}.
-\footnotetext{\Cref{chp:clean_for_haskell_programmers} contains a guide for \gls{CLEAN} tailored to \gls{HASKELL} programmers.}
+The \gls{ITASK} system is implemented as an \gls{EDSL} in the programming language \gls{CLEAN}\footnote{\Cref{chp:clean_for_haskell_programmers} contains a guide for \gls{CLEAN} tailored to \gls{HASKELL} programmers.} \citep{plasmeijer_itasks:_2007,plasmeijer_task-oriented_2012}.
From the structural properties of the data types and the current status of the work to be done, the entire \gls{UI} is automatically generated.
Browsers are powering \gls{ITASK}'s presentation layer.
The framework is built on top of standard web techniques such as JavaScript, HTML, and {CSS}.
The actions either change the value, sorting or clearing it, or terminate the task by returning the current value of the \gls{SDS}.
Special combinators (e.g.\ \cleaninline{@>>} at \cref{lst:todo_ui}) are used to tweak the \gls{UI} to display informative labels.
-\cleaninputlisting[float=,firstline=6,lastline=22,tabsize=3,numbers=left,caption={The code for the shared to-do list in \gls{ITASK}.},label={lst:todo}]{lst/sharedlist.icl}
+\cleaninputlisting[float=,firstline=6,lastline=22,tabsize=3,numbers=left,caption={The code for a shared to-do list in \gls{ITASK}.},label={lst:todo}]{lst/sharedlist.icl}
\begin{figure}
\centering
\subsection{The \texorpdfstring{\gls{MTASK}}{mTask} system}
The work for \gls{IOT} edge devices can often be succinctly described by \gls{TOP} programs.
Software on microcontrollers is usually composed of smaller basic tasks, are interactive, and share data with other components or the server.
-The \gls{ITASK} system seems an obvious candidate at first glance for bringing \gls{TOP} to \gls{IOT} edge devices.
-An \gls{ITASK} application contains many features that are not needed on \emph{edge devices} such as higher-order tasks, support for a distributed architecture, or a multi-user web server.
+The \gls{ITASK} system seems an obvious candidate for bringing \gls{TOP} to \gls{IOT} edge devices.
+However, an \gls{ITASK} application contains many features that are not needed on \emph{edge devices} such as higher-order tasks, support for a distributed architecture, or a multi-user web server.
Furthermore, \gls{IOT} edge devices are in general not powerful enough to run or interpret \gls{CLEAN}\slash\gls{ABC} code, they just lack the processor speed and memory.
To bridge this gap, \gls{MTASK} is developed, a domain-specific \gls{TOP} system for \gls{IOT} edge devices that is integrated in \gls{ITASK} \citep{koopman_task-based_2018}.
The \gls{ITASK} language abstracts away from details such as user interfaces, data storage, client-side platforms, and persistent workflows.
The \gls{MTASK} system is seamlessly integrated with \gls{ITASK}.
Tasks in \gls{MTASK} are integrated in such a way that they function as regular \gls{ITASK} tasks.
-Furthermore, \glspl{SDS} on the device can tether \gls{ITASK} \glspl{SDS}.
+Furthermore, \glspl{SDS} on the device can proxy \gls{ITASK} \glspl{SDS}.
Using \gls{MTASK}, the programmer can define all layers of an \gls{IOT} system as a single declarative specification.
The \gls{MTASK} language is written in \gls{CLEAN} as a multi-view \gls{EDSL} and hence there are multiple interpretations possible.
The byte code compiler is the most relevant for this thesis.
\cleaninputlisting[aboveskip=0pt,firstnumber=4,firstline=23,numbers=left,caption={The \gls{MTASK} code for the interactive blinking application.},label={lst:intro_blink_mtask}]{lst/blink.icl}
\subsection{Other \texorpdfstring{\glsxtrshort{TOP}}{TOP} languages}
-While \gls{ITASK} conceived \gls{TOP}, it is not the only \gls{TOP} system anymore.
+While \gls{ITASK} conceived \gls{TOP}, it is no longer the only \gls{TOP} system.
Some \gls{TOP} languages were created to fill a gap encountered in practise.
-Toppyt \citep{lijnse_toppyt_2022} is a general purpose \gls{TOP} language written in \gls{PYTHON} used to host frameworks for modelling \emph{Command\&Control} systems, and hTask \citep{lubbers_htask_2022}, a vessel for experimenting with asynchronous \glspl{SDS}.
+Toppyt \citep{lijnse_toppyt_2022} is a general purpose \gls{TOP} language written in \gls{PYTHON} used to host frameworks for modelling \emph{command \& control} systems, and hTask \citep{lubbers_htask_2022}, a vessel for experimenting with asynchronous \glspl{SDS}.
Furthermore, some \gls{TOP} systems arose from Master's and Bachelor's thesis projects.
For example, \textmu{}Task \citep{piers_task-oriented_2016}, a \gls{TOP} language for modelling non-interruptible embedded systems in \gls{HASKELL}, and LTasks \citep{van_gemert_task_2022}, a \gls{TOP} language written in the dynamically typed programming language {LUA}.
Finally, there are \gls{TOP} languages with strong academic foundations.
It provides an overview of the initial \gls{TOP} \gls{MTASK} language and shows first versions of some interpretations.
\item \emph{Task Oriented Programming for the Internet of Things} \citep{lubbers_task_2018}\footnotetext{This work is an extension of my Master's thesis \citep{lubbers_task_2017}.}
shows how a simple imperative variant of \gls{MTASK} was integrated with \gls{ITASK}.
- While the language was a lot different from later versions, the integration mechanism is still used in \gls{MTASK} today.
+ While the language differs a lot from the current version, the integration mechanism is still used.
% \paragraph{Contribution}
% The research in this paper and writing the paper was performed by me, though there were weekly meetings with Pieter Koopman and Rinus Plasmeijer in which we discussed and refined the ideas.
\item \emph{Multitasking on Microcontrollers using Task Oriented Programming} \citep{lubbers_multitasking_2019}\footnote{This work acknowledges the support of the \erasmusplus{} project ``Focusing Education on Composability, Comprehensibility and Correctness of Working Software'', no.\ 2017--1--SK01--KA203--035402.}
% \paragraph{Contribution}
% The research in this paper and writing the paper was performed by me, though there were weekly meetings with Pieter Koopman and Rinus Plasmeijer.
\item \emph{Simulation of a Task-\kern-1.25ptBased Embedded Domain Specific Language for the Internet of Things} \citep{koopman_simulation_2023}\footnotemark[\value{footnote}]
- are the revised lecture notes for a course on the \gls{MTASK} simulator was provided at the 2018 \gls{CEFP}\slash{}\gls{3COWS} winter school in Ko\v{s}ice, Slovakia, January 22--26, 2018.
+ are the revised lecture notes for a course on the \gls{MTASK} simulator provided at the 2018 \gls{CEFP}\slash{}\gls{3COWS} winter school in Ko\v{s}ice, Slovakia, January 22--26, 2018.
% \paragraph{Contribution}
% Pieter Koopman wrote and taught it, I helped with the software and research.
\item \emph{Writing Internet of Things Applications with Task Oriented Programming} \citep{lubbers_writing_2023}\footnotemark[\value{footnote}]
\paragraph{Contribution:}
The original \gls{MTASK} language, and their initial interpretations were developed by Pieter Koopman and Rinus Plasmeijer.
I extended the language, developed the byte code interpreter, the integration with \gls{ITASK}, and the \gls{RTS}.
-The paper of which I am first author are solely written by me, there were weekly meetings with the co-authors in which we discussed and refined the ideas.
+The papers of which I am first author are solely written by me, there were weekly meetings with the co-authors in which we discussed and refined the ideas.
\subsection{\Fullref{prt:tvt}}
-\Cref{prt:tvt} is based on a journal paper that quantitatively and qualitatively compares traditional \gls{IOT} architectures with \gls{IOT} systems using \gls{TOP} and contains a single chapter.
+\Cref{prt:tvt} is based on a journal paper that quantitatively and qualitatively compares traditional \gls{IOT} architectures with \gls{IOT} systems using \gls{TOP}.
This chapter is based on the conference paper and a journal paper extending it:
\begin{enumerate}[resume]
\item \emph{Tiered versus Tierless \glsxtrshort{IOT} Stacks: Comparing Smart Campus Software Architectures} \citep{lubbers_tiered_2020}\footnote{This work was partly funded by the 2019 Radboud-Glasgow Collaboration Fund.}\label{enum:iot20} compares traditional tiered programming to tierless architectures by comparing two implementations of a smart-campus application.
repositories / phd-thesis.git / blobdiff