This section describes some of the history of \gls{CLEAN} and provides a crash course in \gls{CLEAN} pecularities written for \gls{HASKELL} programmers.
It is based on the
-\Gls{CLEAN}---acronym for Clean \acrlong{LEAN}~\cite{barendregt_towards_1987}---, was originally designed as a \gls{GRS} core language but quickly served as an intermediate language for other functional languages~\cite{brus_clean_1987}.
-In the early days it has also been called \emph{Concurrent} \gls{CLEAN}~\cite{nocker_concurrent_1991} but these days the language has no support for this anymore.
-Fast forward thirty years, \gls{CLEAN} is now a robust language with state-of-the-art features and is actually used in industry as well as academia---albeit in select areas of the world~\cite{plasmeijer_clean_2021}.
+\Gls{CLEAN}---acronym for Clean \acrlong{LEAN}~\citep{barendregt_towards_1987}---, was originally designed as a \gls{GRS} core language but quickly served as an intermediate language for other functional languages~\citep{brus_clean_1987}.
+In the early days it has also been called \emph{Concurrent} \gls{CLEAN}~\citep{nocker_concurrent_1991} but these days the language has no support for this anymore.
+Fast forward thirty years, \gls{CLEAN} is now a robust language with state-of-the-art features and is actually used in industry as well as academia---albeit in select areas of the world~\citep{plasmeijer_clean_2021}.
Initially, when it was used mostly as an intermediate language, it had a fairly spartan syntax.
However, over the years, the syntax got friendlier and it currently it looks a lot like \gls{HASKELL}.
-In the past, a \emph{double-edged} fronted even existed that allowed \gls{CLEAN} to be extended with \gls{HASKELL98} syntax and vice versa, however this frontend is no longer maintained~\cite{groningen_exchanging_2010}.
-This chapter therefore gives a brief syntactical and functional comparison, a complete specification of the \gls{CLEAN} language can be found in the latest language report~\cite{plasmeijer_clean_2021}.
-Many of this is based on work by Achten although that was based on \gls{CLEAN} 2.1 and \gls{HASKELL98}~\cite{achten_clean_2007}.
+In the past, a \emph{double-edged} fronted even existed that allowed \gls{CLEAN} to be extended with \gls{HASKELL98} syntax and vice versa, however this frontend is no longer maintained~\citep{groningen_exchanging_2010}.
+This chapter therefore gives a brief syntactical and functional comparison, a complete specification of the \gls{CLEAN} language can be found in the latest language report~\citep{plasmeijer_clean_2021}.
+Many of this is based on work by Achten although that was based on \gls{CLEAN} 2.1 and \gls{HASKELL98}~\citep{achten_clean_2007}.
When \gls{HASKELL} is mentioned we actually mean \gls{GHC}'s \gls{HASKELL}\footnote{If an extension is enabled, a footnote is added stating that \GHCmod{SomeExtension} is required.} this is denoted and by \gls{CLEAN} we mean \gls{CLEAN} 3.1's compiler with the \gls{ITASK} extensions.
\section{Features}
Within functions the strict let (\cleaninline{#!}) can be used to force evaluate an expression, in \gls{HASKELL} \haskellinline{seq} or \haskellinline{\$!} is used for this.
\subsection{Uniqueness typing}
-Types in \gls{CLEAN} may be \emph{unique}, which means that they may not be shared~\cite{barendsen_uniqueness_1996}.
+Types in \gls{CLEAN} may be \emph{unique}, which means that they may not be shared~\citep{barendsen_uniqueness_1996}.
The uniqueness type system allows the compiler to generate efficient code because unique data structures can be destructively updated.
-Furthermore, uniqueness typing serves as a model for side effects as well~\cite{achten_high_1993,achten_ins_1995}.
-\Gls{CLEAN} uses the \emph{world-as-value} paradigm where \cleaninline{World} represents the external environment and is always unique~\cite{backus_introduction_1990}.
+Furthermore, uniqueness typing serves as a model for side effects as well~\citep{achten_high_1993,achten_ins_1995}.
+\Gls{CLEAN} uses the \emph{world-as-value} paradigm where \cleaninline{World} represents the external environment and is always unique~\citep{backus_introduction_1990}.
A program with side effects is characterised by a \cleaninline{Start :: *World -> *World} start function.
-In \gls{HASKELL}, interaction with the world is done using the \haskellinline{IO} monad~\cite{peyton_jones_imperative_1993}.
+In \gls{HASKELL}, interaction with the world is done using the \haskellinline{IO} monad~\citep{peyton_jones_imperative_1993}.
The \haskellinline{IO} monad could very well be---and actually is---implemented in \gls{CLEAN} using a state monad with the \cleaninline{World} as a state.
Besides marking types as unique, it is also possible to mark them with uniqueness attributes variables \cleaninline{u:} and define constraints on them.
For example, to make sure that an argument of a function is at least as unique as another argument.
\end{lstClean}
\subsection{Generics}
-Polytypic functions~\cite{jeuring_polytypic_1996}---also known as generic or kind-indexed fuctions---are built into \gls{CLEAN}~\cite[Chp.~7.1]{plasmeijer_clean_2021}\cite{alimarine_generic_2005} whereas in \gls{HASKELL} they are implemented as a library~\cite[Chp.~6.19.1]{ghc_team_ghc_2021}.
-The implementation of generics in \gls{CLEAN} is very similar to that of Generic H$\forall$skell~\cite{hinze_generic_2003}.
+Polytypic functions~\citep{jeuring_polytypic_1996}---also known as generic or kind-indexed fuctions---are built into \gls{CLEAN}~\cite[Chp.~7.1]{plasmeijer_clean_2021}\citep{alimarine_generic_2005} whereas in \gls{HASKELL} they are implemented as a library~\cite[Chp.~6.19.1]{ghc_team_ghc_2021}.
+The implementation of generics in \gls{CLEAN} is very similar to that of Generic H$\forall$skell~\citep{hinze_generic_2003}.
%When calling a generic function, the kind must always be specified and depending on the kind, the function may require more arguments.
For example, defining a generic equality is done as in \cref{lst:generic_eq}.
\lstinputlisting[language=Clean,firstline=4,label={lst:generic_print},caption={Generic print function in \gls{CLEAN}.}]{lst/generic_print.icl}
\subsection{\texorpdfstring{\glsentrytext{GADT}}{GADT}s}
-\Glspl{GADT} are enriched data types that allow the type instantiation of the constructor to be explicitly defined~\cite{cheney_first-class_2003,hinze_fun_2003}.
+\Glspl{GADT} are enriched data types that allow the type instantiation of the constructor to be explicitly defined~\citep{cheney_first-class_2003,hinze_fun_2003}.
While \glspl{GADT} are not natively supported in \gls{CLEAN}, they can be simulated using embedding-projection pairs or equivalence types~\cite[Sec.~2.2]{cheney_lightweight_2002}.
To illustrate this, \cref{lst:gadt_clean} shows an example \gls{GADT} that would be implemented in \gls{HASKELL} as done in \cref{lst:gadt_haskell}\requiresGHCmod{GADTs}.
\end{chapterabstract}
\section{Introduction}%
-The two flavours of DSL embedding are deep and shallow embedding~\cite{boulton_experience_1992}.
+The two flavours of DSL embedding are deep and shallow embedding~\citep{boulton_experience_1992}.
In functional programming languages, shallow embedding models language constructs as functions in the host language.
As a result, adding new language constructs---extra functions---is easy.
However, the semantics of the language is embedded in these functions, making it troublesome to add semantics since it requires updating all existing language constructs.
It can be stated that the language constructs are embedded in the functions that form a semantics.
If one wants to add a language construct, all semantics functions must be revisited and revised to avoid ending up with partial functions.
-This juxtaposition has been known for many years~\cite{reynolds_user-defined_1978} and discussed by many others~\cite{krishnamurthi_synthesizing_1998} but most famously dubbed the \emph{expression problem} by Wadler~\cite{wadler_expression_1998}:
+This juxtaposition has been known for many years~\citep{reynolds_user-defined_1978} and discussed by many others~\citep{krishnamurthi_synthesizing_1998} but most famously dubbed the \emph{expression problem} by Wadler~\citep{wadler_expression_1998}:
\begin{quote}
The \emph{expression problem} is a new name for an old problem.
\end{quote}
In shallow embedding, abstracting the functions to type classes disentangles the language constructs from the semantics, allowing extension both ways.
-This technique is dubbed tagless-final embedding~\cite{carette_finally_2009}, nonetheless it is no silver bullet.
+This technique is dubbed tagless-final embedding~\citep{carette_finally_2009}, nonetheless it is no silver bullet.
Some semantics that require an intensional analysis of the syntax tree, such as transformation and optimisations, are difficult to implement in shallow embedding due to the lack of an explicit data structure representing the abstract syntax tree.
-The semantics of the DSL have to be combined and must hold some kind of state or context, so that structural information is not lost~\cite{kiselyov_typed_2012}.
+The semantics of the DSL have to be combined and must hold some kind of state or context, so that structural information is not lost~\citep{kiselyov_typed_2012}.
\subsection{Research contribution}
This paper shows how to apply the technique observed in tagless-final embedding to deep embedding.
The presented basic technique, christened \emph{classy deep embedding}, does not require advanced type system extensions to be used.
However, it is suitable for type system extensions such as generalised algebraic data types.
While this paper is written as a literate
-Haskell~\cite{peyton_jones_haskell_2003} program using some minor extensions provided by GHC~\cite{ghc_team_ghc_2021}, the idea is applicable to other languages as well\footnotemark{}.
+Haskell~\citep{peyton_jones_haskell_2003} program using some minor extensions provided by GHC~\citep{ghc_team_ghc_2021}, the idea is applicable to other languages as well\footnotemark{}.
\footnotetext{Lubbers, M. (2021): Literate Haskell/lhs2\TeX{} source code of the paper ``Deep Embedding
with Class'': TFP 2022.\ DANS.\ \url{https://doi.org/10.5281/zenodo.5081386}.}
Adding semantics on the other hand---e.g.\ a printer---is not that simple because the semantics are part of the functions representing the language constructs.
One way to add semantics is to change all functions to execute both semantics at the same time.
In our case this means changing the type of \haskelllhstexinline{Sem_s} to be \haskelllhstexinline{(Int, String)} so that all functions operate on a tuple containing the result of the evaluator and the printed representation at the same time. %chktex 36
-Alternatively, a single semantics can be defined that represents a fold over the language constructs~\cite{gibbons_folding_2014}, delaying the selection of semantics to the moment the fold is applied.
+Alternatively, a single semantics can be defined that represents a fold over the language constructs~\citep{gibbons_folding_2014}, delaying the selection of semantics to the moment the fold is applied.
\subsection{Tagless-final embedding}
Tagless-final embedding overcomes the limitations of standard shallow embedding.
The recursive knot is left untied and as a result, \haskelllhstexinline{Sub_1} can never be reached from an \haskelllhstexinline{Expr_1}.
Luckily, we can reconnect them by adding a special constructor to the \haskelllhstexinline{Expr_1} data type for housing extensions.
-It contains an existentially quantified~\cite{mitchell_abstract_1988} type with type class constraints~\cite{laufer_combining_1994,laufer_type_1996} for all semantics type classes~\cite[Chp.~6.4.6]{ghc_team_ghc_2021} to allow it to house not just subtraction but any future extension.
+It contains an existentially quantified~\citep{mitchell_abstract_1988} type with type class constraints~\citep{laufer_combining_1994,laufer_type_1996} for all semantics type classes~\cite[Chp.~6.4.6]{ghc_team_ghc_2021} to allow it to house not just subtraction but any future extension.
\begin{lstHaskellLhstex}
data Expr_2 = Lit_2 Int
The class alias removes the need for the programmer to visit the main data type when adding additional semantics.
Unfortunately, the compiler does need to visit the main data type again.
Some may argue that adding semantics happens less frequently than adding language constructs but in reality it means that we have to concede that the language is not as easily extensible in semantics as in language constructs.
-More exotic type system extensions such as constraint kinds~\cite{bolingbroke_constraint_2011,yorgey_giving_2012} can untangle the semantics from the data types by making the data types parametrised by the particular semantics.
+More exotic type system extensions such as constraint kinds~\citep{bolingbroke_constraint_2011,yorgey_giving_2012} can untangle the semantics from the data types by making the data types parametrised by the particular semantics.
However, by adding some boilerplate, even without this extension, the language constructs can be parametrised by the semantics by putting the semantics functions in a data type.
First the data types for the language constructs are parametrised by the type variable \haskelllhstexinline{d} as follows.
Pattern matching within datatypes and from an extension to the main data type works out of the box.
Cross-extensional pattern matching on the other hand---matching on a particular extension---is something that requires a bit of extra care.
Take for example negation propagation and double negation elimination.
-Pattern matching on values with an existential type is not possible without leveraging dynamic typing~\cite{abadi_dynamic_1991,baars_typing_2002}.
-To enable dynamic typing support, the \haskelllhstexinline{Typeable} type class as provided by \haskelllhstexinline{Data.Dynamic}~\cite{ghc_team_datadynamic_2021} is added to the list of constraints in all places where we need to pattern match across extensions.
+Pattern matching on values with an existential type is not possible without leveraging dynamic typing~\citep{abadi_dynamic_1991,baars_typing_2002}.
+To enable dynamic typing support, the \haskelllhstexinline{Typeable} type class as provided by \haskelllhstexinline{Data.Dynamic}~\citep{ghc_team_datadynamic_2021} is added to the list of constraints in all places where we need to pattern match across extensions.
As a result, the \haskelllhstexinline{Typeable} type class constraints are added to the quantified type variable \haskelllhstexinline{x} of the \haskelllhstexinline{Ext_4} constructor and to \haskelllhstexinline{d}s in the smart constructors.
\begin{lstHaskellLhstex}
\end{lstHaskellLhstex}
\section{Generalised algebraic data types}%
-Generalised algebraic data types (GADTs) are enriched data types that allow the type instantiation of the constructor to be explicitly defined~\cite{cheney_first-class_2003,hinze_fun_2003}.
+Generalised algebraic data types (GADTs) are enriched data types that allow the type instantiation of the constructor to be explicitly defined~\citep{cheney_first-class_2003,hinze_fun_2003}.
Leveraging GADTs, deeply embedded DSLs can be made statically type safe even when different value types are supported.
Even when GADTs are not supported natively in the language, they can be simulated using embedding-projection pairs or equivalence types~\cite[Sec.~2.2]{cheney_lightweight_2002}.
Where some solutions to the expression problem do not easily generalise to GADTs (see \cref{sec:cde:related}), classy deep embedding does.
Now that the shape of the type classes has changed, the dictionary data types and the type classes need to be adapted as well.
The introduced type variable \haskelllhstexinline{a} is not an argument to the type class, so it should not be an argument to the dictionary data type.
-To represent this type class function, a rank-2 polymorphic function is needed~\cite[Chp.~6.4.15]{ghc_team_ghc_2021}\cite{odersky_putting_1996}.
+To represent this type class function, a rank-2 polymorphic function is needed~\cite[Chp.~6.4.15]{ghc_team_ghc_2021}\citep{odersky_putting_1996}.
Concretely, for the evaluatior this results in the following definitions:
\begin{lstHaskellLhstex}
Embedded DSL techniques in functional languages have been a topic of research for many years, thus we do not claim a complete overview of related work.
-Clearly, classy deep embedding bears most similarity to the \emph{Datatypes \`a la Carte}~\cite{swierstra_data_2008}.
+Clearly, classy deep embedding bears most similarity to the \emph{Datatypes \`a la Carte}~\citep{swierstra_data_2008}.
In Swierstra's approach, semantics are lifted to type classes similarly to classy deep embedding.
Each language construct is their own datatype parametrised by a type parameter.
This parameter contains some type level representation of language constructs that are in use.
In return, pattern matching is easier and does not require dynamic typing.
Classy deep embedding only strains the programmer with writing the extension case for the main data type and the occasional loopback constructor.
-L\"oh and Hinze proposed a language extension that allows open data types and open functions, i.e.\ functions and data types that can be extended with more cases later on~\cite{loh_open_2006}.
+L\"oh and Hinze proposed a language extension that allows open data types and open functions, i.e.\ functions and data types that can be extended with more cases later on~\citep{loh_open_2006}.
They hinted at the possibility of using type classes for open functions but had serious concerns that pattern matching would be crippled because constructors are becoming types, thus ultimately becoming impossible to type.
In contrast, this paper shows that pattern matching is easily attainable---albeit using dynamic types---and that the terms can be typed without complicated type system extensions.
-A technique similar to classy deep embedding was proposed by Najd and Peyton~Jones to tackle a slightly different problem, namely that of reusing a data type for multiple purposes in a slightly different form~\cite{najd_trees_2017}.
+A technique similar to classy deep embedding was proposed by Najd and Peyton~Jones to tackle a slightly different problem, namely that of reusing a data type for multiple purposes in a slightly different form~\citep{najd_trees_2017}.
For example to decorate the abstract syntax tree of a compiler differently for each phase of the compiler.
They propose to add an extension descriptor as a type variable to a data type and a type family that can be used to decorate constructors with extra information and add additional constructors to the data type using an extension constructor.
Classy deep embedding works similarly but uses existentially quantified type variables to describe possible extensions instead of type variables and type families.
In classy deep embedding, the extensions do not need to be encoded in the type system and less boilerplate is required.
Furthermore, pattern matching on extensions becomes a bit more complicated but in return it allows for multiple extensions to be added orthogonally and avoids the necessity for type system extensions.
-Tagless-final embedding is the shallowly embedded counterpart of classy deep embedding and was invented for the same purpose; overcoming the issues with standard shallow embedding~\cite{carette_finally_2009}.
+Tagless-final embedding is the shallowly embedded counterpart of classy deep embedding and was invented for the same purpose; overcoming the issues with standard shallow embedding~\citep{carette_finally_2009}.
Classy deep embedding was organically grown from observing the evolution of tagless-final embedding.
The main difference between tagless-final embedding and classy deep embedding---and in general between shallow and deep embedding---is that intensional analyses of the abstract syntax tree is more difficult because there is no tangible abstract syntax tree data structure.
In classy deep embedding, it is possible to define transformations even across extensions.
Hybrid approaches between deep and shallow embedding exist as well.
-For example, Svenningson et al.\ show that by expressing the deeply embedded language in a shallowly embedded core language, extensions can be made orthogonally as well~\cite{svenningsson_combining_2013}.
+For example, Svenningson et al.\ show that by expressing the deeply embedded language in a shallowly embedded core language, extensions can be made orthogonally as well~\citep{svenningsson_combining_2013}.
This paper differs from those approaches in the sense that it does not require a core language in which all extensions need to be expressible.
\section*{Acknowledgements}
In the given language it is possible to create an expression such as \cleaninline{Plus (LitI 4) (LitB True)} that adds a boolean to an integer.
Extending the \gls{ADT} is easy and convenient but extending the views accordingly is tedious since it has to be done individually for all views.
-The first downside of this type of \gls{EDSL} can be overcome by using \glspl{GADT}~\cite{cheney_first-class_2003}.
+The first downside of this type of \gls{EDSL} can be overcome by using \glspl{GADT}~\citep{cheney_first-class_2003}.
Example~\ref{lst:exdeepgadt} shows the same language, but type-safe with a \gls{GADT}.
\glspl{GADT} are not supported in the current version of \gls{CLEAN} and therefore the syntax is hypothetical (See \todo{insert link to appendix}).
-However, it has been shown that \glspl{GADT} can be simulated using bimaps or projection pairs~\cite{cheney_first-class_2003}.
+However, it has been shown that \glspl{GADT} can be simulated using bimaps or projection pairs~\citep{cheney_first-class_2003}.
Unfortunately the lack of extendability remains a problem.
If a language construct is added, no compile time guarantee can be given that all views support it.
Shallow embedding on the other hand models the language constructs as functions with the semantics embedded.
Consequently, adding a construct is easy, i.e.\ it only entails adding another function.
Contrarily, adding semantics requires adapting all language constructs.
-Lifting the functions to type classes, i.e.\ parametrising the constructs over the semantics, allows extension of the language both in constructs and in semantics orthogonally. This advanced style of embedding is called tagless-final or class-based shallow embedding~\cite{kiselyov_typed_2012}.
+Lifting the functions to type classes, i.e.\ parametrising the constructs over the semantics, allows extension of the language both in constructs and in semantics orthogonally. This advanced style of embedding is called tagless-final or class-based shallow embedding~\citep{kiselyov_typed_2012}.
While it is often possible to lift values of a user-defined data type to a value in the \gls{DSL}, it is not possible to interact with it using \gls{DSL} constructs, they are not first-class citizens.
\end{enumerate*}
The functions for this are simply not available automatically in the embedded language.
For some semantics---such as an interpreter---it is possible to directly lift the functions from the host language to the \gls{DSL}.
-In other cases---e.g.\ \emph{compiling} \glspl{DSL} such as a compiler or a printer---this is not possible~\cite{elliott_compiling_2003}. %the torget this is not possible. cannot just be lifted from the host language to the \gls{DSL} so it requires a lot of boilerplate to define and implement them.
+In other cases---e.g.\ \emph{compiling} \glspl{DSL} such as a compiler or a printer---this is not possible~\citep{elliott_compiling_2003}. %the torget this is not possible. cannot just be lifted from the host language to the \gls{DSL} so it requires a lot of boilerplate to define and implement them.
Thus, all of the operations on the data type have to be defined by hand requiring a lot of plumbing and resulting in a lot of boilerplate code.
To relieve the burden of adding all these functions, metaprogramming\nobreak---\nobreak\hskip0pt and custom quasiquoters---can be used.
As a result, views on the \gls{DSL} are data types implementing these classes.
To illustrate the technique, a simple \gls{DSL}, a language consisting of literals and addition, is outlined.
-This language, implemented according to the tagless-final style~\cite{carette_finally_2009} in \gls{HASKELL}~\cite{peyton_jones_haskell_2003} consists initially only of one type class containing two functions.
+This language, implemented according to the tagless-final style~\citep{carette_finally_2009} in \gls{HASKELL}~\citep{peyton_jones_haskell_2003} consists initially only of one type class containing two functions.
The \haskellinline{lit} function lifts values from the host language to the \gls{DSL} domain.
The class constraint \haskellinline{Show} is enforced on the type variable \haskellinline{a} to make sure that the value can be printed.
The infix function \haskellinline{+.} represents the addition of two expressions in the \gls{DSL}.
\haskellinline{newtype}s are special data types only consisting a single constructor with one field to which the type is isomorphic.
During compilation the constructor is completely removed resulting in no overhead~\cite[Sec.~4.2.3]{peyton_jones_haskell_2003}.
}
-Since the language is typed, the printer data type has to have a type variable but it is only used during typing---i.e.\ a phantom type~\cite{leijen_domain_2000}:
+Since the language is typed, the printer data type has to have a type variable but it is only used during typing---i.e.\ a phantom type~\citep{leijen_domain_2000}:
\begin{lstHaskell}
newtype Printer a = P { runPrinter :: String }
Adding functions to the language is achieved by adding a multi-parameter class to the \gls{DSL}.
The type of the class function allows for the implementation to only allow first order function by supplying the arguments in a tuple.
Furthermore, by defining the \haskellinline{Main} type, the \gls{DSL} forces all functions to be defined at the top level and with the \haskellinline{:-} operator the syntax becomes usable.
-Finally, by defining the functions as a \gls{HOAS} type safety is achieved~\cite{pfenning_higher-order_1988,chlipala_parametric_2008}.
+Finally, by defining the functions as a \gls{HOAS} type safety is achieved~\citep{pfenning_higher-order_1988,chlipala_parametric_2008}.
The complete definition looks as follows:
\begin{lstHaskell}
}
Adding these classes and their corresponding instances is tedious and results in boilerplate code.
-We therefore resort to metaprogramming, and in particular \gls{TH}~\cite{sheard_template_2002} to alleviate this burden.
+We therefore resort to metaprogramming, and in particular \gls{TH}~\citep{sheard_template_2002} to alleviate this burden.
\section{Template metaprogramming}
Metaprogramming is a special flavour of programming where programs have the ability to treat and manipulate programs or program fragments as data.
-There are several techniques to facilitate metaprogramming, moreover it has been around for many years now~\cite{lilis_survey_2019}.
-Even though it has been around for many years, it is considered complex~\cite{sheard_accomplishments_2001}.
+There are several techniques to facilitate metaprogramming, moreover it has been around for many years now~\citep{lilis_survey_2019}.
+Even though it has been around for many years, it is considered complex~\citep{sheard_accomplishments_2001}.
-\gls{TH} is GHC's de facto metaprogramming system, implemented as a compiler extension together with a library~\cite{sheard_template_2002}\cite[Sec.~6.13.1]{ghc_team_ghc_2021}.
+\gls{TH} is GHC's de facto metaprogramming system, implemented as a compiler extension together with a library~\citep{sheard_template_2002}\cite[Sec.~6.13.1]{ghc_team_ghc_2021}.
Readers already familiar with \gls{TH} can safely skip this section.
\gls{TH} adds four main concepts to the language, na\-me\-ly AST data types, splicing, quasiquotation and reification.
With this machinery, regular \gls{HASKELL} functions can be defined that are called at compile time, inserting generated code into the {AST}.
These functions are monadic functions operating in the \haskellinline{Q} monad.
-The \haskellinline{Q} monad facilitates failure, reification and fresh identifier generation for hygienic macros~\cite{kohlbecker_hygienic_1986}.
+The \haskellinline{Q} monad facilitates failure, reification and fresh identifier generation for hygienic macros~\citep{kohlbecker_hygienic_1986}.
Within the \haskellinline{Q} monad, capturable and non-capturable identifiers can be generated using the \haskellinline{mkName} and \haskellinline{newName} functions respectively.
The \emph{Peter Parker principle}\footnote{With great power comes great responsibility.} holds for the \haskellinline{Q} monad as well because it executes at compile time and is very powerful.
-For example it can subvert module boundaries, thus accessing constructors that were hidden; access the structure of abstract types; and it may cause side effects during compilation because it is possible to call \haskellinline{IO} operations~\cite{terei_safe_2012}.
+For example it can subvert module boundaries, thus accessing constructors that were hidden; access the structure of abstract types; and it may cause side effects during compilation because it is possible to call \haskellinline{IO} operations~\citep{terei_safe_2012}.
To achieve the goal of embedding data types in a \gls{DSL} we refrain from using these \emph{unsafe} features.
\subsubsection{Data types}
\end{lstHaskell}
\subsubsection{Quasiquotation}
-Another key concept of \gls{TH} is Quasiquotation, the dual of splicing~\cite{bawden_quasiquotation_1999}.
+Another key concept of \gls{TH} is Quasiquotation, the dual of splicing~\citep{bawden_quasiquotation_1999}.
While it is possible to construct entire programs using the provided data types, it is a little cumbersome.
Using \emph{Oxford brackets} (\verb#[|# \ldots\verb#|]#) or single or double apostrophes, verbatim \gls{HASKELL} code can be entered that is converted automatically to the corresponding AST nodes easing the creation of language constructs.
Depending on the context, different quasiquotes are used:
\subsection{Custom quasiquoters}
The syntax burden of \glspl{EDSL} can be reduced using quasiquotation.
In \gls{TH}, quasiquotation is a convenient way to create \gls{HASKELL} language constructs by entering them verbatim using Oxford brackets.
-However, it is also possible to create so-called custom quasiquoters~\cite{mainland_why_2007}.
+However, it is also possible to create so-called custom quasiquoters~\citep{mainland_why_2007}.
If the programmer writes down a fragment of code between tagged \emph{Oxford brackets}, the compiler executes the associated quasiquoter functions at compile time.
A quasiquoter is a value of the following data type:
They correspond to a multi-way conditional expressions and can thus be converted to \gls{DSL} constructs straightforwardly~\cite[Chp.~4.4]{peyton_jones_implementation_1987}.
In contrast to the binary literal quasiquoter example, we do not create the parser by hand.
-The parser combinator library \emph{parsec} is used instead to ease the creation of the parser~\cite{leijen_parsec_2001}.
+The parser combinator library \emph{parsec} is used instead to ease the creation of the parser~\citep{leijen_parsec_2001}.
First the location of the quasiquoted code is retrieved using the \haskellinline{location} function that operates in the \haskellinline{Q} monad.
This location is inserted in the parsec parser so that errors are localised in the source code.
Then, the \haskellinline{expr} parser is called that returns an \haskellinline{Exp} in the \haskellinline{Q} monad.
\end{lstHaskell}
\section{Related work}
-Generic or polytypic programming is a promising technique at first glance for automating the generation of function implementations~\cite{lammel_scrap_2003}.
+Generic or polytypic programming is a promising technique at first glance for automating the generation of function implementations~\citep{lammel_scrap_2003}.
However, while it is possible to define a function that works on all first-order types, adding a new function with a new name to the language is not possible.
This does not mean that generic programming is not useable for embedding pattern matches.
In generic programming, types are represented as sums of products and using this representation it is possible to define pattern matching functions.
-For example, Rhiger showed a method for expressing statically typed pattern matching using typed higher-order functions~\cite{rhiger_type-safe_2009}.
+For example, Rhiger showed a method for expressing statically typed pattern matching using typed higher-order functions~\citep{rhiger_type-safe_2009}.
If not the host language but the \gls{DSL} contains higher order functions, the same technique could be applied to port pattern matching to \glspl{DSL} though using an explicit sums of products representation.
Atkey et al.\ describe embedding pattern matching in a \gls{DSL} by giving patterns an explicit representation in the \gls{DSL} by using pairs, sums and injections~\cite[Section~3.3]{atkey_unembedding_2009}.
-McDonell et al.\ extends on this idea, resulting in a very similar but different solution to ours~\cite{mcdonell_embedded_2021}.
+McDonell et al.\ extends on this idea, resulting in a very similar but different solution to ours~\citep{mcdonell_embedded_2021}.
They used the technique that Atkey et al.\ showed and applied it to deep embedding using the concrete syntax of the host language.
The scaffolding---e.g.\ generating the pairs, sums and injections---for embedding is automated using generics but the required pattern synonyms are generated using \gls{TH}.
The key difference to our approach is that we specialise the implementation for each of the backends instead of providing a general implementation of data type handling operations.
Furthermore, our implementation does not require a generic function to trace all constructors, resulting in problems with (mutual) recursion.
-Young et al.\ added pattern matching to a deeply \gls{EDSL} using a compiler plugin~\cite{young_adding_2021}.
+Young et al.\ added pattern matching to a deeply \gls{EDSL} using a compiler plugin~\citep{young_adding_2021}.
This plugin implements an \haskellinline{externalise :: a -> E a} function that allows lifting all machinery required for pattern matching automatically from the host language to the \gls{DSL}.
Under the hood, this function translates the pattern match to constructors, deconstructors, and constructor predicates.
The main difference with this work is that it requires a compiler plugin while our metaprogramming approach works on any compiler supporting a metaprogramming system similar to \gls{TH}.
We therefore do not claim an exhaustive overview of related work on all aspects of metaprogramming.
However, we have have tried to present most research on metaprogramming in \gls{TH}.
Czarnecki et al.\ provide a more detailed comparison of different metaprogramming techniques.
-They compare staged interpreters, metaprogramming and templating by comparing MetaOCaml, \gls{TH} and C++ templates~\cite{czarnecki_dsl_2004}.
+They compare staged interpreters, metaprogramming and templating by comparing MetaOCaml, \gls{TH} and C++ templates~\citep{czarnecki_dsl_2004}.
\gls{TH} has been used to implement related work.
They all differ slightly in functionality from our domain and can be divided into several categories.
\subsubsection{Generating extra code}
Using \gls{TH} or other metaprogramming systems it is possible to add extra code to your program.
-The original \gls{TH} paper showed that it is possible to create variadic functions such as \haskellinline{printf} using \gls{TH} that would be almost impossible to define without~\cite{sheard_template_2002}.
-Hammond et al.\ used \gls{TH} to generate parallel programming skeletons~\cite{hammond_automatic_2003}.
+The original \gls{TH} paper showed that it is possible to create variadic functions such as \haskellinline{printf} using \gls{TH} that would be almost impossible to define without~\citep{sheard_template_2002}.
+Hammond et al.\ used \gls{TH} to generate parallel programming skeletons~\citep{hammond_automatic_2003}.
In practise, this means that the programmer selects a skeleton and, at compile time, the code is massaged to suit the pattern and information about the environment is inlined for optimisation.
-Polak et al.\ implemented automatic GUI generation using \gls{TH}~\cite{polak_automatic_2006}.
-Dureg\aa{}rd et al.\ wrote a parser generator using \gls{TH} and the custom quasiquoting facilities~\cite{duregard_embedded_2011}.
+Polak et al.\ implemented automatic GUI generation using \gls{TH}~\citep{polak_automatic_2006}.
+Dureg\aa{}rd et al.\ wrote a parser generator using \gls{TH} and the custom quasiquoting facilities~\citep{duregard_embedded_2011}.
From a specification of the grammar, given in verbatim using a custom quasiquoter, a parser is generated at compile time.
-Shioda et al.\ used metaprogramming in the D programming language to create a \gls{DSL} toolkit~\cite{shioda_libdsl_2014}.
+Shioda et al.\ used metaprogramming in the D programming language to create a \gls{DSL} toolkit~\citep{shioda_libdsl_2014}.
They also programmatically generate parsers and a backend for either compiling or interpreting the \gls{IR}.
-Blanchette et al.\ use \gls{TH} to simplify the development of Liquid \gls{HASKELL} proofs~\cite{blanchette_liquid_2022}.
-Folmer et al.\ used \gls{TH} to synthesize C$\lambda$aSH~\cite{baaij_digital_2015} abstract syntax trees to be processed~\cite{folmer_high-level_2022}.
-In similar fashion, Materzok used \gls{TH} to translate YieldFSM programs to C$\lambda$aSH~\cite{materzok_generating_2022}.
+Blanchette et al.\ use \gls{TH} to simplify the development of Liquid \gls{HASKELL} proofs~\citep{blanchette_liquid_2022}.
+Folmer et al.\ used \gls{TH} to synthesize C$\lambda$aSH~\citep{baaij_digital_2015} abstract syntax trees to be processed~\citep{folmer_high-level_2022}.
+In similar fashion, Materzok used \gls{TH} to translate YieldFSM programs to C$\lambda$aSH~\citep{materzok_generating_2022}.
\subsubsection{Optimisation}
Besides generating code, it is also possible to analyse existing code and perform optimisations.
Yet, this is dangerous territory because unwantedly the semantics of the optimised program may be slightly different from the original program.
-For example, Lynagh implemented various optimisations in \gls{TH} such as automatic loop unrolling~\cite{lynagh_unrolling_2003}.
+For example, Lynagh implemented various optimisations in \gls{TH} such as automatic loop unrolling~\citep{lynagh_unrolling_2003}.
The compile-time executed functions analyse the recursive function and unroll the recursion to a fixed depth to trade execution speed for program space.
-Also, O'Donnell embedded Hydra, a hardware description language, in \gls{HASKELL} utilising \gls{TH}~\cite{odonnell_embedding_2004}.
+Also, O'Donnell embedded Hydra, a hardware description language, in \gls{HASKELL} utilising \gls{TH}~\citep{odonnell_embedding_2004}.
Using intensional analysis of the AST, it detects cycles by labelling nodes automatically so that it can generate \emph{netlists}.
The authors mention that alternatively this could have be done using a monad but this hampers equational reasoning greatly, which is a key property of Hydra.
-Finally, Viera et al.\ present a way of embedding attribute grammars in \gls{HASKELL} in a staged fashion~\cite{viera_staged_2018}.
+Finally, Viera et al.\ present a way of embedding attribute grammars in \gls{HASKELL} in a staged fashion~\citep{viera_staged_2018}.
Checking several aspects of the grammar is done at compile time using \gls{TH} while other safety checks are performed at runtime.
\subsubsection{Compiler extension}
Sometimes, expressing certain functionalities in the host languages requires a lot of boilerplate, syntax wrestling, or other pains.
Metaprogramming can relieve some of this stress by performing this translation to core constructs automatically.
For example, implementing generic---or polytypic--- functions in the compiler is a major effort.
-Norell et al.\ used \gls{TH} to implement the machinery required to implement generic functions at compile time~\cite{norell_prototyping_2004}.
-Adams et al.\ also explores implementing generic programming using \gls{TH} to speed things up considerably compared to regular generic programming~\cite{adams_template_2012}.
-Clifton et al.\ use \gls{TH} with a custom quasiquoter to offer skeletons for workflows and embed foreign function interfaces in a \gls{DSL}~\cite{clifton-everest_embedding_2014}.
-Eisenberg et al.\ showed that it is possible to programmatically lift some functions from the function domain to the type domain at compile time, i.e.\ type families~\cite{eisenberg_promoting_2014}.
-Furthermore, Seefried et al.\ argued that it is difficult to do some optimisations in \glspl{EDSL} and that metaprogramming can be of use there~\cite{seefried_optimising_2004}.
+Norell et al.\ used \gls{TH} to implement the machinery required to implement generic functions at compile time~\citep{norell_prototyping_2004}.
+Adams et al.\ also explores implementing generic programming using \gls{TH} to speed things up considerably compared to regular generic programming~\citep{adams_template_2012}.
+Clifton et al.\ use \gls{TH} with a custom quasiquoter to offer skeletons for workflows and embed foreign function interfaces in a \gls{DSL}~\citep{clifton-everest_embedding_2014}.
+Eisenberg et al.\ showed that it is possible to programmatically lift some functions from the function domain to the type domain at compile time, i.e.\ type families~\citep{eisenberg_promoting_2014}.
+Furthermore, Seefried et al.\ argued that it is difficult to do some optimisations in \glspl{EDSL} and that metaprogramming can be of use there~\citep{seefried_optimising_2004}.
They use \gls{TH} to change all types to unboxed types, unroll loops to a certain depth and replace some expressions by equivalent more efficient ones.
-Torrano et al.\ showed that it is possible to use \gls{TH} to perform a strictness analysis and perform let-to-case translation~\cite{torrano_strictness_2005}.
+Torrano et al.\ showed that it is possible to use \gls{TH} to perform a strictness analysis and perform let-to-case translation~\citep{torrano_strictness_2005}.
Both applications are examples of compiler extensions that can be implemented using \gls{TH}.
-Another example of such a compiler extension is shown by Gill et al.~\cite{gill_haskell_2009}.
+Another example of such a compiler extension is shown by Gill et al.~\citep{gill_haskell_2009}.
They created a meta level \gls{DSL} to describe rewrite rules on \gls{HASKELL} syntax that are applied on the source code at compile time.
\subsubsection{Quasiquotation}
By means of quasiquotation, the host language syntax that usually seeps through the embedding can be hidden.
-The original \gls{TH} quasiquotation paper~\cite{mainland_why_2007} shows how this can be done for regular expressions, not only resulting in a nicer syntax but syntax errors are also lifted to compile time instead of run time.
-Also, Kariotis et al.\ used \gls{TH} to automatically construct monad stacks without having to resort to the monad transformers library which requires advanced type system extensions~\cite{kariotis_making_2008}.
+The original \gls{TH} quasiquotation paper~\citep{mainland_why_2007} shows how this can be done for regular expressions, not only resulting in a nicer syntax but syntax errors are also lifted to compile time instead of run time.
+Also, Kariotis et al.\ used \gls{TH} to automatically construct monad stacks without having to resort to the monad transformers library which requires advanced type system extensions~\citep{kariotis_making_2008}.
-Najd uses the compile time to be able to do normalisation for a \gls{DSL}, dubbing it \glspl{QDSL}~\cite{najd_everything_2016}.
+Najd uses the compile time to be able to do normalisation for a \gls{DSL}, dubbing it \glspl{QDSL}~\citep{najd_everything_2016}.
They utilise the quasiquation facilities of \gls{TH} to convert \gls{HASKELL} \gls{DSL} code to constructs in the \gls{DSL}, applying optimisations such as eliminating lambda abstractions and function applications along the way.
-Egi et al.\ extended \gls{HASKELL} to support non-free data type pattern matching---i.e.\ data type with no standard form, e.g.\ sets, graphs---using \gls{TH}~\cite{egi_embedding_2022}.
+Egi et al.\ extended \gls{HASKELL} to support non-free data type pattern matching---i.e.\ data type with no standard form, e.g.\ sets, graphs---using \gls{TH}~\citep{egi_embedding_2022}.
Using quasiquotation, they make a complicated embedding of non-linear pattern matching available through a simple lens.
\subsubsection{\gls{TTH}}\label{ssec_fcd:typed_template_haskell}
-\gls{TTH} is a very recent extension/alternative to normal \gls{TH}~\cite{pickering_multi-stage_2019,xie_staging_2022}.
+\gls{TTH} is a very recent extension/alternative to normal \gls{TH}~\citep{pickering_multi-stage_2019,xie_staging_2022}.
Where in \gls{TH} you can manipulate arbitrary parts of the syntax tree, add top-level splices of data types, definitions and functions, in \gls{TTH} the programmer can only splice expressions but the abstract syntax tree fragments representing the expressions are well-typed by construction instead of untyped.
-Pickering et al.\ implemented staged compilation for the \emph{generics-sop}~\cite{de_vries_true_2014} generics library to improve the efficiency of the code using \gls{TTH}~\cite{pickering_staged_2020}.
-Willis et al.\ used \gls{TTH} to remove the overhead of parsing combinators~\cite{willis_staged_2020}.
+Pickering et al.\ implemented staged compilation for the \emph{generics-sop}~\citep{de_vries_true_2014} generics library to improve the efficiency of the code using \gls{TTH}~\citep{pickering_staged_2020}.
+Willis et al.\ used \gls{TTH} to remove the overhead of parsing combinators~\citep{willis_staged_2020}.
\section{Discussion}
Functional programming languages are especially suitable for embedding \glspl{DSL} but adding user-defined data types is still an issue.
For future work, it would be interesting to see how generating boilerplate for user-defined data types translates from shallow embedding to deep embedding.
In deep embedding, the language constructs are expressed as data types in the host language.
Adding new constructs, e.g.\ constructors, deconstructors, and constructor tests, for the user-defined data type therefore requires extending the data type.
-Techniques such as data types \`a la carte~\cite{swierstra_data_2008} and open data types~\cite{loh_open_2006} show that it is possible to extend data types orthogonally but whether metaprogramming can still readily be used is something that needs to be researched.
+Techniques such as data types \`a la carte~\citep{swierstra_data_2008} and open data types~\citep{loh_open_2006} show that it is possible to extend data types orthogonally but whether metaprogramming can still readily be used is something that needs to be researched.
It may also be possible to implemented (parts) of the boilerplate generation using \gls{TTH} (see \cref{ssec_fcd:typed_template_haskell}) to achieve more confidence in the type correctness of the implementation.
Another venue of research is to try to find the limits of this technique regarding richer data type definitions.
-It would be interesting to see whether it is possible to apply the technique on data types with existentially quantified type variables or full-fledged generalised \glspl{ADT}~\cite{hinze_fun_2003}.
+It would be interesting to see whether it is possible to apply the technique on data types with existentially quantified type variables or full-fledged generalised \glspl{ADT}~\citep{hinze_fun_2003}.
It is not possible to straightforwardly lift the deconstructors to type classes because existentially quantified type variables will escape.
Rank-2 polymorphism offers tools to define the types in such a way that this is not the case anymore.
However, implementing compiling views on the \gls{DSL} is complicated because it would require inventing values of an existentially quantified type variable to satisfy the type system which is difficult.
%\epigraphhead[30]{
% A \textbf{rhapsody} in music is a one-movement work that is episodic yet integrated, free-flowing in structure, featuring a range of highly contrasted moods, colour, and tonality. An air of spontaneous inspiration and a sense of improvisation make it freer in form than a set of variations.
%}{%
-% Wikipedia~\cite{wikipedia_contributors_rhapsody_2022}
+% Wikipedia~\citep{wikipedia_contributors_rhapsody_2022}
%}%
The sheer number of connected devices around us is increasing exponentially.
First and foremost, these devices are driven by software.
\section{Internet of Things}
\todo[inline]{add more citations and rewrite to make modern}
The \gls{IOT} is growing rapidly and it is changing the way people and machines interact with the world.
-While the term \gls{IOT} briefly gained interest around 1999 to describe the communication of \gls{RFID} devices~\todo{cite}, it probably already popped up halfway the eigthies in a speech by Peter T. Lewis~\cite{peter_t_lewis_speech_1985}.
+While the term \gls{IOT} briefly gained interest around 1999 to describe the communication of \gls{RFID} devices~\todo{cite}, it probably already popped up halfway the eigthies in a speech by Peter T. Lewis~\citep{peter_t_lewis_speech_1985}.
\begin{quote}
The \acrlong{IOT}, or \acrshort{IOT}, is the integration of people, processes and technology with connectable devices and sensors to enable remote monitoring, status, manipulation and evaluation of trends of such devices.
\end{quote}
-CISCO states that the \gls{IOT} only started when there where as many connected devices as there were people on the globe, i.e.\ around 2008~\cite{evans_internet_2011}.
+CISCO states that the \gls{IOT} only started when there where as many connected devices as there were people on the globe, i.e.\ around 2008~\citep{evans_internet_2011}.
Today, the \gls{IOT} is the term for a system of devices that sense the environment, act upon it and communicate with each other and the world.
These connected devices are already in everyone's household in the form of smart electricity meters, smart fridges, smartphones, smart watches, home automation, \etc.
A \gls{SN} is a collection of sensors connected by a mesh network or central hub.
\end{description}
-While the number of devices seems to be growing exponentially fast, programming \gls{IOT} systems remains difficult as there is a lot of semantic friction~\cite{ireland_classification_2009}.
+While the number of devices seems to be growing exponentially fast, programming \gls{IOT} systems remains difficult as there is a lot of semantic friction~\citep{ireland_classification_2009}.
-The devices are a large heterogeneous collection of different platforms, protocols and languages resulting in impedance problems or semantic friction between layers~\cite{ireland_classification_2009}.
+The devices are a large heterogeneous collection of different platforms, protocols and languages resulting in impedance problems or semantic friction between layers~\citep{ireland_classification_2009}.
Additionaly, the perception layer often is a heterogeneous collections of microcontrollers as well, each having their own peculiarities, language of choice and hardware interfaces.
The hardware needs to be cheap, small-scale and energy efficient.
As a result, the \glspl{MCU} used to power these devices do not have a lot of computational power, a soup\c{c}on of memory, and little communication bandwidth.
Typically the devices do not run a full fledged \gls{OS} but a compiled firmware.
This firmware is often written in an imperative language that needs to be flashed to the program memory.
-It is possible to dynamically send the program to the program memory using \gls{OTA} programming~\cite{baccelli_scripting_2018,baccelli_reprogramming_2018}.
+It is possible to dynamically send the program to the program memory using \gls{OTA} programming~\citep{baccelli_scripting_2018,baccelli_reprogramming_2018}.
Program memory typically is flash based and only lasts a couple of thousand writes before it wears out.
While devices are getting a bit faster, smaller, and cheaper, they keep these properties to an extent.
The properties of the device greatly reduce the flexibility for dynamic systems where tasks are created on the fly, executed on demand and require parallel execution.
%weiser_computer_1991
\section{Domain-specific languages}
% General
-Programming languages can be divided up into two categories: \glspl{DSL} and \glspl{GPL}~\cite{fowler_domain_2010}.
+Programming languages can be divided up into two categories: \glspl{DSL} and \glspl{GPL}~\citep{fowler_domain_2010}.
Where \glspl{GPL} are not made with a demarcated area in mind, \glspl{DSL} are tailor-made for a specific domain.
Writing idiomatic domain-specific code in an \gls{DSL} is easy but this may come at the cost of the \gls{DSL} being less expressive to an extent that it may not even be Turing complete.
\Glspl{DSL} come in two main flavours: standalone and embedded\footnote{Also called external and internal respectively.} of which \glspl{EDSL} can again be classified into heterogeneous and homogeneous languages (see \cref{fig:hyponymy_of_dsls} for this hyponymy).
Unfortunately it also means that they need to develop a compiler or interpreter for the language to be usable making standalone \glspl{DSL} costly to create.
Examples of standalone \glspl{DSL} are regular expressions, make, yacc, XML, SQL, \etc.
-A radically different approach is embeddding the \gls{DSL} in a host language, i.e.\ \glspl{EDSL}~\cite{hudak_modular_1998}.
+A radically different approach is embeddding the \gls{DSL} in a host language, i.e.\ \glspl{EDSL}~\citep{hudak_modular_1998}.
By defining the language as constructs in the host language, much of the machinery is inherited and the cost of creating embedded languages is very low.
There are however two sides to the this coin.
If the syntax of the host language is not very flexible, the syntax of the \gls{DSL} may become clumsy.
Furthermore, errors shown to the programmer may be larded with host language errors, making it difficult for a non-expert of the host language to work with the \gls{DSL}.
-Tratt applied a notion from metaprogramming~\cite{sheard_accomplishments_2001} to \glspl{EDSL} to define homogeneity and heterogeneity of \glspl{EDSL} as follows~\cite{tratt_domain_2008}:
+Tratt applied a notion from metaprogramming~\citep{sheard_accomplishments_2001} to \glspl{EDSL} to define homogeneity and heterogeneity of \glspl{EDSL} as follows~\citep{tratt_domain_2008}:
\begin{quote}
a homogeneous system is one where all the components are specifically designed to work with each other, whereas in heterogeneous systems at least one of the components is largely, or completely, ignorant of the existence of the other parts of the system.
\end{quote}
Examples of homogeneous \glspl{EDSL} are libraries such as ones for \glspl{GUI} creation, LISP's macro system, \etc.
On the other hand, heterogeneous \glspl{EDSL} are languages that are not executed in the host language.
-For example, Elliott et al.\ describe the language Pan, for which the final representation in the host language is a compiler that will, when executed, generate code for a completely different target platform~\cite{elliott_compiling_2003}.
+For example, Elliott et al.\ describe the language Pan, for which the final representation in the host language is a compiler that will, when executed, generate code for a completely different target platform~\citep{elliott_compiling_2003}.
In fact, \gls{ITASK} and \gls{MTASK} are both heterogeneous \glspl{EDSL}.
\section{Task-oriented programming}
-\Gls{TOP} is a declarative programming paradigm designed to model interactive systems~\cite{plasmeijer_task-oriented_2012}.
+\Gls{TOP} is a declarative programming paradigm designed to model interactive systems~\citep{plasmeijer_task-oriented_2012}.
Instead of dividing problems into layers or tiers, as is done in \gls{IOT} architectures as well, it deals with separation of concerns in a novel way.
From the data types, utilising various \emph{type-parametrised} concepts, all other aspects are handled (see \cref{fig:tosd}).
-This approach to software development is also called \gls{TOSD}~\cite{wang_maintaining_2018}.
+This approach to software development is also called \gls{TOSD}~\citep{wang_maintaining_2018}.
\begin{figure}[ht]
\centering
%\epigraph{%
% \textbf{rhapsody} /\textipa{['r\ae{}ps@di]}/ \emph{noun} (pl.\ \textbf{-ies}) a piece of music that is full of feeling and is not regular in form: Liszt's Hungarian Rhapsodies.
%}{%
-% Oxford Advanced Learners Dictionary~\cite{margaret_deuter_rhapsody_2015}.
+% Oxford Advanced Learners Dictionary~\citep{margaret_deuter_rhapsody_2015}.
%}
This thesis is structured as a purely functional rhapsody.
-On Wikipedia, a rhapsody is defined as follows~\cite{wikipedia_contributors_rhapsody_2022}:
+On Wikipedia, a rhapsody is defined as follows~\citep{wikipedia_contributors_rhapsody_2022}:
\begin{quote}
A \textbf{rhapsody} in music is a one-movement work that is episodic yet integrated, free-flowing in structure, featuring a range of highly contrasted moods, colour, and tonality. An air of spontaneous inspiration and a sense of improvisation make it freer in form than a set of variations.
\end{quote}
\subsubsection{\Cref{chp:classy_deep_embedding}}
This chapter is based on the paper: \emph{Deep Embedding with Class}~\todo{cite when published}.
-During a Master's thesis supervision~\cite{amazonas_cabral_de_andrade_developing_2018}, focussing on an early version of \gls{MTASK}, a seed was planted for a novel deep embedding technique for \glspl{DSL} where the resulting language is extendible both in constructs and in interpretation using type classes and existential data types.
+During a Master's thesis supervision~\citep{amazonas_cabral_de_andrade_developing_2018}, focussing on an early version of \gls{MTASK}, a seed was planted for a novel deep embedding technique for \glspl{DSL} where the resulting language is extendible both in constructs and in interpretation using type classes and existential data types.
Slowly the ideas organically grew to form the technique shown in the paper.
The research from this paper and writing the paper was solely performed by me.
Pieter Koopman wrote it but I helped with the software and research.
\item \bibentry{lubbers_task_2018}.
- This paper was an extension of my Master's thesis~\cite{lubbers_task_2017}.
+ This paper was an extension of my Master's thesis~\citep{lubbers_task_2017}.
It shows how a simple imperative variant of \gls{MTASK} was integrated with \gls{ITASK}.
While the language was a lot different than later versions, the integration mechanism is still used in \gls{MTASK} today.
This paper shows how to create a scheduler so that devices running \gls{MTASK} tasks can go to sleep more automatically.
Furthermore, it shows how to integrate hardware interrupts into \gls{MTASK}.
- The research was carried out by Sjoerd Crooijmans during his Master's thesis~\cite{crooijmans_reducing_2021}.
+ The research was carried out by Sjoerd Crooijmans during his Master's thesis~\citep{crooijmans_reducing_2021}.
I did the daily supervision and helped with the research, Pieter Koopman was the formal supervisor and wrote most of the paper.
\item \emph{Asynchronous Shared Data Sources}~\todo{cite when accepted}
The example application shown in \cref{prt:tvt} heavily uses asynchronous \glspl{SDS}.
This paper shows how to lift \glspl{SDS} to operate asynchronously independent of \gls{ITASK} or another \gls{TOP} implementation.
- The initial research was carried out by Haye B\"ohm during his Master's thesis~\cite{bohm_asynchronous_2019}.
+ The initial research was carried out by Haye B\"ohm during his Master's thesis~\citep{bohm_asynchronous_2019}.
I did the daily supervision and helped with the research, Pieter Koopman and Rinus Plasmeijer were the formal supervisors.
I extended and generalised the research and wrote the paper.
\item \emph{Green Computing for the Internet of Things}~\todo[inline]{cite when done}
The research in these papers and writing them was performed by all authors.
I created the server application, the \gls{CLEAN}/\gls{ITASK}/\gls{MTASK} implementation (\acrshort{CWS}) and the \gls{CLEAN}/\gls{ITASK} implementation (\acrshort{CRS})
-Adrian Ramsingh created the micropython implementation (\acrshort{PWS}), the original python implementation (\acrshort{PRS}) and the server application were created by Jeremy Singer, Dejice Jacob and Kristian Hentschel~\cite{hentschel_supersensors:_2016}.
+Adrian Ramsingh created the micropython implementation (\acrshort{PWS}), the original python implementation (\acrshort{PRS}) and the server application were created by Jeremy Singer, Dejice Jacob and Kristian Hentschel~\citep{hentschel_supersensors:_2016}.
\begin{itemize}
\item \bibentry{lubbers_tiered_2020}.
\item \secondauthor{}
A Task-Based DSL for Microcomputers
- P. Koopman\footnote{\orcid{0000-0002-3688-0957}}, M. Lubbers\footnote{\orcid{0000-0002-4015-4878}}, and R. Plasmeijer (RWDSL 2018)~\cite{koopman_task-based_2018}.
+ P. Koopman\footnote{\orcid{0000-0002-3688-0957}}, M. Lubbers\footnote{\orcid{0000-0002-4015-4878}}, and R. Plasmeijer (RWDSL 2018)~\citep{koopman_task-based_2018}.
\item Task Oriented Programming and the Internet of Things
- M. Lubbers, P. Koopman, and R. Plasmeijer (IFL 2018)~\cite{lubbers_task_2017} (extension of Master's thesis~\cite{lubbers_task_2018}).
+ M. Lubbers, P. Koopman, and R. Plasmeijer (IFL 2018)~\citep{lubbers_task_2017} (extension of Master's thesis~\citep{lubbers_task_2018}).
\item Multitasking on Microcontrollers using Task Oriented Programming
- M. Lubbers, P. Koopman, and R. Plasmeijer (MIPRO/4COWS 2019)~\cite{lubbers_multitasking_2019}.
+ M. Lubbers, P. Koopman, and R. Plasmeijer (MIPRO/4COWS 2019)~\citep{lubbers_multitasking_2019}.
\item \secondauthor{}
Simulation of a Task-Based Embedded Domain Specific Language for the Internet of Things
M. Lubbers, P. Koopman, and R. Plasmeijer (CEFP/3COWS 2019).
\item Interpreting Task Oriented Programs on Tiny Computers
- M. Lubbers, P. Koopman, and R. Plasmeijer (IFL 2019)~\cite{lubbers_interpreting_2019}.
+ M. Lubbers, P. Koopman, and R. Plasmeijer (IFL 2019)~\citep{lubbers_interpreting_2019}.
\item Tiered versus Tierless IoT Stacks: Comparing Smart Campus Software Architectures
- M. Lubbers, P. Koopman, A. Ramsingh\footnote{\orcid{0000-0003-3501-902X}}, J. Singer\footnote{\orcid{0000-0001-9462-6802}}, and P. Trinder\footnote{\orcid{0000-0003-0190-7010}} (IoT 2020)~\cite{lubbers_tiered_2020}.
+ M. Lubbers, P. Koopman, A. Ramsingh\footnote{\orcid{0000-0003-3501-902X}}, J. Singer\footnote{\orcid{0000-0001-9462-6802}}, and P. Trinder\footnote{\orcid{0000-0003-0190-7010}} (IoT 2020)~\citep{lubbers_tiered_2020}.
% in press
\item Deep Embedding with Class
%Under review
\item \underreview{} Could Tierless Languages Reduce IoT Development Grief?
- M. Lubbers, P. Koopman, A. Ramsingh, J. Singer, and P. Trinder (ACM TIOT) (extension of~\cite{lubbers_tiered_2020}).
+ M. Lubbers, P. Koopman, A. Ramsingh, J. Singer, and P. Trinder (ACM TIOT) (extension of~\citep{lubbers_tiered_2020}).
\item \secondauthor\underreview{}
Strongly-Typed Multi-View Stack-Based Computations
Unfortunately, this does not work because the \arduinoinline{delay} function blocks all further execution.
The resulting program will blink the \glspl{LED} after each other instead of at the same time.
-To overcome this, it is necessary to slice up the blinking behaviour in very small fragments so it can be manually interleaved~\cite{feijs_multi-tasking_2013}.
+To overcome this, it is necessary to slice up the blinking behaviour in very small fragments so it can be manually interleaved~\citep{feijs_multi-tasking_2013}.
Listing~\ref{lst:blinkthread} shows how three different blinking patterns might be achieved in \gls{ARDUINO} using the slicing method.
If we want the blink function to be a separate parametrizable function we need to explicitly provide all references to the required state.
Furthermore, the \arduinoinline{delay} function can not be used and polling \arduinoinline{millis} is required.
\subsection{Functions}
Adding functions to the language is achieved by one multi-parameter class to the \gls{DSL}.
-By using \gls{HOAS}, both the function definition and the calls to the function can be controlled by the \gls{DSL}~\cite{pfenning_higher-order_1988,chlipala_parametric_2008}.
+By using \gls{HOAS}, both the function definition and the calls to the function can be controlled by the \gls{DSL}~\citep{pfenning_higher-order_1988,chlipala_parametric_2008}.
As \gls{MTASK} only supports first-order functions and does not allow partial function application.
Using a type class of this form, this restriction can be enforced on the type level.
Instead of providing one instance for all functions, a single instance per function arity is defined.
\chapter{\texorpdfstring{\gls{MTASK}}{mTask} history}
\section{Generating \texorpdfstring{\gls{C}/\gls{CPP}}{C/C++} code}
-A first throw at a class-based shallowly \gls{EDSL} for microprocessors was made by Pieter Koopman and Rinus Plasmijer in 2016~\cite{plasmeijer_shallow_2016}.
+A first throw at a class-based shallowly \gls{EDSL} for microprocessors was made by Pieter Koopman and Rinus Plasmijer in 2016~\citep{plasmeijer_shallow_2016}.
The language was called \gls{ARDSL} and offered a type safe interface to \gls{ARDUINO} \gls{CPP} dialect.
A \gls{CPP} code generation backend was available together with an \gls{ITASK} simulation backend.
There was no support for tasks or even functions.
-Some time later in the 2015 CEFP summer school, an extended version was created that allowed the creation of imperative tasks, \glspl{SDS} and the usage of functions~\cite{koopman_type-safe_2019}.
+Some time later in the 2015 CEFP summer school, an extended version was created that allowed the creation of imperative tasks, \glspl{SDS} and the usage of functions~\citep{koopman_type-safe_2019}.
The name then changed from \gls{ARDSL} to \gls{MTASK}.
\section{Integration with \texorpdfstring{\gls{ITASK}}{iTask}}
-Mart Lubbers extended this in his Master's Thesis by adding integration with \gls{ITASK} and a bytecode compiler to the language~\cite{lubbers_task_2017}.
+Mart Lubbers extended this in his Master's Thesis by adding integration with \gls{ITASK} and a bytecode compiler to the language~\citep{lubbers_task_2017}.
\Gls{SDS} in \gls{MTASK} could be accessed on the \gls{ITASK} server.
In this way, entire \gls{IOT} systems could be programmed from a single source.
However, this version used a simplified version of \gls{MTASK} without functions.
-This was later improved upon by creating a simplified interface where \glspl{SDS} from \gls{ITASK} could be used in \gls{MTASK} and the other way around~\cite{lubbers_task_2018}.
-It was shown by Matheus Amazonas Cabral de Andrade that it was possible to build real-life \gls{IOT} systems with this integration~\cite{amazonas_cabral_de_andrade_developing_2018}.
-Moreover, a course on the \gls{MTASK} simulator was provided at the 2018 CEFP/3COWS winter school in Ko\v{s}ice, Slovakia~\cite{koopman_simulation_2018}.
+This was later improved upon by creating a simplified interface where \glspl{SDS} from \gls{ITASK} could be used in \gls{MTASK} and the other way around~\citep{lubbers_task_2018}.
+It was shown by Matheus Amazonas Cabral de Andrade that it was possible to build real-life \gls{IOT} systems with this integration~\citep{amazonas_cabral_de_andrade_developing_2018}.
+Moreover, a course on the \gls{MTASK} simulator was provided at the 2018 CEFP/3COWS winter school in Ko\v{s}ice, Slovakia~\citep{koopman_simulation_2018}.
\section{Transition to \texorpdfstring{\gls{TOP}}{TOP}}
-The \gls{MTASK} language as it is now was introduced in 2018~\cite{koopman_task-based_2018}.
+The \gls{MTASK} language as it is now was introduced in 2018~\citep{koopman_task-based_2018}.
This paper updated the language to support functions, tasks and \glspl{SDS} but still compiled to \gls{CPP} \gls{ARDUINO} code.
-Later the bytecode compiler and \gls{ITASK} integration was added to the language~\cite{lubbers_interpreting_2019}.
-Moreover, it was shown that it is very intuitive to write \gls{MCU} applications in a \gls{TOP} language~\cite{lubbers_multitasking_2019}.
+Later the bytecode compiler and \gls{ITASK} integration was added to the language~\citep{lubbers_interpreting_2019}.
+Moreover, it was shown that it is very intuitive to write \gls{MCU} applications in a \gls{TOP} language~\citep{lubbers_multitasking_2019}.
One reason for this is that a lot of design patterns that are difficult using standard means are for free in \gls{TOP} (e.g.\ multithreading).
-In 2019, the CEFP summer school in Budapest, Hungary hosted a course on developing \gls{IOT} applications with \gls{MTASK} as well~\cite{lubbers_writing_2019}.
+In 2019, the CEFP summer school in Budapest, Hungary hosted a course on developing \gls{IOT} applications with \gls{MTASK} as well~\citep{lubbers_writing_2019}.
\section{\texorpdfstring{\gls{TOP}}{TOP}}
In 2022, the SusTrainable summer school in Rijeka, Croatia hosted a course on developing greener \gls{IOT} applications using \gls{MTASK} as well (the lecture notes are to be written).
Several students worked on extending \gls{MTASK} with many useful features:
-Erin van der Veen did preliminary work on a green computer analysis, built a simulator and explored the possibilities for adding bounded datatypes~\cite{veen_van_der_mutable_2020}; Michel de Boer investigated the possibilities for secure communication channels~\cite{boer_de_secure_2020}; and Sjoerd Crooijmans added abstractions for low-power operation to \gls{MTASK} such as hardware interrupts and power efficient scheduling~\cite{crooijmans_reducing_2021}.
-Elina Antonova defined a preliminary formal semantics for a subset of \gls{MTASK}~\cite{antonova_MTASK_2022}.
+Erin van der Veen did preliminary work on a green computer analysis, built a simulator and explored the possibilities for adding bounded datatypes~\citep{veen_van_der_mutable_2020}; Michel de Boer investigated the possibilities for secure communication channels~\citep{boer_de_secure_2020}; and Sjoerd Crooijmans added abstractions for low-power operation to \gls{MTASK} such as hardware interrupts and power efficient scheduling~\citep{crooijmans_reducing_2021}.
+Elina Antonova defined a preliminary formal semantics for a subset of \gls{MTASK}~\citep{antonova_MTASK_2022}.
Moreover, plans for student projects and improvements include exploring integrating \gls{TINYML} into \gls{MTASK}; and adding intermittent computing support to \gls{MTASK}.
In 2023, the SusTrainable summer school in Coimbra, Portugal will host a course on \gls{MTASK} as well.
\section{\texorpdfstring{\gls{MTASK}}{mTask} in practise}
Funded by the Radboud-Glasgow Collaboration Fund, collaborative work was executed with Phil Trinder, Jeremy Singer and Adrian Ravi Kishore Ramsingh.
-An existing smart campus application was developed using \gls{MTASK} and quantitively and qualitatively compared to the original application that was developed using a traditional \gls{IOT} stack~\cite{lubbers_tiered_2020}.
+An existing smart campus application was developed using \gls{MTASK} and quantitively and qualitatively compared to the original application that was developed using a traditional \gls{IOT} stack~\citep{lubbers_tiered_2020}.
The collaboration is still ongoing and a journal article is under review comparing four approaches for the edge layer: \gls{PYTHON}, \gls{MICROPYTHON}, \gls{ITASK} and \gls{MTASK}.
Furthermore, power efficiency behaviour of traditional versus \gls{TOP} \gls{IOT} stacks is being compared as well adding a \gls{FREERTOS} implementation to the mix as well
\section{Introduction}%
\label{sec_t4t:Intro}
-Conventional \gls{IOT} software stacks are notoriously complex and pose very significant software development, reliability, and maintenance challenges. \Gls{IOT} software architectures typically comprise multiple components organised in four or more tiers or layers~\cite{sethi2017internet,Ravulavaru18,Alphonsa20}. This is due to the highly distributed nature of typical \gls{IOT} applications that must read sensor data from end points (the \emph{perception} layer), aggregate and select the data and communicate over a network (the \emph{network} layer), store the data in a database and analyse it (the \emph{application} layer) and display views of the data, commonly on web pages (the \emph{presentation} layer).
+Conventional \gls{IOT} software stacks are notoriously complex and pose very significant software development, reliability, and maintenance challenges. \Gls{IOT} software architectures typically comprise multiple components organised in four or more tiers or layers~\citep{sethi2017internet,Ravulavaru18,Alphonsa20}. This is due to the highly distributed nature of typical \gls{IOT} applications that must read sensor data from end points (the \emph{perception} layer), aggregate and select the data and communicate over a network (the \emph{network} layer), store the data in a database and analyse it (the \emph{application} layer) and display views of the data, commonly on web pages (the \emph{presentation} layer).
Conventional \gls{IOT} software architectures require the development of separate programs in various programming languages for each of the components/tiers in the stack. This is modular, but a significant burden for developers, and some key challenges are as follows.
\begin{enumerate*}
\item The developer must deal with the potentially diverse failure modes of each component, and of component interoperations.
\end{enumerate*}
-A radical alternative development paradigm uses a single \emph{tierless} language that synthesizes all components/tiers in the software stack. There are established \emph{tierless} languages for web stacks, e.g.\ Links~\cite{cooper2006links} or Hop~\cite{serrano2006hop}.
+A radical alternative development paradigm uses a single \emph{tierless} language that synthesizes all components/tiers in the software stack. There are established \emph{tierless} languages for web stacks, e.g.\ Links~\citep{cooper2006links} or Hop~\citep{serrano2006hop}.
In a tierless language the developer writes the application as a single program. The code for different tiers is simultaneously checked by the compiler, and compiled to the required component languages. For example, Links compiles to HTML and JavaScript for the web client and to SQL on the server to interact with the database system. Tierless languages for \gls{IOT} stacks are more recent and less common, examples include
-Potato~\cite{troyer_building_2018} and \gls{CLEAN} with \gls{ITASK}\slash\gls{MTASK}~\cite{lubbers_interpreting_2019}.
+Potato~\citep{troyer_building_2018} and \gls{CLEAN} with \gls{ITASK}\slash\gls{MTASK}~\citep{lubbers_interpreting_2019}.
\Gls{IOT} sensor nodes may be microcontrollers with very limited compute resources, or supersensors: resource-rich single board computers like a Raspberry Pi. A tierless language may target either class of sensor node, and microcontrollers are the more demanding target due to the limited resources, e.g.\ small memory, executing on bare metal etc.
Potentially a tierless language both reduces the development effort and improves correctness as correct interoperation and communication is automatically generated by the compiler. A tierless language may, however, introduce other problems. How expressive is the language? That is, can it readily express the required functionality? How maintainable is the software? Is the generated code efficient in terms of time, space, and power?
-This paper reports a systematic comparative evaluation of two tierless language technologies for \gls{IOT} stacks: one targeting resource-constrained microcontrollers, and the other resource-rich supersensors. The basis of the comparison is four implementations of a typical smart campus \gls{IOT} stack~\cite{hentschel_supersensors:_2016}. Two implementations are conventional tiered \gls{PYTHON}-based stacks: \gls{PRS} and \gls{PWS}. The other two implementations are tierless: \gls{CRS} and \gls{CWS}. Our work makes the following research contributions, and the key results are summarised, discussed, and quantified in \cref{sec_t4t:Conclusion}.
+This paper reports a systematic comparative evaluation of two tierless language technologies for \gls{IOT} stacks: one targeting resource-constrained microcontrollers, and the other resource-rich supersensors. The basis of the comparison is four implementations of a typical smart campus \gls{IOT} stack~\citep{hentschel_supersensors:_2016}. Two implementations are conventional tiered \gls{PYTHON}-based stacks: \gls{PRS} and \gls{PWS}. The other two implementations are tierless: \gls{CRS} and \gls{CWS}. Our work makes the following research contributions, and the key results are summarised, discussed, and quantified in \cref{sec_t4t:Conclusion}.
\begin{description}
\item[C1] We show that \emph{tierless languages have the potential to significantly reduce the development effort for \gls{IOT} systems}.
- We systematically compare code size (\gls{SLOC}) of the four smart campus implementations as a measure of development effort and maintainability~\cite{alpernaswonderful,rosenberg1997some}.
+ We systematically compare code size (\gls{SLOC}) of the four smart campus implementations as a measure of development effort and maintainability~\citep{alpernaswonderful,rosenberg1997some}.
The tierless implementations require 70\% less code than the tiered implementations. We analyse the codebases to attribute the code reduction to three factors.
\begin{enumerate*}
\item Tierless languages benefit from reduced interoperation, requiring far fewer languages, paradigms and source code files e.g.\ \gls{CWS} uses two languages, one paradigm and three source code files where \gls{PWS} uses seven languages, two paradigms and 35 source code files
We show that the bare metal execution environment enforces some restrictions on \gls{MTASK} although they remain high level. Moreover, the environment conveys some advantages, e.g.\ better control over timing (\cref{sec_t4t:ComparingTierless}).
\end{description}
-The current work extends~\cite{lubbers_tiered_2020} as follows. Contributions C3 and C4 are entirely new, and C1 is enhanced by being based on the analysis of four rather than two languages and implementations.
+The current work extends~\citep{lubbers_tiered_2020} as follows. Contributions C3 and C4 are entirely new, and C1 is enhanced by being based on the analysis of four rather than two languages and implementations.
\section{Background and related work}%
\label{sec_t4t:Background}
access to sensor data through a RESTful \gls{API}.
This allows campus stakeholders to add functionality at a business layer above the layers that we consider here. To date,
simple apps have been developed including room temperature
-monitors and campus utilization maps~\cite{hentschel_supersensors:_2016}.
+monitors and campus utilization maps~\citep{hentschel_supersensors:_2016}.
A longitudinal study of sensor accuracy has also been
-conducted~\cite{harth_predictive_2018}.
+conducted~\citep{harth_predictive_2018}.
\subsection{\texorpdfstring{\Gls{IOT}}{IoT} applications}%
\label{sec_t4t:Stacks}
are even more complex as they combine a web application with a second distributed system of sensor and actuator nodes that collect and aggregate data, operate on it, and communicate with the server.
Both web and \gls{IOT} applications are commonly structured into tiers, e.g.\ the classical four-tier Linux, Apache, MySQL and PHP (LAMP) stack.
-\Gls{IOT} stacks typically have more tiers than webapps, with the number depending on the complexity of the application~\cite{sethi2017internet}. While other tiers, like the business layer~\cite{muccini2018iot} may be added above them, the focus of our study is on programming the lower four tiers of the \gls{PRS}, \gls{CRS}, \gls{PWS} and \gls{CWS} stacks, as illustrated in \cref{fig_t4t:iot_arch}.
+\Gls{IOT} stacks typically have more tiers than webapps, with the number depending on the complexity of the application~\citep{sethi2017internet}. While other tiers, like the business layer~\citep{muccini2018iot} may be added above them, the focus of our study is on programming the lower four tiers of the \gls{PRS}, \gls{CRS}, \gls{PWS} and \gls{CWS} stacks, as illustrated in \cref{fig_t4t:iot_arch}.
\begin{landscape}
\begin{figure}[ht]
structure complex software is a common software engineering practice that provides significant architectural benefits for \gls{IOT} and other software. The tiered \gls{PYTHON} \gls{PRS} and \gls{PWS} stacks exhibit these benefits.
\begin{enumerate}
- \item Modularity: tiers allow a system to be structured as a set of components with clearly defined functionality. They can be implemented independently, and may be interchanged with other components that have similar functionality~\cite{maccormack2007impact}. In \gls{PRS} and \gls{PWS}, for example, a different NoSQL DBMS could relatively easily be substituted for {MongoDB}
+ \item Modularity: tiers allow a system to be structured as a set of components with clearly defined functionality. They can be implemented independently, and may be interchanged with other components that have similar functionality~\citep{maccormack2007impact}. In \gls{PRS} and \gls{PWS}, for example, a different NoSQL DBMS could relatively easily be substituted for {MongoDB}
- \item Abstraction: the hierarchical composition of components in the stack abstracts the view of the system as a whole. Enough detail is provided to understand the roles of each layer and how the components relate to one another~\cite{belle2013layered}. \Cref{fig_t4t:iot_arch} illustrates the abstraction of \gls{PRS} and \gls{PWS} into four tiers.
+ \item Abstraction: the hierarchical composition of components in the stack abstracts the view of the system as a whole. Enough detail is provided to understand the roles of each layer and how the components relate to one another~\citep{belle2013layered}. \Cref{fig_t4t:iot_arch} illustrates the abstraction of \gls{PRS} and \gls{PWS} into four tiers.
- \item Cohesion: well-defined boundaries ensure each tier contains functionality directly related to the task of the component~\cite{lee2001component}. The tiers in \gls{PRS} and \gls{PWS} contain all the functionality associated with perception, networking, application and presentation respectively.
+ \item Cohesion: well-defined boundaries ensure each tier contains functionality directly related to the task of the component~\citep{lee2001component}. The tiers in \gls{PRS} and \gls{PWS} contain all the functionality associated with perception, networking, application and presentation respectively.
\end{enumerate}
\begin{enumerate}
- \item Polyglot Development {--} the developer must be fluent in all the languages and components in the stack, known as being a full stack developer for webapps~\cite{mazzei2018full}. That is, the developer must correctly use multiple languages that have different paradigms, i.e.\ manage significant \emph{semantic friction}~\cite{ireland2009classification}. For example the \gls{PWS} developer must integrate components written in seven languages with two paradigms (\cref{sec_t4t:interoperation}).
+ \item Polyglot Development {--} the developer must be fluent in all the languages and components in the stack, known as being a full stack developer for webapps~\citep{mazzei2018full}. That is, the developer must correctly use multiple languages that have different paradigms, i.e.\ manage significant \emph{semantic friction}~\citep{ireland2009classification}. For example the \gls{PWS} developer must integrate components written in seven languages with two paradigms (\cref{sec_t4t:interoperation}).
\item Correct Interoperation {--} the developer must adhere to the \gls{API} or communication protocols between components. \Cref{sec_t4t:codesize,sec_t4t:resourcerich} show that communication requires some 17\% of \gls{PRS} and \gls{PWS} code, so around 100 \gls{SLOC}. \Cref{sec_t4t:Communication} discusses the complexity of writing this distributed communication code.
\end{enumerate}
-Beyond \gls{PRS} and \gls{PWS} the challenges of tiered polyglot software development are evidenced in real world studies. As recent examples, a study of GitHub open source projects found an average of five different languages in each project, with many using tiered architectures~\cite{mayer2017multi}.
-An earlier empirical study of GitHub shows that using more languages to implement a project has a significant effect on project quality, since it increases defects~\cite{kochhar2016large}.
-A study of \gls{IOT} stack developers found that interoperation poses a real challenge, that microservices blur the abstraction between tiers, and that both testing and scaling \gls{IOT} applications to more devices are hard~\cite{motta2018challenges}.
+Beyond \gls{PRS} and \gls{PWS} the challenges of tiered polyglot software development are evidenced in real world studies. As recent examples, a study of GitHub open source projects found an average of five different languages in each project, with many using tiered architectures~\citep{mayer2017multi}.
+An earlier empirical study of GitHub shows that using more languages to implement a project has a significant effect on project quality, since it increases defects~\citep{kochhar2016large}.
+A study of \gls{IOT} stack developers found that interoperation poses a real challenge, that microservices blur the abstraction between tiers, and that both testing and scaling \gls{IOT} applications to more devices are hard~\citep{motta2018challenges}.
One way of minimising the challenges of developing tiered polyglot \gls{IOT} software is to standardise and reuse components. This approach has been hugely successful for web stacks, e.g.\ browser standards. The W3C
-Web of Things aims to facilitate re-use by providing standardised metadata and other re-usable technological \gls{IOT} building blocks~\cite{guinard_building_2016}. However, the Web of Things has yet to gain widespread adoption. Moreover, as it is based on web technology, it requires the \emph{thing} to run a web server, significantly increasing the hardware requirements.
+Web of Things aims to facilitate re-use by providing standardised metadata and other re-usable technological \gls{IOT} building blocks~\citep{guinard_building_2016}. However, the Web of Things has yet to gain widespread adoption. Moreover, as it is based on web technology, it requires the \emph{thing} to run a web server, significantly increasing the hardware requirements.
\section{Tierless languages}%
\label{sec_t4t:TiredvsTierless}
A radical approach to overcoming the challenges raised by tiered distributed software is to use a tierless programming language that eliminates the semantic friction between tiers by generating code for all tiers, and all communication between tiers, from a single program.
-%\adriancomment{Also referred to as multi-tier programming, tierless language applications usually utilise a single language, paradigm and type system, and the entire system is simultaneously checked by the compiler~\cite{weisenburger2020survey}.}
+%\adriancomment{Also referred to as multi-tier programming, tierless language applications usually utilise a single language, paradigm and type system, and the entire system is simultaneously checked by the compiler~\citep{weisenburger2020survey}.}
Typically a tierless program uses a single language, paradigm and type system, and the entire distributed system is simultaneously checked by the compiler.
-There is intense interest in developing tierless, or multitier, language technologies with a number of research languages developed over the last fifteen years, e.g.\ \cite{cooper2006links, serrano2006hop, troyer_building_2018, 10.1145/2775050.2633367}. These languages demonstrate the
-advantages of the paradigm, including less development effort, better maintainability, and sound semantics of distributed execution. At the same time a number of industrial technologies incorporate tierless concepts, e.g.\ \cite{balat2006ocsigen, bjornson2010composing, strack2015getting}. These languages demonstrate the benefits of the paradigm in practice. Some tierless languages use (embedded) \glspl{DSL} to specify parts of the multi-tier software.
+There is intense interest in developing tierless, or multitier, language technologies with a number of research languages developed over the last fifteen years, e.g.\ \citep{cooper2006links, serrano2006hop, troyer_building_2018, 10.1145/2775050.2633367}. These languages demonstrate the
+advantages of the paradigm, including less development effort, better maintainability, and sound semantics of distributed execution. At the same time a number of industrial technologies incorporate tierless concepts, e.g.\ \citep{balat2006ocsigen, bjornson2010composing, strack2015getting}. These languages demonstrate the benefits of the paradigm in practice. Some tierless languages use (embedded) \glspl{DSL} to specify parts of the multi-tier software.
-Tierless languages have been developed for a range of distributed paradigms, including web applications, client-server applications, mobile applications, and generic distributed systems. A recent and substantial survey of these tierless technologies is available~\cite{weisenburger2020survey}. Here we provide a brief introduction to tierless languages with a focus on \gls{IOT} software.
+Tierless languages have been developed for a range of distributed paradigms, including web applications, client-server applications, mobile applications, and generic distributed systems. A recent and substantial survey of these tierless technologies is available~\citep{weisenburger2020survey}. Here we provide a brief introduction to tierless languages with a focus on \gls{IOT} software.
\subsection{Tierless web languages}
% Standalone DSLs
There are established tierless languages for web development, both standalone languages and \glspl{DSL} embedded in a host language.
-Example standalone tierless web languages are Links~\cite{cooper2006links} and Hop~\cite{serrano2006hop}.
+Example standalone tierless web languages are Links~\citep{cooper2006links} and Hop~\citep{serrano2006hop}.
From a single declarative program the client, server and database code is simultaneously checked by the compiler, and compiled to the required component languages. For example, Links compiles to HTML and JavaScript for the client side and to SQL on the server-side to interact with the database system.
-An example tierless web framework that uses a \gls{DSL} is Haste~\cite{10.1145/2775050.2633367}, that embeds the \gls{DSL} in \gls{HASKELL}. Haste programs are compiled multiple times: the server code is generated by the standard \gls{GHC} \gls{HASKELL} compiler~\cite{hall1993glasgow}; Javascript for the client is generated by a custom \gls{GHC} compiler backend. The design leverages \gls{HASKELL}'s high-level programming abstractions and strong typing, and benefits from \gls{GHC}: a mature and sophisticated compiler.
+An example tierless web framework that uses a \gls{DSL} is Haste~\citep{10.1145/2775050.2633367}, that embeds the \gls{DSL} in \gls{HASKELL}. Haste programs are compiled multiple times: the server code is generated by the standard \gls{GHC} \gls{HASKELL} compiler~\citep{hall1993glasgow}; Javascript for the client is generated by a custom \gls{GHC} compiler backend. The design leverages \gls{HASKELL}'s high-level programming abstractions and strong typing, and benefits from \gls{GHC}: a mature and sophisticated compiler.
\subsection{Tierless \texorpdfstring{\gls{IOT}}{IoT} languages}
\subsubsection{\texorpdfstring{\Glspl{DSL}}{DSLs} for microcontrollers}
Many \glspl{DSL} provide high-level programming for microcontrollers, for example providing strong typing and memory safety.
-For example Copilot~\cite{hess_arduino-copilot_2020}
-and Ivory~\cite{elliott_guilt_2015} are imperative \glspl{DSL} embedded in a functional language that compile to \gls{C}\slash\gls{CPP}. In contrast to \gls{CLEAN}/\gls{ITASK}/\gls{MTASK} such \glspl{DSL} are not tierless \gls{IOT} languages as they have no automatic integration with the server, i.e.\ with the application and presentation layers.
+For example Copilot~\citep{hess_arduino-copilot_2020}
+and Ivory~\citep{elliott_guilt_2015} are imperative \glspl{DSL} embedded in a functional language that compile to \gls{C}\slash\gls{CPP}. In contrast to \gls{CLEAN}/\gls{ITASK}/\gls{MTASK} such \glspl{DSL} are not tierless \gls{IOT} languages as they have no automatic integration with the server, i.e.\ with the application and presentation layers.
\subsubsection{\texorpdfstring{\Gls{FRP}}{Functional reactive programming}}
\Gls{FRP} is a declarative paradigm often used for implementing the perception layer of an \gls{IOT} stack.
-Examples include mfrp~\cite{sawada_emfrp:_2016}, CFRP~\cite{suzuki_cfrp_2017}, XFRP~\cite{10.1145/3281366.3281370}, Juniper~\cite{helbling_juniper:_2016}, Hailstorm~\cite{sarkar_hailstorm_2020}, and Haski~\cite{valliappan_towards_2020}.
+Examples include mfrp~\citep{sawada_emfrp:_2016}, CFRP~\citep{suzuki_cfrp_2017}, XFRP~\citep{10.1145/3281366.3281370}, Juniper~\citep{helbling_juniper:_2016}, Hailstorm~\citep{sarkar_hailstorm_2020}, and Haski~\citep{valliappan_towards_2020}.
None of these languages are tierless \gls{IOT} languages as they have no automatic integration with the server.
-Potato goes beyond other \gls{FRP} languages to provide a tierless \gls{FRP} \gls{IOT} language for resource rich sensor nodes~\cite{troyer_building_2018}. It does so using the Erlang programming language and sophisticated virtual machine.
+Potato goes beyond other \gls{FRP} languages to provide a tierless \gls{FRP} \gls{IOT} language for resource rich sensor nodes~\citep{troyer_building_2018}. It does so using the Erlang programming language and sophisticated virtual machine.
-TOP allows for more complex collaboration patterns than \gls{FRP}~\cite{wang_maintaining_2018}, and in consequence is unable to provide the strong guarantees on memory usage available in a restricted variant of \gls{FRP} such as arrowized \gls{FRP}~\cite{nilsson_functional_2002}.
+TOP allows for more complex collaboration patterns than \gls{FRP}~\citep{wang_maintaining_2018}, and in consequence is unable to provide the strong guarantees on memory usage available in a restricted variant of \gls{FRP} such as arrowized \gls{FRP}~\citep{nilsson_functional_2002}.
\subsubsection{Erlang/Elixir \texorpdfstring{\gls{IOT}}{IoT} systems}
A number of production \gls{IOT} systems are engineered in Erlang or Elixir, and many are mostly tierless.
That is the perception, network and application layers are sets of distributed Erlang processes, although the presentation layer typically uses some conventional web technology.
A resource-rich sensor node may support many Erlang processes on an Erlang VM, or low level code (typically \gls{C}\slash\gls{CPP}) on a resource-constrained microcontroller can emulate an Erlang process.
-Only a small fraction of these systems are described in the academic literature, example exceptions are~\cite{sivieri2012drop,shibanai_distributed_2018}, with many described only in grey literature or not at all.
+Only a small fraction of these systems are described in the academic literature, example exceptions are~\citep{sivieri2012drop,shibanai_distributed_2018}, with many described only in grey literature or not at all.
\subsection{Characteristics of tierless \texorpdfstring{\gls{IOT}}{IoT} languages}%
\label{sec_t4t:characteristics}
\subsubsection{Program splitting}
A key challenge for an automatically segmented tierless language is to determine which parts of the program correspond to a particular tier and hence should be executed by a specific component on a specific host, so-called tier splitting.
-For example a tierless web language must identify client code to ship to browsers, database code to execute in the DBMS, and application code to run on the server. Some tierless languages split programs using types, others use syntactic markers, e.g.\ pragmas like \cleaninline{server} or \cleaninline{client}, to split the program~\cite{cooper2006links,10.1145/2775050.2633367}. It may be possible to infer the splitting between tiers, relieving the developers from the need specify it, as illustrated for Javascript as a tierless web language~\cite{10.1145/2661136.2661146}.
+For example a tierless web language must identify client code to ship to browsers, database code to execute in the DBMS, and application code to run on the server. Some tierless languages split programs using types, others use syntactic markers, e.g.\ pragmas like \cleaninline{server} or \cleaninline{client}, to split the program~\citep{cooper2006links,10.1145/2775050.2633367}. It may be possible to infer the splitting between tiers, relieving the developers from the need specify it, as illustrated for Javascript as a tierless web language~\citep{10.1145/2661136.2661146}.
In \gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} and \gls{CLEAN}/\gls{ITASK} tier splitting is specified by functions, and hence is a first-class language construct.
For example in \gls{CLEAN}\slash\gls{ITASK} the \cleaninline{asyncTask} function identifies a task for execution on a remote device and \cleaninline{liftmTask} executes the given task on an \gls{IOT} device. The tier splitting functions are illustrated in examples in the next section, e.g.\ on \cref{lst_t4t:itaskTempFull:startdevtask} in \cref{lst_t4t:itaskTempFull} and \cref{lst_t4t:mtaskTemp:liftmtask} in \cref{lst_t4t:mtaskTemp}.
\subsubsection{Communication}\label{ssec_t4t:communication}
-Tierless languages may adopt a range of communication paradigms for communicating between components. Different tierless languages specify communication in different ways~\cite{weisenburger2020survey}. Remote procedures are the most common communication mechanism: a procedure/function executing on a remote host/machine is called as if it was local. The communication of the arguments to, and the results from, the remote procedure is automatically provided by the language implementation. Other mechanisms include explicit message passing between components; publish/subscribe where components subscribe to topics of interest from other components; reactive programming defines event streams between remote components; finally shared state makes changes in a shared and potentially remote data structure visible to components.
+Tierless languages may adopt a range of communication paradigms for communicating between components. Different tierless languages specify communication in different ways~\citep{weisenburger2020survey}. Remote procedures are the most common communication mechanism: a procedure/function executing on a remote host/machine is called as if it was local. The communication of the arguments to, and the results from, the remote procedure is automatically provided by the language implementation. Other mechanisms include explicit message passing between components; publish/subscribe where components subscribe to topics of interest from other components; reactive programming defines event streams between remote components; finally shared state makes changes in a shared and potentially remote data structure visible to components.
\Gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} and \gls{CLEAN}/\gls{ITASK} communicate using a combination of remote task invocation, similar to remote procedures, and shared state through \glspl{SDS}.
\Cref{lst_t4t:itaskTempFull} illustrates: \cref{lst_t4t:itaskTempFull:startdevtask} shows a server task launching a remote task, \cleaninline{devTask}, on to a sensor node; and \cref{lst_t4t:itaskTempFull:remoteShare} shows the sharing of the remote \cleaninline{latestTemp} \gls{SDS}.
Hence, most \gls{IOT} systems compile sensor node code directly for the target architecture or via an existing language such as \gls{C}\slash\gls{CPP}.
Techniques such as over-the-air programming and interpreters allow microcontrollers to be dynamically provisioned, increasing their maintainability and resilience.
-For example Baccelli et al.\ provide a single language \gls{IOT} system based on the RIOT \gls{OS} that allows runtime deployment of code snippets called containers~\cite{baccelli_reprogramming_2018}.
+For example Baccelli et al.\ provide a single language \gls{IOT} system based on the RIOT \gls{OS} that allows runtime deployment of code snippets called containers~\citep{baccelli_reprogramming_2018}.
Both client and server are written in JavaScript. However, there is no integration between the client and the server other than that they are programmed from a single source.
-Mat\`e is an example of an early tierless sensor network framework where devices are provided with a virtual machine using TinyOS for dynamic provisioning~\cite{levis_mate_2002}.
+Mat\`e is an example of an early tierless sensor network framework where devices are provided with a virtual machine using TinyOS for dynamic provisioning~\citep{levis_mate_2002}.
Placement specifies how data and computations in a tierless program are assigned to the
-devices/hosts in the distributed system. Different tierless languages specify placement in different ways, e.g.\ code annotations or configuration files, and at different granularities, e.g.\ per function or per class~\cite{weisenburger2020survey}.
+devices/hosts in the distributed system. Different tierless languages specify placement in different ways, e.g.\ code annotations or configuration files, and at different granularities, e.g.\ per function or per class~\citep{weisenburger2020survey}.
\Gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} and \gls{CLEAN}/\gls{ITASK} both use dynamic task placement, similar to dynamic function placement.
In \gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} sensor nodes are programmed once with the \gls{MTASK} \gls{RTS}, and possibly some precompiled tasks.
Thereafter a sensor node can dynamically receive \gls{MTASK} programs, compiled at runtime by the server.
-In \gls{CLEAN}\slash\gls{ITASK} the sensor node runs an \gls{ITASK} server that recieves and executes code from the (\gls{IOT}) server~\cite{oortgiese_distributed_2017}.
-%The \gls{ITASK} server decides what code to execute depending on the serialised execution graph that the server sends~\cite{oortgiese_distributed_2017}.
+In \gls{CLEAN}\slash\gls{ITASK} the sensor node runs an \gls{ITASK} server that recieves and executes code from the (\gls{IOT}) server~\citep{oortgiese_distributed_2017}.
+%The \gls{ITASK} server decides what code to execute depending on the serialised execution graph that the server sends~\citep{oortgiese_distributed_2017}.
Placement happens automatically as part of the first-class splitting constructs outlined in \cref{ssec_t4t:communication}, so \cref{lst_t4t:mtaskTemp:liftmtask} in \cref{lst_t4t:mtaskTemp} places \cleaninline{devTask} onto the \cleaninline{dev} sensor node.
\subsubsection{Security}
-Security is a major issue and a considerable challenge for many \gls{IOT} systems~\cite{10.1145/3437537}. There are potentially security issues at each layer in an \gls{IOT} application (\cref{fig_t4t:iot_arch}). The security issues and defence mechanisms at the application and presentation layers are relatively standard, e.g.\ defending against SQL injection attacks. The security issues at the network and perception layers are more challenging. Resource-rich sensor nodes can adopt some standard security measures like encrypting messages, and regularly applying software patches to the operating system. However microcontrollers often lack the computational resources for encryption, and it is hard to patch their system software because the program is often stored in flash memory. In consequence there are infamous examples of \gls{IOT} systems being hijacked to create botnets~\cite{203628,herwig_measurement_2019}.
+Security is a major issue and a considerable challenge for many \gls{IOT} systems~\citep{10.1145/3437537}. There are potentially security issues at each layer in an \gls{IOT} application (\cref{fig_t4t:iot_arch}). The security issues and defence mechanisms at the application and presentation layers are relatively standard, e.g.\ defending against SQL injection attacks. The security issues at the network and perception layers are more challenging. Resource-rich sensor nodes can adopt some standard security measures like encrypting messages, and regularly applying software patches to the operating system. However microcontrollers often lack the computational resources for encryption, and it is hard to patch their system software because the program is often stored in flash memory. In consequence there are infamous examples of \gls{IOT} systems being hijacked to create botnets~\citep{203628,herwig_measurement_2019}.
Securing the entire stack in a conventional tiered \gls{IOT} application is particularly challenging as the stack is implemented in a collection of programming languages with low level programming and communication abstractions. In such polyglot distributed systems it is hard to determine, and hence secure, the flow of data between components. In consequence a small mistake may have severe security implications.
-A number of characteristics of tierless languages help to improve security. Communication and placement vulnerabilities are minimised as communication and placement are automatically generated and checked by the compiler. So injection attacks and the exploitation of communication/placement protocol bugs are less likely. Vulnerabilities introduced by mismatched types are avoided as the entire system is type checked. Moreover, tierless languages can exploit language level security techniques. For example languages like Jif/split~\cite{zdancewic2002secure} and Swift~\cite{chong2007secure} place components to protect the security of data. Another example are programming language technologies for controlling information flow, and these can be used to improve security. For example Haski uses them to improve the security of \gls{IOT} systems~\cite{valliappan_towards_2020}.
+A number of characteristics of tierless languages help to improve security. Communication and placement vulnerabilities are minimised as communication and placement are automatically generated and checked by the compiler. So injection attacks and the exploitation of communication/placement protocol bugs are less likely. Vulnerabilities introduced by mismatched types are avoided as the entire system is type checked. Moreover, tierless languages can exploit language level security techniques. For example languages like Jif/split~\citep{zdancewic2002secure} and Swift~\citep{chong2007secure} place components to protect the security of data. Another example are programming language technologies for controlling information flow, and these can be used to improve security. For example Haski uses them to improve the security of \gls{IOT} systems~\citep{valliappan_towards_2020}.
-However many tierless languages have yet to provide a comprehensive set of security technologies, despite its importance in domains like web and \gls{IOT} applications. For example Erlang and many Erlang-based systems~\cite{shibanai_distributed_2018,sivieri2012drop}, lack important security measures. Indeed security is not covered in a recent, otherwise comprehensive, survey of tierless technologies~\cite{weisenburger2020survey}.
+However many tierless languages have yet to provide a comprehensive set of security technologies, despite its importance in domains like web and \gls{IOT} applications. For example Erlang and many Erlang-based systems~\citep{shibanai_distributed_2018,sivieri2012drop}, lack important security measures. Indeed security is not covered in a recent, otherwise comprehensive, survey of tierless technologies~\citep{weisenburger2020survey}.
-\Gls{CLEAN}\slash\gls{ITASK} and \gls{CLEAN}/\gls{ITASK}/\gls{MTASK} are typical in this respect: little effort has yet been expended on improving their security. Of course as tierless languages they benefit from static type safety and automatically generated communication and placement. Some preliminary work shows that, as the communication between layers is protocol agnostic, more secure alternatives can be used. One example is to run the \gls{ITASK} server behind a reverse proxy implementing TLS/SSL encryption~\cite{wijkhuizen_security_2018}. A second is to add integrity checks or even encryption to the communication protocol for resource-rich sensor nodes~\cite{boer_de_secure_2020}.
+\Gls{CLEAN}\slash\gls{ITASK} and \gls{CLEAN}/\gls{ITASK}/\gls{MTASK} are typical in this respect: little effort has yet been expended on improving their security. Of course as tierless languages they benefit from static type safety and automatically generated communication and placement. Some preliminary work shows that, as the communication between layers is protocol agnostic, more secure alternatives can be used. One example is to run the \gls{ITASK} server behind a reverse proxy implementing TLS/SSL encryption~\citep{wijkhuizen_security_2018}. A second is to add integrity checks or even encryption to the communication protocol for resource-rich sensor nodes~\citep{boer_de_secure_2020}.
\section{Task-oriented and \texorpdfstring{\gls{IOT}}{IoT} programming in \texorpdfstring{\gls{CLEAN}}{Clean}}
To make this paper self-contained we provide a concise overview of \gls{CLEAN}, \gls{TOP}, and \gls{IOT} programming in \gls{ITASK} and \gls{MTASK}. The minor innovations reported here are the interface to the \gls{IOT} sensors, and the \gls{CLEAN} port for the Raspberry Pi.
-\Gls{CLEAN} is a statically typed functional programming language similar to \gls{HASKELL}: both languages are pure and non-strict~\cite{achten_clean_2007}.
-A key difference is how state is handled: \gls{HASKELL} typically embeds stateful actions in the \haskellinline{IO} Monad~\cite{peyton_jones_imperative_1993,wiki:IO}.
-In contrast, \gls{CLEAN} has a uniqueness type system to ensure the single-threaded use of stateful objects like files and windows~\cite{barendsen_smetsers_1996}.
-Both \gls{CLEAN} and \gls{HASKELL} support fairly similar models of generic programming~\cite{ComparingGenericProgramming}, enabling functions to work on many types. As we shall see generic programming is heavily used in task-oriented programming~\cite{GenericProgrammingExtensionForClean,HinzeGenericFunctionalProgramming}, for example to construct web editors and communication protocols that work for any user-defined datatype.
+\Gls{CLEAN} is a statically typed functional programming language similar to \gls{HASKELL}: both languages are pure and non-strict~\citep{achten_clean_2007}.
+A key difference is how state is handled: \gls{HASKELL} typically embeds stateful actions in the \haskellinline{IO} Monad~\citep{peyton_jones_imperative_1993,wiki:IO}.
+In contrast, \gls{CLEAN} has a uniqueness type system to ensure the single-threaded use of stateful objects like files and windows~\citep{barendsen_smetsers_1996}.
+Both \gls{CLEAN} and \gls{HASKELL} support fairly similar models of generic programming~\citep{ComparingGenericProgramming}, enabling functions to work on many types. As we shall see generic programming is heavily used in task-oriented programming~\citep{GenericProgrammingExtensionForClean,HinzeGenericFunctionalProgramming}, for example to construct web editors and communication protocols that work for any user-defined datatype.
\subsection{\texorpdfstring{\Acrlong{TOP}}{Task-oriented programming}}
-\Gls{TOP} is a declarative programming paradigm for constructing interactive distributed systems~\cite{plasmeijer_task-oriented_2012}.
+\Gls{TOP} is a declarative programming paradigm for constructing interactive distributed systems~\citep{plasmeijer_task-oriented_2012}.
Tasks are the basic blocks of \gls{TOP} and represent work that needs to be done in the broadest sense.
Examples of typical tasks range from allowing a user to complete a form, controlling peripherals, moderating other tasks, or monitoring a database.
From a single declarative description of tasks all of the required software components are generated.
This may include web servers, client code for browsers or \gls{IOT} devices, and for their interoperation.
That is, from a single \gls{TOP} program the language implementation automatically generates an \emph{integrated distributed system}.
-Application areas range from simple web forms or blinking \glspl{LED} to multi-user distributed collaboration between people and machines~\cite{oortgiese_distributed_2017}.
+Application areas range from simple web forms or blinking \glspl{LED} to multi-user distributed collaboration between people and machines~\citep{oortgiese_distributed_2017}.
\Gls{TOP} adds three concepts: tasks, task combinators, and \glspl{SDS}. Example basic tasks are web editors for user-defined datatypes, reading some \gls{IOT} sensor, or controlling peripherals like a servo motor.
Task combinators compose tasks into more advanced tasks, either in parallel or sequential and allow task values to be observed by other tasks.
As tasks can be returned as the result of a function, recursion can be freely used, e.g.\ to express the repetition of tasks.
There are also standard combinators for common patterns.
-Tasks can exchange information via \glspl{SDS}~\cite{ParametricLenses}.
+Tasks can exchange information via \glspl{SDS}~\citep{ParametricLenses}.
All tasks involved can atomically observe and change the value of a typed \gls{SDS}, allowing more flexible communication than with task combinators.
\glspl{SDS} offer a general abstraction of data shared by different tasks, analogous to variables, persistent values, files, databases and peripherals like sensors. Combinators compose \glspl{SDS} into a larger \gls{SDS}, and
parametric lenses define a specific view on an \gls{SDS}.
\label{sec_t4t:itasks}
-The \gls{ITASK} \gls{EDSL} is designed for constructing multi-user distributed applications, including web~\cite{TOP-ICFP07} or \gls{IOT} applications.
+The \gls{ITASK} \gls{EDSL} is designed for constructing multi-user distributed applications, including web~\citep{TOP-ICFP07} or \gls{IOT} applications.
Here we present \gls{ITASK} by example, and the first is a complete program to repeatedly read the room temperature from a digital humidity and temperature (DHT) sensor attached to the machine and display it on a web page (\cref{lst_t4t:itaskTemp}).
The first line is the module name, the third imports the \cleaninline{iTask} module, and the main function (\cref{lst_t4t:itaskTemp:systemfro,lst_t4t:itaskTemp:systemto}) launches \cleaninline{readTempTask} and the \gls{ITASK} system to generate the web interface in \cref{fig_t4t:itaskTempSimple}.
relatively powerful Raspberry Pi 3 Model Bs. There is a simple object-oriented \gls{PYTHON} collector for configuring the sensors and reading their values. The collector daemon service marshals the sensor data and transmits using \gls{MQTT} to the central monitoring server at a preset frequency.
The collector caches sensor data locally when the server is unreachable.
-In contrast to \gls{PRS}, \gls{PWS}'s sensor nodes are microcontrollers running \gls{MICROPYTHON}, a dialect of \gls{PYTHON} specifically designed to run on small, low powered embedded devices~\cite{kodali2016low}.
+In contrast to \gls{PRS}, \gls{PWS}'s sensor nodes are microcontrollers running \gls{MICROPYTHON}, a dialect of \gls{PYTHON} specifically designed to run on small, low powered embedded devices~\citep{kodali2016low}.
To enable a fair comparison between the software stacks we are careful to use the same object-oriented software architecture, e.g.\ using the same classes in \gls{PWS} and \gls{PRS}.
\Gls{PYTHON} and \gls{MICROPYTHON} are appropriate tiered comparison languages. Tiered \gls{IOT} systems are implemented in a whole range of programming languages, with \gls{PYTHON}, \gls{MICROPYTHON}, \gls{C} and \gls{CPP} being popular for some tiers in many implementations. \gls{C}\slash\gls{CPP} implementations would probably result in more verbose programs and even less type safety.
Communication between a sensor node and the server is initiated by the server.
\Gls{CRS}'s sensor nodes are Raspberry Pi 4s, and execute \gls{CLEAN}\slash\gls{ITASK} programs.
-Communication from the sensor node to the server is implicit and happens via \glspl{SDS} over \gls{TCP} using platform independent execution graph serialisation~\cite{oortgiese_distributed_2017}.
+Communication from the sensor node to the server is implicit and happens via \glspl{SDS} over \gls{TCP} using platform independent execution graph serialisation~\citep{oortgiese_distributed_2017}.
\Gls{CWS}'s sensor nodes are Wemos microcontrollers running \gls{MTASK} tasks. Communication and serialisation is, by design, very similar to \gls{ITASK}, i.e.\ via \glspl{SDS} over either a serial port connection, raw \gls{TCP}, or \gls{MQTT} over \gls{TCP}.
\end{table}
\Cref{tbl_t4t:mem} shows the maximum memory residency after garbage collection of the sensor node for all four smart campus implementations. The smart campus sensor node programs executing on the Wemos microcontrollers have low maximum residencies: \qty{20270}{\byte} for \gls{PWS} and \qty{880}{\byte} for \gls{CWS}. In \gls{CWS} the \gls{MTASK} system generates very high level \gls{TOP} byte code that is interpreted by the \gls{MTASK} virtual machine and uses a small and predictable amount of heap memory.
-In \gls{PWS}, the hand-written \gls{MICROPYTHON} is compiled to byte code for execution on the virtual machine. Low residency is achieved with a fixed size heap and efficient memory management. For example both \gls{MICROPYTHON} and \gls{MTASK} use fixed size allocation units and mark\&sweep garbage collection to minimise memory usage at the cost of some execution time~\cite{plamauer2017evaluation}.
+In \gls{PWS}, the hand-written \gls{MICROPYTHON} is compiled to byte code for execution on the virtual machine. Low residency is achieved with a fixed size heap and efficient memory management. For example both \gls{MICROPYTHON} and \gls{MTASK} use fixed size allocation units and mark\&sweep garbage collection to minimise memory usage at the cost of some execution time~\citep{plamauer2017evaluation}.
The smart campus sensor node programs executing on the Raspberry Pis have far higher maximum residencies than those executing on the microcontrollers: \qty{3.5}{\mebi\byte} for \gls{PRS} and \qty{2.7}{\mebi\byte} for \gls{CRS}. In \gls{CRS} the sensor node code is a set of \gls{ITASK} executing on a full-fledged \gls{ITASK} server running in distributed child mode and this consumes far more memory.
%The memory used is actually very similar to the memory usage of the server with a single client connected.
In \gls{PRS} the sensor node program is written in \gls{PYTHON}, a language far less focused on minimising memory usage than \gls{MICROPYTHON}. For example an object like a string is larger in \gls{PYTHON} than in \gls{MICROPYTHON} and consequently does not support all features such as \emph{f-strings}.
-Furthermore, not all advanced \gls{PYTHON} feature regarding classes are available in \gls{MICROPYTHON}, i.e.\ only a subset of the \gls{PYTHON} specification is supported~\cite{diffmicro}.%\mlcomment{reference \url{https://docs.micropython.org/en/latest/genrst/index.html} ? It contains an overview of supported features}
+Furthermore, not all advanced \gls{PYTHON} feature regarding classes are available in \gls{MICROPYTHON}, i.e.\ only a subset of the \gls{PYTHON} specification is supported~\citep{diffmicro}.%\mlcomment{reference \url{https://docs.micropython.org/en/latest/genrst/index.html} ? It contains an overview of supported features}
In summary the sensor node code generated by both tierless languages, \gls{ITASK} and \gls{MTASK}, is sufficiently memory efficient for the target sensor node hardware. Indeed, the maximum residencies of the \gls{CLEAN} sensor node code is less than the corresponding hand-written (Micro)\gls{PYTHON} code. Of course in a tiered stack the hand-written code can be more easily optimised to minimise residency, and this could even entail using a memory efficienthat thet language like \gls{C}\slash\gls{CPP}. However, such optimisation requires additional developer effort, and a new language would introduce additional semantic friction.
\label{sec_t4t:codesize}
%A comparison of the Temperature sensor in \gls{PYTHON} Micropyton, Itask \& \gls{MTASK}.
-\paragraph{Code Size} is widely recognised as an approximate measure of the development and maintenance effort required for a software system~\cite{rosenberg1997some}. \gls{SLOC} is a common code size metric, and is especially useful for multi-paradigm systems like \gls{IOT} systems. It is based on the simple principle that the more \gls{SLOC}, the more developer effort and the increased likelihood of bugs~\cite{rosenberg1997some}. It is a simple measure, not dependent on some formula, and can be automatically computed~\cite{sheetz2009understanding}.
+\paragraph{Code Size} is widely recognised as an approximate measure of the development and maintenance effort required for a software system~\citep{rosenberg1997some}. \gls{SLOC} is a common code size metric, and is especially useful for multi-paradigm systems like \gls{IOT} systems. It is based on the simple principle that the more \gls{SLOC}, the more developer effort and the increased likelihood of bugs~\citep{rosenberg1997some}. It is a simple measure, not dependent on some formula, and can be automatically computed~\citep{sheetz2009understanding}.
-Of course \gls{SLOC} must be used carefully as it is easily influenced by programming style, language paradigm, and counting method~\cite{alpernaswonderful}. Here we are counting code to compare development effort, use the same idiomatic programming style in each component, and only count lines of code, omitting comments and blank lines.
+Of course \gls{SLOC} must be used carefully as it is easily influenced by programming style, language paradigm, and counting method~\citep{alpernaswonderful}. Here we are counting code to compare development effort, use the same idiomatic programming style in each component, and only count lines of code, omitting comments and blank lines.
\Cref{table_t4t:multi} enumerates the \gls{SLOC} required to implement the \gls{UOG} smart campus functionalities in \gls{PWS}, \gls{PRS}, \gls{CWS} and \gls{CRS}. Both \gls{PYTHON} and \gls{CLEAN} implementations use the same server and communication code for Raspberry Pi and for Wemos sensor nodes (rows 5--7 of the table).
The Sensor Interface (SI) refers to code facilitating the communication between the peripherals and the sensor node software. % formerly hardware interface
Before exploring the reasons for the smaller tierless codebase we compare the implementations for resource-rich and resource-constrained sensor nodes, again using \gls{SLOC} and code proportions. \Cref{table_t4t:multi} shows that the two tiered implementations are very similar in size: with \gls{PWS} for microcontrollers requiring 562 \gls{SLOC} and \gls{PRS} for supersensors requiring 576 \gls{SLOC}.
The two tierless implementations are also similar in size: \gls{CWS} requiring 166 and \gls{CRS} 155 \gls{SLOC}.
-There are several main reasons for the similarity. One is that the server-side code, i.e.\ for the presentation and application layers, is identical for both resource rich/constrained implementations. The identical server code accounts for approximately 40\% of the \gls{PWS} and \gls{PRS} codebases, and approximately 85\% of the \gls{CWS} and \gls{CRS} codebases (\cref{fig_t4t:multipercentage}). For the perception and network layers on the sensor nodes, the \gls{PYTHON} and \gls{MICROPYTHON} implementations have the same structure, e.g.\ a class for each type of sensor, and use analogous libraries. Indeed, approaches like CircuitPython~\cite{CircuitPython} allow the same code to execute on both resource-rich and resource-constrained sensor nodes.
+There are several main reasons for the similarity. One is that the server-side code, i.e.\ for the presentation and application layers, is identical for both resource rich/constrained implementations. The identical server code accounts for approximately 40\% of the \gls{PWS} and \gls{PRS} codebases, and approximately 85\% of the \gls{CWS} and \gls{CRS} codebases (\cref{fig_t4t:multipercentage}). For the perception and network layers on the sensor nodes, the \gls{PYTHON} and \gls{MICROPYTHON} implementations have the same structure, e.g.\ a class for each type of sensor, and use analogous libraries. Indeed, approaches like CircuitPython~\citep{CircuitPython} allow the same code to execute on both resource-rich and resource-constrained sensor nodes.
Like \gls{PYTHON} and \gls{MICROPYTHON}, \gls{ITASK} and \gls{MTASK} are designed to be similar, as elaborated in \cref{sec_t4t:ComparingTierless}. The similarity is apparent when comparing the \gls{ITASK} \gls{CRTS} and \gls{ITASK}\slash\gls{MTASK} \gls{CWTS} room temperature systems in \cref{lst_t4t:itaskTempFull,lst_t4t:mtaskTemp}. That is, both implementations use similar \glspl{SDS} and lenses; they have similar \cleaninline{devTask}s that execute on the sensor node, and the server-side \cleaninline{mainTask}s are almost identical: they deploy the remote \cleaninline{devTask} before generating the web page to report the readings.
The vast majority of \gls{IOT} systems are implemented using a number of different programming languages and paradigms, and these must be effectively used and interoperated. A major reason that the tierless \gls{IOT} implementations are simpler and shorter than the tiered implementations is that they use far fewer programming languages and paradigms. Here we use language to distinguish \glspl{EDSL} from their host language: so \gls{ITASK} and \gls{MTASK} are considered distinct from \gls{CLEAN}; and to distinguish dialects: so \gls{MICROPYTHON} is considered distinct from \gls{PYTHON}.
-The tierless implementations use just two conceptually-similar \glspl{DSL} embedded in the same host language, and a single paradigm (\cref{table_t4t:languages,table_t4t:paradigms}). In contrast, the tiers in \gls{PRS} and \gls{PWS} use six or more very different languages, and both imperative and declarative paradigms. Multiple languages are commonly used in other typical software systems like web stacks, e.g.\ a recent survey of open source projects reveals that on average at least five different languages are used~\cite{mayer2015empirical}. Interoperating components in multiple languages and paradigms raises a plethora of issues.
+The tierless implementations use just two conceptually-similar \glspl{DSL} embedded in the same host language, and a single paradigm (\cref{table_t4t:languages,table_t4t:paradigms}). In contrast, the tiers in \gls{PRS} and \gls{PWS} use six or more very different languages, and both imperative and declarative paradigms. Multiple languages are commonly used in other typical software systems like web stacks, e.g.\ a recent survey of open source projects reveals that on average at least five different languages are used~\citep{mayer2015empirical}. Interoperating components in multiple languages and paradigms raises a plethora of issues.
-Interoperation \emph{increases the cognitive load on the developer} who must simultaneously think in multiple languages and paradigms. This is commonly known as semantic friction or impedance mismatch~\cite{ireland2009classification}. A simple illustration of this is that the tiered \gls{PRS} source code comprises some 38 source and configuration files, whereas the tierless \gls{CRS} requires just 3 files (\cref{table_t4t:multi}). The source could be structured as a single file, but to separate concerns is structured into three modules, one each for \glspl{SDS}, types, and control logic~\cite{wang_maintaining_2018}.
+Interoperation \emph{increases the cognitive load on the developer} who must simultaneously think in multiple languages and paradigms. This is commonly known as semantic friction or impedance mismatch~\citep{ireland2009classification}. A simple illustration of this is that the tiered \gls{PRS} source code comprises some 38 source and configuration files, whereas the tierless \gls{CRS} requires just 3 files (\cref{table_t4t:multi}). The source could be structured as a single file, but to separate concerns is structured into three modules, one each for \glspl{SDS}, types, and control logic~\citep{wang_maintaining_2018}.
The developer must \emph{correctly interoperate the components}, e.g.\ adhere to the \gls{API} or communication protocols between components. The interoperation often entails additional programming tasks like marshalling or demarshalling data between components. For example, in the tiered \gls{PRS} and \gls{PWS} architectures, \gls{JSON} is used to serialise and deserialise data strings from the \gls{PYTHON} collector component before storing the data in the Redis database (\cref{lst_t4t:json}).
%e.g.\ to marshall and demarshall data between components.
\subsection{Automatic Communication}%
\label{sec_t4t:Communication}
-In conventional tiered \gls{IOT} implementations the developer must write and maintain code to communicate between tiers. For example \gls{PRS} and \gls{PWS} create, send and read \gls{MQTT}~\cite{light2017mosquitto} messages
+In conventional tiered \gls{IOT} implementations the developer must write and maintain code to communicate between tiers. For example \gls{PRS} and \gls{PWS} create, send and read \gls{MQTT}~\citep{light2017mosquitto} messages
between the perception and application layers. \Cref{table_t4t:multi} shows that communication between these layers require some 94 \gls{SLOC} in \gls{PWS} and 98 in \gls{PRS}, accounting for 17\% of the codebase (bottom bars in \cref{fig_t4t:multipercentage}). To illustrate, \cref{lst_t4t:mwssmqtt} shows part of the code to communicate sensor readings from the \gls{PWS} sensor node to the Redis store on the server.
Not only must the tiered developer write additional code, but \gls{IOT} communication code is often intricate. In such a distributed system the sender and receiver must be correctly configured, correctly follow the communication protocol through all execution states, and deal with potential failures. For example line 3 of \cref{lst_t4t:mwssmqtt}: \pythoninline{redis host = config.get('Redis', 'Host')} will fail if either the host or IP are incorrect.
Firstly, functional programming languages are generally more concise than most other programming languages because their powerful abstractions like higher-order and/or polymorphic functions require less code to describe a computation.
Secondly, the \gls{TOP} paradigm used in \gls{ITASK} and \gls{MTASK} reduces the code size further by making it easy to specify \gls{IOT} functionality concisely.
As examples, the step combinator \cleaninline{>>*.} allows the task value on the left-hand side to be observed until one of the steps is enabled;
-and the \cleaninline{viewSharedInformation} (line 31 of \cref{lst_t4t:mtaskTemp}) part of the UI will be automatically updated when the value of the \gls{SDS} changes. Moreover, each \gls{SDS} provides automatic updates to all coupled \glspl{SDS} and associated tasks. Thirdly, the amount of explicit type information is minimised in comparison to other languages, as much is automatically inferred~\cite{hughes1989functional}.
+and the \cleaninline{viewSharedInformation} (line 31 of \cref{lst_t4t:mtaskTemp}) part of the UI will be automatically updated when the value of the \gls{SDS} changes. Moreover, each \gls{SDS} provides automatic updates to all coupled \glspl{SDS} and associated tasks. Thirdly, the amount of explicit type information is minimised in comparison to other languages, as much is automatically inferred~\citep{hughes1989functional}.
\section{Could Tierless \texorpdfstring{\gls{IOT}}{IoT} Programming Be More Reliable than Tiered?}%
\label{sec_t4t:Discussion}
\subsection{Type Safety}%
\label{sec_t4t:typesafety}
Strong typing identifies errors early in the development cycle, and hence plays a crucial role in improving software quality. In consequence almost all modern languages provide strong typing, and encourage static typing to minimise runtime errors.
-% Phil: so widely known that a citation is unnecessary~\cite{madsen1990strong}.
-That said, many distributed system components written in languages that primarily use static typing, like \gls{HASKELL} and Scala, use some dynamic typing, e.g.\ to ensure that the data arriving in a message has the anticipated type~\cite{epstein2011towards,gupta2012akka}.
+% Phil: so widely known that a citation is unnecessary~\citep{madsen1990strong}.
+That said, many distributed system components written in languages that primarily use static typing, like \gls{HASKELL} and Scala, use some dynamic typing, e.g.\ to ensure that the data arriving in a message has the anticipated type~\citep{epstein2011towards,gupta2012akka}.
-In a typical tiered multi-language \gls{IOT} system the developer must integrate software in different languages with very different type systems, and potentially executing on different hardware. The challenges of maintaining type safety have long been recognised as a major component of the semantic friction in multi-language systems, e.g.\ \cite{ireland2009classification}.
+In a typical tiered multi-language \gls{IOT} system the developer must integrate software in different languages with very different type systems, and potentially executing on different hardware. The challenges of maintaining type safety have long been recognised as a major component of the semantic friction in multi-language systems, e.g.\ \citep{ireland2009classification}.
-Even if the different languages used in two components are both strongly typed, they may attribute, often quite subtly, different types to a value. Such type errors can lead to runtime errors, or the application silently reporting erroneous data. Such errors can be hard to find. Automatic detection of such errors is sometimes possible, but requires an addition tool like Jinn~\cite{Jinn,Furr2005}.
+Even if the different languages used in two components are both strongly typed, they may attribute, often quite subtly, different types to a value. Such type errors can lead to runtime errors, or the application silently reporting erroneous data. Such errors can be hard to find. Automatic detection of such errors is sometimes possible, but requires an addition tool like Jinn~\citep{Jinn,Furr2005}.
%Such errors can be hard to debug, partly because there is very limited tool support for detecting them
-%Phil: another possible source to discuss ~\cite{egyed1999automatically}
+%Phil: another possible source to discuss ~\citep{egyed1999automatically}
\begin{lstPython}[caption={\Gls{PRS} loses type safety as a sensor node sends a {\tt\footnotesize double}, and the server stores a {\tt\footnotesize string}.},label={lst_t4t:float},morekeywords={message,enum,uint64,double}]
message SensorData {
Analysis of the \gls{PRS} codebase reveals an instance where it, fairly innocuously, loses type safety. The fragment in \cref{lst_t4t:float} first shows a \pythoninline{double} sensor value being sent from the sensor node, and then shows the value being stored in Redis as a \pythoninline{string} on the server. As \gls{PWS} preserves the same server components it also suffers from the same loss of type safety.
-\emph{A tierless language makes it possible to guarantee type safety across an entire \gls{IOT} stack}. For example the \gls{CLEAN} compiler guarantees static type safety as the entire \gls{CWS} software stack is type checked, and generated, from a single source. Tierless web stack languages like Links~\cite{cooper2006links} and Hop~\cite{serrano2006hop} provide the same guarantee for web stacks.
+\emph{A tierless language makes it possible to guarantee type safety across an entire \gls{IOT} stack}. For example the \gls{CLEAN} compiler guarantees static type safety as the entire \gls{CWS} software stack is type checked, and generated, from a single source. Tierless web stack languages like Links~\citep{cooper2006links} and Hop~\citep{serrano2006hop} provide the same guarantee for web stacks.
\subsection{Failure Management}%
\subsection{Support}%
\label{sec_t4t:support}
%\mlcomment{I've shortened this quite a bit}
-Community and tool support are essential for engineering reliable production software. \gls{PRS} and \gls{PWS} are both \gls{PYTHON} based, and \gls{PYTHON}\slash\gls{MICROPYTHON} are among the most popular programming languages~\cite{cass2020top}. \gls{PYTHON} is also a common choice for some tiers of \gls{IOT} applications~\cite{tanganelli2015coapthon}.
+Community and tool support are essential for engineering reliable production software. \gls{PRS} and \gls{PWS} are both \gls{PYTHON} based, and \gls{PYTHON}\slash\gls{MICROPYTHON} are among the most popular programming languages~\citep{cass2020top}. \gls{PYTHON} is also a common choice for some tiers of \gls{IOT} applications~\citep{tanganelli2015coapthon}.
Hence, there are a wide range of development tools like \glspl{IDE} and debuggers, a thriving community and a wealth of training material. There are even specialised \gls{IOT} Boards like PyBoard \& WiPy that are specifically programmed using \gls{PYTHON} variations like \gls{MICROPYTHON}.
In contrast, tierless languages are far less mature than the languages used in tiered stacks, and far less widely adopted.
\subsection{Language Restrictions for Resource-Constrained Execution}
-Executing components on a resource-constrained sensor node imposes restrictions on programming abstractions available in a tierless \gls{IOT} language or \gls{DSL}. The small and fixed-size memory are key limitations. The limitations are shared by any high-level language that targets microcontrollers such as BIT, PICBIT, PICOBIT, Microscheme and uLisp~\cite{dube_bit:_2000,feeley_picbit:_2003,st-amour_picobit:_2009,suchocki_microscheme:_2015, johnson-davies_lisp_2020}.
+Executing components on a resource-constrained sensor node imposes restrictions on programming abstractions available in a tierless \gls{IOT} language or \gls{DSL}. The small and fixed-size memory are key limitations. The limitations are shared by any high-level language that targets microcontrollers such as BIT, PICBIT, PICOBIT, Microscheme and uLisp~\citep{dube_bit:_2000,feeley_picbit:_2003,st-amour_picobit:_2009,suchocki_microscheme:_2015, johnson-davies_lisp_2020}.
Even in low level languages some language features are disabled by default when targeting microcontrollers, such as runtime type information (RTTI) in \gls{CPP}.
Here we investigate the restrictions imposed by resource-constrained sensor nodes on \gls{MTASK}, in comparison with \gls{ITASK}. While \gls{ITASK} and \gls{MTASK} are by design superficially similar languages, to execute on resource-constrained sensor nodes \gls{MTASK} tasks are more restricted, and have a different semantics.
However, even when using multiple \gls{MTASK} tasks, it is easier to control the number of tasks on a device than controlling the number of processes and threads executing under an \gls{OS}.
An \gls{MTASK} program has more control over energy consumption.
-The \gls{MTASK} \gls{EDSL} and the \gls{MTASK} \gls{RTS} are designed to minimise energy usage~\cite{crooijmans_reducing_2021}.
+The \gls{MTASK} \gls{EDSL} and the \gls{MTASK} \gls{RTS} are designed to minimise energy usage~\citep{crooijmans_reducing_2021}.
Intensional analysis of the declarative task description and current progress at run time allow the \gls{RTS} to schedule tasks and maximise idle time.
As the \gls{RTS} is the only program running on the device, it can enforce deep sleep and wake up without having to worry about influencing other processes.
\Cref{table_t4t:languagecomparison} summarises the differences between the \gls{CLEAN} \gls{IOT} \gls{EDSL} and their host language.
The restrictions imposed by a resource-constrained execution environment on the tierless \gls{IOT} language are relatively minor. Moreover the \gls{MTASK} programming abstraction is broadly compatible with \gls{ITASK}. As a simple example compare the \gls{ITASK} and \gls{MTASK} temperature sensors in \cref{lst_t4t:itaskTempFull,lst_t4t:mtaskTemp}. As a more realistic example, the \gls{MTASK} based \gls{CWS} smart campus implementation is similar to the \gls{ITASK} based \gls{CRS}, and requires less than 10\% additional code: 166 \gls{SLOC} compared with 155 \gls{SLOC} (\cref{table_t4t:multi}).
-Even with these restrictions, \gls{MTASK} programming is at a far higher level of abstraction than almost all bare metal languages, e.g.\ BIT, PICBIT, PICOBIT and Microscheme. That is \gls{MTASK} provides a set of higher order task combinators, shared distributed data stores, etc. (\cref{sec_t4t:mtasks}). Moreover, it seems that common sensor node programs are readily expressed using \gls{MTASK}. In addition to the \gls{CWTS} and \gls{CWS} systems outlined here, other case studies include Arduino examples as well as some bigger tasks~\cite{koopman_task-based_2018,lubbers_writing_2019,LubbersMIPRO}. We conclude that the programming of sensor tasks is well-supported by both \glspl{DSL}.
+Even with these restrictions, \gls{MTASK} programming is at a far higher level of abstraction than almost all bare metal languages, e.g.\ BIT, PICBIT, PICOBIT and Microscheme. That is \gls{MTASK} provides a set of higher order task combinators, shared distributed data stores, etc. (\cref{sec_t4t:mtasks}). Moreover, it seems that common sensor node programs are readily expressed using \gls{MTASK}. In addition to the \gls{CWTS} and \gls{CWS} systems outlined here, other case studies include Arduino examples as well as some bigger tasks~\citep{koopman_task-based_2018,lubbers_writing_2019,LubbersMIPRO}. We conclude that the programming of sensor tasks is well-supported by both \glspl{DSL}.
\section{Conclusion}%
\label{sec_t4t:Conclusion}
\item Tierless developers benefit from automatically generated, and hence correct, communication (\cref{lst_t4t:mtaskTemp}), and write 6$\times$ less communication code (\cref{fig_t4t:multipercentage}).
%and TODO).%~\ref{lst_t4t:mqtt}).
\item Tierless developers can exploit powerful high-level declarative and task-oriented \gls{IOT} programming abstractions (\cref{table_t4t:temp}), specifically the composable, higher-order task combinators outlined in \cref{sec_t4t:itasks}.
-Our empirical results for \gls{IOT} systems are consistent with the benefits claimed for tierless languages in other application domains. Namely that a tierless language provides a \textit{Higher Abstraction Level}, \textit{Improved Software Design}, and improved \textit{Program Comprehension}~\cite{weisenburger2020survey}.
+Our empirical results for \gls{IOT} systems are consistent with the benefits claimed for tierless languages in other application domains. Namely that a tierless language provides a \textit{Higher Abstraction Level}, \textit{Improved Software Design}, and improved \textit{Program Comprehension}~\citep{weisenburger2020survey}.
\end{enumerate*}
We show that \emph{tierless languages have the potential to significantly improve the reliability of \gls{IOT} systems}. We illustrate how \gls{CLEAN} maintains type safety, contrasting this with a loss of type safety in \gls{PRS}.
-We illustrate higher order failure management in \gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} in contrast to the \gls{PYTHON}-based failure management in \gls{PRS}. For maintainability a tiered approach makes replacing components easy, but refactoring within the components is far harder than in a tierless \gls{IOT} language. Again our findings are consistent with the simplied \textit{Code Maintenance} benefits claimed for tierless languages~\cite{weisenburger2020survey}.
+We illustrate higher order failure management in \gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} in contrast to the \gls{PYTHON}-based failure management in \gls{PRS}. For maintainability a tiered approach makes replacing components easy, but refactoring within the components is far harder than in a tierless \gls{IOT} language. Again our findings are consistent with the simplied \textit{Code Maintenance} benefits claimed for tierless languages~\citep{weisenburger2020survey}.
Finally, we contrast community support for the technologies (\cref{sec_t4t:Discussion}).
-%\pwtcomment{Pieter: please check discussion of ~\cite{weisenburger2020survey} in preceding 2 paragraphs}
+%\pwtcomment{Pieter: please check discussion of ~\citep{weisenburger2020survey} in preceding 2 paragraphs}
We report \emph{the first comparison of a tierless \gls{IOT} codebase for resource-rich sensor nodes with one for resource-constrained sensor nodes}.
\begin{enumerate*}
\end{enumerate*}
We present \emph{the first comparison of two tierless \gls{IOT} languages: one designed for resource-constrained sensor nodes (\gls{CLEAN} with \gls{ITASK} and \gls{MTASK}), and the other for resource-rich sensor nodes (\gls{CLEAN} with \gls{ITASK}).} \gls{CLEAN}\slash\gls{ITASK} can implement all layers of the \gls{IOT} stack if the sensor nodes have the computational resources, as the Raspberry Pis do in \gls{CRS}. On resource constrained sensor nodes \gls{MTASK} are required to implement the perception and network layers, as on the Wemos minis in \gls{CWS}. We show that a bare metal execution environment allows \gls{MTASK} to have better control of peripherals, timing and energy consumption. The memory available on a microcontroller restricts the programming abstractions available in \gls{MTASK} to a fixed set of combinators, no user defined or recursive data types, strict evaluation, and makes it harder to add new abstractions. Even with these restrictions \gls{MTASK} provide a higher level of abstraction than most bare metal languages, and can readily express many \gls{IOT} applications including the \gls{CWS} \gls{UOG} smart campus application (\cref{sec_t4t:ComparingTierless}).
-Our empirical results are consistent with the benefits of tierless languages listed in Section 2.1 of~\cite{weisenburger2020survey}.
+Our empirical results are consistent with the benefits of tierless languages listed in Section 2.1 of~\citep{weisenburger2020survey}.
\subsection{Reflections}
This study is based on a specific pair of tierless \gls{IOT} languages, and the \gls{CLEAN} language frameworks represent a specific set of tierless language design decisions. Many alternative tierless \gls{IOT} language designs are possible, and some are outlined in \cref{sec_t4t:characteristics}. Crucially the limitations of the tierless \gls{CLEAN} languages, e.g.\ that they currently provide limited security, should not be seen as limitations of tierless technologies in general.
-This study has explored some, but not all, of the potential benefits of tierless languages for \gls{IOT} systems. An \gls{IOT} system specified as a single tierless program is amenable to a host of programming language technologies. For example, if the language has a formal semantics, as Links, Hop and \gls{CLEAN} tasks do~\cite{cooper2006links,serrano2006hop,plasmeijer_task-oriented_2012}, it is possible to prove properties of the system, e.g.\ \cite{Steenvoorden2019tophat}. As another example program analyses can be applied, and \cref{sec_t4t:characteristics} and~\cite{weisenburger2020survey} outline some of the analyses could be, and in some cases have been, used to improve \gls{IOT} systems. Examples include automatic tier splitting~\cite{10.1145/2661136.2661146}, and controlling information flow to enhance security~\cite{valliappan_towards_2020}.
+This study has explored some, but not all, of the potential benefits of tierless languages for \gls{IOT} systems. An \gls{IOT} system specified as a single tierless program is amenable to a host of programming language technologies. For example, if the language has a formal semantics, as Links, Hop and \gls{CLEAN} tasks do~\citep{cooper2006links,serrano2006hop,plasmeijer_task-oriented_2012}, it is possible to prove properties of the system, e.g.\ \citep{Steenvoorden2019tophat}. As another example program analyses can be applied, and \cref{sec_t4t:characteristics} and~\citep{weisenburger2020survey} outline some of the analyses could be, and in some cases have been, used to improve \gls{IOT} systems. Examples include automatic tier splitting~\citep{10.1145/2661136.2661146}, and controlling information flow to enhance security~\citep{valliappan_towards_2020}.
While offering real benefits for \gls{IOT} systems development, tierless languages also raise some challenges. Programmers must master new tierless programming abstractions, and the semantics of these automatic multi-tier behaviours are necessarily relatively complex. In the \gls{CLEAN} context this entails becoming proficient with the \gls{ITASK} and \gls{MTASK} \glspl{DSL}. Moreover, specifying a behaviour that is not already provided by the tierless language requires either a workaround, or extending a \gls{DSL}. However, implementing the relatively simple smart campus application required no such adaption. Finally, tierless \gls{IOT} technology is very new, and both tool and community support have yet to mature.
repositories / phd-thesis.git / commitdiff