\cleaninline:{}
\pythoninline:{}
\haskellinline:{}
- \haskelllshtexinline:{}
+ \haskelllhstexinline:{}
\arduinoinline:{}
\texttt:{}
\url:{}
This thesis research has been carried out under the research data management policy of the Institute for Computing and Information Science of Radboud University, the Netherlands\footnote{\url{https://www.ru.nl/icis/research-data-management/}, last accessed \formatdate{20}{1}{2020}.}.
The following research datasets have been produced during this PhD research:
-\todo{reference correct chapters}
\begin{itemize}
- \item \ldots
- \item \rdmentry{\Cref{chp:classy_deep_embedding}}
- {\mlubbers}
- {2022}
- {Literate Haskell/lhs2TeX source code for the paper ``Deep Embedding with Class'': TFP 2022}
- {Zenodo}{10.5281/zenodo.6650880} %chktex 8
- \item \rdmentry{Chapter 0}
- {\mlubbers; \pkoopman; \rplasmeijer}
- {2021}
- {Source code for the interpreted mTask language}
- {DANS}{10.17026/dans-zrn-2wv3} %chktex 8
- \item \rdmentry{\Cref{chp:smart_campus}}
- {\mlubbers; \pkoopman; Ramsingh, A.\ (University of Glasgow); Singer, dr.\ J.\ (University of Glasgow); Trinder, prof.~dr.\ P.\ (University of Glasgow)}
- {2021}
- {Source code of the PRSS and CWSS applications}
- {DANS}{10.17026/dans-zvf-4p9m} %chktex 8
- \item \rdmentry{\Cref{prt:top}}
- {\mlubbers; \pkoopman; \rplasmeijer}
- {2020}
- {Source code for the multitasking mTask language integrated with the iTask system}
- {DANS}{10.17026/dans-x2y-rtxx} %chktex 8
- \item \rdmentry{\Cref{prt:top}}
- {\mlubbers; \pkoopman; \rplasmeijer}
- {2020}
- {Source code for a simplified mTask language integrated with the iTask system}
- {DANS}{10.17026/dans-xv6-fvxd} %chktex 8
- \item \rdmentry{\Cref{prt:top}}
- {\mlubbers; \pkoopman; \rplasmeijer}
- {2020}
- {Source code for the mTask language}
- {DANS}{10.17026/dans-xx4-8zs9} %chktex 8
+ \item \fullref{chp:classy_deep_embedding}
+ \begin{itemize}
+ \item \rdmentry{\mlubbers}{2022}
+ {Literate Haskell/lhs2\TeX{} source code for the paper ``Deep Embedding with Class'': TFP 2022}
+ {Zenodo}{10.5281/zenodo.6650880} %chktex 8
+ \item \fullref{sec:classy_reprise}:
+ \begin{itemize}
+ \item \rdmentry{\mlubbers}{2022}
+ {\todo[inline]{replace with actual name}}
+ {Zenodo}{10.5281/zenodo.7277498} %chktex 8
+ \end{itemize}
+ \end{itemize}
+ \item \fullref{chp:first-class_datatypes}:
+ \begin{itemize}
+ \item \rdmentry{\mlubbers}{2022}
+ {Code for the paper ``First-Class Data Types in Shallow Embedded Domain-Specific Languages using Metaprogramming'': IFL 2022}
+ {Zenodo}{10.5281/zenodo.6416747}
+ \end{itemize}
+ \item \fullref{prt:top}:
+ \begin{itemize}
+ \item \rdmentry{\mlubbers; \pkoopman; \rplasmeijer}{2020}
+ {Source code for the mTask language}
+ {DANS}{10.17026/dans-xx4-8zs9} %chktex 8
+ \item \rdmentry{\mlubbers; \pkoopman; \rplasmeijer}{2020}
+ {Source code for the multitasking mTask language integrated with the iTask system}
+ {DANS}{10.17026/dans-x2y-rtxx} %chktex 8
+ \item \rdmentry{\mlubbers; \pkoopman; \rplasmeijer}{2020}
+ {Source code for a simplified mTask language integrated with the iTask system}
+ {DANS}{10.17026/dans-xv6-fvxd} %chktex 8
+ \item \rdmentry{\mlubbers; \pkoopman; \rplasmeijer}{2021}
+ {Source code for the interpreted mTask language}
+ {DANS}{10.17026/dans-zrn-2wv3} %chktex 8
+ \end{itemize}
+ \item \fullref{prt:tvt}:
+ \begin{itemize}
+ \item \rdmentry{\mlubbers; \pkoopman; \aramsingh; \jsinger; \ptrinder}{2021}
+ {Source code, line counts and memory statistics for CRS, CWS, CRTS and CWTS}
+ {Zenodo}{10.5281/zenodo.5040754}
+ \item \rdmentry{\mlubbers; \pkoopman; \aramsingh; \jsinger; \ptrinder}{2021}
+ {Source code, line counts and memory stats for PRS, PWS, PRT and PWT}
+ {Zenodo}{10.5281/zenodo.5081386}
+ \item \rdmentry{\mlubbers; \pkoopman; \aramsingh; \jsinger; \ptrinder}{2021}
+ {Source code of the PRSS and CWSS applications}
+ {DANS}{10.17026/dans-zvf-4p9m} %chktex 8
+ \end{itemize}
\end{itemize}
\input{subfilepostamble}
\cleaninline:{}
\pythoninline:{}
\haskellinline:{}
- \haskelllshtexinline:{}
+ \haskelllhstexinline:{}
\arduinoinline:{}
\texttt:{}
\url:{}
Embedded \gls{DSL} techniques in functional languages have been a topic of research for many years, thus we do not claim a complete overview of related work.
Clearly, classy deep embedding bears most similarity to the \emph{Datatypes \`a la Carte}~\citep{swierstra_data_2008}.
-In Swierstra's approach, semantics are lifted to type classes similarly to classy deep embedding.
+In \citeauthor{swierstra_data_2008}'s approach, semantics are lifted to type classes similarly to classy deep embedding.
Each language construct is their own datatype parametrised by a type parameter.
This parameter contains some type level representation of language constructs that are in use.
In classy deep embedding, extensions only have to be enumerated at the type level when the term is required to be overloaded, in all other cases they are captured in the extension case.
Furthermore, I would like to thank Pieter and Rinus for the fruitful discussions, Ralf for inspiring me to write a functional pearl, and the anonymous reviewers for their valuable and honest comments.
\begin{subappendices}
-\section{Reprise: reducing boilerplate}\label{sec:classy_reprise}
+\section{Reprise: reducing boilerplate}%
+\label{sec:classy_reprise}
\todo{Improve text}
One of the unique selling points of this novel \gls{DSL} embedding technique is that it, in its basic form, does not require advanced type system extensions nor a lot of boilerplate.
However, generalising the technique to \glspl{GADT} arguably unleashes a cesspool of \emph{unsafe} compiler extensions.
-If we are willing to work with extensions, almost all of the boilerplate can be inferred or generated\footnote{The source code for this extension can be found here: \url{https://gitlab.com/mlubbers/classydeepembedding}.}.
+If we are willing to work with extensions, almost all of the boilerplate can be inferred or generated.
-The \gls{DSL} datatype is parametrised by a type variable providing a witness to the interpretation on the language.
+In classy deep embedding, the \gls{DSL} datatype is parametrised by a type variable providing a witness to the interpretation on the language.
When using multiple interpretations, these need to be bundled in a data type.
Using the \gls{GHC}'s \GHCmod{ConstraintKind} extension, we can make these witnesses explicit, tying into \gls{HASKELL}'s type system immediately.
Furthermore, this constraint does not necessarily has to be a single constraint, after enabling \GHCmod{DataKinds} and \GHCmod{TypeOperators}, we can encode lists of witnesses instead.
-The data type for this list of witnesses is \haskelllhstexinline{Record}.
-The \haskelllhstexinline{Record} \gls{GADT} is parametrised by two type variables, the first type variable (\haskelllhstexinline{dt}) is the data type on which the constraints can be applied.
-The second type variable (\haskelllhstexinline{clist}) is the list of constraints itself.
-It is not just a list of \haskelllhstexinline{Constraint} but it is a list containing constraint constructors that will, when given a type of the polymorphic kind \haskelllhstexinline{k}, produce a constraint.
-This means that when \haskelllhstexinline{Cons} is pattern matched, the type class constraint for \haskelllhstexinline{c dt} can be solved by the compiler.
-\GHCmod{KindSignatures} is used to force the kinds of the type parameters and the kind of the data type is polymorphic (\GHCmod{PolyKinds}) so that the \haskelllhstexinline{Record} data type can be used for \glspl{DSL}s using type classes but also type constructor classes (e.g.\ when using \glspl{GADT})..
+The data type for this list of witnesses is \haskelllhstexinline{Record} as shown in \cref{lst_cbde:record_type}.
+This \gls{GADT} is parametrised by two type variables.
+The first type variable (\haskelllhstexinline{dt}) is the type or type constructor on which the constraints can be applied and the second type variable (\haskelllhstexinline{clist}) is the list of constraints constructors itself.
+This means that when \haskelllhstexinline{Cons} is pattern matched, the overloading of the type class constraint for \haskelllhstexinline{c dt} can be solved by the compiler.
+\GHCmod{KindSignatures} is used to force the kinds of the type parameters and the kind of \haskelllhstexinline{dt} is polymorphic (\GHCmod{PolyKinds}) so that the \haskelllhstexinline{Record} data type can be used for \glspl{DSL} using type classes but also type constructor classes (e.g.\ when using \glspl{GADT}).
-\begin{lstHaskellLhstex}[caption={Data type for a list of constraints}]
+\begin{lstHaskellLhstex}[label={lst_cbde:record_type},caption={Data type for a list of constraints}]
data Record (dt :: k) (clist :: [k -> Constraint]) where
Nil :: Record dt '[]
Cons :: c dt => Record dt cs -> Record dt (c ': cs)
\end{lstHaskellLhstex}
-To incorporate this type in the \haskelllhstexinline{Expr} type, the \haskelllhstexinline{Ext} constructor changes as follows:
+To incorporate this type in the \haskelllhstexinline{Expr} type, the \haskelllhstexinline{Ext} constructor changes from containing a single witness dictionary to a \haskelllhstexinline{Record} type containing all the required dictionaries.
\begin{lstHaskellLhstex}[caption={Data type for a list of constraints}]
data Expr c
| Ext (Record x c) x
\end{lstHaskellLhstex}
-Furthermore, we define a type class that allows us to extract explicit dictionaries \haskelllhstexinline{Dict} from these records if the constraint can is present in the list.
+Furthermore, we define a type class (\haskelllhstexinline{In}) that allows us to extract explicit dictionaries \haskelllhstexinline{Dict} from these records if the constraint can is present in the list.
+Since the constraints become available as soon as the \haskelllhstexinline{Cons} constructor is matched, the implementation is a trivial type-level list traversal.
\begin{lstHaskellLhstex}[caption={Membership functions for constraints}]
class c `In` cs where
project (Cons xs) = project xs
\end{lstHaskellLhstex}
-Finally, creating these \haskelllhstexinline{Record} witnesses is a chore so this can be automated as well using a \haskelllhstexinline{CreateRecord} multi-parameter type class (requiring the \GHCmod{MultiParamTypeclasses} and \GHCmod{FlexibleInstances} extension).
+The final scaffolding is a multi-parameter type class \haskelllhstexinline{CreateRecord} (requiring the \GHCmod{MultiParamTypeclasses} and \GHCmod{FlexibleInstances} extension) to create these \haskelllhstexinline{Record} witnesses automatically.
This type class creates a record structure cons by cons if and only if all type class constraints are available in the list of constraints.
+It is not required to provide instances for this for specific records or type classes, the two instances describe all the required constraints.
\begin{lstHaskellLhstex}[caption={Membership functions for constraints}]
class CreateRecord dt c where
Using \haskelllhstexinline{`In`}'s \haskelllhstexinline{project} function, a dictionary can be brought into scope.
This dictionary can then subsequently be used to apply the type class function on the extension using the \haskelllhstexinline{withDict} function from the \haskelllhstexinline{Data.Constraint} library\footnote{\haskelllhstexinline{withDict :: Dict c -> (c => r) -> r}}.
The \GHCmod{ScopedTypeVariables} extension is used to make sure the existentially quantified type variable for the extension is matched to the type of the dictionary.
-Furthermore, because the class constraint is seemingly not smaller than the instance head, \GHCmod{UndecidableInstances} should be enabled.
+Furthermore, because the class constraint is not smaller than the instance head, \GHCmod{UndecidableInstances} should be enabled.
\begin{lstHaskellLhstex}[caption={Evaluation instance for the main data type}]
class Eval v where
\end{lstHaskellLhstex}
Smart constructors need to be adapted as well, as can be seen from the smart constructor \haskelllhstexinline{subst}.
+Instead of a \haskelllhstexinline{GDict} class constraint, a \haskelllhstexinline{CreateRecord} class constraint needs to be added.
\begin{lstHaskellLhstex}[caption={Substitution smart constructor}]
subst :: (Typeable c, CreateRecord (Subt c) c) => Expr c -> Expr c -> Expr c
subst l r = Ext createRecord (l `Subt` r)
\end{lstHaskellLhstex}
-Finally, defining terms in the language is can be done immediately if the interpretations are known.
-For example, if we want to print and/or optimise the term $~(~(42+(38-4)))$, we can define it as follows.
+Finally, defining terms in the language can be done immediately if the interpretations are known.
+For example, if we want to print and/or optimise the term $~(~(42+(38-4)))$, we can define it as follows:
\begin{lstHaskellLhstex}[caption={Substitution smart constructor}]
e0 :: Expr '[Print,Opt]
\end{lstHaskellLhstex}
It is also possible to define terms in the \gls{DSL} as being overloaded in the interpretation.
-This does require enumerating all the \haskelllhstexinline{CreateRecord} type classes for every extension.
+This does require enumerating all the \haskelllhstexinline{CreateRecord} type classes for every extension in a similar fashion as was required for \haskelllhstexinline{GDict}.
At the call site, the concrete list of constraints must be known.
\begin{lstHaskellLhstex}[caption={Substitution smart constructor}]
e1 :: (Typeable c, UsingExt '[Neg, Subst]) => Expr c
\end{lstHaskellLhstex}
-Giving an instance for \haskelllhstexinline{Interp} for \haskelllhstexinline{DataType} that uses the extensions \haskelllhstexinline{e_1,e2,...} and depends on interpretations \haskelllhstexinline{i_1,i_2,...} is done as follows:
+Giving an instance for \haskelllhstexinline{Interp} for \haskelllhstexinline{DataType} that uses the extensions \haskelllhstexinline{e_1, e2, ...} and depends on interpretations \haskelllhstexinline{i_1,i_2, ...} is done as follows:
\begin{lstHaskellLhstex}
instance ( UsingExt '[e_1,e_2,...] s
...
\end{lstHaskellLhstex}
+With these enhancements, there is hardly any boilerplate required to use classy deep embedding.
+The \haskelllhstexinline{Record} data type; the \haskelllhstexinline{CreateRecord} type class; and the \haskelllhstexinline{UsingExt} and \haskelllhstexinline{DependsOn} type families can be provided as a library only requiring the programmer to create the extension constructors with their respective implementations and smart constructors for language construct extensions.
+The source code for this extension can be found here: \url{https://gitlab.com/mlubbers/classydeepembedding}.
+
\section{Data types and definitions}%
\label{sec:cde:appendix}
\begin{lstHaskellLhstex}[caption={Data type definitions.}]
\begin{document}
\chapter{\texorpdfstring{\Glsxtrshort{DSL}}{DSL} embedding techniques}%
\label{chp:dsl_embedding_techniques}%
-An \gls{EDSL} is a language embedded in a host language created for a specific domain\todo{citation needed?}.
-Properties such as referential transparency, minimal syntax, powerful type systems and rich data types make \gls{FP} languages excellent candidates for hosting \glspl{EDSL}.
-
-There are two flavours of \gls{DSL} embedding: deep- and shallow embedding~\citep{boulton_experience_1992}.
-Shallow embedding---also called tagless embedding---models language constructs as functions in the host language.
-As a result, adding new language constructs---extra functions---is easy.
-However, the interpretation of the language is embedded in these functions, making it troublesome to add semantics since it requires updating all existing language constructs.
-
-In contrast to shallow embedding, deep embedding---also called tagged embedding---models terms in the language as data types.
-Interpretations are functions over these data types.
-
-Consequently, adding new semantics, i.e.\ novel functions, is straightforward.
-It can be stated that the language constructs are embedded in the functions that form a semantics.
-If one wants to add a language construct, all semantics functions must be revisited and revised to avoid ending up with partial functions.
-
-This juxtaposition has been known for many years~\citep{reynolds_user-defined_1978} and discussed by many others~\citep{krishnamurthi_synthesizing_1998} but most famously dubbed the \emph{expression problem} by Wadler~\citep{wadler_expression_1998}:
-
-\begin{quote}
- The \emph{expression problem} is a new name for an old problem.
- The goal is to define a data type by cases, where one can add new cases to the data type and new functions over the data type, without recompiling existing code, and while retaining static type safety (e.g., no casts).
-\end{quote}
-
-Terms in an \glspl{EDSL} can have multiple interpretations\footnote{Interpretations are also called backends or views}, i.e.\ a term in the \gls{DSL} is just an interface.
-Commonly used intepretations are printing, compiling, simulating, optimising, verifying, proving the program\etc.
-There are two main flavours of embedding \glspl{DSL}.
-Deep embedding---also called shallow--- models terms in the language as data types, interpretations are functions over these terms.
-Shallow embedding---also called tagless---models terms in the language as functions, interpretations are embedded in these functions.
-
-Most importantly, the two flavours differ on two axes: extensibility of language constructs and extensibility of interpretations.
-\todo{elaborate}
-
-\Cref{sec:deep_embedding} shows the basics of deep embedding.
-\Cref{sec:shallow_embedding} shows the basics of shallow embedding including tagless embedding.
-\Cref{sec:compare_embedding} compares the embedding technique.
-
-In the following sections the basics of both techniques are explained.
-A simple language with integers, booleans and some arithmetic operators is used as a running example.
+\begin{chapterabstract}
+ An \gls{EDSL} is a language embedded in a host language created for a specific domain\todo{citation needed?}.
+ Properties such as referential transparency, minimal syntax, powerful type systems and rich data types make \gls{FP} languages excellent candidates for hosting \glspl{EDSL}.
+ Terms in an \glspl{EDSL} can have multiple interpretations\footnote{Interpretations are also called backends or views}, i.e.\ a term in the \gls{DSL} is just an interface.
+ Commonly used intepretations are printing, compiling, simulating, optimising, verifying, proving the program\etc.
+
+ There are two flavours of \gls{DSL} embedding: deep- and shallow embedding~\citep{boulton_experience_1992}.
+ Shallow or tagless embedding models language constructs as functions in the host language.
+ As a result, adding new language constructs---extra functions---is easy.
+ However, the interpretation of the language is embedded in these functions, making it troublesome to add semantics since it requires updating all existing language constructs.
+
+ In contrast to shallow embedding, deep embedding or tagged models terms in the language as data types.
+ Interpretations are functions over these data types.
+
+ Consequently, adding new semantics, i.e.\ novel functions, is straightforward.
+ It can be stated that the language constructs are embedded in the functions that form a semantics.
+ If one wants to add a language construct, all semantics functions must be revisited and revised to avoid ending up with partial functions.
+
+ This juxtaposition has been known for many years~\citep{reynolds_user-defined_1978} and discussed by many others~\citep{krishnamurthi_synthesizing_1998} but most famously dubbed the \emph{expression problem} by \citet{wadler_expression_1998}:
+
+ \begin{quote}
+ The \emph{expression problem} is a new name for an old problem.
+ The goal is to define a data type by cases, where one can add new cases to the data type and new functions over the data type, without recompiling existing code, and while retaining static type safety (e.g., no casts).
+ \end{quote}
+
+% Most importantly, the two flavours differ on two axes: extensibility of language constructs and extensibility of interpretations.
+% \todo{elaborate}
+
+ Using a simple language with integers, booleans and some arithmetic operators, \cref{sec:deep_embedding} shows some deep embedding variants and \cref{sec:shallow_embedding} shows the relevant shallow embedding variants.
+ \Cref{sec:compare_embedding} compares the embedding techniques.
+\end{chapterabstract}
\section{Deep embedding}\label{sec:deep_embedding}
-In a deeply embedded \gls{DSL}, the language terms are represented as data type{(s)} in the host language.
+In a deeply embedded \gls{DSL}, the language terms are represented as data types in the host language.
Therefore, interpretations of the terms are functions that operate on these data types.
\Cref{lst:exdeep} shows an implementation for the example \gls{DSL}.
\end{lstHaskell}
Adding a construct---for example subtraction---reveals the Achilles' heel of deep embedding, namely that we need to revisit the original data type \emph{and} all the existing views.
-I.e.\ we need to add \haskellinline{| Sub Expr Expr} to the \haskellinline{Expr} data type.
+I.e.\ we need to add \haskellinline{\| Sub Expr Expr} to the \haskellinline{Expr} data type.
Furthermore, we need to add \haskellinline{print (Sub l r) = ...} to the \haskellinline{print} view in order to not end up with a partial function.
-This limitation can be overcome by lifting the views to classes (See \cref{chp:classy_deep_embedding}).
+Using a novel variant of deep embedding, this limitation can be overcome by lifting the views to classes (see \cref{chp:classy_deep_embedding}).
-Implementing an evaluator for the language is possible without touching any original code, we just add a function operating on the \haskellinline{Expr} data type.
-To store variables, it has an extra environment argument.
+Implementing an extra view, an evaluator as shown in \cref{lst:deep_simple}, for the language is possible without touching any original code, we just add a function operating on the \haskellinline{Expr} data type.
Here another downside of basic deep embedding arises immediately, the expressions are not typed, and therefore there has to be some type checking in the evaluation code.
Luckily this problem can be overcome by switching from regular \glspl{ADT} to \glspl{GADT}, resulting in the following data type and evaluator.
-\begin{lstHaskell}[caption={An evaluator for the deeply embedded expression \gls{DSL}.}]
+\begin{lstHaskell}[label={lst:deep_simple},caption={An evaluator for the deeply embedded expression \gls{DSL}.}]
eval :: Expr -> Value
eval (Lit i) = i
eval (Plus l r) = case (eval l, eval r) of
(Lit (I l), Lit (I r)) -> I (l+r))
- (l, r) -> error ("Can't add " ++ show l ++ " to " ++ show r)
+ (l, r) -> error ("Can't add " ++ show l ++ " to " ++ show r)
eval (Eq l r) = case (eval l, eval r) of
(Lit (I l), Lit (I r)) -> B (l==r)
(Lit (B l), Lit (B r)) -> B (l==r)
- (l, r) -> error ("Can't compare " ++ show l ++ " to " ++ show r)
+ (l, r) -> error ("Can't compare " ++ show l ++ " to " ++ show r)
\end{lstHaskell}
\subsection{\texorpdfstring{\Glsxtrlongpl{GADT}}{Generalised algebraic data types}}
Deep embedding has the advantage that it is easy to build and views are easy to add.
On the downside, the expressions created with this language are not necessarily type-safe.
In the given language it is possible to create an expression such as \haskellinline{LitI 4 `Plus` LitB True} that adds a boolean to an integer.
-Extending the \gls{ADT} is easy and convenient but extending the views accordingly is tedious since it has to be done individually for all views.
-
-The first downside of this type of \gls{EDSL} can be overcome by using \glspl{GADT}~\citep{cheney_first-class_2003}.
-\Cref{lst:exdeepgadt} shows the same language, but type-safe with a \gls{GADT}.
-\glspl{GADT} are not supported in the current version of \gls{CLEAN} and therefore the syntax is hypothetical (See \todo{insert link to appendix}).
-However, it has been shown that \glspl{GADT} can be simulated using bimaps or projection pairs~\citep[\citesection{2.2}]{cheney_lightweight_2002}.
-Unfortunately the lack of extendability remains a problem.
-If a language construct is added, no compile time guarantee can be given that all views support it.
+This downside of the \gls{EDSL} technique can be overcome by using \glspl{GADT} instead of \glspl{ADT}~\citep{cheney_first-class_2003}.
+Even if the host language does not support \glspl{GADT}, it has been shown that they can be simulated using bimaps or projection pairs~\citep[\citesection{2.2}]{cheney_lightweight_2002}.
+\Cref{lst:exdeepgadt} shows the same language, but made type-safe with a \gls{GADT}.
+The data types are annotated with a type variable representing the type of the expression.
+This restriction makes the evaluator's implementation much more concise.
+For the printer, the implementation can even remain the same.
+Only the type signature needs an update.
\begin{lstHaskell}[label={lst:exdeepgadt},caption={A deeply embedded expression \gls{DSL} using \glspl{GADT}.}]
data Expr a where
eval (Lit i) = i
eval (Plus l r) = eval l + eval r
eval (Eq l r) = eval l == eval r
+
+print :: Expr a -> String
+...
\end{lstHaskell}
\section{Shallow embedding}\label{sec:shallow_embedding}
-In a shallowly \gls{EDSL} all language constructs are expressed as functions in the host language.
+In a shallowly embedded \gls{DSL} the language constructs are expressed as functions in the host language.
An evaluator view for the example language then can be implemented as the code shown in \cref{lst:exshallow}.
-Note that the internals of the language could have been hidden using a reader monad.
\begin{lstHaskell}[label={lst:exshallow}, caption={A minimal shallow \gls{EDSL}.}]
-type Env = String -> Int
-type DSL a = Env -> a
-
-lit :: a -> DSL a
-lit x = \e->x
+data Eval a = Eval {runEval :: a}
-var :: String -> DSL Int
-var i = \e->retrEnv e i
+lit :: a -> Eval a
+lit x = Eval x
-plus :: DSL Int -> DSL Int -> DSL Int
-plus x y = \e->x e + y e
+plus :: Num a => Eval a -> Eval a -> Eval a
+plus x y = Eval (runEval x + runEval y)
-eq :: Eq a => DSL a -> DSL a -> DSL Bool
-eq x y = \e->x e == y e
+eq :: Eq a => Eval a -> Eval a -> Eval Bool
+eq x y = Eval (runEval x == runEval y)
\end{lstHaskell}
One of the advantages of shallowly embedding a language in a host language is its extendability.
For example, adding a new construct---such as subtraction---is done as follows:
\begin{lstHaskell}[label={lst:exshallowsubst},caption={Adding subtraction to the shallow \gls{EDSL}.}]
-sub :: DSL Int -> DSL Int -> DSL Int
-sub x y = \e->x e - y e
+sub :: Num a => Eval a -> Eval a -> Eval a
+sub x y = Eval (runEval x - runEval y)
\end{lstHaskell}
Moreover, the language is type safe as it is directly typed in the host language, i.e.\ \haskellinline{lit True `plus` lit 4} is rejected.
Another advantage is the intimate link with the host language, allowing for a lot more linguistic reuse such as the support of implicit sharing \citep{krishnamurthi_linguistic_2001}.
The downside of this method is extending the language with views.
-It is nearly impossible to add views to a shallowly embedded language.
+Since the views are embedded, adding a view requires embedding combining the two views in some way.
The only way of achieving this is by reimplementing all functions so that they run all backends at the same time or to create a single interpretation that produces a fold function \citep{gibbons_folding_2014}.
\subsection{Tagless-final embedding}\label{ssec:tagless}
By lifting the functions representing the \gls{DSL} terms to type classes, interpretations can be added.
-This technique is called tagless-final---or class-based shallow---embedding.
+This technique is called tagless-final---or class-based shallow---embedding~\citep{carette_finally_2009}.
The interface for the \gls{DSL} looks as follows:
\begin{lstHaskell}[label={lst:extagless},caption={A minimal tagless-final \gls{EDSL}.}]
class DSL v where
- lit :: a -> v a
- var :: String -> v a
- plus :: v Int -> v Int -> v Int
- eq :: Eq a => v a -> v a -> v Bool
+ lit :: Show a => a -> v a
+ plus :: Num a => v a -> v a -> v a
+ eq :: Eq a => v a -> v a -> v Bool
\end{lstHaskell}
An interpretation of this view is a data type that implements the type class.
data Print a = P {runPrint :: String}
instance DSL Print where
lit a = P (show a)
- var i = P i
plus x y = P ("(" ++ runPrint x ++ "+" ++ runPrint y ++ ")"
eq x y = P ("(" ++ runPrint x ++ "==" ++ runPrint y ++ ")"
\end{lstHaskell}
\begin{lstHaskell}[label={lst:extaglesssubt},caption={Adding subtraction to the shallow \gls{EDSL}.}]
class Sub v where
- sub :: v Int -> v Int -> v Int
+ sub :: Num a => v a -> v a -> v a
instance Sub Print where
sub x y = P ("(" ++ runPrint x ++ "-" ++ runPrint y ++ ")"
Adding an interpretation means adding a data type and providing instances for the language constructs.
\begin{lstHaskell}[label={lst:extaglesseval},caption={An evaluator interpretation of the minimal tagless-final \gls{EDSL}.}]
-data Eval a = Eval {runEval :: Env -> a}
+data Eval a = Eval {runEval :: a}
instance DSL v where
- lit a = Eval (\_->a)
- var i = Eval (\e->retrEnv e i)
- plus x y = Eval (\e->runEval x e + runEval y e)
- eq x y = Eval (\e->runEval x e == runEval y e)
+ lit a = Eval a
+ plus x y = Eval (runEval x + runEval y)
+ eq x y = Eval (runEval x == runEval y)
instance Sub Eval where
- sub x y = Eval (\e->runEval x e - runEval y e)
+ sub x y = Eval (runEval x - runEval y)
\end{lstHaskell}
+\section{Other embedding techniques}\label{sec:other}
\section{Comparison}\label{sec:compare_embedding}
-Both flavours have their strengths and weaknesses and both flavours can be improved in order to mitigate (some of the) downsides.
+Both flavours have their strengths and weaknesses.
+As we have seen with \glspl{GADT} for deep embedding and tagless-final for shallow embedding, both flavours can be improved in some way in order to mitigate (some of the) downsides.
+Besides deep and shallow embedding, there are some more advanced variants that try to mitigate some of the problems.
\begin{table}[ht]
\begin{threeparttable}[b]
\label{tbl:dsl_comparison}
\begin{tabular}{lllllll}
\toprule
- & Shallow & Deep & Hybrid & Poly & Comp. & Classy\\
+ & Shallow & Deep & Hybrid & Poly & Comp. & Classy\\
\midrule
Transcoding free & yes & yes & no & yes & yes & yes\\
Linguistic reuse & yes & no & partly\tnote{1} & yes & yes & no?\\
\chapter{First-class data types in shallow \glsxtrlongpl{EDSL} using metaprogramming}%
\label{chp:first-class_datatypes}%
\begin{chapterabstract}
- \Gls{FP} languages are excellent candidates for hosting \glspl{EDSL} because of their rich type systems, minimal syntax, and referential transparency.
+ \Gls{FP} languages are excellent for hosting \glspl{EDSL} because of their rich type systems, minimal syntax, and referential transparency.
However, data types defined in the host language are not automatically available in the embedded language.
To do so, all the operations on the data type must be ported to the \gls{EDSL} resulting in a lot of boilerplate.
\subsection{Functions}
Adding functions to the language is achieved by adding a multi-parameter class to the \gls{DSL}.
The type of the class function allows for the implementation to only allow first order function by supplying the arguments in a tuple.
-Furthermore, by defining the \haskellinline{Main} type, the \gls{DSL} forces all functions to be defined at the top level and with the \haskellinline{:-} operator the syntax becomes usable.
+Furthermore, with the \haskellinline{:-} operator the syntax becomes usable.
Finally, by defining the functions as a \gls{HOAS} type safety is achieved~\citep{pfenning_higher-order_1988,chlipala_parametric_2008}.
The complete definition looks as follows:
\begin{lstHaskell}
class Function a v where
- fun :: ( (a -> v s) -> In (a -> v s) (Main (v u)) ) -> Main (v u)
-newtype Main a = Main { unmain :: a }
+ fun :: ((a -> v s) -> In (a -> v s) (v u)) -> v u
data In a b = a :- b
infix 1 :-
\end{lstHaskell}
\begin{lstHaskell}
fun \increment-> (\x ->x +. lit 1)
:- fun \divide-> (\(x, y)->x /. y )
-:- Main (increment (divide (lit 38, lit 5)))
+:- increment (divide (lit 38, lit 5))
\end{lstHaskell}
The interpreter only requires one instance of the \haskellinline{Function} class that works for any argument type.
\begin{lstHaskell}
instance Function a Maybe where
- fun def = Main (let g :- m = def g in unmain m)
+ fun def = let g :- m = def g in m
\end{lstHaskell}
The given \haskellinline{Printer} type is not sufficient to implement the instances for the \haskellinline{Function} class, it must be possible to generate fresh function names.
\begin{lstHaskell}
instance Function () Printer where ...
instance Function (Printer a) Printer where ...
- fun def = Main $ freshLabel >>= \f->
+ fun def = freshLabel >>= \f->
let g :- m = def $ \a0->const undefined
<$> (tell ["f", show f, " ("]
>> a0 >> tell [")"])
in tell ["let f", f, " a0 = "]
>> g (const undefined <$> tell ["a0"])
- >> tell [" in "] >> unmain m
+ >> tell [" in "] >> m
instance Function (Printer a, Printer b) Printer where ...
\end{lstHaskell}
\footnotetext{\usebox{\LstBox}}
\begin{lstHaskell}
-program :: (ListDSL v, Support v, ...) => Main (v Int)
+program :: (ListDSL v, Support v, ...) => v Int
program
= fun \sum->(\l->if'(isNil l)
(lit 0)
(unCons l (\hd tl->hd +. sum tl)))
- :- Main (sum (cons (lit 38) (cons (lit 4) nil)))
+ :- sum (cons (lit 38) (cons (lit 4) nil))
\end{lstHaskell}
A similar \gls{HASKELL} implementation is much more elegant and less cluttered because of the support for pattern matching.
As a byproduct, syntactic cruft such as the special symbols for the operators and calls to \haskellinline{lit} can be removed as well resulting in the following summation implementation:
\begin{lstHaskell}
-program :: (ListDSL v, DSL v, ...) => Main (v Int)
+program :: (ListDSL v, DSL v, ...) => v Int
program
= fun \sum->(\l->[dsl|case l of
Cons hd tl -> hd + sum tl
Nil -> 0|])
- :- Main (sum (cons (lit 38) (cons (lit 4) nil)))
+ :- sum (cons (lit 38) (cons (lit 4) nil))
\end{lstHaskell}
\section{Related work}
This does not mean that generic programming is not useable for embedding pattern matches.
In generic programming, types are represented as sums of products and using this representation it is possible to define pattern matching functions.
-For example, Rhiger showed a method for expressing statically typed pattern matching using typed higher-order functions~\citep{rhiger_type-safe_2009}.
+For example, \citet{rhiger_type-safe_2009} showed a method for expressing statically typed pattern matching using typed higher-order functions.
If not the host language but the \gls{DSL} contains higher order functions, the same technique could be applied to port pattern matching to \glspl{DSL} though using an explicit sums of products representation.
-Atkey et al.\ describe embedding pattern matching in a \gls{DSL} by giving patterns an explicit representation in the \gls{DSL} by using pairs, sums and injections~\citep[\citesection{3.3}]{atkey_unembedding_2009}.
+\Citeauthor{atkey_unembedding_2009} describe embedding pattern matching in a \gls{DSL} by giving patterns an explicit representation in the \gls{DSL} by using pairs, sums and injections~\citep[\citesection{3.3}]{atkey_unembedding_2009}.
-McDonell et al.\ extends on this idea, resulting in a very similar but different solution to ours~\citep{mcdonell_embedded_2022}.
-They used the technique that Atkey et al.\ showed and applied it to deep embedding using the concrete syntax of the host language.
+\Citet{mcdonell_embedded_2022} extends on this idea, resulting in a very similar but different solution to ours.
+They used the technique that \citeauthor{atkey_unembedding_2009} showed and applied it to deep embedding using the concrete syntax of the host language.
The scaffolding---e.g.\ generating the pairs, sums and injections---for embedding is automated using generics but the required pattern synonyms are generated using \gls{TH}.
The key difference to our approach is that we specialise the implementation for each of the backends instead of providing a general implementation of data type handling operations.
Furthermore, our implementation does not require a generic function to trace all constructors, resulting in problems with (mutual) recursion.
-Young et al.\ added pattern matching to a deeply \gls{EDSL} using a compiler plugin~\citep{young_adding_2021}.
+\Citet{young_adding_2021} added pattern matching to a deeply \gls{EDSL} using a compiler plugin.
This plugin implements an \haskellinline{externalise :: a -> E a} function that allows lifting all machinery required for pattern matching automatically from the host language to the \gls{DSL}.
Under the hood, this function translates the pattern match to constructors, deconstructors, and constructor predicates.
The main difference with this work is that it requires a compiler plugin while our metaprogramming approach works on any compiler supporting a metaprogramming system similar to \gls{TH}.
-\subsection{Related work on \gls{TH}}
+\subsection{Related work on \texorpdfstring{\glsxtrlong{TH}}{Template Haskell}}
Metaprogramming in general is a very broad research topic and has been around for years already.
We therefore do not claim an exhaustive overview of related work on all aspects of metaprogramming.
However, we have have tried to present most research on metaprogramming in \gls{TH}.
-Czarnecki et al.\ provide a more detailed comparison of different metaprogramming techniques.
-They compare staged interpreters, metaprogramming and templating by comparing MetaOCaml, \gls{TH} and C++ templates~\citep{czarnecki_dsl_2004}.
+\Citet{czarnecki_dsl_2004} provide a more detailed comparison of different metaprogramming techniques.
+They compare staged interpreters, metaprogramming and templating by comparing MetaOCaml, \gls{TH} and \gls{CPP} templates.
\gls{TH} has been used to implement related work.
They all differ slightly in functionality from our domain and can be divided into several categories.
\subsubsection{Generating extra code}
Using \gls{TH} or other metaprogramming systems it is possible to add extra code to your program.
The original \gls{TH} paper showed that it is possible to create variadic functions such as \haskellinline{printf} using \gls{TH} that would be almost impossible to define without~\citep{sheard_template_2002}.
-Hammond et al.\ used \gls{TH} to generate parallel programming skeletons~\citep{hammond_automatic_2003}.
+\Citet{hammond_automatic_2003} used \gls{TH} to generate parallel programming skeletons.
In practise, this means that the programmer selects a skeleton and, at compile time, the code is massaged to suit the pattern and information about the environment is inlined for optimisation.
-Polak et al.\ implemented automatic GUI generation using \gls{TH}~\citep{polak_automatic_2006}.
-Dureg\aa{}rd et al.\ wrote a parser generator using \gls{TH} and the custom quasiquoting facilities~\citep{duregard_embedded_2011}.
+\Citet{polak_automatic_2006} implemented automatic GUI generation using \gls{TH}.
+\Citet{duregard_embedded_2011} wrote a parser generator using \gls{TH} and the custom quasiquoting facilities.
From a specification of the grammar, given in verbatim using a custom quasiquoter, a parser is generated at compile time.
-Shioda et al.\ used metaprogramming in the D programming language to create a \gls{DSL} toolkit~\citep{shioda_libdsl_2014}.
+\Citet{shioda_libdsl_2014} used metaprogramming in the D programming language to create a \gls{DSL} toolkit.
They also programmatically generate parsers and a backend for either compiling or interpreting the \gls{IR}.
-Blanchette et al.\ use \gls{TH} to simplify the development of Liquid \gls{HASKELL} proofs~\citep{blanchette_liquid_2022}.
-Folmer et al.\ used \gls{TH} to synthesize C$\lambda$aSH~\citep{baaij_digital_2015} abstract syntax trees to be processed~\citep{folmer_high-level_2022}.
-In similar fashion, Materzok used \gls{TH} to translate YieldFSM programs to C$\lambda$aSH~\citep{materzok_generating_2022}.
+\Citet{blanchette_liquid_2022} use \gls{TH} to simplify the development of Liquid \gls{HASKELL} proofs.
+\Citet{folmer_high-level_2022} used \gls{TH} to synthesize C$\lambda$aSH~\citep{baaij_digital_2015} abstract syntax trees to be processed.
+In similar fashion, \citet{materzok_generating_2022} used \gls{TH} to translate YieldFSM programs to {C$\lambda$aSH}.
\subsubsection{Optimisation}
Besides generating code, it is also possible to analyse existing code and perform optimisations.
Yet, this is dangerous territory because unwantedly the semantics of the optimised program may be slightly different from the original program.
-For example, Lynagh implemented various optimisations in \gls{TH} such as automatic loop unrolling~\citep{lynagh_unrolling_2003}.
+For example, \citet{lynagh_unrolling_2003} implemented various optimisations in \gls{TH} such as automatic loop unrolling.
The compile-time executed functions analyse the recursive function and unroll the recursion to a fixed depth to trade execution speed for program space.
-Also, O'Donnell embedded Hydra, a hardware description language, in \gls{HASKELL} utilising \gls{TH}~\citep{odonnell_embedding_2004}.
+Also, \citet{odonnell_embedding_2004} embedded Hydra, a hardware description language, in \gls{HASKELL} utilising \gls{TH}.
Using intensional analysis of the AST, it detects cycles by labelling nodes automatically so that it can generate \emph{netlists}.
The authors mention that alternatively this could have be done using a monad but this hampers equational reasoning greatly, which is a key property of Hydra.
-Finally, Viera et al.\ present a way of embedding attribute grammars in \gls{HASKELL} in a staged fashion~\citep{viera_staged_2018}.
+Finally, \citet{viera_staged_2018} present a way of embedding attribute grammars in \gls{HASKELL} in a staged fashion.
Checking several aspects of the grammar is done at compile time using \gls{TH} while other safety checks are performed at runtime.
\subsubsection{Compiler extension}
Sometimes, expressing certain functionalities in the host languages requires a lot of boilerplate, syntax wrestling, or other pains.
Metaprogramming can relieve some of this stress by performing this translation to core constructs automatically.
For example, implementing generic---or polytypic--- functions in the compiler is a major effort.
-Norell et al.\ used \gls{TH} to implement the machinery required to implement generic functions at compile time~\citep{norell_prototyping_2004}.
-Adams et al.\ also explores implementing generic programming using \gls{TH} to speed things up considerably compared to regular generic programming~\citep{adams_template_2012}.
-Clifton et al.\ use \gls{TH} with a custom quasiquoter to offer skeletons for workflows and embed foreign function interfaces in a \gls{DSL}~\citep{clifton-everest_embedding_2014}.
-Eisenberg et al.\ showed that it is possible to programmatically lift some functions from the function domain to the type domain at compile time, i.e.\ type families~\citep{eisenberg_promoting_2014}.
-Furthermore, Seefried et al.\ argued that it is difficult to do some optimisations in \glspl{EDSL} and that metaprogramming can be of use there~\citep{seefried_optimising_2004}.
+\Citet{norell_prototyping_2004} used \gls{TH} to implement the machinery required to implement generic functions at compile time.
+\Citet{adams_template_2012} also explores implementing generic programming using \gls{TH} to speed things up considerably compared to regular generic programming.
+\Citet{clifton-everest_embedding_2014} use \gls{TH} with a custom quasiquoter to offer skeletons for workflows and embed foreign function interfaces in a \gls{DSL}.
+\Citet{eisenberg_promoting_2014} showed that it is possible to programmatically lift some functions from the function domain to the type domain at compile time, i.e.\ type families.
+Furthermore, \citet{seefried_optimising_2004} argued that it is difficult to do some optimisations in \glspl{EDSL} and that metaprogramming can be of use there.
They use \gls{TH} to change all types to unboxed types, unroll loops to a certain depth and replace some expressions by equivalent more efficient ones.
-Torrano et al.\ showed that it is possible to use \gls{TH} to perform a strictness analysis and perform let-to-case translation~\citep{torrano_strictness_2005}.
+\Citet{torrano_strictness_2005} showed that it is possible to use \gls{TH} to perform a strictness analysis and perform let-to-case translation.
Both applications are examples of compiler extensions that can be implemented using \gls{TH}.
-Another example of such a compiler extension is shown by Gill et al.~\citep{gill_haskell_2009}.
+Another example of such a compiler extension is shown by \citet{gill_haskell_2009}.
They created a meta level \gls{DSL} to describe rewrite rules on \gls{HASKELL} syntax that are applied on the source code at compile time.
\subsubsection{Quasiquotation}
By means of quasiquotation, the host language syntax that usually seeps through the embedding can be hidden.
The original \gls{TH} quasiquotation paper~\citep{mainland_why_2007} shows how this can be done for regular expressions, not only resulting in a nicer syntax but syntax errors are also lifted to compile time instead of run time.
-Also, Kariotis et al.\ used \gls{TH} to automatically construct monad stacks without having to resort to the monad transformers library which requires advanced type system extensions~\citep{kariotis_making_2008}.
+Also, \citet{kariotis_making_2008} used \gls{TH} to automatically construct monad stacks without having to resort to the monad transformers library which requires advanced type system extensions.
-Najd uses the compile time to be able to do normalisation for a \gls{DSL}, dubbing it \glspl{QDSL}~\citep{najd_everything_2016}.
+\Citet{najd_everything_2016} uses the compile time to be able to do normalisation for a \gls{DSL}, dubbing it \glspl{QDSL}.
They utilise the quasiquation facilities of \gls{TH} to convert \gls{HASKELL} \gls{DSL} code to constructs in the \gls{DSL}, applying optimisations such as eliminating lambda abstractions and function applications along the way.
-Egi et al.\ extended \gls{HASKELL} to support non-free data type pattern matching---i.e.\ data type with no standard form, e.g.\ sets, graphs---using \gls{TH}~\citep{egi_embedding_2022}.
+\Citet{egi_embedding_2022} extended \gls{HASKELL} to support non-free data type pattern matching---i.e.\ data type with no standard form, e.g.\ sets, graphs---using \gls{TH}.
Using quasiquotation, they make a complicated embedding of non-linear pattern matching available through a simple lens.
-\subsubsection{\gls{TTH}}\label{ssec_fcd:typed_template_haskell}
+\subsubsection{\texorpdfstring{\glsxtrlong{TTH}}{Typed Template Haskell}}\label{ssec_fcd:typed_template_haskell}
\gls{TTH} is a very recent extension/alternative to normal \gls{TH}~\citep{pickering_multi-stage_2019,xie_staging_2022}.
Where in \gls{TH} you can manipulate arbitrary parts of the syntax tree, add top-level splices of data types, definitions and functions, in \gls{TTH} the programmer can only splice expressions but the abstract syntax tree fragments representing the expressions are well-typed by construction instead of untyped.
-Pickering et al.\ implemented staged compilation for the \emph{generics-sop}~\citep{de_vries_true_2014} generics library to improve the efficiency of the code using \gls{TTH}~\citep{pickering_staged_2020}.
-Willis et al.\ used \gls{TTH} to remove the overhead of parsing combinators~\citep{willis_staged_2020}.
+\Citet{pickering_staged_2020} implemented staged compilation for the \emph{generics-sop}~\citep{de_vries_true_2014} generics library to improve the efficiency of the code using \gls{TTH}.
+\Citet{willis_staged_2020} used \gls{TTH} to remove the overhead of parsing combinators.
\section{Discussion}
+This paper aims to be twofold, first, it shows how to inherit data types in a \gls{DSL} as first-class citizens by generating the boilerplate at compile time using \gls{TH}.
+Secondly, it introduces the reader to \gls{TH} by giving an overview of the literature in which \gls{TH} is used and provides a gentle introduction by explaining the case study.
+
\Gls{FP} languages are especially suitable for embedding \glspl{DSL} but adding user-defined data types is still an issue.
The tagless-final style of embedding offers great modularity, extensibility and flexibility.
However, user-defined data types are awkward to handle because the built-in operations on them---construction, deconstruction and constructor tests---are not inherited from the host language.
-By giving an implementation, we showed how to create a \gls{TH} function that will splice the required class definitions and view instances.
+We showed how to create a \gls{TH} function that will splice the required class definitions and view instances.
The code dataset also contains an implementation for defining field selectors and provides an implementation for a compiler (see \cref{chp:research_data_management}).
Furthermore, by writing a custom quasiquoter, pattern matches in natural syntax can be automatically converted to the internal representation of the \gls{DSL}, thus removing the syntax burden of the facilities.
The use of a custom quasiquoter does require the \gls{DSL} programmer to write a parser for their \gls{DSL}, i.e.\ the parser is not inherited from the host language as is often the case in an embedded \gls{DSL}.
H.P. (Howard) Lovecraft
}
\epigraph{%
- \selectlanguage{russian}
- Рукописи не горят\\
- \selectlanguage{british}
- (Manuscripts don't burn)
+ \selectlanguage{russian}Рукописи не горят\\
+ \selectlanguage{british} (Manuscripts don't burn)
}{%
- \selectlanguage{russian}
- М.А. (Mихаил) Булгаков\\
- \selectlanguage{british}
- (M.A. (Mikhail) Bulgakov)
+ \selectlanguage{russian}М.А.\ (Mихаил) Булгаков\\
+ \selectlanguage{british} (M.A.\ (Mikhail) Bulgakov)
}
\epigraph{%
You start a question, and it's like starting a stone.
\item ISBN:\ 111{-}11{-}11111{-}11{-}1
\item Copyright \copyright{} Mart Lubbers, 2023
\item \href{https://martlubbers.net}{martlubbers.net}
- \item This work is licensed under the Creative Commons Attribution-NoDerivatives 4.0 International License.
+ \item \small This work is licensed under the Creative Commons Attribution-NoDerivatives 4.0 International License.
To view a copy of this license, visit \url{http://creativecommons.org/licenses/by-nd/4.0/} or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, {USA}.
\item
\includegraphics[scale=.5]{cc}%
\cleaninline:{}
\pythoninline:{}
\haskellinline:{}
- \haskelllshtexinline:{}
+ \haskelllhstexinline:{}
\arduinoinline:{}
\texttt:{}
\url:{}
\begin{document}
\chapter{Introduction}%
\label{chp:introduction}
-
\begin{chapterabstract}
- The sheer number of connected devices around us is mind boggling and seems increases exponentially for many years.
- In 2022, there is an estimated number of 13.4 billion of devices connected that sense, act or otherwise interact with people and the physical world surrounding us\footnote{\url{https://transformainsights.com/research/tam/market}, accessed on: \formatdate{2022}{10}{13}}.
- These devices, together with all the scaffolding and integration such as the various networks providing the communication, (cloud) computers realising the back end or administration and the devices in our pockets providing us with a view on the system are called the \gls{IOT}.
- \Gls{IOT} systems can be seen as layered systems, where each layer is powered by different types of computers; programming languages and even programming paradigms.
- This thesis shows a novel way of orchestrating these brobdingnagian systems using the \gls{TOP} paradigm.
- It does so by giving a proof-of-concept implementation for a \gls{TOP} system specifically designed for the \gls{IOT}: \gls{MTASK}.
- At the core of the \gls{MTASK} system is a \gls{DSL}, embedded in the general purpose \gls{TOP} system \gls{ITASK}.
- Using the \gls{MTASK} system, all layers of an \gls{IOT} system can be programmed from a single declarative specification.
+
+ \noindent%
+ \begin{itemize}
+ \item How many devices are there?
+ \begin{itemize}
+ \item Number of devices is big\footnote{\url{https://transformainsights.com/research/tam/market}, accessed on: \formatdate{2022}{10}{13}}
+ \item It only grows
+ \item they are powered by software
+ \item These devices live in layered systems
+ \end{itemize}
+ \item What is the {IoT}?
+ \begin{itemize}
+ \item IoT is such a system
+ \item Layers, device layer
+ \end{itemize}
+ \item What is impedance mismatch/semantic friction?
+ \begin{itemize}
+ \item heterogeneous between layers
+ \item heterogeneous on the device/edge/perception layer
+ \item Results in problems in software
+ \item We see this also in web systems
+ \end{itemize}
+ \item What is TOP?\todo{hier al TOP uitleggen in \`e\`en zin? of alleen benoemen}
+ \begin{itemize}
+ \item declarative workflow language (partiture AND conductor)
+ \item iTask for distributed web applications.
+ \end{itemize}
+ \item This thesis: how to orchestrate this concerto of devices?
+ \begin{itemize}
+ \item Embedded devices require special code (different clef/key)
+ \item DSL is a special language in a language to facilitate this
+ \item mTask is a TOP language for IoT
+ \end{itemize}
+ \end{itemize}
This chapter provides the required background material, an overview of the concrete contributions and a reading guide.
+
+% The sheer number of connected devices around us is mind boggling and seems increases exponentially for many years.
+% In 2022, there is an estimated number of 13.4 billion of devices connected that sense, act or otherwise interact with people and the physical world surrounding us\footnote{\url{https://transformainsights.com/research/tam/market}, accessed on: \formatdate{2022}{10}{13}}.
+% These devices, together with all the scaffolding and integration such as the various networks providing the communication, (cloud) computers realising the back end or administration and the devices in our pockets providing us with a view on the system are called the \gls{IOT}.
+% \Gls{IOT} systems can be seen as layered systems, where each layer is powered by different types of computers; programming languages and even programming paradigms.
+% This thesis shows a novel way of orchestrating these brobdingnagian systems using the \gls{TOP} paradigm.
+% It does so by giving a proof-of-concept implementation for a \gls{TOP} system specifically designed for the \gls{IOT}: \gls{MTASK}.
+% At the core of the \gls{MTASK} system is a \gls{DSL}, embedded in the general purpose \gls{TOP} system \gls{ITASK}.
+% Using the \gls{MTASK} system, all layers of an \gls{IOT} system can be programmed from a single declarative specification.
+
\end{chapterabstract}
-\todo{Introduction in the abstract doen zoals nu?}
\section{Internet of Things}
The \gls{IOT} is growing rapidly and it is changing the way people and machines interact with the world.
While the term \gls{IOT} briefly gained interest around 1999 to describe the communication of \gls{RFID} devices~\citep{ashton_internet_1999,ashton_that_2009}, it probably already popped up halfway the eigthies in a speech by \citet{peter_t_lewis_speech_1985}:
\begin{quote}
- \emph{The \acrlong{IOT}, or \acrshort{IOT}, is the integration of people, processes and technology with connectable devices and sensors to enable remote monitoring, status, manipulation and evaluation of trends of such devices.}
+ \emph{The \glsxtrlong{IOT}, or \glsxtrshort{IOT}, is the integration of people, processes and technology with connectable devices and sensors to enable remote monitoring, status, manipulation and evaluation of trends of such devices.}
\end{quote}
CISCO states that the \gls{IOT} only started when there where as many connected devices as there were people on the globe, i.e.\ around 2008~\citep{evans_internet_2011}.
With interpretation, a specialized interpreter is flashed in the program memory once that receives the program code to execute at runtime.
%weiser_computer_1991
-\section{\texorpdfstring{\Acrlongpl{DSL}}{Domain-specific languages}}
+\section{\texorpdfstring{\Glsxtrlongpl{DSL}}{Domain-specific languages}}
% General
Programming languages can be divided up into two categories: \glspl{DSL}\footnote{Historically this has been called DSEL as well.} and \glspl{GPL}~\citep{fowler_domain_2010}.
Where \glspl{GPL} are not made with a demarcated area in mind, \glspl{DSL} are tailor-made for a specific domain.
\end{itemize}
\subsection*{\nameref{prt:tvt}}
-This chapter is based on the journal paper: \citeentry{lubbers_could_2022}\footnote{This work is an extension of the conference article: \citeentry{lubbers_tiered_2020}\footnotemark{}}.
+This chapter is based on the journal paper: \citeentry{lubbers_could_2022}\todo{change when published}\footnote{This work is an extension of the conference article: \citeentry{lubbers_tiered_2020}\footnotemark{}}.
\footnotetext{This paper was partly funded by the Radboud-Glasgow Collaboration Fund.}
It compares programming traditional tiered architectures to tierless architectures by showing a qualitative and a quantitative four-way comparison of a smart campus application.
Writing the paper was performed by all authors.
-I created the server application, the \gls{CLEAN}/\gls{ITASK}/\gls{MTASK} implementation (\acrshort{CWS}) and the \gls{CLEAN}/\gls{ITASK} implementation (\acrshort{CRS})
-Adrian Ramsingh created the \gls{MICROPYTHON} implementation (\acrshort{PWS}), the original \gls{PYTHON} implementation (\acrshort{PRS}) and the server application were created by Jeremy Singer, Dejice Jacob and Kristian Hentschel~\citep{hentschel_supersensors:_2016}.
+I created the server application, the \gls{CLEAN}/\gls{ITASK}/\gls{MTASK} implementation (\glsxtrshort{CWS}) and the \gls{CLEAN}/\gls{ITASK} implementation (\glsxtrshort{CRS})
+Adrian Ramsingh created the \gls{MICROPYTHON} implementation (\glsxtrshort{PWS}), the original \gls{PYTHON} implementation (\glsxtrshort{PRS}) and the server application were created by Jeremy Singer, Dejice Jacob and Kristian Hentschel~\citep{hentschel_supersensors:_2016}.
\input{subfilepostamble}
\end{document}
% Titlecase glossary commands
\newcommand{\glst}[1]{\titlecap{\glsentrylong{#1}}}
\newcommand{\Glst}[1]{\glst{#1}}
-% Fix gls in hyperlink errors
+\usepackage{glossary-mcols}
\pdfstringdefDisableCommands{%
\def\glsxtrlong#1{}%
\def\glsxtrlongpl#1{}%
\def\Glsentrytext#1{}%
\def\titlecap#1{}%
}
-\usepackage{glossary-mcols}
% Index
%\usepackage{makeidx}
\newcommand{\requiresGHCmod}[2][]{\footnote{Requires \GHCmod{#2} to be enabled. #1}}
%\newcommand{\etc}{{\fontfamily{cmr}\selectfont{\itshape\/\&c}}}
\newcommand{\etc}{{\fontfamily{cmr}\selectfont{\itshape\/\&\kern-0.2em c}}}
-\newcommand{\rdmentry}[6]{#1: #2 (#3): #4. #5.\ \href{https://doi.org/#6}{#6}}
+\newcommand{\rdmentry}[5]{#1 (#2): #3. #4.\ \href{https://doi.org/#5}{#5}}
\newcommand{\mlubbers}{Lubbers, M.\ (Radboud University)}
\newcommand{\pkoopman}{Koopman, dr.\ P.\ (Radboud University)}
\newcommand{\rplasmeijer}{Plasmeijer, prof.\ dr.\ ir.\ R.\ (Radboud University)}
+\newcommand{\aramsingh}{Ramsingh, A.\ (University of Glasgow)}
+\newcommand{\jsinger}{Singer, dr.\ J.\ (University of Glasgow)}
+\newcommand{\ptrinder}{Trinder, prof.~dr.\ P.\ (University of Glasgow)}
\newcommand{\mypart}[3]{
\part[#2: #3]{#2\\[2ex]\smaller{}#3}%
year = {2022},
note = {Place: New York, NY, USA
Publisher: Association for Computing Machinery
-under-review},
+in-press},
keywords = {access control, internet-of-things, policy language, privilege escalation, Smart home system},
}
\documentclass[../thesis.tex]{subfiles}
+\input{subfilepreamble}
+
\begin{document}
\ifSubfilesClassLoaded{
\pagenumbering{arabic}
There is also a special type of basic task for delaying execution.
The \cleaninline{delay} task---given a number of milliseconds---yields an unstable value while the time has not passed.
Once the specified time has passed, the time it overshot the planned time is yielded as a stable task value.
+See \cref{sec:repeat} for an example task using \cleaninline{delay}.
\begin{lstClean}[label={lst:basic_tasks},caption={Function examples in \gls{MTASK}.}]
class rtrn v :: (v t) -> MTask v t
The type classes for these tasks are not included in the \cleaninline{mtask} class collection as not all devices nor all language interpretations have such peripherals connected.
\todo{Historically, peripheral support has been added \emph{by need}.}
-\Cref{lst:dht} and \cref{lst:gpio} show the type classes for \glspl{DHT} sensors and \gls{GPIO} access.
+\Cref{lst:dht,lst:gpio} show the type classes for \glspl{DHT} sensors and \gls{GPIO} access.
Other peripherals have similar interfaces, they are available in the \cref{sec:aux_peripherals}.
For the \gls{DHT} sensor there are two basic tasks, \cleaninline{temperature} and \cleaninline{humidity}, that---will given a \cleaninline{DHT} object---produce a task that yields the observed temperature in \unit{\celcius} or relative humidity as a percentage as an unstable value.
Currently, two different types of \gls{DHT} sensors are supported, the \emph{DHT} family of sensors connect through $1$-wire protocol and the \emph{SHT} family of sensors connected using \gls{I2C} protocol.
Writing to a pin is also a task that immediately finishes but yields the written value instead.
Reading a pin is a task that yields an unstable value---i.e.\ the value read from the actual pin.
-\begin{lstClean}[label={lst:gpio},caption{The \gls{MTASK} interface for \gls{GPIO} access.}]
+\begin{lstClean}[label={lst:gpio},caption={The \gls{MTASK} interface for \gls{GPIO} access.}]
:: APin = A0 | A1 | A2 | A3 | A4 | A5
:: DPin = D0 | D1 | D2 | D3 | D4 | D5 | D6 | D7 | D8 | D9 | D10 | D11 | D12 | D13
:: PinMode = PMInput | PMOutput | PMInputPullup
declarePin :: p PinMode ((v p) -> Main (v a)) -> Main (v a) | pin p
\end{lstClean}
+\begin{lstClean}[label={lst:gpio_examples},caption={\Gls{GPIO} example in \gls{MTASK}.}]
+task1 :: MTask v Int | mtask v
+task1 = declarePin A3 PMInput \a3->{main=readA a3}
+
+task2 :: MTask v Int | mtask v
+task2 = declarePin D3 PMOutput \d3->{main=writeD d3 true}
+\end{lstClean}
+
\subsection{Task combinators}
Task combinators are used to combine multiple tasks into one to describe workflows.
There are three main types of task combinators, namely:
In {main = monitor d0 .||. monitor d1}
\end{lstClean}
-\subsubsection{Repeat}
+\subsubsection{Repeat}\label{sec:repeat}
The \cleaninline{rpeat} combinator executes the child task.
If a stable value is observed, the task is reinstated.
The functionality of \cleaninline{rpeat} can also be simulated by using functions and sequential task combinators and even made to be stateful as can be seen in \cref{lst:blinkthreadmtask}.
task =
declarePin A1 PMInput \a1->
declarePin A2 PMOutput \a2->
- {main = rpeat (readA a1 >>~. writeA a2)}
+ {main = rpeat (readA a1 >>~. writeA a2 >>|. delay (lit 1000))}
\end{lstClean}
-\subsection{\texorpdfstring{\Acrlongpl{SDS}}{Shared data sources}}
-\Glspl{SDS} in \gls{MTASK} are by default references to shared memory.
+\subsection{\texorpdfstring{\Glsxtrlongpl{SDS}}{Shared data sources}}
+\Glspl{SDS} in \gls{MTASK} are by default references to shared memory but can also be references to \glspl{SDS} in \gls{ITASK} (see \cref{sec:liftsds}).
Similar to peripherals (see \cref{sssec:peripherals}), they are constructed at the top level and are accessed through interaction tasks.
The \cleaninline{getSds} task yields the current value of the \gls{SDS} as an unstable value.
This behaviour is similar to the \cleaninline{watch} task in \gls{ITASK}.
Writing a new value to an \gls{SDS} is done using \cleaninline{setSds}.
This task yields the written value as a stable result after it is done writing.
-Getting and immediately after setting an \gls{SDS} is not an \emph{atomic} operation.
-It is possible that another task accesses the \gls{SDS} in between.
+Getting and immediately after setting an \gls{SDS} is not necessarily an \emph{atomic} operation in \gls{MTASK} because it is possible that another task accesses the \gls{SDS} in between.
To circumvent this issue, \cleaninline{updSds} is created, this task atomically updates the value of the \gls{SDS}.
+The \cleaninline{updSds} task only guarantees atomicity within \gls{MTASK}.
\begin{lstClean}[label={lst:mtask_sds},caption={\Glspl{SDS} in \gls{MTASK}.}]
:: Sds a // abstract
updSds :: (v (Sds t)) ((v t) -> v t) -> MTask v t
\end{lstClean}
-\todo{examples sdss}
+\Cref{lst:mtask_sds_examples} shows an example using \glspl{SDS}.
+The \cleaninline{count} function takes a pin and returns a task that counts the number of times the pin is observed as high by incrementing the \cleaninline{share} \gls{SDS}.
+In the \cleaninline{main} expression, this function is called twice and the results are combined using the parallel or combinator (\cleaninline{.||.}).
+Using a sequence of \cleaninline{getSds} and \cleaninline{setSds} would be unsafe here because the other branch might write their old increment value immediately after writing, effectively missing a count.\todo{beter opschrijven}
+
+\begin{lstClean}[label={lst:mtask_sds_examples},caption={Examples with \glspl{SDS} in \gls{MTASK}.}]
+task :: MTask v Int | mtask v
+task = declarePin D3 PMInput \d3->
+ declarePin D5 PMInput \d5->
+ sds \share=0
+ In fun \count=(\pin->
+ readD pin
+ >>* [IfValue (\x->x) (\_->updSds (\x->x +. lit 1) share)]
+ >>| delay (lit 100) // debounce
+ >>| count pin)
+ In {main=count d3 .||. count d5}
+\end{lstClean}
\chapter{Green computing with \texorpdfstring{\gls{MTASK}}{mTask}}%
\label{chp:green_computing_mtask}
liftmTask :: (Main (BCInterpret (TaskValue u))) MTDevice -> Task u | iTask u
\end{lstClean}
-\section{Lifting \texorpdfstring{\acrlongpl{SDS}}{shared data sources}}\label{sec:liftsds}
+\section{Lifting \texorpdfstring{\glsxtrlongpl{SDS}}{shared data sources}}\label{sec:liftsds}
\begin{lstClean}[label={lst:mtask_itasksds},caption={Lifted \gls{ITASK} \glspl{SDS} in \gls{MTASK}.}]
class liftsds v where
liftsds :: ((v (Sds t))->In (Shared sds t) (Main (MTask v u)))
The \gls{MTASK} language as it is now was introduced in 2018~\citep{koopman_task-based_2018}.
This paper updated the language to support functions, tasks and \glspl{SDS} but still compiled to \gls{CPP} \gls{ARDUINO} code.
Later the bytecode compiler and \gls{ITASK} integration was added to the language~\citep{lubbers_interpreting_2019}.
-Moreover, it was shown that it is very intuitive to write \gls{MCU} applications in a \gls{TOP} language~\citep{lubbers_multitasking_2019}.
+Moreover, it was shown that it is very intuitive to write microprocessor applications in a \gls{TOP} language~\citep{lubbers_multitasking_2019}.
One reason for this is that a lot of design patterns that are difficult using standard means are for free in \gls{TOP} (e.g.\ multithreading).
In 2019, the \gls{CEFP} summer school in Budapest, Hungary hosted a course on developing \gls{IOT} applications with \gls{MTASK} as well~\citep{lubbers_writing_2019}.
\cleaninline:{}
\pythoninline:{}
\haskellinline:{}
- \haskelllshtexinline:{}
+ \haskelllhstexinline:{}
\arduinoinline:{}
\texttt:{}
\url:{}
\begin{chapterabstract}
\Gls{IOT} software is notoriously complex, conventionally comprising multiple tiers.
- The developer must use multiple programming languages and ensure that the components interoperate correctly. A novel alternative is to use a single \emph{tierless} language with a compiler that generates the code for each component and ensures their correct interoperation.
+ Traditionally an \gls{IOT} developer must use multiple programming languages and ensure that the components interoperate correctly. A novel alternative is to use a single \emph{tierless} language with a compiler that generates the code for each component and ensures their correct interoperation.
We report a systematic comparative evaluation of two tierless language technologies for \gls{IOT} stacks: one for resource-rich sensor nodes (\gls{CLEAN} with \gls{ITASK}), and one for resource-constrained sensor nodes (\gls{CLEAN} with \gls{ITASK} and \gls{MTASK}).
The evaluation is based on four implementations of a typical smart campus application: two tierless and two \gls{PYTHON}-based tiered.
This study compares a pair of tierless \gls{IOT} languages with conventional tiered \gls{PYTHON} \gls{IOT} software. \gls{CLEAN}\slash\gls{ITASK} and \gls{CLEAN}/\gls{ITASK}/\gls{MTASK} represent a specific set of tierless language design decisions, however many alternative designs are available. Crucially the limitations of the tierless \gls{CLEAN} languages, e.g.\ that they currently provide limited security, should not be seen as limitations of tierless technologies in general. This section briefly outlines key design decisions for tierless \gls{IOT} languages, discusses alternative designs, and describes the \gls{CLEAN} designs. The \gls{CLEAN} designs are illustrated in the examples in the following section.
-\subsubsection{Program splitting}
-A key challenge for an automatically segmented tierless language is to determine which parts of the program correspond to a particular tier and hence should be executed by a specific component on a specific host, so-called tier splitting.
-For example a tierless web language must identify client code to ship to browsers, database code to execute in the DBMS, and application code to run on the server. Some tierless languages split programs using types, others use syntactic markers, e.g.\ pragmas like \cleaninline{server} or \cleaninline{client}, to split the program~\citep{cooper2006links,10.1145/2775050.2633367}. It may be possible to infer the splitting between tiers, relieving the developers from the need specify it, as illustrated for Javascript as a tierless web language~\citep{10.1145/2661136.2661146}.
+\subsubsection{Tier splitting and placement}
+A key challenge for a tierless language is to determine which parts of the program correspond to a particular tier and hence should be executed by a specific component on a specific host.
-In \gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} and \gls{CLEAN}/\gls{ITASK} tier splitting is specified by functions, and hence is a first-class language construct.
-For example in \gls{CLEAN}\slash\gls{ITASK} the \cleaninline{asyncTask} function identifies a task for execution on a remote device and \cleaninline{liftmTask} executes the given task on an \gls{IOT} device. The tier splitting functions are illustrated in examples in the next section, e.g.\ on \cref{lst_t4t:itaskTempFull:startdevtask} in \cref{lst_t4t:itaskTempFull} and \cref{lst_t4t:mtaskTemp:liftmtask} in \cref{lst_t4t:mtaskTemp}.
+For example a tierless web language must identify client code to ship to browsers, database code to execute in the DBMS, and application code to run on the server.
+Tierless web languages can make this determination statically, so-called \emph{tier splitting} using types or syntactic markers like \texttt{server} or \texttt{client} pragmas \citep{cooper2006links,10.1145/2775050.2633367}.
+It is even possible to infer the splitting, relieving the developers from the need to specify it, as illustrated for Javascript as a tierless web language \citep{10.1145/2661136.2661146}.
+
+In \gls{CLEAN}\slash\gls{ITASK}\slash\gls{MTASK} and \gls{CLEAN}\slash\gls{ITASK} tier splitting is specified by functions, e.g.\ the \gls{CLEAN}\slash\gls{ITASK} \cleaninline{asyncTask} function identifies a task for execution on a remote device and \cleaninline{liftmTask} executes the given task on an \gls{IOT} device.
+The tier splitting functions are illustrated in examples in the next section, e.g.\ on \cref{lst_tvtitaskTempFull:startdevtask} in \cref{lst_tvtitaskTempFull} and \cref{lst_tvtmtasktemp:liftmtask} in \cref{lst_tvtmtasktemp}.
Specifying splitting as functions means that new splitting functions can be composed, and that splitting is under program control, e.g.\ during execution a program can decide to run a task locally or remotely.
+As \gls{IOT} stacks are more complex than web stacks, the \emph{placement} of data and computations onto the devices/hosts in the system is more challenging.
+In many \gls{IOT} systems placement is manual: the sensor nodes are microcontrollers that are programmed by writing the program to flash memory.
+So physical access to the microcontroller is normally required to change the program, making updates challenging.
+%Hence, most IoT systems compile sensor node code directly for the target architecture or via an existing language such as C/C++.
+
+Techniques like over-the-air programming and interpreters allow microcontrollers to be dynamically provisioned, increasing their maintainability and resilience.
+For example \citet{baccelli_reprogramming_2018} provide a single language \gls{IOT} system based on the RIOT \gls{OS} that allows runtime deployment of code snippets called containers.
+Both client and server are written in JavaScript.
+However, there is no integration between the client and the server other than that they are programmed from a single source.
+Mat\`e is an example of an early tierless sensor network framework where devices are provided with a virtual machine using TinyOS for dynamic provisioning \citep{levis_mate_2002}.
+
+In general different tierless languages specify placement in different ways, e.g.\ code annotations or configuration files, and at different granularities, e.g.\ per function or per class \citep{weisenburger2020survey}.
+
+\Gls{CLEAN}\slash\gls{ITASK}\slash\gls{MTASK} and \Gls{CLEAN}\slash\gls{ITASK} both use dynamic task placement.
+In \gls{CLEAN}\slash\gls{ITASK}\slash\gls{MTASK} sensor nodes are programmed once with the \gls{MTASK} \gls{RTS}, and possibly some precompiled tasks.
+Thereafter a sensor node can dynamically receive \gls{MTASK} programs, compiled at runtime by the server.
+In \gls{CLEAN}\slash\gls{ITASK} the sensor node runs an \gls{ITASK} server that recieves and executes code from the (\gls{IOT}) server \citep{oortgiese_distributed_2017}.
+Placement happens automatically as part of the first-class splitting constructs, so \cref{lst_tvtmtasktemp:liftmtask} in \cref{lst_tvtmtasktemp} places \cleaninline{devTask} onto the \cleaninline{dev} sensor node.
+
+%\subsubsection{Program splitting}
+%
+%A key challenge for an automatically segmented tierless language is to determine which parts of the program correspond to a particular tier and hence should be executed by a specific component on a specific host, so-called tier splitting.
+%For example a tierless web language must identify client code to ship to browsers, database code to execute in the DBMS, and application code to run on the server. Some tierless languages split programs using types, others use syntactic markers, e.g.\ pragmas like \cleaninline{server} or \cleaninline{client}, to split the program~\citep{cooper2006links,10.1145/2775050.2633367}. It may be possible to infer the splitting between tiers, relieving the developers from the need specify it, as illustrated for Javascript as a tierless web language~\citep{10.1145/2661136.2661146}.
+%
+%In \gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} and \gls{CLEAN}/\gls{ITASK} tier splitting is specified by functions, and hence is a first-class language construct.
+%For example in \gls{CLEAN}\slash\gls{ITASK} the \cleaninline{asyncTask} function identifies a task for execution on a remote device and \cleaninline{liftmTask} executes the given task on an \gls{IOT} device. The tier splitting functions are illustrated in examples in the next section, e.g.\ on \cref{lst_t4t:itaskTempFull:startdevtask} in \cref{lst_t4t:itaskTempFull} and \cref{lst_t4t:mtaskTemp:liftmtask} in \cref{lst_t4t:mtaskTemp}.
+%Specifying splitting as functions means that new splitting functions can be composed, and that splitting is under program control, e.g.\ during execution a program can decide to run a task locally or remotely.
+
\subsubsection{Communication}\label{ssec_t4t:communication}
Tierless languages may adopt a range of communication paradigms for communicating between components. Different tierless languages specify communication in different ways~\citep{weisenburger2020survey}. Remote procedures are the most common communication mechanism: a procedure/function executing on a remote host/machine is called as if it was local. The communication of the arguments to, and the results from, the remote procedure is automatically provided by the language implementation. Other mechanisms include explicit message passing between components; publish/subscribe where components subscribe to topics of interest from other components; reactive programming defines event streams between remote components; finally shared state makes changes in a shared and potentially remote data structure visible to components.
\Cref{lst_t4t:itaskTempFull} illustrates: \cref{lst_t4t:itaskTempFull:startdevtask} shows a server task launching a remote task, \cleaninline{devTask}, on to a sensor node; and \cref{lst_t4t:itaskTempFull:remoteShare} shows the sharing of the remote \cleaninline{latestTemp} \gls{SDS}.
%\mlcomment{I think this is fine}
-\subsubsection{Placement}
-
-In many \gls{IOT} systems the sensor nodes are microprocessors that are programmed by writing the program to flash memory. This means that without extra effort, physical access to the microcontroller is needed to change the program making updates challenging.
-Hence, most \gls{IOT} systems compile sensor node code directly for the target architecture or via an existing language such as \gls{C}\slash\gls{CPP}.
-
-Techniques such as over-the-air programming and interpreters allow microprocessors to be dynamically provisioned, increasing their maintainability and resilience.
-For example Baccelli et al.\ provide a single language \gls{IOT} system based on the RIOT \gls{OS} that allows runtime deployment of code snippets called containers~\citep{baccelli_reprogramming_2018}.
-Both client and server are written in JavaScript. However, there is no integration between the client and the server other than that they are programmed from a single source.
-Mat\`e is an example of an early tierless sensor network framework where devices are provided with a virtual machine using TinyOS for dynamic provisioning~\citep{levis_mate_2002}.
-
-Placement specifies how data and computations in a tierless program are assigned to the
-devices/hosts in the distributed system. Different tierless languages specify placement in different ways, e.g.\ code annotations or configuration files, and at different granularities, e.g.\ per function or per class~\citep{weisenburger2020survey}.
-
-\Gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} and \gls{CLEAN}/\gls{ITASK} both use dynamic task placement, similar to dynamic function placement.
-In \gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} sensor nodes are programmed once with the \gls{MTASK} \gls{RTS}, and possibly some precompiled tasks.
-Thereafter a sensor node can dynamically receive \gls{MTASK} programs, compiled at runtime by the server.
-In \gls{CLEAN}\slash\gls{ITASK} the sensor node runs an \gls{ITASK} server that recieves and executes code from the (\gls{IOT}) server~\citep{oortgiese_distributed_2017}.
-%The \gls{ITASK} server decides what code to execute depending on the serialised execution graph that the server sends~\citep{oortgiese_distributed_2017}.
-Placement happens automatically as part of the first-class splitting constructs outlined in \cref{ssec_t4t:communication}, so \cref{lst_t4t:mtaskTemp:liftmtask} in \cref{lst_t4t:mtaskTemp} places \cleaninline{devTask} onto the \cleaninline{dev} sensor node.
+%\subsubsection{Placement}
+%
+%In many \gls{IOT} systems the sensor nodes are microprocessors that are programmed by writing the program to flash memory. This means that without extra effort, physical access to the microcontroller is needed to change the program making updates challenging.
+%Hence, most \gls{IOT} systems compile sensor node code directly for the target architecture or via an existing language such as \gls{C}\slash\gls{CPP}.
+%
+%Techniques such as over-the-air programming and interpreters allow microprocessors to be dynamically provisioned, increasing their maintainability and resilience.
+%For example \citet{baccelli_reprogramming_2018} provide a single language \gls{IOT} system based on the RIOT \gls{OS} that allows runtime deployment of code snippets called containers.
+%Both client and server are written in JavaScript. However, there is no integration between the client and the server other than that they are programmed from a single source.
+%Mat\`e is an example of an early tierless sensor network framework where devices are provided with a virtual machine using TinyOS for dynamic provisioning~\citep{levis_mate_2002}.
+%
+%Placement specifies how data and computations in a tierless program are assigned to the
+%devices/hosts in the distributed system. Different tierless languages specify placement in different ways, e.g.\ code annotations or configuration files, and at different granularities, e.g.\ per function or per class~\citep{weisenburger2020survey}.
+%
+%\Gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} and \gls{CLEAN}/\gls{ITASK} both use dynamic task placement, similar to dynamic function placement.
+%In \gls{CLEAN}\slash\gls{ITASK}/\gls{MTASK} sensor nodes are programmed once with the \gls{MTASK} \gls{RTS}, and possibly some precompiled tasks.
+%Thereafter a sensor node can dynamically receive \gls{MTASK} programs, compiled at runtime by the server.
+%In \gls{CLEAN}\slash\gls{ITASK} the sensor node runs an \gls{ITASK} server that recieves and executes code from the (\gls{IOT}) server~\citep{oortgiese_distributed_2017}.
+%%The \gls{ITASK} server decides what code to execute depending on the serialised execution graph that the server sends~\citep{oortgiese_distributed_2017}.
+%Placement happens automatically as part of the first-class splitting constructs outlined in \cref{ssec_t4t:communication}, so \cref{lst_t4t:mtaskTemp:liftmtask} in \cref{lst_t4t:mtaskTemp} places \cleaninline{devTask} onto the \cleaninline{dev} sensor node.
\subsubsection{Security}
Another reason that the tierless \gls{CLEAN} implementations are concise is because they use powerful higher order \gls{IOT} programming abstractions.
For comprehensibility the simple temperature sensor from \cref{sec_t4t:mtasks} (\cref{lst_t4t:mtaskTemp}) is used to compare the expressive power of \gls{CLEAN} and \gls{PYTHON}-based \gls{IOT} programming abstractions.
-There are implementations for all four configurations: \gls{PRTS} (\gls{PYTHON} Raspberry Pi Temperature Sensor)\footnotemark, \gls{PWTS}\footnotemark[\value{footnote}]\todo{the dataset will be uploaded, the DOI is reserved.}
-\footnotetext{Lubbers, M.; Koopman, P.; Ramsingh, A.; Singer, J.; Trinder, P. (2021): Source code, line counts and memory stats for PRS, PWS, PRT and PWT.\ Zenodo.\ \href{https://doi.org/10.5281/zenodo.5081386}{10.5281/zenodo.5081386}.}, \gls{CRTS}\footnotemark{} and \gls{CWTS}\footnotemark[\value{footnote}] \todo{the dataset will be uploaded, the DOI is reserved.}
+There are implementations for all four configurations: \gls{PRTS} (\gls{PYTHON} Raspberry Pi Temperature Sensor)\footnotemark, \gls{PWTS}\footnotemark[\value{footnote}]
+\footnotetext{Lubbers, M.; Koopman, P.; Ramsingh, A.; Singer, J.; Trinder, P. (2021): Source code, line counts and memory stats for PRS, PWS, PRT and PWT.\ Zenodo.\ \href{https://doi.org/10.5281/zenodo.5081386}{10.5281/zenodo.5081386}.}, \gls{CRTS}\footnotemark{} and \gls{CWTS}\footnotemark[\value{footnote}].
\footnotetext{Lubbers, M.; Koopman, P.; Ramsingh, A.; Singer, J.; Trinder, P. (2021): Source code, line counts and memory stats for CRS, CWS, CRTS and CWTS.\ Zenodo.\ \href{https://doi.org/10.5281/zenodo.5040754}{10.5281/zenodo.5040754}.}
but as the programming abstractions are broadly similar, we compare only the \gls{PWTS} and \gls{CWTS} implementations.
repositories / phd-thesis.git / commitdiff