Shallow embedding on the other hand models the language constructs as functions with the semantics embedded.
Consequently, adding a construct is easy, i.e.\ it only entails adding another function.
Contrarily, adding semantics requires adapting all language constructs.
-Lifting the functions to type classes, i.e.\ parametrising the constructs over the semantics, allows extension of the language both in constructs and in semantics orthogonally. This advanced style of embedding is called tagless-final or class-based shallow embedding~\citep{kiselyov_typed_2012}.
+Lifting the functions to type classes, i.e.\ parametrising the constructs over the semantics, allows extension of the language both in constructs and in semantics orthogonally. This advanced style of embedding is called tagless-final or class-based shallow embedding \citep{kiselyov_typed_2012}.
While it is often possible to lift values of a user-defined data type to a value in the \gls{DSL}, it is not possible to interact with it using \gls{DSL} constructs, they are not first-class citizens.
\end{enumerate*}
The functions for this are simply not available automatically in the embedded language.
For some semantics---such as an interpreter---it is possible to directly lift the functions from the host language to the \gls{DSL}.
-In other cases---e.g.\ \emph{compiling} \glspl{DSL} such as a compiler or a printer---this is not possible~\citep{elliott_compiling_2003}. %the torget this is not possible. cannot just be lifted from the host language to the \gls{DSL} so it requires a lot of boilerplate to define and implement them.
+In other cases---e.g.\ \emph{compiling} \glspl{DSL} such as a compiler or a printer---this is not possible \citep{elliott_compiling_2003}. %the torget this is not possible. cannot just be lifted from the host language to the \gls{DSL} so it requires a lot of boilerplate to define and implement them.
Thus, all of the operations on the data type have to be defined by hand requiring a lot of plumbing and resulting in a lot of boilerplate code.
To relieve the burden of adding all these functions, metaprogramming\nobreak---\nobreak\hskip0pt and custom quasiquoters---can be used.
As a result, views on the \gls{DSL} are data types implementing these classes.
To illustrate the technique, a simple \gls{DSL}, a language consisting of literals and addition, is outlined.
-This language, implemented according to the tagless-final style~\citep{carette_finally_2009} in \gls{HASKELL}~\citep{peyton_jones_haskell_2003} consists initially only of one type class containing two functions.
+This language, implemented according to the tagless-final style \citep{carette_finally_2009} in \gls{HASKELL} \citep{peyton_jones_haskell_2003} consists initially only of one type class containing two functions.
The \haskellinline{lit} function lifts values from the host language to the \gls{DSL} domain.
The class constraint \haskellinline{Show} is enforced on the type variable \haskellinline{a} to make sure that the value can be printed.
The infix function \haskellinline{+.} represents the addition of two expressions in the \gls{DSL}.
\footnotetext{%
In this case a \haskellinline{newtype} is used instead of regular \haskellinline{data} declarations.
\haskellinline{newtype}s are special data types only consisting a single constructor with one field to which the type is isomorphic.
- During compilation the constructor is completely removed resulting in no overhead~\citep[\citesection{4.2.3}]{peyton_jones_haskell_2003}.
+ During compilation the constructor is completely removed resulting in no overhead \citep[\citesection{4.2.3}]{peyton_jones_haskell_2003}.
}
-Since the language is typed, the printer data type has to have a type variable but it is only used during typing---i.e.\ a phantom type~\citep{leijen_domain_2000}:
+Since the language is typed, the printer data type has to have a type variable but it is only used during typing---i.e.\ a phantom type \citep{leijen_domain_2000}:
\begin{lstHaskell}
newtype Printer a = P { runPrinter :: String }
instance Expr Printer where
lit a = P (show a)
(+.) l r = P ("(" ++ runPrinter l
- ++ "+" ++ runPrinter r ++ ")")
+ ++ "+" ++ runPrinter r ++ ")")
instance Div Printer where
(/.) l r = P ("(" ++ runPrinter l
- ++ "/" ++ runPrinter r ++ ")")
+ ++ "/" ++ runPrinter r ++ ")")
\end{lstHaskell}
\subsection{Functions}
Adding functions to the language is achieved by adding a multi-parameter class to the \gls{DSL}.
The type of the class function allows for the implementation to only allow first order function by supplying the arguments in a tuple.
Furthermore, with the \haskellinline{:-} operator the syntax becomes usable.
-Finally, by defining the functions as a \gls{HOAS} type safety is achieved~\citep{pfenning_higher-order_1988,chlipala_parametric_2008}.
+Finally, by defining the functions as a \gls{HOAS} type safety is achieved \citep{pfenning_higher-order_1988,chlipala_parametric_2008}.
The complete definition looks as follows:
\begin{lstHaskell}
cons :: v a -> v (List a) -> v (List a)
-- deconstructors
unNil :: v (List a) -> v b -> v b
- unCons :: v (List a)
- -> (v a -> v (List a) -> v b) -> v b
+ unCons :: v (List a) -> (v a -> v (List a) -> v b) -> v b
-- constructor predicates
isNil :: v (List a) -> v Bool
isCons :: v (List a) -> v Bool
nil = Just Nil
cons hd tl = Cons <$> hd <*> tl
unNil d f = d >>= \Nil->f
- unCons d f = d
- >>= \(Cons hd tl)->f (Just hd) (Just tl)
+ unCons d f = d >>= \(Cons hd tl)->f (Just hd) (Just tl)
isNil d = d >>= \case[+\footnotemark+]
Nil -> Just True
_ -> Just False
}
Adding these classes and their corresponding instances is tedious and results in boilerplate code.
-We therefore resort to metaprogramming, and in particular \gls{TH}~\citep{sheard_template_2002} to alleviate this burden.
+We therefore resort to metaprogramming, and in particular \gls{TH} \citep{sheard_template_2002} to alleviate this burden.
\section{Template metaprogramming}
Metaprogramming is a special flavour of programming where programs have the ability to treat and manipulate programs or program fragments as data.
-There are several techniques to facilitate metaprogramming, moreover it has been around for many years now~\citep{lilis_survey_2019}.
-Even though it has been around for many years, it is considered complex~\citep{sheard_accomplishments_2001}.
+There are several techniques to facilitate metaprogramming, moreover it has been around for many years now \citep{lilis_survey_2019}.
+Even though it has been around for many years, it is considered complex \citep{sheard_accomplishments_2001}.
-\gls{TH} is GHC's de facto metaprogramming system, implemented as a compiler extension together with a library~\citep{sheard_template_2002}\citep[\citesection{6.13.1}]{ghc_team_ghc_2021}.
+\gls{TH} is GHC's de facto metaprogramming system, implemented as a compiler extension together with a library \citep{sheard_template_2002}\citep[\citesection{6.13.1}]{ghc_team_ghc_2021}.
Readers already familiar with \gls{TH} can safely skip this section.
\gls{TH} adds four main concepts to the language, na\-me\-ly AST data types, splicing, quasiquotation and reification.
With this machinery, regular \gls{HASKELL} functions can be defined that are called at compile time, inserting generated code into the {AST}.
These functions are monadic functions operating in the \haskellinline{Q} monad.
-The \haskellinline{Q} monad facilitates failure, reification and fresh identifier generation for hygienic macros~\citep{kohlbecker_hygienic_1986}.
+The \haskellinline{Q} monad facilitates failure, reification and fresh identifier generation for hygienic macros \citep{kohlbecker_hygienic_1986}.
Within the \haskellinline{Q} monad, capturable and non-capturable identifiers can be generated using the \haskellinline{mkName} and \haskellinline{newName} functions respectively.
The \emph{Peter Parker principle}\footnote{With great power comes great responsibility.} holds for the \haskellinline{Q} monad as well because it executes at compile time and is very powerful.
-For example it can subvert module boundaries, thus accessing constructors that were hidden; access the structure of abstract types; and it may cause side effects during compilation because it is possible to call \haskellinline{IO} operations~\citep{terei_safe_2012}.
+For example it can subvert module boundaries, thus accessing constructors that were hidden; access the structure of abstract types; and it may cause side effects during compilation because it is possible to call \haskellinline{IO} operations \citep{terei_safe_2012}.
To achieve the goal of embedding data types in a \gls{DSL} we refrain from using these \emph{unsafe} features.
\subsubsection{Data types}
To give an impression of these data types, a selection of data types available in \gls{TH} is given below:
\begin{lstHaskell}
-data Dec = FunD Name [Clause] | DataD Cxt Name ...
- | SigD Name Type | ClassD Cxt Name | ...
+data Dec = FunD Name [Clause] | DataD Cxt Name ... | SigD Name Type
+ | ClassD Cxt Name | ...
data Clause = Clause [Pat] Body [Dec]
-data Pat = LitP Lit | VarP Name | TupP [Pat]
- | WildP | ListP [Pat] | ...
+data Pat = LitP Lit | VarP Name | TupP [Pat] | WildP | ListP [Pat] | ...
data Body = GuardedB [(Guard, Exp)] | NormalB Exp
data Guard = NormalG Exp | PatG [Stmt]
-data Exp = VarE Name | LitE Lit | AppE Exp Exp
- | TupE [Maybe Exp] | LamE [Pat] Exp | ...
-data Lit = CharL Char | StringL String
- | IntegerL Integer | ...
+data Exp = VarE Name | LitE Lit | AppE Exp Exp | TupE [Maybe Exp]
+ | LamE [Pat] Exp | ...
+data Lit = CharL Char | StringL String | IntegerL Integer | ...
\end{lstHaskell}
To ease creating AST data types in the \haskellinline{Q} monad, lowercase variants of the constructors are available that lift the constructor to the \haskellinline{Q} monad as.
\end{lstHaskell}
\subsubsection{Quasiquotation}
-Another key concept of \gls{TH} is Quasiquotation, the dual of splicing~\citep{bawden_quasiquotation_1999}.
+Another key concept of \gls{TH} is Quasiquotation, the dual of splicing \citep{bawden_quasiquotation_1999}.
While it is possible to construct entire programs using the provided data types, it is a little cumbersome.
Using \emph{Oxford brackets} (\verb#[|# \ldots\verb#|]#) or single or double apostrophes, verbatim \gls{HASKELL} code can be entered that is converted automatically to the corresponding AST nodes easing the creation of language constructs.
Depending on the context, different quasiquotes are used:
\end{itemize*}.
It is possible to escape the quasiquotes again by splicing.
Variables defined within quasiquotes are always fresh---as if defined with \haskellinline{newName}---but it is possible to capture identifiers using \haskellinline{mkName}.
-For example, \haskellinline{[|\\x->x|]} translates to \haskellinline{newName "x" >>= \\x->lamE [varP x] (varE x)} and does not interfere with other \haskellinline{x}s already defined.
+For example, \haskellinline{[\|\\x->x\|]} translates to \haskellinline{newName "x" >>= \\x->lamE [varP x] (varE x)} and does not interfere with other \haskellinline{x}s already defined.
\subsubsection{Reification}
Reification is the act of querying the compiler for information about a certain name.
\begin{lstHaskell}
genDSL :: Name -> Q [Dec]
genDSL name = reify name >>= \case
- TyConI (DataD cxt typeName tvs mkind
- constructors derives)
- -> mapM getConsName constructors
- >>= \d->genDSL' tvs typeName d
+ TyConI (DataD cxt typeName tvs mkind constructors derives)
+ -> mapM getConsName constructors >>= \d->genDSL' tvs typeName d
t -> fail ("genDSL does not support: " ++ show t)
getConsName :: Con -> Q (Name, [VarBangType])
getConsName (NormalC consName fs) = pure (consName,
- [(adtFieldName consName i, b, t)
- | (i, (b, t))<-[0..] `zip` fs])
+ [(adtFieldName consName i, b, t) | (i, (b, t))<-[0..] `zip` fs])
getConsName (RecC consName fs) = pure (consName, fs)
-getConsName c
- = fail ("genDSL does not support: " ++ show c)
+getConsName c = fail ("genDSL does not support: " ++ show c)
-genDSL' :: [TyVarBndr] -> Name -> [(Name, [VarBangType])]
- -> Q [Dec]
+genDSL' :: [TyVarBndr] -> Name -> [(Name, [VarBangType])] -> Q [Dec]
genDSL' typeVars typeName constructors = sequence
[ mkClass, mkInterpreter, mkPrinter, ... ]
where
\begin{lstHaskell}
mkConstructor :: Name -> [VarBangType] -> Q Dec
-mkConstructor n fs
- = sigD (constructorName n) (mkCFun fs res)
+mkConstructor n fs = sigD (constructorName n) (mkCFun fs res)
mkCFun :: [VarBangType] -> Q Type -> Q Type
mkCFun fs res = foldr (\x y->[t|$x -> $y|])
\begin{lstHaskell}
mkPredicate :: Name -> Q Dec
-mkPredicate n = sigD (predicateName n)
- [t|$res -> $v Bool|]
+mkPredicate n = sigD (predicateName n) [t|$res -> $v Bool|]
\end{lstHaskell}
\subsection{Interpreter instance generation}\label{sec_fcd:interpreter}
fresh <- mapM (newName . nameBase . fst3) fs
fun (deconstructorName consName) [varP d, varP f]
[|$(varE d) >>= \($(match f))->$(fapp f fresh)|]
- where fapp f = foldl appE (varE f)
- . map (\f->[|pure $(varE f)|])
+ where fapp f = foldl appE (varE f) . map (\f->[|pure $(varE f)|])
match f = pure (ConP consName (map VarP f))
\end{lstHaskell}
\subsection{Custom quasiquoters}
The syntax burden of \glspl{EDSL} can be reduced using quasiquotation.
In \gls{TH}, quasiquotation is a convenient way to create \gls{HASKELL} language constructs by entering them verbatim using Oxford brackets.
-However, it is also possible to create so-called custom quasiquoters~\citep{mainland_why_2007}.
+However, it is also possible to create so-called custom quasiquoters \citep{mainland_why_2007}.
If the programmer writes down a fragment of code between tagged \emph{Oxford brackets}, the compiler executes the associated quasiquoter functions at compile time.
A quasiquoter is a value of the following data type:
Custom quasiquoters allow the \gls{DSL} user to enter fragments verbatim, bypassing the syntax of the host language.
Pattern matching in general is not suitable for a custom quasiquoter because it does not really fit in one of the four syntactic categories for which custom quasiquoter support is available.
However, a concrete use of pattern matching, interesting enough to be beneficial, but simple enough for a demonstration is the \emph{simple case expression}, a case expression that does not contain nested patterns and is always exhaustive.
-They correspond to a multi-way conditional expressions and can thus be converted to \gls{DSL} constructs straightforwardly~\citep[\citesection{4.4}]{peyton_jones_implementation_1987}.
+They correspond to a multi-way conditional expressions and can thus be converted to \gls{DSL} constructs straightforwardly \citep[\citesection{4.4}]{peyton_jones_implementation_1987}.
In contrast to the binary literal quasiquoter example, we do not create the parser by hand.
-The parser combinator library \emph{parsec} is used instead to ease the creation of the parser~\citep{leijen_parsec_2001}.
+The parser combinator library \emph{parsec} is used instead to ease the creation of the parser \citep{leijen_parsec_2001}.
First the location of the quasiquoted code is retrieved using the \haskellinline{location} function that operates in the \haskellinline{Q} monad.
This location is inserted in the parsec parser so that errors are localised in the source code.
Then, the \haskellinline{expr} parser is called that returns an \haskellinline{Exp} in the \haskellinline{Q} monad.
\end{lstHaskell}
\section{Related work}
-Generic or polytypic programming is a promising technique at first glance for automating the generation of function implementations~\citep{lammel_scrap_2003}.
+Generic or polytypic programming is a promising technique at first glance for automating the generation of function implementations \citep{lammel_scrap_2003}.
However, while it is possible to define a function that works on all first-order types, adding a new function with a new name to the language is not possible.
This does not mean that generic programming is not useable for embedding pattern matches.
In generic programming, types are represented as sums of products and using this representation it is possible to define pattern matching functions.
For example, \citet{rhiger_type-safe_2009} showed a method for expressing statically typed pattern matching using typed higher-order functions.
If not the host language but the \gls{DSL} contains higher order functions, the same technique could be applied to port pattern matching to \glspl{DSL} though using an explicit sums of products representation.
-\Citeauthor{atkey_unembedding_2009} describe embedding pattern matching in a \gls{DSL} by giving patterns an explicit representation in the \gls{DSL} by using pairs, sums and injections~\citep[\citesection{3.3}]{atkey_unembedding_2009}.
+\Citeauthor{atkey_unembedding_2009} describe embedding pattern matching in a \gls{DSL} by giving patterns an explicit representation in the \gls{DSL} by using pairs, sums and injections \citep[\citesection{3.3}]{atkey_unembedding_2009}.
\Citet{mcdonell_embedded_2022} extends on this idea, resulting in a very similar but different solution to ours.
They used the technique that \citeauthor{atkey_unembedding_2009} showed and applied it to deep embedding using the concrete syntax of the host language.
\subsubsection{Generating extra code}
Using \gls{TH} or other metaprogramming systems it is possible to add extra code to your program.
-The original \gls{TH} paper showed that it is possible to create variadic functions such as \haskellinline{printf} using \gls{TH} that would be almost impossible to define without~\citep{sheard_template_2002}.
+The original \gls{TH} paper showed that it is possible to create variadic functions such as \haskellinline{printf} using \gls{TH} that would be almost impossible to define without \citep{sheard_template_2002}.
\Citet{hammond_automatic_2003} used \gls{TH} to generate parallel programming skeletons.
In practise, this means that the programmer selects a skeleton and, at compile time, the code is massaged to suit the pattern and information about the environment is inlined for optimisation.
\Citet{shioda_libdsl_2014} used metaprogramming in the D programming language to create a \gls{DSL} toolkit.
They also programmatically generate parsers and a backend for either compiling or interpreting the \gls{IR}.
\Citet{blanchette_liquid_2022} use \gls{TH} to simplify the development of Liquid \gls{HASKELL} proofs.
-\Citet{folmer_high-level_2022} used \gls{TH} to synthesize C$\lambda$aSH~\citep{baaij_digital_2015} abstract syntax trees to be processed.
+\Citet{folmer_high-level_2022} used \gls{TH} to synthesize C$\lambda$aSH \citep{baaij_digital_2015} abstract syntax trees to be processed.
In similar fashion, \citet{materzok_generating_2022} used \gls{TH} to translate YieldFSM programs to {C$\lambda$aSH}.
\subsubsection{Optimisation}
\subsubsection{Quasiquotation}
By means of quasiquotation, the host language syntax that usually seeps through the embedding can be hidden.
-The original \gls{TH} quasiquotation paper~\citep{mainland_why_2007} shows how this can be done for regular expressions, not only resulting in a nicer syntax but syntax errors are also lifted to compile time instead of run time.
+The original \gls{TH} quasiquotation paper \citep{mainland_why_2007} shows how this can be done for regular expressions, not only resulting in a nicer syntax but syntax errors are also lifted to compile time instead of run time.
Also, \citet{kariotis_making_2008} used \gls{TH} to automatically construct monad stacks without having to resort to the monad transformers library which requires advanced type system extensions.
\Citet{najd_everything_2016} uses the compile time to be able to do normalisation for a \gls{DSL}, dubbing it \glspl{QDSL}.
Using quasiquotation, they make a complicated embedding of non-linear pattern matching available through a simple lens.
\subsubsection{\texorpdfstring{\glsxtrlong{TTH}}{Typed Template Haskell}}\label{ssec_fcd:typed_template_haskell}
-\gls{TTH} is a very recent extension/alternative to normal \gls{TH}~\citep{pickering_multi-stage_2019,xie_staging_2022}.
+\gls{TTH} is a very recent extension/alternative to normal \gls{TH} \citep{pickering_multi-stage_2019,xie_staging_2022}.
Where in \gls{TH} you can manipulate arbitrary parts of the syntax tree, add top-level splices of data types, definitions and functions, in \gls{TTH} the programmer can only splice expressions but the abstract syntax tree fragments representing the expressions are well-typed by construction instead of untyped.
-\Citet{pickering_staged_2020} implemented staged compilation for the \emph{generics-sop}~\citep{de_vries_true_2014} generics library to improve the efficiency of the code using \gls{TTH}.
+\Citet{pickering_staged_2020} implemented staged compilation for the \emph{generics-sop} \citep{de_vries_true_2014} generics library to improve the efficiency of the code using \gls{TTH}.
\Citet{willis_staged_2020} used \gls{TTH} to remove the overhead of parsing combinators.
\section{Discussion}
For future work, it would be interesting to see how generating boilerplate for user-defined data types translates from shallow embedding to deep embedding.
In deep embedding, the language constructs are expressed as data types in the host language.
Adding new constructs, e.g.\ constructors, deconstructors, and constructor tests, for the user-defined data type therefore requires extending the data type.
-Techniques such as data types \`a la carte~\citep{swierstra_data_2008} and open data types~\citep{loh_open_2006} show that it is possible to extend data types orthogonally but whether metaprogramming can still readily be used is something that needs to be researched.
+Techniques such as data types \`a la carte \citep{swierstra_data_2008} and open data types \citep{loh_open_2006} show that it is possible to extend data types orthogonally but whether metaprogramming can still readily be used is something that needs to be researched.
It may also be possible to implemented (parts) of the boilerplate generation using \gls{TTH} (see \cref{ssec_fcd:typed_template_haskell}) to achieve more confidence in the type correctness of the implementation.
Another venue of research is to try to find the limits of this technique regarding richer data type definitions.
-It would be interesting to see whether it is possible to apply the technique on data types with existentially quantified type variables or full-fledged generalised \glspl{ADT}~\citep{hinze_fun_2003}.
+It would be interesting to see whether it is possible to apply the technique on data types with existentially quantified type variables or full-fledged generalised \glspl{ADT} \citep{hinze_fun_2003}.
It is not possible to straightforwardly lift the deconstructors to type classes because existentially quantified type variables will escape.
Rank-2 polymorphism offers tools to define the types in such a way that this is not the case anymore.
However, implementing compiling views on the \gls{DSL} is complicated because it would require inventing values of an existentially quantified type variable to satisfy the type system which is difficult.
repositories / phd-thesis.git / blobdiff