--- /dev/null
+\begin{enumerate}
+ % Question 1a
+ \item Disfluencies are annotated by surrounding them with square braces.
+ The first bit shows the \emph{reparandum}, the second bit denoted with
+ the \texttt{+} shows the \emph{editing phase} and the last bit shows
+ the \emph{repair}. We want to only keep the repair since that depicts
+ the correct, meant by the speaker, speech.
+
+ \verb#s/\[.*?\+\{.*?\}(.*?)\]/\1/g#
+
+ Bit by bit:
+ \begin{itemize}
+ \item \verb#s/# Substitution.
+ \item \verb#\[# Matches the opening square bracket. We escape this
+ because \verb#[# is a regular expression control character and
+ we want to match a literal.
+ \item \verb#.*?\+# Matches non-greedily everything up to the plus
+ mark. Thus the \emph{reparandum}. Note that the
+ \emph{reparandum} can be empty (in case the speaker immediately
+ start editing). We escape the \verb#+# for the same reason as
+ the previous segment.
+ \item \verb#\{.*?\}# Matches everything between the curly braces.
+ Thus the \emph{editing phase}. Note again that this match can
+ only contain empty curly braces since the \emph{editing phase}
+ can be empty.
+ \item \verb#(.*?)# Matches non-greedily everything up to the
+ closing square brace and captures it in the group. Thus the
+ \emph{repair}. Note that we do not require this group to be the
+ exact same as the \emph{reparandum}.
+ \item \verb#\]/# Matches the closing square bracket and we proceed
+ to the replacement. We escape this for the same reason as
+ before.
+ \item \verb#\1/g# We replace the entire match with only the
+ captured \emph{repair} group and do this globally since there
+ can be multiple repairs in an utterance.
+ \end{itemize}
+
+ % Question 1b
+ \item \textsc{MEMM}'s use features to add extra information to words.
+ \textsc{IOB} tagging is a partial parsing or chunking method that only
+ discriminates between \emph{Beginning} (\texttt{B}), \emph{Internal}
+ (\texttt{I}) and \emph{Outside} (\texttt{O}) categories.
+
+ Say we use the same segmentation as before, we should mark the
+ \emph{reparandum} and \emph{editing phase} as \emph{Outside}
+ (\texttt{O}) parts and the repair should be parsed as usual. Note that
+ a chunk then can include \texttt{O} marked segments. For example in ``a
+ car uh plane'' the ``car uh'' part will be tagged as \texttt{O}, ``a''
+ as \texttt{B\_NP} and ``plane'' as \texttt{I\_NP}.
+
+ For the algorithms it might be necessary to add a different tag to
+ denote internal \texttt{O} segments. This can be done by adding a
+ suffix to the \texttt{O} tag. In the previous example the text will
+ then be chunked as: \texttt{B\_NP O\_NP I\_NP}.
+
+ Concerning the \textsc{MEMM} features, obviously editing phase segments
+ should be marked as such but also the reparandum should be tagged as
+ such to not confuse it with a regular segment.
+
+ % Question 1c
+ \item Repairs are only noticed when you can lookahead to the \emph{editing
+ phase} markers. It might be necessary to either lookahead a little bit
+ or to work outwards from the identified \emph{editing phase}.
+ Right-to-left has the same problem as left-to-right in the sense that
+ it will see the repair first and also has to lookahead to know whether
+ it is part of a repair.
+\end{enumerate}
repositories / itlast1617.git / blobdiff