\subsection{Backend}
\subsubsection{Program description}
The backend consists of a main module and a set of libraries all written in
-\textit{Python}\footnote{\url{https://www.python.org/}}. The main module can,
+\textit{Python}\cite{Python}. The main module can,
and is, be embedded in an apache
webserver\footnote{\url{https://httpd.apache.org/}} via the
-\textit{mod\_python} apache module\footnote{\url{http://modpython.org/}}. The
-module \textit{mod\_python} allows the webserver to execute \textit{Python}
-code in the webserver. We chose Python because of the rich set of standard
-libraries and solid cross platform capabilities. We chose Python 2 because it
-is still the default Python version on all major operating systems and stays
-supported until at least the year 2020 meaning that the program can function
-safe at least 5 full years. The application consists of a main Python module
-that is embedded in the webserver. Finally there are some libraries and there
-is a standalone program that does the periodic crawling.
+\textit{mod\_python} apache module\cite{Modpython}. The module
+\textit{mod\_python} allows the webserver to execute Python code in
+the webserver. We chose Python because of the rich set of standard libraries
+and solid cross platform capabilities. We chose Python 2 because it is still
+the default Python version on all major operating systems and stays supported
+until at least the year 2020 meaning that the program can function safe at
+least 5 full years. The application consists of a main Python module that is
+embedded in the webserver. Finally there are some libraries and there is a
+standalone program that does the periodic crawling.
\subsubsection{Main module}
The main module is the program that deals with the requests, controls the
dictionary where all the entries are hashed so that the crawler knows which
ones are already present in the database and which ones are new so that it
does not have to process all the old entries when they appear in the feed. The
-crawler also has a function to export the database to XML format. The XML
-format is specified in an XSD file for minimal ambiguity.
+RSS's GUID could also have been used but since it is an optional value in the
+feed not every feed uses the GUID and therefore it is not reliable to use it.
+The crawler also has a function to export the database to XML format. The XML
+format is specified in an XSD\cite{Xsd} file for minimal ambiguity.
+
+An XSD file is a file that precisely describes the field the XML file uses.
+As almost every programming language contains an XML library most of the
+languages also contain a XSD library that allows the XML library to parse files
+according to the scheme and in that way the programmer knows exactly what to
+expect as XML fields. The XSD file for this program can be found in the
+appendices in Listing~\ref{scheme.xsd}.
repositories / bsc-thesis1415.git / blobdiff