-Hello world
+The test process consists of several stages. The results of the first stage,
+planning, are descibed in this document. On the basis of this document actual
+test cases will be designed. Afterwards the actual tests will be implemented
+and executed. The results of these tests will then be evaluated against the
+exit criteria found in Section~\ref{sec:exitcriteria} and depending on the
+outcome of these evaluations further tests might be deemed necessary.
+
+\subsection{Quality Characteristics}
+The quality characteristics that are to be tested are described using the
+ISO/IEC 25010 \cite{iso25010} as a guideline. In the following sections we will
+discuss the relevant \textit{product quality} and \textit{quality in use}
+characteristics.
+
+\subsubsection{Product quality}
+Product quality is divided into eight main categories which are divided into
+several subcategories. Below we will discuss the qualities which are relevant
+to the SUT.
+\begin{itemize}
+ \item \textbf{Functional suitability}\\
+ As described in Section~\ref{sec:risks} the SUT is core functionality of
+ the networking capable system. Because many other systems running on the
+ system could rely on it it is very important that the SUT is functionality
+ suitable. Therefore all three sub characteristics of Functional
+ Suitability (\textit{Functional completeness, Functional correctness,
+ Functional appropriateness}) are of vital importance. As was previously
+ mentioned in Section~\ref{sec:risks} extra emphasis should be placed on
+ testing \emph{Functional Correctness} as recovery from Failures in
+ computer-to-computer systems is problematic.
+ \item \textbf{Performance efficiency} \label{sec:perf_eff}\\
+ As the SUT runs as a service on a system with other programs it must have
+ efficient \emph{resource utilisation}. It can not contain any memory leaks
+ or use other resources more than necessary.
+ \item \textbf{Compatibility}\\
+ \emph{Interoperability} is the key feature of the SUT as it's purpose is to
+ communicate with other systems implementing the TCP protocol. Therefore it
+ is of vital importance that the SUT implements the TCP protocol correctly.
+ Furthermore it is very important that the SUT can \emph{co-exist} with
+ other programs on the system it runs on, since it is used as a service by
+ those programs. This means that the SUT has to handle preemption as well as
+ having multiple programs requesting it's services at once.
+ \item \textbf{Reliability}\\
+ As stated before, the SUT is used as a core service, this means it has to
+ be very \emph{mature}. It needs to behave as expected under normal working
+ conditions. As it can continually be requested the SUT needs to have
+ constant
+ \emph{availability}. As the SUT relies on a potentially unreliable channel
+ to send and receive data it needs to be \emph{fault tolerant}. The SUT
+ needs to properly handle errors in received data or complete unavailability
+ of the underlying channel.
+ \item \textbf{Security}\\
+ \emph{Confidentiality} is an important aspect for the SUT. Received data
+ should only be delivered to the program to which it is addressed.
+\end{itemize}
+This leaves three categories which are not relevant the SUT. Below we will
+shortly discuss per category why these are not relevant. \emph{Maintainability}
+is an important aspect of any software system, however for the SUT it is not a
+core aspect, as it is first and foremost of importance that the implementation
+is correct, furthermore TCP does not change often. \emph{Usability} isn't a
+core aspect either, as the SUT is not used directly by humans, but is a service
+which is addressed when another program needs it. \emph{Portability} isn't
+either as the SUT is installed on a system once and intended to work on that
+system.
+
+\subsubsection{Quality in use}
+Quality in use is dived into five subcategories. Below we will discuss the
+categories which are relevant to the SUT.
+\begin{itemize}
+ \item \textbf{Effectiveness}\\
+ This is the core aspect of the SUT, users (other programs) need to be able
+ to effectively use the SUT to send and receive data.
+ \item \textbf{Efficiency}\\
+ This issue has already been covered above under
+ ``performance efficiency''~\ref{sec:perf_eff}.
+ \item \textbf{Satisfaction}\\
+ It is important that programs using the SUT can \emph{trust} that the SUT
+ provides the promised services. This means that data is send and received
+ reliably and the SUT provides clear and unambiguous errors when this
+ service
+ can not be provided.
+ \item \textbf{Context Coverage}\\
+ The SUT needs to behave as expected in all specified contexts
+ (\emph{context completeness}).
+\end{itemize}
+This leaves \emph{freedom from risk}, which we consider not relevant as the SUT
+itself does not pose any risks, and correct implementation (which is covered in
+the other categories) gives clear guarantees to programs using the services of
+the SUT.
+
+\subsection{Levels and types of testing} \label{levels}
+The client will deliver a product for certification. This means our team will
+only conduct acceptance testing and assume that the client who requested
+certification has conducted unit, module and integration testing. We will only
+be conducting black-box testing and the client is not required to handover any
+source-code. Initially we will conduct several basic test cases based on
+experience acquired from previous certification requests (error guessing). If
+the product fails these basic tests we reject it and seize all further
+activities. If the product is not rejected we will proceed with more thorough
+testing. For each test we produce a test report. If any of the test cases fail
+the product is still rejected but in order to deliver usable feedback to the
+client we will still produce a test report. For each test case a performance
+analysis will be included.
+
+\subsubsection{Test generation}
+The basic tests mentioned in Section \ref{levels} are conducted using a
+checklist. If any of the checks fail we immediately reject the product.
+
+\begin{enumerate}
+ \item Is the product complete?
+ \item Does the product come with a manual or quick start guide?
+ \item Is it possible to get the product in a usable state?
+ \item Can we use the product to initiate a connection in a corruption
+ free
+ environment?
+ \item ....
+\end{enumerate}
+
+For the remaining tests we first use equivalence partitioning to reduce the
+overall number of test cases.
+
+\begin{enumerate}
+ \item Valid requests:
+ \begin{enumerate}
+ \item Single request.
+ \item Multiple requests.
+ \end{enumerate}
+ \item Invalid requests:
+ \begin{enumerate}
+ \item Single request.
+ \item Multiple requests.
+ \end{enumerate}
+\end{enumerate}
+
+For these requests we can introduce more cases using equivalence partitioning
+for the different packets that are sent during one request.
+
+\begin{enumerate}
+ \item Packets received in order.
+ \item Packets received out of order.
+\end{enumerate}
+
+For each individual packet we can specify the follow equivalent classes.
+
+\begin{enumerate}
+ \item Valid packet.
+ \item Corrupted packet.
+ \item Missing packets.
+\end{enumerate}
+
+We will test all possible combinations of requests/packet order/packet content.
+For each combination we will use boundary value analysis to reduce the total
+number of test cases. Boundary values are constructed using the following
+parameters:
+
+\begin{enumerate}
+ \item Checksum: valid/invalid
+ \item Header: valid/invalid
+ \item Payload: valid/invalid
+ \item ...
+\end{enumerate}
+
+\subsubsection{Test environment and automatization}
+\label{section:testenv}
+%%%Java\footnote{\url{http://www.java.com}} TCP driven echo server that executed
+%%%on a virtualized Ubuntu system\footnote{\url{http://www.ubuntu.com}} running on
+%%%the error behaviour custom iptables output policies have to be set
+%%%Listing~\ref{listing:iptables}. This is needed because the kernel by default
+%%%closes all connections from unknown sources and the manually created TCP
+%%%packets used in testing the implementation are from a source unknown to the
+%%%kernel.
+%%%
+%%%\begin{lstlisting}[label={listing:iptables},caption={settings iptables}]
+%%%Chain OUTPUT (policy ACCEPT)
+%%%target prot opt source destination
+%%%ACCEPT tcp -- anywhere anywhere tcp flags:PSH/PSH
+%%%DROP tcp -- anywhere anywhere tcp flags:RST/RST
+%%%\end{lstlisting}
+%%%All the tools we are going to use together with the SUT gives us the following
+%%%collection of software.
+
+\begin{enumerate}
+ \item Windows, used as a host OS.
+ \item Linux, used as both a host and guest OS.
+ \item VirtualBox, used to run the guest OS containing the SUT.
+ \item Wireshark, used on the guest in order to capture and analyze network
+ traffic.
+ \item Tcpdump, used to prepare network packets.
+\end{enumerate}
+
+All test will be conducted in a virtual environment. We will use VirtualBox to
+run a Linux distribution with the product installed. All the tests are
+performed from within the VirtualBox environment. When testing network
+transmissions we will only analyze the packets sent/received to/from the guest
+OS. The host system is disconnected from the Internet or any other network in
+order to prevent unnecessary traffic.
+% Dit is niet nodig omdat het via loopback gaat
+
+For each test case (except for the basic tests) a file containing previously
+captured network traffic will be replayed using Wireshark. We will use tcpdump
+to update the prepared packets with the MAC address of the guest network
+adapter. The response packets coming from the guest OS will be recorded and
+analyzed at a later stage. The valid packets are obtained by capturing traffic
+between known working alternatives to the SUT. Invalid packets are generated
+from this valid traffic using tcpdump. The boundary values for the different
+parameters (fields in packets) are determined by hand. Automated scripts are
+written that will generate packets with some fields replaced with these
+boundary values. The performance analysis will consists of measured latencies
+for all packets sent.
+
+% Dit is mooier om footnotes van te maken en te gebruiken als het voor het
+% eerst gerefereerd is
+\begin{enumerate}
+ \item VirtualBox, https://www.virtualbox.org/
+ \item Whireshark, https://www.wireshark.org/
+ \item Tcpdump, http://www.tcpdump.org/
+\end{enumerate}
repositories / tt2015.git / blobdiff