The test process consists of several stages. The results of the first stage,
planning, are descibed in this document. On the basis of this document actual
test cases will be designed. Afterwards the actual tests will be implemented
and executed. The results of these tests will then be evaluated against the
exit criteria found in Section~\ref{sec:exitcriteria} and depending on the
outcome of these evaluations further tests might be deemed necessary.

\subsection{Quality Characteristics}
The quality characteristics that are to be tested are described using the
ISO/IEC 25010~\cite{iso25010} as a guideline. In the following sections we will
discuss the relevant \textit{product quality} and \textit{quality in use}
characteristics.

\subsubsection{Product quality}
Product quality is divided into eight main categories which are divided into
several subcategories. Below we will discuss the qualities which are relevant
to the SUT.
\begin{itemize}
	\item \textbf{Functional suitability}\\
	As described in Section~\ref{sec:risks} the SUT is core functionality of
	the networking capable system. Because many other systems running on the
	system could rely on it it is very important that the SUT is functionality
	suitable. Therefore all three sub characteristics of Functional
	Suitability (\textit{Functional completeness, Functional correctness,
	Functional appropriateness}) are of vital importance. As was previously
	mentioned in Section~\ref{sec:risks} extra emphasis should be placed on
	testing \emph{Functional Correctness} as recovery from Failures in
	computer-to-computer systems is problematic.
	\item \textbf{Performance efficiency} \label{sec:perf_eff}\\ 
	As the SUT runs as a service on a system with other programs it must have
	efficient \emph{resource utilisation}. It can not contain any memory leaks
	or use other resources more than necessary.
	\item \textbf{Compatibility}\\ 
	\emph{Interoperability} is the key feature of the SUT as it's purpose is to
	communicate with other systems implementing the TCP protocol. Therefore it
	is of vital importance that the SUT implements the TCP protocol correctly.
	Furthermore it is very important that the SUT can \emph{co-exist} with
	other programs on the system it runs on, since it is used as a service by
	those programs. This means that the SUT has to handle preemption as well as
	having multiple programs requesting it's services at once.
	\item \textbf{Reliability}\\ 
	As stated before, the SUT is used as a core service, this means it has to
	be very \emph{mature}. It needs to behave as expected under normal working
	conditions. As it can continually be requested the SUT needs to have
	constant
	\emph{availability}. As the SUT relies on a potentially unreliable channel
	to send and receive data it needs to be \emph{fault tolerant}. The SUT
	needs to properly handle errors in received data or complete unavailability
	of the underlying channel. 
\end{itemize}
This leaves four categories which are not relevant the SUT. Below we will
shortly discuss per category why these are not relevant. \emph{Maintainability}
is an important aspect of any software system, however for the SUT it is not a
core aspect, as it is first and foremost of importance that the implementation
is correct, furthermore TCP does not change often. \emph{Usability} isn't a
core aspect either, as the SUT is not used directly by humans, but is a service
which is addressed when another program needs it. \emph{Portability} isn't
either as the SUT is installed on a system once and intended to work on that
system. \emph{Security} isn't a feature of the SUT either, systems using the
SUT can add their own security mechanisms on top of it. 


\subsubsection{Quality in use}
Quality in use is dived into five subcategories. Below we will discuss the 
categories which are relevant to the SUT. 
\begin{itemize}
	\item \textbf{Effectiveness}\\ 
	This is the core aspect of the SUT, users (other programs) need to be able
	to effectively use the SUT to send and receive data.
	\item \textbf{Efficiency}\\ 
	This issue has already been covered above under 
	``performance efficiency''~\ref{sec:perf_eff}.
	\item \textbf{Satisfaction}\\ 
	It is important that programs using the SUT can \emph{trust} that the SUT
	provides the promised services. This means that data is send and received
	reliably and the SUT provides clear and unambiguous errors when this
	service 
	can not be provided.
	\item \textbf{Context Coverage}\\ 
	The SUT needs to behave as expected in all specified contexts 
	(\emph{context completeness}).
\end{itemize}
This leaves \emph{freedom from risk}, which we consider not relevant as the SUT
itself does not pose any risks, and correct implementation (which is covered in
the other categories) gives clear guarantees to programs using the services of
the SUT. 

\subsection{Levels and types of testing} \label{section:levels}
The client will deliver a product for certification. This means our team will
only conduct \emph{acceptance testing} and assume that the client who requested
certification has conducted \emph{unit}, \emph{module} and \emph{integration
testing}. We will only be conducting \emph{black-box testing} and the client
is not required to handover any source-code.

Initially we will conduct a few basic \emph{manual tests} based on experience
acquired from previous certification requests (\emph{error guessing}). If the
product fails these basic tests we immediately reject it and seize all further
activities. 

If the product is not rejected after the basic \emph{manual tests} we will
proceed with the second stage of testing. For these follow-up tests we will use
\emph{equivalence partitioning} to reduce the number of test cases. Every test
case will result in a test report. If any of the test cases fail the product
is rejected. In order to deliver usable feedback to the client we will still
produce a test report. 

\subsubsection{Manual tests}

The basic tests mentioned in Section~\ref{section:levels} are conducted using a
checklist. If any of the checks fail we immediately reject the product.

\begin{enumerate}
	\item Is the product complete?
	\item Does the product come with a manual or quick start guide?
	\item Is it possible to get the product in a usable state?
	\item Can we use the product to initiate a connection in a corruption
	free
	environment?
\end{enumerate}

These\emph{ manual tests} are performed in order to ensure that the client has
delivered a usable product.

\subsubsection{Test generation}

For the second state of testing we first use \emph{equivalence partitioning} to
reduce the overall number of test cases as mentioned in
Section~\ref{section:levels}. At the highest level we can define the following
equivalent \emph{input classes} for the SUT.

\begin{enumerate}
	\item Valid requests:
		\begin{enumerate}
			\item Single request.
			\item Multiple requests.
		\end{enumerate}
	\item Invalid requests:
		\begin{enumerate}
			\item Single request.
			\item Multiple requests.
		\end{enumerate}	
\end{enumerate}

For these requests we can introduce more cases by applying \emph{equivalence
partitioning} to the different packets that are sent during one request.

\begin{enumerate}
	\item Packets received in order.
	\item Packets received out of order.
\end{enumerate}

For each individual packet we can specify the follow \emph{equivalent classes}.

\begin{enumerate}
	\item Valid packet.
	\item Corrupted packet.
	\item Missing packets.
\end{enumerate}

A \emph{decision table} is used in order to ensure all different equivalent
classes are covered in our tests. Valid packets can only exist in one form,
the valid form and there is no need to divide these into additional classes.
For the invalid packets we will construct a test case where only one parameter
is made invalid. For the invalid value of this parameter we use \emph{boundary
value analysis} to reduce the total number of test cases. The SUT input
parameters correspond to different fields in the network packets. The
following fields and there boundary values are considered during the testing.

\begin{enumerate}
	\item Checksum: valid, invalid.
	\item Header: valid, even or odd number of bits corrupted.
	\item Payload: valid, even or odd number of bits corrupted.
\end{enumerate}

\subsubsection{Test environment and automatization}
\label{section:testenv}
All the tools we are going to use together with the SUT gives us the following
collection of software 

\begin{itemize}
	\item Windows\footnote{\url{http://www.microsoft.com/en-us/windows}}, used
			as a host OS.
	\item Ubuntu\footnote{\url{http://www.ubuntu.com}}, used as the guest OS
			running the SUT.
	\item VirtualBox\footnote{\url{https://www.virtualbox.org/}}, used to run
			the guest OS containing the SUT.
	\item Wireshark\footnote{\url{https://www.wireshark.org/}}, used on the
			guest OS in order to capture and analyze network traffic.
	\item Bit-Twist\footnote{\url{http://bittwist.sourceforge.net/}}, used to
			prepare network packets.
	\item Java\footnote{\url{http://www.java.com}} TCP driven echo server.
\end{itemize}

\begin{figure}[H]
		\label{fig:sut}
		\centering
		\includegraphics[width=0.5\linewidth]{SUTsetup.eps}
		\caption{Test environment}
\end{figure}


All test will be conducted in a virtual environment. We will use VirtualBox to
run a Linux distribution with the product installed. The Linux distribution in
question is Ubuntu. All the tests are performed from within the VirtualBox
environment. When testing network transmissions we will only analyze the
packets sent/received to/from the SUT. The host system is disconnected from the
Internet or any other network in order to prevent unnecessary traffic.
% Dit is niet nodig omdat het via loopback gaat
% 	Zeker weten? de SUT ontvangt ook niet loopback packets toch?
%		Ja maar we geven de host gewoon geen interface?

For each test case (except for the \emph{manual tests}) a file containing
previously captured network traffic will be replayed using Wireshark and sent
to the \emph{Java TCP driven echo server}. We will use Bit-Twist to update the
prepared packets with the MAC address of the guest network adapter and provide
them with a valid source address. This updating step is needed because the
kernel would otherwise reject the packets and prevent them from reaching the
SUT. The response packets sent by the \emph{Java TCP driven echo server} and
passing trough the SUT will be recorded, analyzed and validated according to
the \textit{RFC793} specification. The valid packets are build manually from
the \textit{RFC793} specification. Invalid packets are generated from this
valid traffic using Bit-Twist. The boundary values for the different
parameters (fields in packets) are determined by hand. Automated scripts are
used in order to generate packets with some fields replaced with these
\emph{boundary values}.
