to send and receive data it needs to be \emph{fault tolerant}. The SUT
needs to properly handle errors in received data or complete unavailability
of the underlying channel.
- \item \textbf{Security}\\
- \emph{Confidentiality} is an important aspect for the SUT. Received data
- should only be delivered to the program to which it is addressed.
\end{itemize}
-This leaves three categories which are not relevant the SUT. Below we will
+This leaves four categories which are not relevant the SUT. Below we will
shortly discuss per category why these are not relevant. \emph{Maintainability}
is an important aspect of any software system, however for the SUT it is not a
core aspect, as it is first and foremost of importance that the implementation
core aspect either, as the SUT is not used directly by humans, but is a service
which is addressed when another program needs it. \emph{Portability} isn't
either as the SUT is installed on a system once and intended to work on that
-system.
+system. \emph{Security} isn't a feauture of the SUT either, systems using the
+SUT can add their own security mechanisms on top of it.
+
\subsubsection{Quality in use}
Quality in use is dived into five subcategories. Below we will discuss the
\subsection{Levels and types of testing} \label{levels}
The client will deliver a product for certification. This means our team will
-only conduct acceptance testing and assume that the client who requested
-certification has conducted unit, module and integration testing. We will only
-be conducting black-box testing and the client is not required to handover any
-source-code. Initially we will conduct several basic test cases based on
-experience acquired from previous certification requests (error guessing). If
-the product fails these basic tests we reject it and seize all further
-activities. If the product is not rejected we will proceed with more thorough
-testing. For each test we produce a test report. If any of the test cases fail
-the product is still rejected but in order to deliver usable feedback to the
-client we will still produce a test report. For each test case a performance
-analysis will be included.
+only conduct \emph{acceptance testing} and assume that the client who requested
+certification has conducted \emph{unit}, \emph{module} and \emph{integration testing}.
+We will only be conducting \emph{black-box testing} and the client is not required to handover any
+source-code.
+
+Initially we will conduct a few basic \emph{manual tests} based on
+experience acquired from previous certification requests (\emph{error guessing}). If
+the product fails these basic tests we immediately reject it and seize all further
+activities.
+
+If the product is not rejected after the basic \emph{manual tests} we will proceed with the
+second stage of testing. For these follow-up tests we will use \emph{equivalence partitioning}
+to reduce the number of test cases. Every test case will result in a test report.
+If any of the test cases fail the product is rejected. In order to deliver
+usable feedback to the client we will still produce a test report.
+
+\subsubsection{Manual tests}
-\subsubsection{Test generation}
The basic tests mentioned in Section \ref{levels} are conducted using a
checklist. If any of the checks fail we immediately reject the product.
\item Can we use the product to initiate a connection in a corruption
free
environment?
- \item ....
\end{enumerate}
-For the remaining tests we first use equivalence partitioning to reduce the
-overall number of test cases.
+These\emph{ manual tests} are performed in order to ensure that the client has delivered a usable product.
+
+\subsubsection{Test generation}
+
+For the second state of testing we first use \emph{equivalence partitioning} to reduce the
+overall number of test cases as mentioned in Section \ref{levels}. At the highest level we can define the following equivalent \emph{input classes} for the SUT.
\begin{enumerate}
\item Valid requests:
\end{enumerate}
\end{enumerate}
-For these requests we can introduce more cases using equivalence partitioning
-for the different packets that are sent during one request.
+For these requests we can introduce more cases by applying \emph{equivalence partitioning}
+to the different packets that are sent during one request.
\begin{enumerate}
\item Packets received in order.
\item Packets received out of order.
\end{enumerate}
-For each individual packet we can specify the follow equivalent classes.
+For each individual packet we can specify the follow \emph{equivalent classes}.
\begin{enumerate}
\item Valid packet.
\item Missing packets.
\end{enumerate}
-We will test all possible combinations of requests/packet order/packet content.
-For each combination we will use boundary value analysis to reduce the total
-number of test cases. Boundary values are constructed using the following
-parameters:
+A \emph{decision table} is used in order to ensure all different equivalent classes are covered in our tests.
+Valid packets can only exist in one form, the valid form and there is
+no need to divide these into additional classes.
+For the invalid packets we will construct a test case where only one parameter is made invalid.
+For the invalid value of this parameter we use \emph{boundary value analysis} to reduce the total
+number of test cases.
+The SUT input parameters correspond to different fields in the network packets.
+The following fields and there boundary values are considered during the testing.
\begin{enumerate}
- \item Checksum: valid/invalid
- \item Header: valid/invalid
- \item Payload: valid/invalid
- \item ...
+ \item Checksum: valid, invalid.
+ \item Header: valid, even or odd number of bits corrupted.
+ \item Payload: valid, even or odd number of bits corrupted.
\end{enumerate}
\subsubsection{Test environment and automatization}
+\label{section:testenv}
+%%%Java\footnote{\url{http://www.java.com}} TCP driven echo server that executed
+%%%on a virtualized Ubuntu system\footnote{\url{http://www.ubuntu.com}} running on
+%%%the error behaviour custom iptables output policies have to be set
+%%%Listing~\ref{listing:iptables}. This is needed because the kernel by default
+%%%closes all connections from unknown sources and the manually created TCP
+%%%packets used in testing the implementation are from a source unknown to the
+%%%kernel.
+%%%
+%%%\begin{lstlisting}[label={listing:iptables},caption={settings iptables}]
+%%%Chain OUTPUT (policy ACCEPT)
+%%%target prot opt source destination
+%%%ACCEPT tcp -- anywhere anywhere tcp flags:PSH/PSH
+%%%DROP tcp -- anywhere anywhere tcp flags:RST/RST
+%%%\end{lstlisting}
+
All the tools we are going to use together with the SUT gives us the following
collection of software.
-\begin{enumerate}
- \item Windows, used as a host OS.
- \item Linux, used as both a host and guest OS.
- \item VirtualBox, used to run the guest OS containing the SUT.
- \item Wireshark, used on the guest in order to capture and analyze network
+\begin{itemize}
+ \item Windows\footnote{\url{http://www.microsoft.com/en-us/windows}}, used as a host OS.
+ \item Ubuntu\footnote{\url{http://www.ubuntu.com}}, used as the guest OS running the SUT.
+ \item VirtualBox\footnote{\url{https://www.virtualbox.org/}}, used to run the guest OS containing the SUT.
+ \item Wireshark\footnote{\url{https://www.wireshark.org/}}, used on the guest in order to capture and analyze network
traffic.
- \item Tcpdump, used to prepare network packets.
-\end{enumerate}
+ \item Bit-Twist\footnote{\url{http://bittwist.sourceforge.net/}}, used to prepare network packets.
+ \item Java\footnote{\url{http://www.java.com}} TCP driven echo server.
+\end{itemize}
All test will be conducted in a virtual environment. We will use VirtualBox to
-run a Linux distribution with the product installed. All the tests are
-performed from within the VirtualBox environment. When testing network
-transmissions we will only analyze the packets sent/received to/from the guest
-OS. The host system is disconnected from the Internet or any other network in
+run a Linux distribution with the product installed.
+The Linux distribution in question is Ubuntu.
+All the tests are performed from within the VirtualBox environment.
+When testing network transmissions we will only analyze the packets sent/received to/from the SUT. The host system is disconnected from the Internet or any other network in
order to prevent unnecessary traffic.
% Dit is niet nodig omdat het via loopback gaat
+% Zeker weten? de SUT ontvangt ook niet loopback packets toch?
-For each test case (except for the basic tests) a file containing previously
-captured network traffic will be replayed using Wireshark. We will use tcpdump
+For each test case (except for the \emph{manual tests}) a file containing previously
+captured network traffic will be replayed using Wireshark and sent to the \emph{Java TCP driven echo server}. We will use Bit-Twist
to update the prepared packets with the MAC address of the guest network
-adapter. The response packets coming from the guest OS will be recorded and
-analyzed at a later stage. The valid packets are obtained by capturing traffic
-between known working alternatives to the SUT. Invalid packets are generated
-from this valid traffic using tcpdump. The boundary values for the different
-parameters (fields in packets) are determined by hand. Automated scripts are
-written that will generate packets with some fields replaced with these
-boundary values. The performance analysis will consists of measured latencies
-for all packets sent.
-
-% Dit is mooier om footnotes van te maken en te gebruiken als het voor het
-% eerst gerefereerd is
-\begin{enumerate}
- \item VirtualBox, https://www.virtualbox.org/
- \item Whireshark, https://www.wireshark.org/
- \item Tcpdump, http://www.tcpdump.org/
-\end{enumerate}
+adapter and provide them with a valid source address. This updating step is needed because the kernel would otherwise reject the packets and prevent them from reaching the SUT. The response packets sent by the \emph{Java TCP driven echo server} and passing trough the SUT will be recorded,
+analyzed and validated according to the \textit{RFC793} specification.
+The valid packets are build manually from the \textit{RFC793} specification.
+Invalid packets are generated from this valid traffic using Bit-Twist.
+The boundary values for the different parameters (fields in packets) are determined by hand. Automated scripts are used in order to generate packets with some fields replaced with these
+\emph{boundary values}.
repositories / tt2015.git / blobdiff