\subsection{Levels and types of testing} \label{levels}
The client will deliver a product for certification. This means our team will
-only conduct acceptance testing and assume that the client who requested
-certification has conducted unit, module and integration testing. We will only
-be conducting black-box testing and the client is not required to handover any
-source-code. Initially we will conduct several basic test cases based on
-experience acquired from previous certification requests (error guessing). If
-the product fails these basic tests we reject it and seize all further
-activities. If the product is not rejected we will proceed with more thorough
-testing. For each test we produce a test report. If any of the test cases fail
-the product is still rejected but in order to deliver usable feedback to the
-client we will still produce a test report. For each test case a performance
-analysis will be included.
+only conduct \emph{acceptance testing} and assume that the client who requested
+certification has conducted \emph{unit}, \emph{module} and \emph{integration testing}.
+We will only be conducting \emph{black-box testing} and the client is not required to handover any
+source-code.
+
+Initially we will conduct a few basic \emph{manual tests} based on
+experience acquired from previous certification requests (\emph{error guessing}). If
+the product fails these basic tests we immediately reject it and seize all further
+activities.
+
+If the product is not rejected after the basic \emph{manual tests} we will proceed with the
+second stage of testing. For these follow-up tests we will use \emph{equivalence partitioning}
+to reduce the number of test cases. Every test case will result in a test report.
+If any of the test cases fail the product is rejected. In order to deliver
+usable feedback to the client we will still produce a test report.
+
+\subsubsection{Manual tests}
-\subsubsection{Test generation}
The basic tests mentioned in Section \ref{levels} are conducted using a
checklist. If any of the checks fail we immediately reject the product.
\item Can we use the product to initiate a connection in a corruption
free
environment?
- \item ....
\end{enumerate}
-For the remaining tests we first use equivalence partitioning to reduce the
-overall number of test cases.
+These\emph{ manual tests} are performed in order to ensure that the client has delivered a usable product.
+
+\subsubsection{Test generation}
+
+For the second state of testing we first use \emph{equivalence partitioning} to reduce the
+overall number of test cases as mentioned in Section \ref{levels}. At the highest level we can define the following equivalent \emph{input classes} for the SUT.
\begin{enumerate}
\item Valid requests:
\end{enumerate}
\end{enumerate}
-For these requests we can introduce more cases using equivalence partitioning
-for the different packets that are sent during one request.
+For these requests we can introduce more cases by applying \emph{equivalence partitioning}
+to the different packets that are sent during one request.
\begin{enumerate}
\item Packets received in order.
\item Packets received out of order.
\end{enumerate}
-For each individual packet we can specify the follow equivalent classes.
+For each individual packet we can specify the follow \emph{equivalent classes}.
\begin{enumerate}
\item Valid packet.
\item Missing packets.
\end{enumerate}
-We will test all possible combinations of requests/packet order/packet content.
-For each combination we will use boundary value analysis to reduce the total
-number of test cases. Boundary values are constructed using the following
-parameters:
+A \emph{decision table} is used in order to ensure all different equivalent classes are covered in our tests.
+Valid packets can only exist in one form, the valid form and there is
+no need to divide these into additional classes.
+For the invalid packets we will construct a test case where only one parameter is made invalid.
+For the invalid value of this parameter we use \emph{boundary value analysis} to reduce the total
+number of test cases.
+The SUT input parameters correspond to different fields in the network packets.
+The following fields and there boundary values are considered during the testing.
\begin{enumerate}
- \item Checksum: valid/invalid
- \item Header: valid/invalid
- \item Payload: valid/invalid
- \item ...
+ \item Checksum: valid, invalid.
+ \item Header: valid, even or odd number of bits corrupted.
+ \item Payload: valid, even or odd number of bits corrupted.
\end{enumerate}
\subsubsection{Test environment and automatization}
\begin{enumerate}
\item Windows, used as a host OS.
- \item Linux, used as both a host and guest OS.
+ \item Ubuntu, used as the guest OS running the SUT.
\item VirtualBox, used to run the guest OS containing the SUT.
\item Wireshark, used on the guest in order to capture and analyze network
traffic.
- \item Tcpdump, used to prepare network packets.
+ \item Bit-Twist, used to prepare network packets.
\end{enumerate}
All test will be conducted in a virtual environment. We will use VirtualBox to
-run a Linux distribution with the product installed. All the tests are
-performed from within the VirtualBox environment. When testing network
-transmissions we will only analyze the packets sent/received to/from the guest
-OS. The host system is disconnected from the Internet or any other network in
+run a Linux distribution with the product installed.
+The Linux distribution in question is Ubuntu.
+All the tests are performed from within the VirtualBox environment.
+When testing network transmissions we will only analyze the packets sent/received to/from the SUT. The host system is disconnected from the Internet or any other network in
order to prevent unnecessary traffic.
% Dit is niet nodig omdat het via loopback gaat
+% Zeker weten? de SUT ontvangt ook niet loopback packets toch?
-For each test case (except for the basic tests) a file containing previously
-captured network traffic will be replayed using Wireshark. We will use tcpdump
+For each test case (except for the \emph{manual tests}) a file containing previously
+captured network traffic will be replayed using Wireshark. We will use Bit-Twist
to update the prepared packets with the MAC address of the guest network
-adapter. The response packets coming from the guest OS will be recorded and
-analyzed at a later stage. The valid packets are obtained by capturing traffic
-between known working alternatives to the SUT. Invalid packets are generated
-from this valid traffic using tcpdump. The boundary values for the different
-parameters (fields in packets) are determined by hand. Automated scripts are
-written that will generate packets with some fields replaced with these
-boundary values. The performance analysis will consists of measured latencies
-for all packets sent.
+adapter. The response packets coming from the SUT will be recorded and
+analyzed and validated according to the \textit{RFC793} specification.
+The valid packets are build manually from the \textit{RFC793} specification.
+Invalid packets are generated from this valid traffic using Bit-Twist.
+The boundary values for the different parameters (fields in packets) are determined by hand. Automated scripts are used in order to generate packets with some fields replaced with these
+\emph{boundary values}.
% Dit is mooier om footnotes van te maken en te gebruiken als het voor het
% eerst gerefereerd is
+% WAT HOE?
\begin{enumerate}
\item VirtualBox, https://www.virtualbox.org/
\item Whireshark, https://www.wireshark.org/
- \item Tcpdump, http://www.tcpdump.org/
+ \item Bit-Twist, http://bittwist.sourceforge.net/
\end{enumerate}
repositories / tt2015.git / commitdiff