+
+\pagebreak
+\begin{table}
\subsection{Preflight checklist}
Before actual certification is commenced we perform a manual test using a
checklist.
If any of the checks fail we immediately reject the product.
+The checklist is given in the table below.
-\begin{enumerate}
- \item Is the product complete?
- \item Does the product come with a manual or quick start guide?
- \item Is it possible to get the product in a usable state?
- \item Can we use the product to initiate a connection in a corruption
- free environment?
-\end{enumerate}
+\begin{tabularx}{\linewidth}{| l | X|}
+ \hline
+Check 1 & Get the SUT in a workable state. \\\hline
+Course of action & \begin{enumerate}
+ \item Import the VirtualBox image into VirtualBox.
+ \item Boot the vm.
+ \item Verify the SUT booted succesfully and the network module(s) are loaded.
+\end{enumerate} \\\hline
+Passed & \textit{Yes/No} \\\hline \hline
+Check 2 & Verify the SUT is complete. \\\hline
+Course of action & \begin{enumerate}
+ \item Use the course of action for \emph{Check 1} in order to boot the SUT.
+ \item Verify the loopback device exists using \emph{ifconfig}
+ \item Verify the \emph{echo-server} is present on the system.
+ \item Verify \emph{Scrappy} is present on the system.
+ \item Verify all scripts used for testing are present on the system.
+\end{enumerate} \\\hline
+Passed & \textit{Yes/No} \\\hline \hline
+Check 3 & Initialize the testing environment. \\\hline
+Course of action & \begin{enumerate}
+ \item Boot the SUT (see \emph{Check 1}).
+ \item Setup iptables by executing \emph{./scripts/setup/iptables.sh}
+ \item Setup ... by executing \emph{./scripts/setup/...}
+\end{enumerate} \\\hline
+Passed & \textit{Yes/No} \\\hline
+\end{tabularx}
+\end{table}
-\textbf{DIT MOET VERANDERD WORDEN, HET PRODUCT ZOU AFGEKEURD WORDEN!}
+\begin{table}
+\begin{tabularx}{\linewidth}{| l | X |}
+\hline
+Check 4 & Test the tool environment. \\\hline
+Course of action & \begin{enumerate}
+ \item Boot and initialize the SUT (see \emph{Check 4})
+ \item Execute the \emph{./scripts/tests/case0-helloworld.sh} script.
+ \item Execute the command \emph{cat logs/case0.log}.
+ \item Verify the console displays a success message.
+\end{enumerate} \\\hline
+Passed & \textit{Yes/No} \\\hline \hline
+Check 5 & All test inputs and scripts are present. \\\hline
+Course of action & \begin{enumerate}
+ \item Boot the SUT (see \emph{Check 1})
+ \item Execute the command \emph{ls scripts/tests}
+ \item Verify that there is a file named case$X$ for each test case $X$
+ \item Use \emph{ls packets}
+ \item Verify that there is a file named \emph{single.pcap} and \emph{multi.pcap}.
+\end{enumerate} \\\hline
+Passed & \textit{Yes/No} \\\hline
+\end{tabularx}
+\end{table}
+
+\newpage
\subsection{Testing of SUT}
\item Packets received in or out of order, or missing packets
\end{enumerate}
-\textbf{hier iets over waarom deze partities relevant zijn!}
+\textbf{hier iets over waarom deze partities relevant zijn! Waarom odd en
+even number of bits flipped bijv interessant?}
+\bigskip
Partitions 2 to 6 are tested using pairwise testing to keep the number of test
cases feasible. The pairs are then all *except some where it does not make sense
to do so) tested with the different request sizes of partition 1.
-This is expressed in the table below.
+This is expressed in Table~\ref{table:testpairs}.
-\vspace{3mm}
-\textbf{Hier daadwerkelijke decision tabel}
-\vspace{3mm}
+\begin{table}
+\begin{tabular} {p{2cm} | l | p{2cm} | l | p{1cm} || c | c | p{1.3cm} | p{1.3cm}}
+\textbf{Payload} & \textbf{checksum} & \textbf{destination port} & \textbf{order} & \textbf{source port} & \textbf{payload 0} & \textbf{payload 1} & \textbf{payload small} & \textbf{payload big} \\ \hline \hline
+Correct & Correct & Correct & In order & Correct & \cmark & \cmark & \cmark & \cmark \\ \hline
+Correct & Incorrect & Incorrect & Missing packet & Incorrect & \xmark & \xmark & \cmark & \cmark \\ \hline
+odd\# of bits flipped & Correct & Incorrect & Out of order & Incorrect & \xmark & \xmark & \cmark & \cmark \\ \hline
+odd\# of bits flipped & Incorrect & Correct & Missing packet & Correct & \xmark & \xmark & \cmark & \cmark \\ \hline
+even\# of bits flipped & Correct & Incorrect & Missing packet & Correct & \xmark & \xmark & \cmark & \cmark \\ \hline
+even\# of bits flipped & Incorrect & Correct & Out of order & Incorrect & \xmark & \xmark & \cmark & \cmark \\ \hline
+odd\# of bits flipped & Incorrect & Incorrect & In order & Incorrect & \cmark & \cmark & \cmark & \cmark \\ \hline
+even\# of bits flipped & Incorrect & Incorrect & In order & Incorrect & \cmark & \cmark & \cmark & \cmark \\ \hline
+Correct & Incorrect & Incorrect & Out of order & Correct & \xmark & \xmark & \cmark & \cmark \\
+\end{tabular}
+\caption{Combinations of test cases}
+\label{table:testpairs}
+\end{table}
\subsection{Quality, completeness and coverage of tests}
To further decrease the number of tests needed test cases are divided into
equivalence partitions and the combination of cases as described in
-Table~(\textbf{referentie naar decision table}) ensures that all partitions are
+Table~\ref{table:testpairs} ensures that all partitions are
covered and the number of individual tests is still feasible.
Input & Pcap file with prerecorded valid packets. \\\hline
Expected output & Pcap file with valid response to request. \\\hline
Course of action & \begin{enumerate}
- \item Load input pcap file.
- \item Replay pcap file using ...
- \item Record SUT response using...
- \item Save recorded packets as a pcap file.
- \item Analyze packets in resulting file.
+ \item Execute \emph{./scripts/tests/case1-single-valid.sh}
+ \item Load \emph{output/case1.pcap} with ...
\end{enumerate} \\\hline
Valid trace & \begin{enumerate}
\item \textbf{Hier packets benoemen?}
repositories / tt2015.git / blobdiff