X-Git-Url: https://git.martlubbers.net/?a=blobdiff_plain;f=a2%2F1cases.tex;h=71b630c975b915c587c8263596383f4ae03d2e4f;hb=0bc421a9f086432789520a67ef47c146d2a21785;hp=5d8fd812403b0397575df35a17dc2f637c47fe66;hpb=8fe08f06fcb65dbd627d8f191bbbbda878bf6896;p=tt2015.git diff --git a/a2/1cases.tex b/a2/1cases.tex index 5d8fd81..71b630c 100644 --- a/a2/1cases.tex +++ b/a2/1cases.tex @@ -1,18 +1,63 @@ +\pagebreak \subsection{Preflight checklist} - Before actual certification is commenced we perform a manual test using a checklist. If any of the checks fail we immediately reject the product. +The checklist is given in the table below. -\begin{enumerate} - \item Is the product complete? - \item Does the product come with a manual or quick start guide? - \item Is it possible to get the product in a usable state? - \item Can we use the product to initiate a connection in a corruption - free environment? -\end{enumerate} +\begin{table} +\begin{tabularx}{\linewidth}{| l | X|} + \hline +Check 1 & Get the SUT in a workable state. \\\hline +Course of action & \begin{enumerate} + \item Import the VirtualBox image into VirtualBox. + \item Boot the vm. + \item Verify the SUT booted succesfully and the network module(s) are loaded. +\end{enumerate} \\\hline +Passed & \textit{Yes/No} \\\hline \hline +Check 2 & Verify the SUT is complete. \\\hline +Course of action & \begin{enumerate} + \item Use the course of action for \emph{Check 1} in order to boot the SUT. + \item Verify the loopback device exists using \emph{ifconfig} + \item Verify the \emph{echo-server} is present on the system. + \item Verify \emph{Scrappy} is present on the system. + \item Verify all scripts used for testing are present on the system. +\end{enumerate} \\\hline +Passed & \textit{Yes/No} \\\hline \hline +Check 3 & Initialize the testing environment. \\\hline +Course of action & \begin{enumerate} + \item Boot the SUT (see \emph{Check 1}). + \item Setup iptables by executing \emph{./scripts/setup/iptables.sh} + \item Setup ... by executing \emph{./scripts/setup/...} +\end{enumerate} \\\hline +Passed & \textit{Yes/No} \\\hline +\end{tabularx} +\end{table} -\textbf{DIT MOET VERANDERD WORDEN, HET PRODUCT ZOU AFGEKEURD WORDEN!} +\begin{table} +\begin{tabularx}{\linewidth}{| l | X |} +\hline +Check 4 & Test the tool environment. \\\hline +Course of action & \begin{enumerate} + \item Boot and initialize the SUT (see \emph{Check 4}) + \item Execute the \emph{./scripts/tests/case0-helloworld.sh} script. + \item Execute the command \emph{cat logs/case0.log}. + \item Verify the console displays a success message. +\end{enumerate} \\\hline +Passed & \textit{Yes/No} \\\hline \hline +Check 5 & All test inputs and scripts are present. \\\hline +Course of action & \begin{enumerate} + \item Boot the SUT (see \emph{Check 1}) + \item Execute the command \emph{ls scripts/tests} + \item Verify that there is a file named case$X$ for each test case $X$ + \item Use \emph{ls packets} + \item Verify that there is a file named \emph{single.pcap} and \emph{multi.pcap}. +\end{enumerate} \\\hline +Passed & \textit{Yes/No} \\\hline +\end{tabularx} +\end{table} + +\newpage \subsection{Testing of SUT} @@ -55,17 +100,32 @@ these partitions are given. \item Packets received in or out of order, or missing packets \end{enumerate} -\textbf{hier iets over waarom deze partities relevant zijn!} +\textbf{hier iets over waarom deze partities relevant zijn! Waarom odd en +even number of bits flipped bijv interessant?} +\bigskip Partitions 2 to 6 are tested using pairwise testing to keep the number of test cases feasible. The pairs are then all *except some where it does not make sense to do so) tested with the different request sizes of partition 1. -This is expressed in the table below. +This is expressed in Table~\ref{table:testpairs}. -\vspace{3mm} -\textbf{Hier daadwerkelijke decision tabel} -\vspace{3mm} +\begin{table} +\begin{tabular} {p{2cm} | l | p{2cm} | l | p{1cm} || c | c | p{1.3cm} | p{1.3cm}} +\textbf{Payload} & \textbf{checksum} & \textbf{destination port} & \textbf{order} & \textbf{source port} & \textbf{payload 0} & \textbf{payload 1} & \textbf{payload small} & \textbf{payload big} \\ \hline \hline +Correct & Correct & Correct & In order & Correct & \cmark & \cmark & \cmark & \cmark \\ \hline +Correct & Incorrect & Incorrect & Missing packet & Incorrect & \xmark & \xmark & \cmark & \cmark \\ \hline +odd\# of bits flipped & Correct & Incorrect & Out of order & Incorrect & \xmark & \xmark & \cmark & \cmark \\ \hline +odd\# of bits flipped & Incorrect & Correct & Missing packet & Correct & \xmark & \xmark & \cmark & \cmark \\ \hline +even\# of bits flipped & Correct & Incorrect & Missing packet & Correct & \xmark & \xmark & \cmark & \cmark \\ \hline +even\# of bits flipped & Incorrect & Correct & Out of order & Incorrect & \xmark & \xmark & \cmark & \cmark \\ \hline +odd\# of bits flipped & Incorrect & Incorrect & In order & Incorrect & \cmark & \cmark & \cmark & \cmark \\ \hline +even\# of bits flipped & Incorrect & Incorrect & In order & Incorrect & \cmark & \cmark & \cmark & \cmark \\ \hline +Correct & Incorrect & Incorrect & Out of order & Correct & \xmark & \xmark & \cmark & \cmark \\ +\end{tabular} +\caption{Combinations of test cases} +\label{table:testpairs} +\end{table} \subsection{Quality, completeness and coverage of tests} @@ -95,7 +155,7 @@ to ensure a test suite. To further decrease the number of tests needed test cases are divided into equivalence partitions and the combination of cases as described in -Table~(\textbf{referentie naar decision table}) ensures that all partitions are +Table~\ref{table:testpairs} ensures that all partitions are covered and the number of individual tests is still feasible. @@ -115,11 +175,8 @@ Title & Single valid request. \\\hline Input & Pcap file with prerecorded valid packets. \\\hline Expected output & Pcap file with valid response to request. \\\hline Course of action & \begin{enumerate} - \item Load input pcap file. - \item Replay pcap file using ... - \item Record SUT response using... - \item Save recorded packets as a pcap file. - \item Analyze packets in resulting file. + \item Execute \emph{./scripts/tests/case1-single-valid.sh} + \item Load \emph{output/case1.pcap} with ... \end{enumerate} \\\hline Valid trace & \begin{enumerate} \item \textbf{Hier packets benoemen?}