X-Git-Url: https://git.martlubbers.net/?a=blobdiff_plain;f=a2%2F1cases.tex;h=47f7acfe796e0bf57587f02811db4663c9e8bf63;hb=8f8cfadb72bfa946be1fb01f0d64f51e1bae80be;hp=d8825ed5ce81eb5b5b6157eefd39fc350c0e7c11;hpb=48364b0cd18f9740aedb204b1c1b571a5cd91e67;p=tt2015.git diff --git a/a2/1cases.tex b/a2/1cases.tex index d8825ed..47f7acf 100644 --- a/a2/1cases.tex +++ b/a2/1cases.tex @@ -1,71 +1,163 @@ \subsection{Preflight checklist} - Before actual certification is commenced we perform a manual test using a checklist. If any of the checks fail we immediately reject the product. - -\begin{enumerate} - \item Is the product complete? - \item Does the product come with a manual or quick start guide? - \item Is it possible to get the product in a usable state? - \item Can we use the product to initiate a connection in a corruption - free environment? -\end{enumerate} - -\textbf{DIT MOET VERANDERD WORDEN, HET PRODUCT ZOU AFGEKEURD WORDEN!} +The checklist is given in the table below. All commands in \texttt{monospace} +are to be run in a terminal. Commands prefixed with a \texttt{\#} should be run +with root permissions. Commands prefixed with a \texttt{\$} should be run with +user permissions. + +\begin{longtable}{|p{.2\linewidth}|p{.8\linewidth}|} + \hline + Check 1 & Get the SUT in a workable state.\\ + \hline + \multirow{3}{*}{Course of action} + & 1. Import the VirtualBox image into VirtualBox.\\ + & 2. Boot the vm.\\ + & 3. Verify the SUT booted successfully and the network modules are + loaded.\\ + \hline + Passed & \textit{Yes/No} \\ + \hline\hline + Check 2 & Verify the SUT is complete. \\ + \hline + \multirow{5}{*}{Course of action} + & 1. Boot the SUT as in \emph{Check 1}.\\ + & 2. Verify the loopback device exists by running + \texttt{\$ ifconfig}.\\ + & 3. Verify the \emph{echo-server} is present on the system by running + \texttt{\$ file code/server/Main.java}\\ + & 4. Verify \emph{Scapy} is present on the system by running + \texttt{\$ scapy}.\\ + & 5. Verify all scripts used for testing are present on the system.\\ + \hline + Passed & \textit{Yes/No}\\ + \hline\hline + Check 3 & Initialize the testing environment.\\ + \hline + \multirow{5}{*}{Course of action} + & 1. Boot the SUT as in \emph{Check 1}.\\ + & 2. Setup iptables by executing + \texttt{\# code/iptables.sh}~\footnote{The IPTables script ensures + that the OS does not drop packets due to an the unknown source.}\\ + & 3. Navigate to the working directory by running + \texttt{\$ cd /home/student/tt2015}\\ + & Compile the echo server by running + \texttt{\# cd code/server \&\& make \&\& cd -}\\ + & Start the echo server by running + \texttt{\# cd code/server \&\& java Main}\\ + \hline + Passed & \textit{Yes/No}\\ + \hline\hline + Check 4 & Test the tool environment.\\ + \hline + \multirow{3}{*}{Course of action} + & 1. Initialize the SUT as in \emph{Check 3}\\ + & 2. Execute the test script by running + \texttt{\# code/client/helloworld.py}\\ + & 3. Verify the console displays a success message.\\ + \hline + Passed & \textit{Yes/No}\\ + \hline\hline + Check 5 & All test inputs and scripts are present.\\\hline + \multirow{2}{*}{Course of action} + & 1. Boot the SUT as in \emph{Check 1}.\\ + & 2. Verify that the test generation script is present by running + \texttt{\$ file code/client/test.py}\\ + \hline + Passed & \textit{Yes/No}\\ + \hline + \caption{Preflight checklist\label{tbl:preflight}} +\end{longtable} \subsection{Testing of SUT} - -The SUT is a series of services for other computer programs with no end-user -facing -interface. Therefore the SUT will be tested solely by calling it's services -through various automated scripts. An automated test suite will be available -which executes all these automated scripts and aggregates their results to -asses whether or not the SUT has passed the test. +The SUT is a series of services for other computer programs with no end-user +facing interface. Therefore the SUT will be tested solely by calling it's +services through various automated scripts. An automated test suite will be +available which executes all these automated scripts and aggregates their +results to asses whether or not the SUT has passed the test. The implementation of the SUT is tested using black box testing techniques. A series of tests asses the correctness of the implementation with regards to the TCP specification. These tests are specified in Table~\textbf{referentie naar -tests-tabel}. The test cases aim to cover the most interesting parts of the -TCP specification. +tests-tabel}. The test cases aim to cover the most interesting parts of the TCP +specification. To cover the TCP specification as complete as possible while still maintaining a feasible test suite the tests are divided into equivalence partitions. Below these partitions are given. \begin{enumerate} - \item \emph{Number of packets} in request\footnotemark - \footnotetext{A request is considered establishing a connection - (handshake) and any n number of payloadpackts} + \item \emph{Number of packets} in request~\footnote{A request is considered + establishing a connection (handshake) and a number of payload packets} \begin{enumerate} \item 0 payload packets \item 1 payload packet \item n=small payload packets \item n=big payload packets \end{enumerate} - \item Correct or Incorrect \emph{source port} - \item Correct or Incorrect \emph{Destination port} - \item Bits flipped in \emph{Payload} + \item \emph{source port} + \begin{enumerate} + \item Correct + \item Incorrect + \end{enumerate} + \item \emph{destination port} + \begin{enumerate} + \item Correct + \item Incorrect + \end{enumerate} + \item Bits flipped in \emph{payload} \begin{enumerate} \item Correct payload \item Payload with even number of bits flipped \item Payload with odd number of bits flipped \end{enumerate} - \item Correct or Incorrect \emph{checksum} - \item Packets received in or out of order, or missing packets + \item \emph{checksum} + \begin{enumerate} + \item Correct + \item Incorrect + \end{enumerate} + \item Packet order + \begin{enumerate} + \item Correct + \item Out of order + \item Missing packets + \end{enumerate} \end{enumerate} -\textbf{hier iets over waarom deze partities relevant zijn!} +\textbf{hier iets over waarom deze partities relevant zijn! Waarom odd en +even number of bits flipped bijv interessant?} +\bigskip Partitions 2 to 6 are tested using pairwise testing to keep the number of test cases feasible. The pairs are then all *except some where it does not make sense to do so) tested with the different request sizes of partition 1. -This is expressed in the table below. - -\vspace{3mm} -\textbf{Hier daadwerkelijke decision tabel} -\vspace{3mm} +This is expressed in Table~\ref{table:testpairs}. + +\begin{table}[H] + \centering + \begin{tabular}{|l|l|l|l|l|l|l|l|l|l|l|} + \hline + & \multicolumn{10}{c|}{\textbf{Partition}}\\ + \hline + & \# & 4 & 5 & 3 & 6 & 2 & 1a & 1b & 1c & 1d\\ + \hline\hline + \multirow{9}{*}{Instance} + & 1 & a & a & a & a & a & \checkmark & \checkmark & \checkmark & \checkmark\\ + & 2 & a & b & b & c & b & \xmark & \xmark & \checkmark & \checkmark\\ + & 3 & c & a & b & a & b & \xmark & \xmark & \checkmark & \checkmark\\ + & 4 & c & b & a & c & a & \xmark & \xmark & \checkmark & \checkmark\\ + & 5 & b & a & b & c & a & \xmark & \xmark & \checkmark & \checkmark\\ + & 6 & b & b & a & b & b & \xmark & \xmark & \checkmark & \checkmark\\ + & 7 & c & b & b & a & b & \checkmark & \checkmark & \checkmark & \checkmark\\ + & 8 & b & b & b & a & b & \checkmark & \checkmark & \checkmark & \checkmark\\ + & 9 & a & b & b & b & a & \xmark & \xmark & \checkmark & \checkmark\\ + \hline +\end{tabular} +\caption{Combinations of test cases} +\label{table:testpairs} +\end{table} \subsection{Quality, completeness and coverage of tests} @@ -95,6 +187,53 @@ to ensure a test suite. To further decrease the number of tests needed test cases are divided into equivalence partitions and the combination of cases as described in -Table~(\textbf{referentie naar decision table}) ensures that all partitions are +Table~\ref{table:testpairs} ensures that all partitions are covered and the number of individual tests is still feasible. + +% +% wat ik ook probeer ik krijg de eerste collum +% zijn tekst niet verticaal gecentered +% + + +\subsection{Test cases} + +%\begin{table} +%\begin{tabularx}{\linewidth}{| l | X|} +%\hline +%Nr & 1 \\\hline +%Title & Single valid request. \\\hline +%Input & Pcap file with prerecorded valid packets. \\\hline +%Expected output & Pcap file with valid response to request. \\\hline +%Course of action & \begin{enumerate} +% \item Execute \emph{./scripts/tests/case1-single-valid.sh} +% \item Load \emph{output/case1.pcap} with ... +%\end{enumerate} \\\hline +%Valid trace & \begin{enumerate} +% \item \textbf{Hier packets benoemen?} +%\end{enumerate} \\\hline +%\end{tabularx} +% +%\begin{tabularx}{\linewidth}{| l | X|} +% \hline +% Nr & 2 \\\hline +%Title & Single request with corrupted checksum. \\\hline +% Input & Pcap file used as \emph{test-case 1} input. \\\hline +% Expected output & No response from SUT, logs with rejected packets. \\\hline +% Course of action & \begin{enumerate} +% \item Load input pcap file into .... +% \item Corrupt checksum of loaded packets. +% \item Save resulting packets as pcap file. +% \item Load new pcap file into ... +% \item Replay new pcap file. +% \item Record SUT response using... +% \item Extract log with rejected packets. +% \item Save recorded packets as a pcap file. +% \item Analyze packets in resulting file. +% \end{enumerate} \\\hline +% Valid trace & \begin{enumerate} +% \item \textbf{Aangeven welke packets corrupted zijn?} +% \end{enumerate} \\\hline +%\end{tabularx} +%\end{table}