\subsection{Preflight checklist}
-
Before actual certification is commenced we perform a manual test using a
checklist.
If any of the checks fail we immediately reject the product.
+The checklist is given in the table below. All commands in \texttt{monospace}
+are to be run in a terminal. Commands prefixed with a \texttt{\#} should be run
+with root permissions. Commands prefixed with a \texttt{\$} should be run with
+user permissions.
-\begin{enumerate}
- \item Is the product complete?
- \item Does the product come with a manual or quick start guide?
- \item Is it possible to get the product in a usable state?
- \item Can we use the product to initiate a connection in a corruption
- free environment?
-\end{enumerate}
-
-\textbf{DIT MOET VERANDERD WORDEN, HET PRODUCT ZOU AFGEKEURD WORDEN!}
+\begin{longtable}{|p{.2\linewidth}|p{.8\linewidth}|}
+ \hline
+ Check 1 & Get the SUT in a workable state.\\
+ \hline
+ \multirow{3}{*}{Course of action}
+ & 1. Import the VirtualBox image into VirtualBox.\\
+ & 2. Boot the vm.\\
+ & 3. Verify the SUT booted successfully and the network modules are
+ loaded.\\
+ \hline
+ Passed & \textit{Yes/No} \\
+ \hline\hline
+ Check 2 & Verify the SUT is complete. \\
+ \hline
+ \multirow{5}{*}{Course of action}
+ & 1. Boot the SUT as in \emph{Check 1}.\\
+ & 2. Verify the loopback device exists by running
+ \texttt{\$ ifconfig}.\\
+ & 3. Verify the \emph{echo-server} is present on the system by running
+ \texttt{\$ file code/server/Main.java}\\
+ & 4. Verify \emph{Scapy} is present on the system by running
+ \texttt{\$ scapy}.\\
+ & 5. Verify all scripts used for testing are present on the system.\\
+ \hline
+ Passed & \textit{Yes/No}\\
+ \hline\hline
+ Check 3 & Initialize the testing environment.\\
+ \hline
+ \multirow{5}{*}{Course of action}
+ & 1. Boot the SUT as in \emph{Check 1}.\\
+ & 2. Setup iptables by executing
+ \texttt{\# code/iptables.sh}~\footnote{The IPTables script ensures
+ that the OS does not drop packets due to an the unknown source.}\\
+ & 3. Navigate to the working directory by running
+ \texttt{\$ cd /home/student/tt2015}\\
+ & Compile the echo server by running
+ \texttt{\# cd code/server \&\& make \&\& cd -}\\
+ & Start the echo server by running
+ \texttt{\# cd code/server \&\& java Main}\\
+ \hline
+ Passed & \textit{Yes/No}\\
+ \hline\hline
+ Check 4 & Test the tool environment.\\
+ \hline
+ \multirow{3}{*}{Course of action}
+ & 1. Initialize the SUT as in \emph{Check 3}\\
+ & 2. Execute the test script by running
+ \texttt{\# code/client/helloworld.py}\\
+ & 3. Verify the console displays a success message.\\
+ \hline
+ Passed & \textit{Yes/No}\\
+ \hline\hline
+ Check 5 & All test inputs and scripts are present.\\\hline
+ \multirow{2}{*}{Course of action}
+ & 1. Boot the SUT as in \emph{Check 1}.\\
+ & 2. Verify that the test generation script is present by running
+ \texttt{\$ file code/client/test.py}\\
+ \hline
+ Passed & \textit{Yes/No}\\
+ \hline
+ \caption{Preflight checklist\label{tbl:preflight}}
+\end{longtable}
\subsection{Testing of SUT}
-
-The SUT is a series of services for other computer programs with no end-user
-facing
-interface. Therefore the SUT will be tested solely by calling it's services
-through various automated scripts. An automated test suite will be available
-which executes all these automated scripts and aggregates their results to
-asses whether or not the SUT has passed the test.
+The SUT is a series of services for other computer programs with no end-user
+facing interface. Therefore the SUT will be tested solely by calling it's
+services through various automated scripts. An automated test suite will be
+available which executes all these automated scripts and aggregates their
+results to asses whether or not the SUT has passed the test.
The implementation of the SUT is tested using black box testing techniques. A
series of tests asses the correctness of the implementation with regards to the
TCP specification. These tests are specified in Table~\textbf{referentie naar
-tests-tabel}. The test cases aim to cover the most interesting parts of the
-TCP specification.
+tests-tabel}. The test cases aim to cover the most interesting parts of the TCP
+specification.
To cover the TCP specification as complete as possible while still maintaining
a feasible test suite the tests are divided into equivalence partitions. Below
these partitions are given.
\begin{enumerate}
- \item \emph{Number of packets} in request\footnotemark
- \footnotetext{A request is considered establishing a connection
- (handshake) and any n number of payloadpackts}
+ \item \emph{Number of packets} in request~\footnote{A request is considered
+ establishing a connection (handshake) and a number of payload packets}
\begin{enumerate}
\item 0 payload packets
\item 1 payload packet
\item n=small payload packets
\item n=big payload packets
\end{enumerate}
- \item Correct or Incorrect \emph{source port}
- \item Correct or Incorrect \emph{Destination port}
- \item Bits flipped in \emph{Payload}
+ \item \emph{source port}
+ \begin{enumerate}
+ \item Correct
+ \item Incorrect
+ \end{enumerate}
+ \item \emph{destination port}
+ \begin{enumerate}
+ \item Correct
+ \item Incorrect
+ \end{enumerate}
+ \item Bits flipped in \emph{payload}
\begin{enumerate}
\item Correct payload
\item Payload with even number of bits flipped
\item Payload with odd number of bits flipped
\end{enumerate}
- \item Correct or Incorrect \emph{checksum}
- \item Packets received in or out of order, or missing packets
+ \item \emph{checksum}
+ \begin{enumerate}
+ \item Correct
+ \item Incorrect
+ \end{enumerate}
+ \item Packet order
+ \begin{enumerate}
+ \item Correct
+ \item Out of order
+ \item Missing packets
+ \end{enumerate}
\end{enumerate}
\textbf{hier iets over waarom deze partities relevant zijn! Waarom odd en
This is expressed in Table~\ref{table:testpairs}.
-\begin{table}
-\begin{tabular} {p{2cm} | l | p{2cm} | l | p{1cm} || c | c | p{1.3cm} | p{1.3cm}}
-\textbf{Payload} & \textbf{checksum} & \textbf{destination port} & \textbf{order} & \textbf{source port} & \textbf{payload 0} & \textbf{payload 1} & \textbf{payload small} & \textbf{payload big} \\ \hline \hline
-Correct & Correct & Correct & In order & Correct & \cmark & \cmark & \cmark & \cmark \\ \hline
-Correct & Incorrect & Incorrect & Missing packet & Incorrect & \xmark & \xmark & \cmark & \cmark \\ \hline
-odd\# of bits flipped & Correct & Incorrect & Out of order & Incorrect & \xmark & \xmark & \cmark & \cmark \\ \hline
-odd\# of bits flipped & Incorrect & Correct & Missing packet & Correct & \xmark & \xmark & \cmark & \cmark \\ \hline
-even\# of bits flipped & Correct & Incorrect & Missing packet & Correct & \xmark & \xmark & \cmark & \cmark \\ \hline
-even\# of bits flipped & Incorrect & Correct & Out of order & Incorrect & \xmark & \xmark & \cmark & \cmark \\ \hline
-odd\# of bits flipped & Incorrect & Incorrect & In order & Incorrect & \cmark & \cmark & \cmark & \cmark \\ \hline
-even\# of bits flipped & Incorrect & Incorrect & In order & Incorrect & \cmark & \cmark & \cmark & \cmark \\ \hline
-Correct & Incorrect & Incorrect & Out of order & Correct & \xmark & \xmark & \cmark & \cmark \\
+\begin{table}[H]
+ \centering
+ \begin{tabular}{|l|l|l|l|l|l|l|l|l|l|l|}
+ \hline
+ & \multicolumn{10}{c|}{\textbf{Partition}}\\
+ \hline
+ & \# & 4 & 5 & 3 & 6 & 2 & 1a & 1b & 1c & 1d\\
+ \hline\hline
+ \multirow{9}{*}{Instance}
+ & 1 & a & a & a & a & a & \checkmark & \checkmark & \checkmark & \checkmark\\
+ & 2 & a & b & b & c & b & \xmark & \xmark & \checkmark & \checkmark\\
+ & 3 & c & a & b & a & b & \xmark & \xmark & \checkmark & \checkmark\\
+ & 4 & c & b & a & c & a & \xmark & \xmark & \checkmark & \checkmark\\
+ & 5 & b & a & b & c & a & \xmark & \xmark & \checkmark & \checkmark\\
+ & 6 & b & b & a & b & b & \xmark & \xmark & \checkmark & \checkmark\\
+ & 7 & c & b & b & a & b & \checkmark & \checkmark & \checkmark & \checkmark\\
+ & 8 & b & b & b & a & b & \checkmark & \checkmark & \checkmark & \checkmark\\
+ & 9 & a & b & b & b & a & \xmark & \xmark & \checkmark & \checkmark\\
+ \hline
\end{tabular}
\caption{Combinations of test cases}
\label{table:testpairs}
% wat ik ook probeer ik krijg de eerste collum
% zijn tekst niet verticaal gecentered
%
-\begin{table}
\subsection{Test cases}
-\begin{tabularx}{\linewidth}{| l | X|}
-\hline
-Nr & 1 \\\hline
-Title & Single valid request. \\\hline
-Input & Pcap file with prerecorded valid packets. \\\hline
-Expected output & Pcap file with valid response to request. \\\hline
-Course of action & \begin{enumerate}
- \item Load input pcap file.
- \item Replay pcap file using ...
- \item Record SUT response using...
- \item Save recorded packets as a pcap file.
- \item Analyze packets in resulting file.
-\end{enumerate} \\\hline
-Valid trace & \begin{enumerate}
- \item \textbf{Hier packets benoemen?}
-\end{enumerate} \\\hline
-\end{tabularx}
-
-\begin{tabularx}{\linewidth}{| l | X|}
+Before every test case use the following steps to initialize the testing environment.
+
+\begin{enumerate}
+ \item Boot the vm using VirtualBox.
+ \item Setup iptables by executing \texttt{\# code/iptables.sh}
+ \item Navigate to the working directory by running \texttt{\$ cd /home/student/tt2015}
+ \item Start the echo server by running \texttt{\# cd code/server \&\& java Main}
+\end{enumerate}
+
+\begin{longtable}{|p{.2\linewidth}|p{.8\linewidth}|}
+ \hline
+ Nr & 1 \\\hline
+ Title & Single valid request with 1byte payload. \\\hline
+ Input & Generated packets. \\\hline
+ Expected output & Packets echoed back by Echo-Server. \\\hline
+ \multirow{2}{*}{Course of action}
+ & 1. Use the steps listed above in order to start the SUT. \\
+ & 2. Execute the script by running \texttt{\# code/client/tests/1.py} \\\hline
+ Valid trace & Verify that the script prints 'Success'. \\\hline
\hline
+
Nr & 2 \\\hline
- Title & Single request with corrupted checksum. \\\hline
- Input & Pcap file used as \emph{test-case 1} input. \\\hline
- Expected output & No response from SUT, logs with rejected packets. \\\hline
- Course of action & \begin{enumerate}
- \item Load input pcap file into ....
- \item Corrupt checksum of loaded packets.
- \item Save resulting packets as pcap file.
- \item Load new pcap file into ...
- \item Replay new pcap file.
- \item Record SUT response using...
- \item Extract log with rejected packets.
- \item Save recorded packets as a pcap file.
- \item Analyze packets in resulting file.
- \end{enumerate} \\\hline
- Valid trace & \begin{enumerate}
- \item \textbf{Aangeven welke packets corrupted zijn?}
- \end{enumerate} \\\hline
-\end{tabularx}
-\end{table}
+ Title & Single valid request with 65495bytes payload. \\\hline
+ Input & Generated packets. \\\hline
+ Expected output & Packets echoed back by Echo-Server. \\\hline
+ \multirow{2}{*}{Course of action}
+ & 1. Use the steps listed above in order to start the SUT. \\
+ & 2. Execute the script by running \texttt{\# code/client/tests/2.py} \\\hline
+ Valid trace & Verify that the script prints 'Success'. \\\hline
+ \hline
+
+ Nr & 3 \\\hline
+ Title & 5 valid requests with 1byte payload. \\\hline
+ Input & Generated packets. \\\hline
+ Expected output & Packets echoed back by Echo-Server, in the same order as the client sent them. \\\hline
+ \multirow{2}{*}{Course of action}
+ & 1. Use the steps listed above in order to start the SUT. \\
+ & 2. Execute the script by running \texttt{\# code/client/tests/3.py} \\\hline
+ Valid trace & Verify that the script prints 'Success'. \\\hline
+ \hline
+
+ Nr & 4 \\\hline
+ Title & 5 valid requests with 65495bytes payload. \\\hline
+ Input & Generated packets with 65495bytes payload. \\\hline
+ Expected output & Packets echoed back by Echo-Server, in the same order as the client sent them. \\\hline
+ \multirow{2}{*}{Course of action}
+ & 1. Use the steps listed above in order to start the SUT. \\
+ & 2. Execute the script by running \texttt{\# code/client/tests/4.py} \\\hline
+ Valid trace & Verify that the script prints 'Success'. \\\hline
+ \hline
+
+ Nr & 5 \\\hline
+ Title & 5 valid requests with 1byte payload sent out of order. \\\hline
+ Input & Generated packets with 1byte payload, two packets are swapped in position. \\\hline
+ Expected output & All requests sent up to and including
+ the swapped packet with the lowest sequence number, the remaining packets are dropped. \\\hline
+ \multirow{2}{*}{Course of action}
+ & 1. Use the steps listed above in order to start the SUT. \\
+ & 2. Execute the script by running \texttt{\# code/client/tests/5.py} \\\hline
+ Valid trace & Verify that the script prints 'Success'. \\\hline
+ \hline
+\end{longtable}
-\newpage
+%\begin{tabularx}{\linewidth}{| l | X|}
+%\hline
+%Nr & 1 \\\hline
+%Title & Single valid request. \\\hline
+%Input & Pcap file with prerecorded valid packets. \\\hline
+%Expected output & Pcap file with valid response to request. \\\hline
+%Course of action & \begin{enumerate}
+% \item Execute \emph{./scripts/tests/case1-single-valid.sh}
+% \item Load \emph{output/case1.pcap} with ...
+%\end{enumerate} \\\hline
+%Valid trace & \begin{enumerate}
+% \item \textbf{Hier packets benoemen?}
+%\end{enumerate} \\\hline
+%\end{tabularx}
+%
+%\begin{tabularx}{\linewidth}{| l | X|}
+% \hline
+% Nr & 2 \\\hline
+%Title & Single request with corrupted checksum. \\\hline
+% Input & Pcap file used as \emph{test-case 1} input. \\\hline
+% Expected output & No response from SUT, logs with rejected packets. \\\hline
+% Course of action & \begin{enumerate}
+% \item Load input pcap file into ....
+% \item Corrupt checksum of loaded packets.
+% \item Save resulting packets as pcap file.
+% \item Load new pcap file into ...
+% \item Replay new pcap file.
+% \item Record SUT response using...
+% \item Extract log with rejected packets.
+% \item Save recorded packets as a pcap file.
+% \item Analyze packets in resulting file.
+% \end{enumerate} \\\hline
+% Valid trace & \begin{enumerate}
+% \item \textbf{Aangeven welke packets corrupted zijn?}
+% \end{enumerate} \\\hline
+%\end{tabularx}
+%\end{table}
repositories / tt2015.git / blobdiff