Merge branch 'master' of github.com:dopefishh/tt2015

[tt2015.git] / a2 / 1cases.tex

diff --git a/a2/1cases.tex b/a2/1cases.tex
index c6afba2..2d27d9b 100644 (file)
--- a/a2/1cases.tex
+++ b/a2/1cases.tex
@@ -91,13 +91,14 @@ a feasible test suite the tests are divided into equivalence partitions. Below
 these partitions are given.
 
 \begin{enumerate}
 these partitions are given.
 
 \begin{enumerate}

-       \item \emph{Number of packets} in request~\footnote{A request is considered
-               establishing a connection (handshake) and a number of payload packets}
+       \item \emph{Number of segments} in request~\footnote{A request is
+               considered establishing a connection (handshake) and a number of
+               payload segments}

                \begin{enumerate}
                \begin{enumerate}

-                       \item 0 payload packets
-                       \item 1 payload packet
-                       \item n=small payload packets
-                       \item n=big payload packets
+                       \item 0 payload segments
+                       \item 1 payload segments
+                       \item n=small payload segments (1 byte)
+                       \item n=big payload segments (65495 bytes)

                \end{enumerate}
        \item \emph{source port}
                \begin{enumerate}
                \end{enumerate}
        \item \emph{source port}
                \begin{enumerate}


@@ -109,53 +110,88 @@ these partitions are given.
                        \item Correct
                        \item Incorrect
                \end{enumerate}
                        \item Correct
                        \item Incorrect
                \end{enumerate}

-       \item Bits flipped in \emph{payload}
+       \item Bit errors in \emph{payload}

                \begin{enumerate}
                        \item Correct payload
                \begin{enumerate}
                        \item Correct payload

-                       \item Payload with even number of bits flipped
-                       \item Payload with odd number of bits flipped
+                       \item Payload with bit flips that do not show in checksum
+                       \item Payload with bit flips that do show in checksum

                \end{enumerate}
        \item \emph{checksum}
                \begin{enumerate}
                        \item Correct
                        \item Incorrect
                \end{enumerate}
                \end{enumerate}
        \item \emph{checksum}
                \begin{enumerate}
                        \item Correct
                        \item Incorrect
                \end{enumerate}

-       \item Packet order
+       \item \emph{Segment order}

                \begin{enumerate}
                        \item Correct
                        \item Out of order
                \begin{enumerate}
                        \item Correct
                        \item Out of order

-                       \item Missing packets
+                       \item Missing Segments

                \end{enumerate}
 \end{enumerate}
 
                \end{enumerate}
 \end{enumerate}
 

-\textbf{hier iets over waarom deze partities relevant zijn! Waarom odd en 
-even number of bits flipped bijv interessant?}
+These partitions were chosen since they correspond to key parts of the TCP
+specification. 
+
+TCP segments are send over a TCP connection from a \emph{source} to a \emph{destination port}. Therefore segments which are received that have a 
+source or destination port set to an incorrect value should not be regarded
+as segments belonging to the connection by the SUT. 
+
+TCP uses a \emph{checksum} to catch any error introduced in headers, when this
+checksum does not match the actual computed checksum the SUT should
+disregard the received segment.
+
+The TCP checksum is also an inherently weak one, as it is simply the 
+bitwise negation of the addition, in ones complement arithmetic, 
+of all 16 bit words in the header and data of the segment (excluding the
+checksum itself). Therefore any \emph{bit error} where the ones complement value
+of one word
+increases by one, and the value of another decreases by one, is undetected. 
+The SUT should exhibit the same behavior and accept packets where these type
+of bit errors occur.
+
+TCP guarantees that segments are delivered \emph{in order}
+,even when they are received
+out of order and that missing segments are resend. The SUT should
+exhibit the same behavior. If segments are received out of order it should
+either reassemble them when the missing packet has arrived or request them to
+be resend when the  Missing segments should be re-requested (by ACK-ing
+the correct sequence number).  
+

 \bigskip
 
 Partitions 2 to 6 are tested using pairwise testing to keep the number of test
 cases feasible. The pairs are then all *except some where it does not make sense
 to do so) tested with the different request sizes of partition 1.
 
 \bigskip
 
 Partitions 2 to 6 are tested using pairwise testing to keep the number of test
 cases feasible. The pairs are then all *except some where it does not make sense
 to do so) tested with the different request sizes of partition 1.
 

-This is expressed in Table~\ref{table:testpairs}.
+This is expressed in Table~\ref{table:testpairs}. In this table the first five
+columns represent the different options for the partitions 2 to 6 of the above
+enumeration. The last four columns are the different number segments as
+described in the partition 1 of the above enumeration. These cells identify
+individual test cases by a number. An \xmark in the cell indicates that this
+test case can not be created as it is not possible with that number of segments
+(eg. sending segments out of order when the number of segments is 1).

 
 

+\newcounter{TCC}
+\setcounter{TCC}{1}
+\newcommand{\doTCC}{\theTCC \stepcounter{TCC}}

 \begin{table}[H]
        \centering
 \begin{table}[H]
        \centering

-       \begin{tabular}{|l|l|l|l|l|l|l|l|l|l|l|}
+       \begin{tabular}{|l|l|l|l|l|l||l|l|l|l|}

                \hline
                \hline

-               & \multicolumn{10}{c|}{\textbf{Partition}}\\
+               & \multicolumn{9}{c|}{\textbf{Partition}}\\

                \hline
                \hline

-               & \# & 4 & 5 & 3 & 6 & 2 & 1a & 1b & 1c & 1d\\
+               & 4 & 5 & 3 & 6 & 2 & 1a & 1b & 1c & 1d\\

                \hline\hline
                \multirow{9}{*}{Instance} 
                \hline\hline
                \multirow{9}{*}{Instance} 

-               & 1 & a & a & a & a & a & \checkmark & \checkmark & \checkmark & \checkmark\\
-               & 2 & a & b & b & c & b & \xmark & \xmark & \checkmark & \checkmark\\
-               & 3 & c & a & b & a & b & \xmark & \xmark & \checkmark & \checkmark\\
-               & 4 & c & b & a & c & a & \xmark & \xmark & \checkmark & \checkmark\\
-               & 5 & b & a & b & c & a & \xmark & \xmark & \checkmark & \checkmark\\
-               & 6 & b & b & a & b & b & \xmark & \xmark & \checkmark & \checkmark\\
-               & 7 & c & b & b & a & b & \checkmark & \checkmark & \checkmark & \checkmark\\
-               & 8 & b & b & b & a & b & \checkmark & \checkmark & \checkmark & \checkmark\\
-               & 9 & a & b & b & b & a & \xmark & \xmark & \checkmark & \checkmark\\
+               & a & a & a & a & a & \doTCC & \doTCC & \doTCC & \doTCC\\
+               & a & b & b & c & b & \xmark & \xmark & \doTCC & \doTCC\\
+               & c & a & b & a & b & \xmark & \xmark & \doTCC & \doTCC\\
+               & c & b & a & c & a & \xmark & \xmark & \doTCC & \doTCC\\
+               & b & a & b & c & a & \xmark & \xmark & \doTCC & \doTCC\\
+               & b & b & a & b & b & \xmark & \xmark & \doTCC & \doTCC\\
+               & c & b & b & a & b & \xmark & \doTCC & \doTCC & \doTCC\\
+               & b & b & b & a & b & \xmark & \doTCC & \doTCC & \doTCC\\
+               & a & b & b & b & a & \xmark & \xmark & \doTCC & \doTCC\\

                \hline
 \end{tabular}
 \caption{Combinations of test cases}
                \hline
 \end{tabular}
 \caption{Combinations of test cases}


@@ -186,7 +222,14 @@ test suite can be clearly assessed.
 
 As always, $100\%$ completeness is not feasible, therefore test cases are
 carefully selected to cover the most interesting parts of the TCP specification
 
 As always, $100\%$ completeness is not feasible, therefore test cases are
 carefully selected to cover the most interesting parts of the TCP specification

-to ensure a test suite.
+to ensure a complete but feasible test suite.
+
+To further increase the coverage of the test suites tests are randomized. The
+tests which test the handling of \emph{bit errors}, changes in the \emph{packet
+order} and \emph{dropped packets} randomize where they introduce an error. The
+test suite runs these tests multiple times to increase the likelihood that they
+discover a fault which is only present when an error occurs in a certain
+position. 

 
 To further decrease the number of tests needed test cases are divided into
 equivalence partitions and the combination of cases as described in 
 
 To further decrease the number of tests needed test cases are divided into
 equivalence partitions and the combination of cases as described in 


@@ -202,7 +245,69 @@ covered and the number of individual tests is still feasible.
 
 \subsection{Test cases}
 
 
 \subsection{Test cases}
 

-%\begin{table}
+Before every test case use the following steps to initialize the testing environment.
+
+\begin{enumerate}
+       \item Boot the vm using VirtualBox.
+       \item Setup iptables by executing \texttt{\# code/iptables.sh}
+       \item Navigate to the working directory by running \texttt{\$ cd /home/student/tt2015}
+       \item Start the echo server by running \texttt{\# cd code/server \&\& java Main}
+\end{enumerate}
+
+\begin{longtable}{|p{.2\linewidth}|p{.8\linewidth}|}
+       \hline
+       Nr & 1 \\\hline
+       Title & Single valid request with 1byte payload. \\\hline
+       Input & Generated packets. \\\hline
+       Expected output & Packets echoed back by Echo-Server. \\\hline
+       \multirow{2}{*}{Course of action}
+ & 1. Use the steps listed above in order to start the SUT. \\
+ & 2. Execute the script by running \texttt{\# code/client/tests/1.py} \\\hline
+       Valid trace & Verify that the script prints 'Success'. \\\hline
+       \hline
+       
+       Nr & 2 \\\hline
+       Title & Single valid request with 65495bytes payload. \\\hline
+       Input & Generated packets. \\\hline
+       Expected output & Packets echoed back by Echo-Server. \\\hline
+       \multirow{2}{*}{Course of action}
+       & 1. Use the steps listed above in order to start the SUT. \\
+       & 2. Execute the script by running \texttt{\# code/client/tests/2.py} \\\hline
+       Valid trace & Verify that the script prints 'Success'. \\\hline
+       \hline
+       
+       Nr & 3 \\\hline
+       Title & 5 valid requests with 1byte payload. \\\hline
+       Input & Generated packets. \\\hline
+       Expected output & Packets echoed back by Echo-Server, in the same order as the client sent them. \\\hline
+       \multirow{2}{*}{Course of action}
+       & 1. Use the steps listed above in order to start the SUT. \\
+       & 2. Execute the script by running \texttt{\# code/client/tests/3.py} \\\hline
+       Valid trace & Verify that the script prints 'Success'. \\\hline
+       \hline
+       
+       Nr & 4 \\\hline
+       Title & 5 valid requests with 65495bytes payload. \\\hline
+       Input & Generated packets with 65495bytes payload. \\\hline
+       Expected output & Packets echoed back by Echo-Server, in the same order as the client sent them. \\\hline
+       \multirow{2}{*}{Course of action}
+       & 1. Use the steps listed above in order to start the SUT. \\
+       & 2. Execute the script by running \texttt{\# code/client/tests/4.py} \\\hline
+       Valid trace & Verify that the script prints 'Success'. \\\hline
+       \hline
+       
+       Nr & 5 \\\hline
+       Title & 5 valid requests with 1byte payload sent out of order. \\\hline
+       Input & Generated packets with 1byte payload, two packets are swapped in position. \\\hline
+       Expected output & All requests sent up to and including
+        the swapped packet with the lowest sequence number, the remaining packets are dropped. \\\hline
+       \multirow{2}{*}{Course of action}
+       & 1. Use the steps listed above in order to start the SUT. \\
+       & 2. Execute the script by running \texttt{\# code/client/tests/5.py} \\\hline
+       Valid trace & Verify that the script prints 'Success'. \\\hline
+       \hline
+\end{longtable}
+

 %\begin{tabularx}{\linewidth}{| l | X|}
 %\hline
 %Nr & 1 \\\hline
 %\begin{tabularx}{\linewidth}{| l | X|}
 %\hline
 %Nr & 1 \\\hline