Add explenation for testcases table

[tt2015.git] / a2 / 1cases.tex

diff --git a/a2/1cases.tex b/a2/1cases.tex
index 9ffc07d..403e4d7 100644 (file)
--- a/a2/1cases.tex
+++ b/a2/1cases.tex
@@ -119,11 +119,11 @@ these partitions are given.
                        \item Correct
                        \item Incorrect
                \end{enumerate}
-       \item Packet order
+       \item \emph{Segment order}
                \begin{enumerate}
                        \item Correct
                        \item Out of order
-                       \item Missing packets
+                       \item Missing Segments
                \end{enumerate}
 \end{enumerate}
 
@@ -147,10 +147,12 @@ increases by one, and the value of another decreases by one, is undetected.
 The SUT should exhibit the same behavior and accept packets where these type
 of bit errors occur.
 
-TCP guarantees that segments are delivered in order,even when they are received
-\emph{out of order} and that missing segments are resend. The SUT should
+TCP guarantees that segments are delivered \emph{in order}
+,even when they are received
+out of order and that missing segments are resend. The SUT should
 exhibit the same behavior. If segments are received out of order it should
-deliver them in order. Missing segments should be re-requested (by ACK-ing
+either reassemble them when the missing packet has arrived or request them to
+be resend when the  Missing segments should be re-requested (by ACK-ing
 the correct sequence number).  
 
 \bigskip
@@ -159,26 +161,35 @@ Partitions 2 to 6 are tested using pairwise testing to keep the number of test
 cases feasible. The pairs are then all *except some where it does not make sense
 to do so) tested with the different request sizes of partition 1.
 
-This is expressed in Table~\ref{table:testpairs}.
+This is expressed in Table~\ref{table:testpairs}. In this table the first five
+columns represent the different options for the partitions 2 to 6 of the above
+enumeration. The last four columns are the different number segments as
+described in the partition 1 of the above enumeration. These cells identify
+individual test cases by a number. An \xmark in the cell indicates that this
+test case can not be created as it is not possible with that number of segments
+(eg. sending segments out of order when the number of segments is 1).
 
+\newcounter{TCC}
+\setcounter{TCC}{1}
+\newcommand{\doTCC}{\theTCC \stepcounter{TCC}}
 \begin{table}[H]
        \centering
-       \begin{tabular}{|l|l|l|l|l|l|l|l|l|l|l|}
+       \begin{tabular}{|l|l|l|l|l|l||l|l|l|l|}
                \hline
-               & \multicolumn{10}{c|}{\textbf{Partition}}\\
+               & \multicolumn{9}{c|}{\textbf{Partition}}\\
                \hline
-               & \# & 4 & 5 & 3 & 6 & 2 & 1a & 1b & 1c & 1d\\
+               & 4 & 5 & 3 & 6 & 2 & 1a & 1b & 1c & 1d\\
                \hline\hline
                \multirow{9}{*}{Instance} 
-               & 1 & a & a & a & a & a & \checkmark & \checkmark & \checkmark & \checkmark\\
-               & 2 & a & b & b & c & b & \xmark & \xmark & \checkmark & \checkmark\\
-               & 3 & c & a & b & a & b & \xmark & \xmark & \checkmark & \checkmark\\
-               & 4 & c & b & a & c & a & \xmark & \xmark & \checkmark & \checkmark\\
-               & 5 & b & a & b & c & a & \xmark & \xmark & \checkmark & \checkmark\\
-               & 6 & b & b & a & b & b & \xmark & \xmark & \checkmark & \checkmark\\
-               & 7 & c & b & b & a & b & \checkmark & \checkmark & \checkmark & \checkmark\\
-               & 8 & b & b & b & a & b & \checkmark & \checkmark & \checkmark & \checkmark\\
-               & 9 & a & b & b & b & a & \xmark & \xmark & \checkmark & \checkmark\\
+               & a & a & a & a & a & \doTCC & \doTCC & \doTCC & \doTCC\\
+               & a & b & b & c & b & \xmark & \xmark & \doTCC & \doTCC\\
+               & c & a & b & a & b & \xmark & \xmark & \doTCC & \doTCC\\
+               & c & b & a & c & a & \xmark & \xmark & \doTCC & \doTCC\\
+               & b & a & b & c & a & \xmark & \xmark & \doTCC & \doTCC\\
+               & b & b & a & b & b & \xmark & \xmark & \doTCC & \doTCC\\
+               & c & b & b & a & b & \xmark & \doTCC & \doTCC & \doTCC\\
+               & b & b & b & a & b & \xmark & \doTCC & \doTCC & \doTCC\\
+               & a & b & b & b & a & \xmark & \xmark & \doTCC & \doTCC\\
                \hline
 \end{tabular}
 \caption{Combinations of test cases}
@@ -209,7 +220,14 @@ test suite can be clearly assessed.
 
 As always, $100\%$ completeness is not feasible, therefore test cases are
 carefully selected to cover the most interesting parts of the TCP specification
-to ensure a test suite.
+to ensure a complete but feasible test suite.
+
+To further increase the coverage of the test suites tests are randomized. The
+tests which test the handling of \emph{bit errors}, changes in the \emph{packet
+order} and \emph{dropped packets} randomize where they introduce an error. The
+test suite runs these tests multiple times to increase the likelihood that they
+discover a fault which is only present when an error occurs in a certain
+position. 
 
 To further decrease the number of tests needed test cases are divided into
 equivalence partitions and the combination of cases as described in