\hline\hline
Check 3 & \multicolumn{2}{l|}{Initialize the testing environment..}\\
\hline
- \multirow{5}{*}{Course of action}
+ \multirow{6}{*}{Course of action}
& 1. & Boot the SUT as in \emph{Check 1}.\\
& 2. & Setup iptables by executing
\texttt{\# code/iptables.sh}~\footnote{The IPTables script ensures
& 3. & Navigate to the working directory by running
\texttt{\$ cd /home/student/tt2015}\\
& 4. & Compile the echo server by running
- \texttt{\# cd code/server \&\& make \&\& cd -}\\
+ \texttt{\# cd code/server \&\& make}\\
& 5. & Start the echo server by running
\texttt{\# cd code/server \&\& java Main}\\
+ & 6. & Generate all test cases by running
+ \texttt{\$ python code/client/gen.py}\\
\hline
Passed & \multicolumn{2}{l|}{\textit{Yes/No}}\\
\hline\hline
\item Correct
\item Incorrect
\end{enumerate}
- \item \emph{Packet order}
+ \item \emph{Segment order}
\begin{enumerate}
\item Correct
\item Out of order
- \item Missing packets
+ \item Missing Segments
\end{enumerate}
\end{enumerate}
cases feasible. The pairs are then all *except some where it does not make sense
to do so) tested with the different request sizes of partition 1.
- This is expressed in Table~\ref{table:testpairs}.
+ This is expressed in Table~\ref{table:testpairs}. In this table the first five
+ columns represent the different options for the partitions 2 to 6 of the above
+ enumeration. The last four columns are the different number segments as
+ described in the partition 1 of the above enumeration. These cells identify
+ individual test cases by a number. An \xmark in the cell indicates that this
+ test case can not be created as it is not possible with that number of segments
+ (eg. sending segments out of order when the number of segments is 1).
\newcounter{TCC}
\setcounter{TCC}{1}
\newcommand{\doTCC}{\theTCC \stepcounter{TCC}}
\begin{table}[H]
\centering
- \begin{tabular}{|l|l|l|l|l|l|l|l|l|l|l|}
+ \begin{tabular}{|l|l|l|l|l|l||l|l|l|l|}
\hline
- & \multicolumn{10}{c|}{\textbf{Partition}}\\
+ & \multicolumn{9}{c|}{\textbf{Partition}}\\
\hline
- & \# & 4 & 5 & 3 & 6 & 2 & 1a & 1b & 1c & 1d\\
+ & 4 & 5 & 3 & 6 & 2 & 1a & 1b & 1c & 1d\\
\hline\hline
\multirow{9}{*}{Instance}
- & 1 & a & a & a & a & a & \doTCC & \doTCC & \doTCC & \doTCC\\
- & 2 & a & b & b & c & b & \xmark & \xmark & \doTCC & \doTCC\\
- & 3 & c & a & b & a & b & \xmark & \xmark & \doTCC & \doTCC\\
- & 4 & c & b & a & c & a & \xmark & \xmark & \doTCC & \doTCC\\
- & 5 & b & a & b & c & a & \xmark & \xmark & \doTCC & \doTCC\\
- & 6 & b & b & a & b & b & \xmark & \xmark & \doTCC & \doTCC\\
- & 7 & c & b & b & a & b & \doTCC & \doTCC & \doTCC & \doTCC\\
- & 8 & b & b & b & a & b & \doTCC & \doTCC & \doTCC & \doTCC\\
- & 9 & a & b & b & b & a & \xmark & \xmark & \doTCC & \doTCC\\
+ & a & a & a & a & a & \doTCC & \doTCC & \doTCC & \doTCC\\
+ & a & b & b & c & b & \xmark & \xmark & \doTCC & \doTCC\\
+ & c & a & b & a & b & \xmark & \xmark & \doTCC & \doTCC\\
+ & c & b & a & c & a & \xmark & \xmark & \doTCC & \doTCC\\
+ & b & a & b & c & a & \xmark & \xmark & \doTCC & \doTCC\\
+ & b & b & a & b & b & \xmark & \xmark & \doTCC & \doTCC\\
+ & c & b & b & a & b & \xmark & \doTCC & \doTCC & \doTCC\\
+ & b & b & b & a & b & \xmark & \doTCC & \doTCC & \doTCC\\
+ & a & b & b & b & a & \xmark & \xmark & \doTCC & \doTCC\\
\hline
\end{tabular}
\caption{Combinations of test cases}
As always, $100\%$ completeness is not feasible, therefore test cases are
carefully selected to cover the most interesting parts of the TCP specification
- to ensure a test suite.
+ to ensure a complete but feasible test suite.
+
+ To further increase the coverage of the test suites tests are randomized. The
+ tests which test the handling of \emph{bit errors}, changes in the \emph{packet
+ order} and \emph{dropped packets} randomize where they introduce an error. The
+ test suite runs these tests multiple times to increase the likelihood that they
+ discover a fault which is only present when an error occurs in a certain
+ position.
To further decrease the number of tests needed test cases are divided into
equivalence partitions and the combination of cases as described in
& 2. Execute the script by running \texttt{\# code/client/tests/5.py} \\\hline
Valid trace & Verify that the script prints 'Success'. \\\hline
\hline
+
+ Nr & 6 \\\hline
+ Title & Request with corrupted source port. \\\hline
+ Input & Generated packets with 1byte payload, in these packets the source port number is increased by one. \\\hline
+ Expected output & - \\\hline
+ \multirow{2}{*}{Course of action}
+ & 1. Use the steps listed above in order to start the SUT. \\
+ & 2. Execute the script by running \texttt{\# code/client/tests/6.py} \\\hline
+ Valid trace & Verify that the script prints 'Success'. \\\hline
+ \hline
+
+
+ Nr & 6 \\\hline
+ Title & Request with corrupted destination port. \\\hline
+ Input & Generated packets with 1byte payload, in these packets the destination port number is increased by one. \\\hline
+ Expected output & - \\\hline
+ \multirow{2}{*}{Course of action}
+ & 1. Use the steps listed above in order to start the SUT. \\
+ & 2. Execute the script by running \texttt{\# code/client/tests/6.py} \\\hline
+ Valid trace & Verify that the script prints 'Success'. \\\hline
+ \hline
+
\end{longtable}
%\begin{tabularx}{\linewidth}{| l | X|}
repositories / tt2015.git / commitdiff