From: Dion Date: Wed, 11 Nov 2015 16:27:00 +0000 (+0100) Subject: Merge branch 'master' of https://github.com/dopefishh/tt2015 X-Git-Url: https://git.martlubbers.net/?a=commitdiff_plain;ds=sidebyside;h=2e920ad2a1fe23bed0a6112c8641584ece920d21;hp=-c;p=tt2015.git Merge branch 'master' of https://github.com/dopefishh/tt2015 --- 2e920ad2a1fe23bed0a6112c8641584ece920d21 diff --combined a2/1cases.tex index 78ae168,2d27d9b..bcea37f --- a/a2/1cases.tex +++ b/a2/1cases.tex @@@ -35,7 -35,7 +35,7 @@@ user permissions \hline\hline Check 3 & \multicolumn{2}{l|}{Initialize the testing environment..}\\ \hline - \multirow{5}{*}{Course of action} + \multirow{6}{*}{Course of action} & 1. & Boot the SUT as in \emph{Check 1}.\\ & 2. & Setup iptables by executing \texttt{\# code/iptables.sh}~\footnote{The IPTables script ensures @@@ -43,9 -43,11 +43,11 @@@ & 3. & Navigate to the working directory by running \texttt{\$ cd /home/student/tt2015}\\ & 4. & Compile the echo server by running - \texttt{\# cd code/server \&\& make \&\& cd -}\\ + \texttt{\# cd code/server \&\& make}\\ & 5. & Start the echo server by running \texttt{\# cd code/server \&\& java Main}\\ + & 6. & Generate all test cases by running + \texttt{\$ python code/client/gen.py}\\ \hline Passed & \multicolumn{2}{l|}{\textit{Yes/No}}\\ \hline\hline @@@ -119,11 -121,11 +121,11 @@@ these partitions are given \item Correct \item Incorrect \end{enumerate} - \item \emph{Packet order} + \item \emph{Segment order} \begin{enumerate} \item Correct \item Out of order - \item Missing packets + \item Missing Segments \end{enumerate} \end{enumerate} @@@ -161,29 -163,35 +163,35 @@@ Partitions 2 to 6 are tested using pair cases feasible. The pairs are then all *except some where it does not make sense to do so) tested with the different request sizes of partition 1. - This is expressed in Table~\ref{table:testpairs}. + This is expressed in Table~\ref{table:testpairs}. In this table the first five + columns represent the different options for the partitions 2 to 6 of the above + enumeration. The last four columns are the different number segments as + described in the partition 1 of the above enumeration. These cells identify + individual test cases by a number. An \xmark in the cell indicates that this + test case can not be created as it is not possible with that number of segments + (eg. sending segments out of order when the number of segments is 1). \newcounter{TCC} \setcounter{TCC}{1} \newcommand{\doTCC}{\theTCC \stepcounter{TCC}} \begin{table}[H] \centering - \begin{tabular}{|l|l|l|l|l|l|l|l|l|l|l|} + \begin{tabular}{|l|l|l|l|l|l||l|l|l|l|} \hline - & \multicolumn{10}{c|}{\textbf{Partition}}\\ + & \multicolumn{9}{c|}{\textbf{Partition}}\\ \hline - & \# & 4 & 5 & 3 & 6 & 2 & 1a & 1b & 1c & 1d\\ + & 4 & 5 & 3 & 6 & 2 & 1a & 1b & 1c & 1d\\ \hline\hline \multirow{9}{*}{Instance} - & 1 & a & a & a & a & a & \doTCC & \doTCC & \doTCC & \doTCC\\ - & 2 & a & b & b & c & b & \xmark & \xmark & \doTCC & \doTCC\\ - & 3 & c & a & b & a & b & \xmark & \xmark & \doTCC & \doTCC\\ - & 4 & c & b & a & c & a & \xmark & \xmark & \doTCC & \doTCC\\ - & 5 & b & a & b & c & a & \xmark & \xmark & \doTCC & \doTCC\\ - & 6 & b & b & a & b & b & \xmark & \xmark & \doTCC & \doTCC\\ - & 7 & c & b & b & a & b & \doTCC & \doTCC & \doTCC & \doTCC\\ - & 8 & b & b & b & a & b & \doTCC & \doTCC & \doTCC & \doTCC\\ - & 9 & a & b & b & b & a & \xmark & \xmark & \doTCC & \doTCC\\ + & a & a & a & a & a & \doTCC & \doTCC & \doTCC & \doTCC\\ + & a & b & b & c & b & \xmark & \xmark & \doTCC & \doTCC\\ + & c & a & b & a & b & \xmark & \xmark & \doTCC & \doTCC\\ + & c & b & a & c & a & \xmark & \xmark & \doTCC & \doTCC\\ + & b & a & b & c & a & \xmark & \xmark & \doTCC & \doTCC\\ + & b & b & a & b & b & \xmark & \xmark & \doTCC & \doTCC\\ + & c & b & b & a & b & \xmark & \doTCC & \doTCC & \doTCC\\ + & b & b & b & a & b & \xmark & \doTCC & \doTCC & \doTCC\\ + & a & b & b & b & a & \xmark & \xmark & \doTCC & \doTCC\\ \hline \end{tabular} \caption{Combinations of test cases} @@@ -214,7 -222,14 +222,14 @@@ test suite can be clearly assessed As always, $100\%$ completeness is not feasible, therefore test cases are carefully selected to cover the most interesting parts of the TCP specification - to ensure a test suite. + to ensure a complete but feasible test suite. + + To further increase the coverage of the test suites tests are randomized. The + tests which test the handling of \emph{bit errors}, changes in the \emph{packet + order} and \emph{dropped packets} randomize where they introduce an error. The + test suite runs these tests multiple times to increase the likelihood that they + discover a fault which is only present when an error occurs in a certain + position. To further decrease the number of tests needed test cases are divided into equivalence partitions and the combination of cases as described in @@@ -291,28 -306,6 +306,28 @@@ Before every test case use the followin & 2. Execute the script by running \texttt{\# code/client/tests/5.py} \\\hline Valid trace & Verify that the script prints 'Success'. \\\hline \hline + + Nr & 6 \\\hline + Title & Request with corrupted source port. \\\hline + Input & Generated packets with 1byte payload, in these packets the source port number is increased by one. \\\hline + Expected output & - \\\hline + \multirow{2}{*}{Course of action} + & 1. Use the steps listed above in order to start the SUT. \\ + & 2. Execute the script by running \texttt{\# code/client/tests/6.py} \\\hline + Valid trace & Verify that the script prints 'Success'. \\\hline + \hline + + + Nr & 6 \\\hline + Title & Request with corrupted destination port. \\\hline + Input & Generated packets with 1byte payload, in these packets the destination port number is increased by one. \\\hline + Expected output & - \\\hline + \multirow{2}{*}{Course of action} + & 1. Use the steps listed above in order to start the SUT. \\ + & 2. Execute the script by running \texttt{\# code/client/tests/6.py} \\\hline + Valid trace & Verify that the script prints 'Success'. \\\hline + \hline + \end{longtable} %\begin{tabularx}{\linewidth}{| l | X|}