2approach.tex

   1 %6
   2 %7
   3 %8
   4 %9
   5 %10
   6 %11
   7 %12
   8
   9 The test process consists of several stages. The results of the first stage,
  10 planning, are descibed in this document. On the basis of this document actual
  11 test cases will be designed. Afterwards the actual tests will be implemented
  12 and executed. The results of these tests will then be evaluated against the
  13 exit criteria (see Section~\ref{sec:exitcriteria}) and depending on the outcome
  14 of these evaluations further tests might be deemed necessary.
  15
  16 \subsection{Quality Characteristics}
  17 The quality characteristics that are to be tested are described using the
  18 ISO/IEC 25010 \cite{iso25010} as a guideline. In the following sections we will
  19 discuss the relevant \textit{product quality} and \textit{quality in use}
  20 characteristics.
  21
  22 \subsubsection{Product quality}
  23 Product quality is divided into eight main categories which are divided into
  24 several subcategories. Below we will discuss the qualities which are relevant
  25 to the SUT.
  26 \begin{itemize}
  27         \item \textbf{Functional suitability}\\
  28         As described in Section~\ref{sec:risks} the SUT is core functionality of
  29         the networking capable system. Because many other systems running on the
  30         system could rely on it it is very important that the SUT is functionality
  31         suitable.  Therefore all three sub characteristics of Functional
  32         Suitability (\textit{Functional completeness, Functional correctness,
  33         Functional appropriateness}) are of vital importance. As was previously
  34         mentioned in Section~\ref{sec:risks} extra emphasis should be placed on
  35         testing \emph{Functional Correctness} as recovery from Failures in
  36         computer-to-computer systems is problematic.
  37 \end{itemize}
  38
  39 \subsubsection{Quality in use}
  40
  41 \subsection{Levels and types of testing} \label{levels}
  42
  43 The client will deliver a product for certification. This means our team will only conduct acceptance testing and assume that the client who requested certification has conducted unit, module and integration testing. We will only be conducting black-box testing and the client is not required to handover any source-code. Initially we will conduct several basic test cases based on experience acquired from previous certification requests (error guessing). If the product fails these basic tests we reject it and seize all further activities. If the product is not rejected we will proceed with more thorough testing. For each test we produce a test report. If any of the test cases fail the product is still rejected but in order to deliver usable feedback to the client we will still produce a test report.
  44
  45 \subsection{Test generation}
  46
  47 The basic tests mentioned in section \ref{levels} are conducted using a checklist. If any of the checks fail we immediately reject the product.
  48
  49 \begin{enumerate}
  50         \item Is the product complete?
  51         \item Does the product come with a manual or quick start guide?
  52         \item Is it possible to get the product in a usable state?
  53         \item Can we use the product to initiate a connection in a corruption free environment?
  54         \item ....
  55 \end{enumerate}
  56
  57 For the remaining tests we first use equivalence partitioning to reduce the overall number of test cases.
  58
  59 \begin{enumerate}
  60         \item Valid requests:
  61                 \begin{enumerate}
  62                         \item Single request.
  63                         \item Multiple requests.
  64                 \end{enumerate}
  65         \item Invalid requests:
  66                 \begin{enumerate}
  67                         \item Single request.
  68                         \item Multiple requests.
  69                 \end{enumerate}
  70 \end{enumerate}
  71
  72 For these requests we can introduce more cases using equivalence partitioning for the different packets that are sent during one request.
  73
  74 \begin{enumerate}
  75         \item Packets received in order.
  76         \item Packets received out of order.
  77 \end{enumerate}
  78
  79 For each individual packet we can specify the follow equivalent classes.
  80
  81 \begin{enumerate}
  82         \item Valid packet.
  83         \item Corrupted packet.
  84         \item Missing packets.
  85 \end{enumerate}
  86
  87 We will test all possible combinations of requests/packet order/packet content. For each combination we will use boundary value analysis to reduce the total number of test cases. Boundary values are constructed using the following parameters:
  88
  89 \begin{enumerate}
  90         \item Checksum: valid/invalid
  91         \item Header: valid/invalid
  92         \item Payload: valid/invalid
  93         \item ...
  94 \end{enumerate}