3eval.tex

   1 \subsection{Exit criteria}
   2 \label{sec:exitcriteria}
   3 The testing will be complete and the SUT will pass if and only if all the
   4 functionality tests are a success. Functionality tests are very important since
   5 the SUT has to adhere to a specification of a computer-to-computer protocol and
   6 therefore the impact of a defect in the system is very high.
   7
   8 Other exit criteria are schedule deadline and cost limits. When these limits
   9 are reached the testing will terminate but there is no confidence in the
  10 quality of the product.
  11
  12 \subsection{Test ware}
  13
  14 \subsection{Issue tracking}
  15 Whenever an issue is detected (a test fails) this issue needs to be recorded
  16 such that this incident is added to the test report. It can then be determined
  17 if the incident was the result of a fault in the SUT or was caused by something
  18 else, e.g. an error in the test.
  19
  20 The record of a incident needs to hold all relevant data such that the incident
  21 can easily be reproduced later. Therefore each incident will be recorded in an
  22 incident report, which will include the following details:
  23 \begin{itemize}
  24         \item When the issue was detected and by whom.
  25         \item Which test detected the issue.
  26         \item Expected results.
  27         \item Actual Results.
  28         \item Relevant logs, screenshots or other data which is needed in
  29                         addition to the test itself to reproduce the incident and which
  30                         might be beneficial to it's resolution.
  31         \item Other areas of the SUT which might be affected by the incident or by
  32                         a change resulting from the incident.
  33 \end{itemize}
  34