testware gedoe

[tt2015.git] / 3eval.tex

diff --git a/3eval.tex b/3eval.tex
old mode 100644 (file)
new mode 100755 (executable)
index 32b0fb9..ee0a965
--- a/3eval.tex
+++ b/3eval.tex
@@ -1,8 +1,3 @@
-Hello world
-%13
-%14
-%15
-
 \subsection{Exit criteria}
 \label{sec:exitcriteria}
 The testing will be complete and the SUT will pass if and only if all the
@@ -10,10 +5,38 @@ functionality tests are a success. Functionality tests are very important since
 the SUT has to adhere to a specification of a computer-to-computer protocol and
 therefore the impact of a defect in the system is very high.
 
-Other exit criteria can be schedule or cost limits. When these limits are
-reached the testing will terminate but there is no confidence in the quality of
-the product.
+Other exit criteria are schedule deadline and cost limits. When these limits
+are reached the testing will terminate but there is no confidence in the
+quality of the product.
 
 \subsection{Test ware}
+As stated in Section \ref{section:levels} a basic \emph{manual test} is performed in order to detect a faulty project early on. 
+If the SUT fails all second stage test cases we will try to expand the checklist in order to make sure that a similar faulty SUT can be detected during the first stage of testing.
+
+The packets composed according to \textit{RFC793} as described in Section \ref{section:testenv} are saved for possible reuse.
+Since they are created by hand this is a labor intensive task. 
+It makes sense to keep them around in case the SUT fails certification and the client would request reevaluation of a updated version of the SUT. 
+If the SUT does in fact pass certification the packets could still be reused in the event an other client would request \textit{RFC793} certification of a similar product.
+The same is true for the \emph{Java TCP driven echo server}.
 
 \subsection{Issue tracking}
+Whenever an issue is detected (a test fails) this issue needs to be recorded
+such that this incident is added to the test report. It can then be determined
+if the incident was the result of a fault in the SUT or was caused by something
+else, e.g. an error in the test.
+
+The record of a incident needs to hold all relevant data such that the incident
+can easily be reproduced later. Therefore each incident will be recorded in an
+incident report, which will include the following details:
+\begin{itemize}
+       \item When the issue was detected and by whom.
+       \item Which test detected the issue.
+       \item Expected results.
+       \item Actual Results.
+       \item Relevant logs, screenshots or other data which is needed in
+                       addition to the test itself to reproduce the incident and which
+                       might be beneficial to it's resolution.
+       \item Other areas of the SUT which might be affected by the incident or by
+                       a change resulting from the incident.
+\end{itemize} 
+