-%6
-%7
-%8
-%9
-%10
-%11
-%12
-
The test process consists of several stages. The results of the first stage,
planning, are descibed in this document. On the basis of this document actual
test cases will be designed. Afterwards the actual tests will be implemented
and executed. The results of these tests will then be evaluated against the
-exit criteria (see Section~\ref{sec:exitcriteria}) and depending on the outcome
-of these evaluations further tests might be deemed necessary.
+exit criteria found in Section~\ref{sec:exitcriteria} and depending on the
+outcome of these evaluations further tests might be deemed necessary.
\subsection{Quality Characteristics}
The quality characteristics that are to be tested are described using the
computer-to-computer systems is problematic.
\item \textbf{Performance efficiency} \label{sec:perf_eff}\\
As the SUT runs as a service on a system with other programs it must have
- efficient \emph{resouce utilisation}. It can not contain any memory leaks
+ efficient \emph{resource utilisation}. It can not contain any memory leaks
or use other resources more than necessary.
\item \textbf{Compatibility}\\
- \emph{Interoperability} is the key feauture of the SUT as it's purpose is to
- communicate with other systems implementing the TCP protocol. Therefore
- it is of vital importance that the SUT impements the TCP procotol correctly.
- Furthermore it is very important that the SUT can \emph{co-exist} with other
- programs on the system it runs on, since it is used as a service by those
- programs. This means that the SUT has to handle preemption as well as having
- multiple programs requesting it's services at once.
+ \emph{Interoperability} is the key feature of the SUT as it's purpose is to
+ communicate with other systems implementing the TCP protocol. Therefore it
+ is of vital importance that the SUT implements the TCP protocol correctly.
+ Furthermore it is very important that the SUT can \emph{co-exist} with
+ other programs on the system it runs on, since it is used as a service by
+ those programs. This means that the SUT has to handle preemption as well as
+ having multiple programs requesting it's services at once.
\item \textbf{Reliability}\\
- As stated before, the SUT is used as a core service, this means it has to be
- very \emph{mature}. It needs to behave as expected under normal working
- conditions. As it can continually be requested the SUT needs to have
+ As stated before, the SUT is used as a core service, this means it has to
+ be very \emph{mature}. It needs to behave as expected under normal working
+ conditions. As it can continually be requested the SUT needs to have
constant
- \emph{availibility}. As the SUT relies on an (potentially) unreliable
- channel to send and receive data it needs to be \emph{fault tolerant}. The
- SUT needs to properly handle errors in received data or complete
- unavailibilty of the underlying channel.
+ \emph{availability}. As the SUT relies on a potentially unreliable channel
+ to send and receive data it needs to be \emph{fault tolerant}. The SUT
+ needs to properly handle errors in received data or complete unavailability
+ of the underlying channel.
\item \textbf{Security}\\
- \emph{Confidentiliality} is an important aspect for the SUT. Received
- data should only be delivered to the program to which it is adressed.
+ \emph{Confidentiality} is an important aspect for the SUT. Received data
+ should only be delivered to the program to which it is addressed.
\end{itemize}
This leaves three categories which are not relevant the SUT. Below we will
-shortly discuss per categorie why these are not relevant. \emph{Maintainability}
+shortly discuss per category why these are not relevant. \emph{Maintainability}
is an important aspect of any software system, however for the SUT it is not a
core aspect, as it is first and foremost of importance that the implementation
-is correct, furthermore TCP does not change often. \emph{Usability} isn't a core
-aspect either, as the SUT is not used directly by humans, but is a service which
-is adressed when another program needs it. \emph{Portability} isn't either as
-the SUT is installed on a system once and intended to work on that system.
+is correct, furthermore TCP does not change often. \emph{Usability} isn't a
+core aspect either, as the SUT is not used directly by humans, but is a service
+which is addressed when another program needs it. \emph{Portability} isn't
+either as the SUT is installed on a system once and intended to work on that
+system.
\subsubsection{Quality in use}
Quality in use is dived into five subcategories. Below we will discuss the
\item \textbf{Satisfaction}\\
It is important that programs using the SUT can \emph{trust} that the SUT
provides the promised services. This means that data is send and received
- reliably and the SUT provides clear and unambigious errors when this service
+ reliably and the SUT provides clear and unambiguous errors when this
+ service
can not be provided.
\item \textbf{Context Coverage}\\
The SUT needs to behave as expected in all specified contexts
(\emph{context completeness}).
\end{itemize}
-This leaves \emph{freedom from risk}, which we consider not relevant as the
-SUT itself does not pose any risks, and correct implementation (which is covered
-in the other categories) gives clear guarantees to programs using the services
-of the SUT.
+This leaves \emph{freedom from risk}, which we consider not relevant as the SUT
+itself does not pose any risks, and correct implementation (which is covered in
+the other categories) gives clear guarantees to programs using the services of
+the SUT.
\subsection{Levels and types of testing} \label{levels}
-
-The client will deliver a product for certification. This means our team will only conduct acceptance testing and assume that the client who requested certification has conducted unit, module and integration testing. We will only be conducting black-box testing and the client is not required to handover any source-code. Initially we will conduct several basic test cases based on experience acquired from previous certification requests (error guessing). If the product fails these basic tests we reject it and seize all further activities. If the product is not rejected we will proceed with more thorough testing. For each test we produce a test report. If any of the test cases fail the product is still rejected but in order to deliver usable feedback to the client we will still produce a test report. For each test case a performance analysis will be included.
+The client will deliver a product for certification. This means our team will
+only conduct acceptance testing and assume that the client who requested
+certification has conducted unit, module and integration testing. We will only
+be conducting black-box testing and the client is not required to handover any
+source-code. Initially we will conduct several basic test cases based on
+experience acquired from previous certification requests (error guessing). If
+the product fails these basic tests we reject it and seize all further
+activities. If the product is not rejected we will proceed with more thorough
+testing. For each test we produce a test report. If any of the test cases fail
+the product is still rejected but in order to deliver usable feedback to the
+client we will still produce a test report. For each test case a performance
+analysis will be included.
\subsubsection{Test generation}
-
-The basic tests mentioned in section \ref{levels} are conducted using a checklist. If any of the checks fail we immediately reject the product.
+The basic tests mentioned in Section \ref{levels} are conducted using a
+checklist. If any of the checks fail we immediately reject the product.
\begin{enumerate}
\item Is the product complete?
\item Does the product come with a manual or quick start guide?
\item Is it possible to get the product in a usable state?
- \item Can we use the product to initiate a connection in a corruption free environment?
+ \item Can we use the product to initiate a connection in a corruption
+ free
+ environment?
\item ....
\end{enumerate}
-For the remaining tests we first use equivalence partitioning to reduce the overall number of test cases.
+For the remaining tests we first use equivalence partitioning to reduce the
+overall number of test cases.
\begin{enumerate}
\item Valid requests:
\end{enumerate}
\end{enumerate}
-For these requests we can introduce more cases using equivalence partitioning for the different packets that are sent during one request.
+For these requests we can introduce more cases using equivalence partitioning
+for the different packets that are sent during one request.
\begin{enumerate}
\item Packets received in order.
\item Missing packets.
\end{enumerate}
-We will test all possible combinations of requests/packet order/packet content. For each combination we will use boundary value analysis to reduce the total number of test cases. Boundary values are constructed using the following parameters:
+We will test all possible combinations of requests/packet order/packet content.
+For each combination we will use boundary value analysis to reduce the total
+number of test cases. Boundary values are constructed using the following
+parameters:
\begin{enumerate}
\item Checksum: valid/invalid
\item ...
\end{enumerate}
-\subsubsection{Test environment en automatization}
-
-All the tools we are going to use together with the SUT gives us the following collection of software.
+\subsubsection{Test environment and automatization}
+All the tools we are going to use together with the SUT gives us the following
+collection of software.
\begin{enumerate}
\item Windows, used as a host OS.
\item Linux, used as both a host and guest OS.
\item VirtualBox, used to run the guest OS containing the SUT.
- \item Wireshark, used on the guest in order to capture and analyze network traffic.
+ \item Wireshark, used on the guest in order to capture and analyze network
+ traffic.
\item Tcpdump, used to prepare network packets.
\end{enumerate}
-All test will be conducted in a virtual environment. We will use VirtualBox to run a Linux distro with the product installed. All the tests are performed from within the VirtualBox environment. When testing network transmissions we will only analyze the packets sent/received to/from the guest OS. The host system is disconnected from the Internet or any other network in order to prevent unnecessary traffic.
-
-For each test case (except for the basic tests) a file containing previously captured network traffic will be replayed using Wireshark. We will use tcpdump to update the prepared packets with the MAC address of the guest network adapter. The response packets coming from the guest OS will be recorded and analyzed at a later stage. The valid packets are obtained by capturing traffic between known working alternatives to the SUT. Invalid packets are generated from this valid traffic using tcpdump. The boundary values for the different parameters (fields in packets) are determined by hand. Automated scripts are written that will generate packets with some fields replaced with these boundary values. The performance analysis will consists of measured latencies for all packets sent.
-
+All test will be conducted in a virtual environment. We will use VirtualBox to
+run a Linux distribution with the product installed. All the tests are
+performed from within the VirtualBox environment. When testing network
+transmissions we will only analyze the packets sent/received to/from the guest
+OS. The host system is disconnected from the Internet or any other network in
+order to prevent unnecessary traffic.
+% Dit is niet nodig omdat het via loopback gaat
+
+For each test case (except for the basic tests) a file containing previously
+captured network traffic will be replayed using Wireshark. We will use tcpdump
+to update the prepared packets with the MAC address of the guest network
+adapter. The response packets coming from the guest OS will be recorded and
+analyzed at a later stage. The valid packets are obtained by capturing traffic
+between known working alternatives to the SUT. Invalid packets are generated
+from this valid traffic using tcpdump. The boundary values for the different
+parameters (fields in packets) are determined by hand. Automated scripts are
+written that will generate packets with some fields replaced with these
+boundary values. The performance analysis will consists of measured latencies
+for all packets sent.
+
+% Dit is mooier om footnotes van te maken en te gebruiken als het voor het
+% eerst gerefereerd is
\begin{enumerate}
\item VirtualBox, https://www.virtualbox.org/
\item Whireshark, https://www.wireshark.org/
repositories / tt2015.git / commitdiff