

\subsection{Fortify's results, summarized}

Fortify's results can be summarized to the following:

\begin{enumerate}[label=(\Alph*)]
  \item 50 cases of \XSS{} vurnerabilities, all labeled \textbf{critical}, because none of the \CMS{}'s forms include nonces / protection against \XSS{} is indeed missing.
  \item \textbf{Password management}. In a user password reset form in \code{reset.php}, if the resetting fails, the password the user just entered reappears in the password field. This is not a database-retrieved password, and hence not actually as \textbf{critical} as Fortify labels it, but of course bad practice nonetheless.
  \item In the \textbf{privacy violation} category, Fortify found errors and warnings printed back to the browser, and labelled it \textbf{critical}. However, this happens in the installer script, which we have decided to treat separately, as explained earlier.
  \item \textbf{\SQL{} injection} attacks are possible on the installer script, labelled \textbf{critical}. Yet again: the installer script.
  \item \textbf{Cookie security}: the \code{HttpOnly} header is not set, labelled \textbf{high}.
  \item \textbf{Privacy violation}: \HTML{} forms do not disable autocompletion. Labelled \textbf{high}. However, autocompletion of \HTML{} forms by means of the \code{autocompletion="none"} attribute notoriously does not really work. The larger problem is that the post/redirect/get pattern is not followed, as stated above at our analysis of OWASP requirement (9.1).
  \item Fortify complains that \PHP{}'s \code{crypt(...)} function is \textbf{weak encryption} and labels the 5 usages \textbf{high}.
\end{enumerate}


\subsection{Analysis}

The main point that must be observed is that all the above results are quite low-level of nature. The majority of the OWASP ASVS requirements are of a more high-level nature. Two good examples are:

\begin{description}
  \item[V4.9] Verify that the same access control rules implied by the presentation layer are enforced on the server side. \\
  (\textit{The \CMS{} failed this requirement in our analysis.})
  \item[V5.17] Verify that the application has defenses against \HTTP{} parameter pollution attacks, particularly if the application framework makes no distinction about the source of request parameters (\GET{}, \POST{}, cookies, headers, environment, etc. \\
  (\textit{The \CMS{} passed this requirement in our analysis.})
\end{description}

For this reason, Fortify was nowhere near able to identifying all the problems we found in the \CMS{}. An overview of our findings, where Fortify's concurrences are outlined explicitly, is given by the table below.


\newcommand{\p}{\textit{pass}}
\newcommand{\X}{\textbf{FAIL}}
\setlength\fboxrule{1pt}
\setlength\fboxsep{4pt}

%\newcommand{\F}[2]{%
%  \hspace*{-5pt}%
%  \boxed{\textrm{#2}}$^{\,\textrm{\scriptsize(#1)}}$%
%  \hspace*{-5pt}%
%}% fortify-found security problem: \F\X

\newlength{\lofF}
\newcommand{\F}[2]{%
  \setlength{\lofF}{\widthof{\;#2\;}}
  \hspace*{-2pt}%
  \framebox[\lofF]{\phantom{K}}%
  \hspace*{-\lofF}%
  \;#2\;%
  $^{\,\textrm{\scriptsize(#1)}}$%
}% fortify-found security problem: \F\X

\begin{table}[th!]
\centering
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{@{}p{20pt}p{35pt}p{35pt}p{35pt}p{35pt}p{35pt}p{35pt}p{35pt}p{35pt}p{35pt}@{}}
\toprule
\# &
\textbf{V2} &
\textbf{V3} &
\textbf{V4} &
\textbf{V5/6} &
\textbf{V7} &
\textbf{V8} &
\textbf{V9} &
\textbf{V11} \\
\midrule
%       V2       V3     V4        V5     V7        V8      V9      V11
 1 &   \X    &   \p &   \p    &   \p &    -    &   \X & \F{B}\X &   \X  \\
 2 & \F{B}\X &   \p &    -    &    - &   \p    &   \p &     -   &   \p  \\
 3 &    -    &   \X &    -    &   \X &    -    &   \X &    \p   &    -  \\
 4 &   \p    &    - &   \X    &    - &    -    &   \X &    \X   &   \X  \\
 5 &    -    &   \p &   \X    &   \p &    -    &   \p &    \p   &   \p  \\
 6 &   \X    &   \p &    -    &    - &   \X    &   \p &     -   &   \X  \\
 7 &   \p    &   \X &    -    &    - &   \X    &   \p &    \p   &   \X  \\
 8 &   \p    &    - &   \p    &    - &    -    &   -  &     -   &   \X  \\
 9 &   \X    &    - &   \X    &    - &   \X    &   -  &    \p   &   -   \\
10 &    -    &    - &   \X    &   \X &    -    &   \X &    \X   &   -   \\
11 &     -   &   \p &     -   &   \p &    -    &   -  &    \p   &   -   \\
12 &   \X    &   \X &   \X    &   \p &   \X    &   -  &     -   &   -   \\
13 &   \X    &   -  & \F{A}\X &   \p &   \X    &   \X &     -   &   -   \\
14 &     -   &    - &   \X    &   \p &   \p    &   -  &     -   &   -   \\
15 &     -   &    - &   \X    &   \X &    -    &   -  &     -   &   -   \\
16 &   \X    &   \p &   \X    &   \p &    -    &   -  &     -   &   -   \\
17 &   \p    &   \X &    -    &   \p &   -     &   -  &    -    &   -   \\
18 &   \X    &   \X &    -    &   \X &   -     &   -  &    -    &   -   \\
19 &   \p    &   -  &    -    &   \X &   -     &   -  &    -    &   -   \\
20 &   \X    &   -  &    -    &   \p &   -     &   -  &    -    &   -   \\
21 &   \X    &   -  &    -    &   \p &   -     &   -  &    -    &   -   \\
22 &   \p    &   -  &    -    &   \X &   -     &   -  &    -    &   -   \\
23 &   \X    &   -  &    -    &   \X &   -     &   -  &    -    &   -   \\
24 &   \p    &   -  &    -    &   \p &   -     &   -  &    -    &   -   \\
25 &   \X    &   -  &    -    &   \p &   -     &   -  &    -    &   -   \\
26 &   \p    &   -  &    -    &   \p &   -     &   -  &    -    &   -   \\
27 &   \X    &   -  &    -    &    - &   -     &   -  &    -    &   -   \\
28 &    -    &   -  &    -    &    - &   -     &   -  &    -    &   -   \\
29 &    -    &   -  &    -    &    - &   -     &   -  &    -    &   -   \\
30 &    -    &   -  &    -    &    - &   -     &   -  &    -    &   -   \\
31 &   \X    &   -  &    -    &    - &   -     &   -  &    -    &   -   \\
32 &   \X    &   -  &    -    &    - &   -     &   -  &    -    &   -   \\
33 &   \p    &   -  &    -    &    - &   -     &   -  &    -    &   -   \\
\bottomrule
\end{tabular}
\caption{Summary of our results. Fortify's findings are outlined and labelled, see our analysis above.}
\end{table}
