

\subsection{Fortify's results, summarized}

Fortify's results can be summarized to the following:

\begin{enumerate}[label=(\Alph*)]
  \item 50 cases of \textbf{XSS} vurnerabilities, all labeled \textbf{critical}, because none of the CMS's forms include nonces / protection against XSS is indeed missing.
  \item \textbf{Password management}. In a user password reset form in \code{reset.php}, if the resetting fails, the password the user just entered reappears in the password field. This is not a database-retrieved password, and hence not actually as \textbf{critical} as Fortify labels it, but of course bad practice nonetheless.
  \item In the \textbf{privact violation} category, Fortify found errors and warnings printed back to the browser, and labelled it \textbf{critical}. However, this happens in the installer script, which we have decided to treat separately, as explained earlier.
  \item \textbf{SQL injection} attacks are possible on the installer script, labelled \textbf{critical}. Yet again: the installer script.
  \item \textbf{Cookie security}: the \code{HttpOnly} header is not set, labelled \textbf{high}.
  \item \textbf{Privacy violation}: HTML forms don't disable autocompletion. Labelled \textbf{high}. However, autocompletion of HTML forms by means of the \code{autocompletion="none"} attribute notoriously doesn't really work. The larger problem is that the post/redirect/get pattern is not followed, as stated above at our analysis of OWASP requirement (9.1).
  \item Fortify complains that PHP's \code{crypt(...)} function is \textbf{weak encryption} and labels the 5 usages \textbf{high}.
\end{enumerate}


\subsection{Analysis}

The main point that must be observed is that all the above results are quite low-level of nature. The majority of the OWASP ASVS requirements are of a more high-level nature. Two good examples are:

\begin{description}
  \item[V4.9] Verify that the same access control rules implied by the presentation layer are enforced on the server side. \\
  (\textit{The CMS failed this requirement in our analysis.})
  \item[V5.17] Verify that the application has defenses against HTTP parameter pollution attacks, particularly if the application framework makes no distinction about the source of request parameters (GET, POST, cookies, headers, environment, etc.. \\
  (\textit{The CMS passed this requirement in our analysis.})
\end{description}

For this reason, Fortify was nowhere near able to identifying all the problems we found in the CMS. An overview of our findings, where Fortify's concurrences are outlined explicitly, is given by the table below.


\newcommand{\p}{{\color{lightgray}\pass}}
\newcommand{\X}{\fail}
\setlength\fboxrule{1pt}
\setlength\fboxsep{4pt}
\newcommand{\F}[2]{%
  \hspace*{-5pt}%
  \boxed{\textrm{#2}}$^{\,\textrm{\small(#1)}}$%
  \hspace*{-5pt}%
}% fortify-found security problem: \F\X

\begin{table}[th!]
\centering
%\renewcommand{\arraystretch}{1}
\begin{tabular}{@{}lllllllllll@{}}
\toprule
\# &
\textbf{V2} &
\textbf{V3} &
\textbf{V4} &
\textbf{V5} &
\textbf{V7} &
\textbf{V8} &
\textbf{V9} &
\textbf{V12} &
\textbf{V17} \\
\midrule
%      V2     V3     V4     V5     V7        V8     V9     V12    V17
 1 &   \X    &   \p &   \p    &   \p &         &   \X & \F{B}\X &   \X &   \TODO \\
 2 & \F{B}\p &   \p &         &      &   \p    &   \p &       &   \p &   \TODO \\
 3 &         &   \X &         &   \X &         &   \X &    \p &      &   \TODO \\
 4 &   \p    &      &   \p    &      &         &   \X &    \X &   \X &   \TODO \\
 5 &         &   \p &   \p    &   \p &         &   \p &    \p &   \p &   \TODO \\
 6 &   \X    &   \p &         &      &         &   \p &       &   \X &   \TODO \\
 7 &   \p    &   \X &         &      &   \TODO &   \p &    \p &   \X &   \TODO \\
 8 &   \p    &      &   \p    &      &         &      &       &   \X &         \\
 9 &   \X    &   \p &   \X    &      &         &      &    \p &      &   \TODO \\
10 &         &   \X &   \p    &   \X &         &   \X &    \p &      &   \TODO \\
11 &         &   \p &         &   \p &         &      &    \p &      &   \TODO \\
12 &   \X    &   \X &   \X    &   \p &         &      &       &      &         \\
13 &   \X    &   \X & \F{A}\X &   \p &         &   \X &       &      &         \\
14 &         &      &   \X    &   \p &         &      &       &      &         \\
15 &         &      &   \X    &   \X &         &      &       &      &         \\
16 &   \X    &      &   \X    &   \p &         &      &       &      &         \\
17 &   \p    &      &         &   \p &         &      &       &      &         \\
18 &   \X    &      &         &   \X &         &      &       &      &         \\
19 &   \p    &      &         &   \X &         &      &       &      &         \\
20 &   \X    &      &         &   \p &         &      &       &      &         \\
21 &   \X    &      &         &   \p &         &      &       &      &         \\
22 &   \p    &      &         &   \X &         &      &       &      &         \\
23 &         &      &         &   \X &         &      &       &      &         \\
24 &         &      &         &   \p &         &      &       &      &         \\
25 &   \X    &      &         &   \p &         &      &       &      &         \\
26 &         &      &         &   \p &         &      &       &      &         \\
27 &   \X    &      &         &      &         &      &       &      &         \\
28 &   \X    &      &         &      &         &      &       &      &         \\
29 &   \X    &      &         &      &         &      &       &      &         \\
30 &         &      &         &      &         &      &       &      &         \\
31 &         &      &         &      &         &      &       &      &         \\
32 &   \X    &      &         &      &         &      &       &      &         \\
33 &   \p    &      &         &      &         &      &       &      &         \\
\bottomrule
\end{tabular}
\caption{Summary of our results. Fortify's findings are outlined and labelled, see our analysis above.}
\end{table}
