report/reflection.bottlenecks.tex

   1  % Wouter:  What were the bottlenecks in doing the security review in your
   2  % experience?
   3
   4 % Only working on part of the code: 'big picture' may be missing
   5 When we divided the work that had to be done for the security audit we
   6 settled on giving each each member two categories to check. Then later we all
   7 verified each other's findings.
   8
   9 This approach meant that everyone got very familiar with a certain
  10 functionality of the web application. But it also created a bottleneck in
  11 the sense that it is easy to lose overview of the application as the sum of
  12 its parts. We feel this bottleneck is an acceptable consequence of the way we
  13 divided the workload.
  14
  15 % Finding the code which handles a requirement (failing is easier than pass)
  16 Another bottleneck we ran into when failing a requirement was not trivial, was
  17 finding the actual code which handles a requirement. When assessing if a certain
  18 requirement passes or fails the OWASP guidelines, one has to find the code which
  19 implements the requirement. For many of the requirements this comes down to
  20 finding a counterexample. In our experience there are not often very hard to
  21 find for this specific web application. However for some requirements
  22 counterexamples will be harder to find. For these we needed to find all the code
  23 which touches upon this requirement and then verify it.
  24
  25 To summarize the previous paragraph: it is almost always easier to fail a
  26 requirement and be sure of it, then to pass it. Because passing means that the
  27 requirement is OK in \emph{all} cases. While failing needs only to mean that an
  28 requirement fails in \emph{one} case.
  29
  30 % Verifying each other's results
  31 The final bottleneck we like to note concerns the verifying of each other's
  32 results. When verifying the results of your group members (colleagues) it may be
  33 difficult to fully double-check their work. This touches upon the previous
  34 point, for when a counterexample is giving it is trivial to see the result is
  35 correct. However when a member of the group passes a requirement it will take
  36 more time to verify this is the correct verdict. In the end we are confident of
  37 our results, but this may be a bottleneck which can be addressed in the initial
  38 organization.