report/reflection.tex

   1 Some categories in the ASVS are easier to check than others. For example
   2 Section~\ref{sec:v6}. A lot of possible attack vectors were not available just
   3 because the were not used. In other cases the verdict was an easy fail since
   4 some components, like input escaping, are just not present.
   5 %
   6 %TODO
   7 %- Vandaag verdict (puntjes) uploaden (iedereen)!
   8 %- Na morgen heeft iedereen de resultaten van Fortify een keer bekeken
   9 %- Aanpak beschrijven, vóór volgende bijeenkomst (Daan)
  10 %- Iedereen een stuk reflectie schrijven (zie onder):
  11 %- Spreadsheet bouwen, vóór volgende bijeenkomst (Charlie)
  12 %
  13 %Uit opdracht:
  14 %
  15 %Reflection
  16 %Reflect on the whole process, including
  17 %
  18 %   the ASVS,
  19 %   the use of static code analysis tools,
  20 %   the way you organised the process,
  21 %   and possibly also the TestCMS code.
  22 %
  23 %For example, questions to consider are:
  24 %   - Kelley:  How good (useful, clear, ...) is the ASVS? How could it be improved?
  25 %   - Charlie: How useful were code analysis tools? How could they be improved? How did you experience the rates and amounts of false and true positives? How might that be improved?
  26 %   - Wouter:  What were the bottlenecks in doing the security review in your experience?
  27 %   - Mart:    Maybe in the points above you can distinguish different (types of) security flaws or verification requirements. E.g., are some (categories of) verification requirements easier to check than others?
  28 %   - Daan:    If you would have to do something like this again, what would you do differently? Eg. about organising things within the group: i.e., in retrospect, what do you think the best approach is to organise and divide the work in a team? (Dividing the verification requirements over the team members? Or by dividing the code? Or letting everyone look at everything, because different people will spot different things? Or work in pairs where one person confirms the findings of the other? ...)
  29 %   - Kelley:  About the TestCMS code: are there important aspects that could (or should) be changed to improve security? Or aspects that could be changed to facilitate doing a security review?
  30 %   - Wouter:  This last question might be generalised, to (web) applications in general, rather than this specific example of the TestCMS: ie. if you were to develop an application that will need to be subjected to a security review, would you do anything differently, given your experience in doing this, and if so, what and why?
  31
  32
  33 % Daan: If you would have to do something like this again? (...)
  34 We have noticed that, when doing an audit in a team, it is not feasible for
  35 everybody to have read all source code. Henceforth, trying this is just a bad
  36 idea. We are happy to have divided the project by ASVS category, instead of
  37 program component. For each requirement the the ASVS, the
  38 team had to verify that there were no mistakes in the code. This would have
  39 taken a lot of time if we had to verify each component for each requirement.
  40 Furthermore, the ASVS is an easy guide for dividing the work\footnote{The
  41 categories in the ASVS are all of similar size. We settled on giving each team
  42 member two categories to check.}. Dividing by component would have been a lot
  43 harder to do fairly, especially because when beginning the project we had
  44 little knowledge of the internals (and component sizes) of the CMS.
  45
  46 We haven't experimented by working in pairs. This might be a good idea to
  47 experiment with. We are confident however that, because we have all checked
  48 each other's finished work (and the final product), we did not miss any
  49 problems.
  50 In the end, we are satisfied with the way we have done things.