Introduction =============================================================================== Testing is: - Measure quality - Increase confidence - Help assess risks Challenges: - Big domains(infinite) - Never exhaustive - When to stop Activities: - Generation - Execution - Organization Quality Functionality, Reliability, Usability, Efficiency, Mainainability, Portability Types: - {Non, }Functional, functions or properties - Dimensionality - Accessibility(black, grey, white box) - Detail(system, integration, module, unit) - Characteristics(functional, reliability, usability etc.) Black & White Box Testing =============================================================================== Black box - Partitioning - Boundary value analysis - Error quessing(exploratory testing) White box - Coverage - Path - Statement - Condition - (multiple=all combinations of) Decicion Test Organization/TMap =============================================================================== TMap: Test Management approach planning->preparation->generation->execution->completion v-model test incomplete things with - Driver: piece that simulates higher level system - Stub: piece that simulates lowel level [L]ife cycle - Strategy/planning - Mission - Strategy - Approach - Plan - Preparation - Study test - Specifications - Testability - System division - Generation - Generation of cases: - purpose, start, input, expected output - Test infrastructure - Implementation - Execution - phase testing - static - pre-test - full - execution - test - repair - retest - discriminate in error - implementation - specification - test case - infrastructure - report - Completion - Report - testware - evaluation [T]echniques, [I]nfrastructure, [O]rganization More Black & White Box Testing =============================================================================== Decision table: combination of input parameters Use case: test according to use cases Other whitebox: function, call, jump, dataflow, objectcode, loop, race Combinatorial Testing =============================================================================== - Pairwise(triples, ...) - MC/DC. Modified condition/decision Test only important, shown to independantly affect outcome Thus if we have a && b && c we test: 0 1 1 and 1 1 1 Mutation Testing =============================================================================== Make mutants, deliberate errors and see if the test catch it Add mutant, if not caught extend tests otherwise throw away mutant Quality and Reliability Challenges =============================================================================== - Systems of systems pose a problem - Integration is a problem - Interface testing - Because of continous delivery regression testing