| Black box systems | |
|---|---|
 | |
| System | |
| Black box, Oracle machine | |
| Methods and techniques | |
| Black-box testing, Blackboxing | |
| Related techniques | |
| Feed forward, Obfuscation, Pattern recognition, White box, White-box testing, Gray-box testing, System identification | |
| Fundamentals | |
| A priori information, Control systems, Open systems, Operations research, Thermodynamic systems | |
|
|
Black-box testing, sometimes referred to as specification-based testing,[1] is a method of software testing that examines the functionality of an application without peering into its internal structures or workings. This method of test can be applied virtually to every level of software testing: unit, integration, system and acceptance. Black-box testing is also used as a method in penetration testing, where an ethical hacker simulates an external hacking or cyber warfare attack with no knowledge of the system being attacked.
Specification-based testing aims to test the functionality of software according to the applicable requirements.[2] This level of testing usually requires thorough test cases to be provided to the tester, who then can simply verify that for a given input, the output value (or behavior), either "is" or "is not" the same as the expected value specified in the test case.
Specific knowledge of the application's code, internal structure and programming knowledge in general is not required.[3] The tester is aware of what the software is supposed to do but is not aware of how it does it. For instance, the tester is aware that a particular input returns a certain, invariable output but is not aware of how the software produces the output in the first place.[4]
Test cases are built around specifications and requirements, i.e., what the application is supposed to do. Test cases are generally derived from external descriptions of the software, including specifications, requirements and design parameters. Although the tests used are primarily functional in nature, non-functional tests may also be used. The test designer selects both valid and invalid inputs and determines the correct output, often with the help of a test oracle or a previous result that is known to be good, without any knowledge of the test object's internal structure.
Typical black-box test design techniques include decision table testing, all-pairs testing, equivalence partitioning, boundary value analysis, cause–effect graph, error guessing, state transition testing, use case testing, user story testing, domain analysis, and syntax testing.[5][6]
Test coverage refers to the percentage of software requirements that are tested by black-box testing for a system or application.[7] This is in contrast with code coverage, which examines the inner workings of a program and measures the degree to which the source code of a program is executed when a test suite is run.[8] Measuring test coverage makes it possible to quickly detect and eliminate defects, to create a more comprehensive test suite. and to remove tests that are not relevant for the given requirements.[8][9]
Black-box testing may be necessary to assure correct functionality, but it is insufficient to guard against complex or high-risk situations.[10] An advantage of the black box technique is that no programming knowledge is required. Whatever biases the programmers may have had, the tester likely has a different set and may emphasize different areas of functionality. On the other hand, black-box testing has been said to be "like a walk in a dark labyrinth without a flashlight."[11] Because they do not examine the source code, there are situations when a tester writes many test cases to check something that could have been tested by only one test case or leaves some parts of the program untested.
