TL;DR
Black-box testing is a software quality method that evaluates what a product does — not how it's built — making it ideal for catching omissions, errors, and security gaps before launch. Teams apply it across multiple stages of the software testing life cycle (STLC), using techniques like equivalence partitioning, boundary value analysis, and use-case testing. Regular black-box testing helps protect customer trust and product reliability without requiring testers to understand the underlying code.
Black-box testing at a glance
Black-box testing lets you analyze how something works without digging into the way it's built.
You might use the method to ensure that your website works perfectly before you launch it. Or you might schedule a black-box test at multiple points during the app development process.
Black-box testing has limits. Once you have the results, you'll need to unpack the code and find the right fixes. But if you perform regular, consistent black-box testing, you can make sure that your tools work both properly and consistently for your customers.
What is black-box testing?
Your team knows what your software, website, or app is supposed to do. As your team members work, they look for hiccups, glitches, or unexpected actions that could indicate a coding error.
You could use black-box testing to find:
- Omissions. Did you promise functionality that's missing?
- Errors. Does the interface work unexpectedly? Is the data source sound?
- Limits. Do certain inputs make the system crash or perform poorly?
A black-box testing team has no idea how you built the testing subject, so they can't give you suggestions to fix any problems they find. Their only job is to highlight items that need more investigation.
If you need input on coding problems, which could make repairs quicker, white-box testing or pen testing might be a better choice. People who work on these teams know all about the code beneath a product, and their tests ensure your product works as you designed it.
When is black-box testing used?
On one day in May 2020, thousands of people tried to launch apps on their phones and experienced crashes instead. Facebook's coders were to blame. Black-box testing could have prevented this catastrophic problem.
Before you launch anything to the public, you should test it carefully and repeatedly. The work you do now could ensure that customers have less to complain about in the future.
If you're building software, you'll run black-box tests multiple times. Testing is part of the software testing life cycle (STLC), and you typically perform it during the following stages:
What are the stages of the software testing life cycle?
- Requirement: Software testers determine what elements of the product they can and can't test.
- Planning: Testers determine what sorts of tests will work best and how they will measure success.
- Analysis: Testers specify ideal test conditions.
- Design: They identify formal rules for each test.
- Implement: Testers create test cases.
- Execution: Tests begin, and testers log defects.
- Conclusion: Formal reports end the cycle.
Your product might go through this same process multiple times during development. Each test tells you a little more about what's working and what's broken.
Black-box testing might also make you less vulnerable to hackers. Some of your tests should involve trying to break past security limitations, and any successful breach indicates that you have more to do to keep your data safe. You might run this form of repeated testing throughout the lifecycle of your product, just to ensure that everything remains secure.
How do you perform black-box testing?
Modern projects involve dozens of people. Simply launching a new website, for example, could mean developing a team of 20 or more. Perform black-box testing, and you'll need even more participants.
Black-box testers need access to:
- Specifications. What should the product do? How should it function?
- Design parameters. What should it look like? How should it feel?
- Inputs. What should the tester do to ensure the product works?
- Outputs. What is an error? What is accurate?
With this information, the tester will run test cases to ensure that everything works properly, and they will log all errors.
What test design techniques are used in black-box testing?
Plenty of black-box test design techniques exist, including:
| Technique | Description |
|---|---|
| All-pairs testing | The tester has a pair of input parameters, and they test all combinations. |
| Boundary value analysis | Testers have a range of inputs, and they use them exclusively as they work. |
| Decision-table testing | Testers use different input combinations to test system behavior. They capture all data in a table. |
| Domain analysis | Testers look over related software systems to find variable and common parts. |
| Equivalence partitioning | The tester divides up input data and tests it in groups. They try to find underlying classes of errors. |
| Error guessing | Experienced testers rely on their experience to find bugs in the software they're analyzing now. |
| Use-case testing | Your testing team tests specific actions or events to ensure they help users reach a specific goal. |
| User-story testing | You give testers personas, and they work to see if that person could get satisfaction from the item they're testing. |
You could give your testers a combination of some of the techniques above. For example, you could blend use-case testing with decision tables.
Why is testing frequently so important?
Experts say inadequate testing causes crashes. Any time your website, app, or program doesn't work as expected, you risk losing the trust and goodwill of your customers. Don't let it happen.
If security issues keep you from doing the testing your customers expect, let us help. At Okta, we're experts at helping companies just like yours to keep valuable data safe and secure. Find out more.
Frequently asked questions
What is the difference between black-box testing and white-box testing?
Black-box testers have no knowledge of the underlying code and can only flag issues they find. White-box testers and pen testers, by contrast, understand the code beneath a product and can suggest fixes, making repairs quicker.
What types of errors can black-box testing uncover?
Black-box testing can uncover omissions (promised functionality that is missing), errors (unexpected interface behavior or unsound data sources), and limits (inputs that cause the system to crash or perform poorly).
At what stages of development should black-box testing be performed?
Black-box testing is performed across the seven stages of the software testing life cycle (STLC): Requirement, Planning, Analysis, Design, Implement, Execution, and Conclusion. A product may go through this cycle multiple times during development.
What information do black-box testers need before they begin?
Before testing begins, black-box testers need access to specifications (what the product should do), design parameters (what it should look like and feel like), inputs (what the tester should do to verify the product works), and outputs (what constitutes an error versus an accurate result).
How can black-box testing improve software security?
Some black-box tests should involve attempting to break past security limitations. Any successful breach indicates that more work is needed to keep data safe. Running this form of repeated testing throughout the lifecycle of a product helps ensure that everything remains secure.
What are the most common black-box test design techniques?
The most common black-box test design techniques include all-pairs testing, boundary value analysis, decision-table testing, domain analysis, equivalence partitioning, error guessing, use-case testing, and user-story testing. Testers may combine multiple techniques — for example, blending use-case testing with decision tables.
References
Several Major iOS Apps Crashing at Launch Due to Facebook SDK Issues. (May 2020). Tech Crunch.
Why Hackers Know More About Our Systems. (2011). System Assurance.
How to Organize and Prepare Your Team for a Web Redesign Project. Fastspot.