Black-box pentesting: what it finds and what it misses

In a black-box test, the pentester gets a domain name, sometimes an IP range or the URL of an application. Nothing else. No account, no architecture diagram, no source code. They see your systems exactly the way anyone on the Internet sees them.

This is the approach most people picture when they hear “ethical hacking”. It has real value, and limits worth knowing before you sign a quote.

If you are new to the topic, Penetration testing (pentest): approaches and methodology puts the three approaches side by side.

What the pentester gets, and what they don’t

“No information” doesn’t mean “no rules”. A black-box test starts like any other, with signed rules of engagement: dates, the IP addresses tests will come from, forbidden actions, who to call if something goes wrong. Scope is the only tricky part. The pentester will find subdomains and servers you may not know about, and some of them don’t belong to you (a service hosted by a supplier, an old website that was sold off). Before any active testing, the list of what was found goes back to you for approval. Only what you own gets tested.

How a black-box test runs

The first half of the work is reconnaissance. The pentester builds the inventory an attacker would build:

• subdomains, many of which show up in the public TLS certificate logs (every certificate issued is recorded there, including the one for the staging server);
• IP addresses and the services answering on them: websites, VPN, email, remote desktop, admin interfaces;
• software versions, checked against known flaws;
• what has already leaked: employee credentials in stolen databases, documents indexed by search engines, code repositories made public by mistake.

OWASP gives this information gathering a whole section of its testing guide. It is often where the outcome of the audit is decided.

Then everything found gets tested: default settings, factory passwords, interfaces that shouldn’t be public, known flaws left unpatched. On a web application, the pentester attacks everything reachable without an account: login form, password reset, public pages, the APIs the site calls. If sign-up is open, they create an account, as an attacker would.

What it finds well

Black box is unbeatable on one question: what are you exposing without knowing it?

A typical case, which shows up in one form or another in many external audits: a staging subdomain set up for a project three years ago, never switched off, holding a copy of the customer database and an admin password still at its install value. Nobody internally watches it, it isn’t in any inventory. An attacker finds it in minutes.

It is also very good at flaws that can be exploited without logging in: an unpatched VPN, an exposed file-sharing service, a vulnerable CMS version. Those are exactly the doors used by attackers who scan the Internet at scale, and many ransomware attacks start through one of them.

What it misses

The first blind spot is time. An attacker has weeks, even months. A pentest lasts a few days. Every hour spent guessing what a ten-minute call would have explained is an hour not spent digging deeper.

The second one hurts more: everything behind authentication. On an application with no public sign-up, a black-box test only sees the login page. The pentester can try to reach URLs that should be protected, but won’t seriously test what matters most in a business application: can one customer read another customer’s data? Can a plain user call an admin-only function? These access control flaws top the OWASP Top 10, and black box misses them almost by design.

You often hear that black box is “more realistic”. True, for an opportunistic attacker who doesn’t know you. A motivated attacker usually ends up with an account: phishing, a reused password, a leak at a supplier. From that point on they are in grey box, and you don’t know what they can do.

Who it is for, and when

Black box works well for:

• a company that has never had an audit and wants a first picture of its Internet exposure on a controlled budget;
• a stocktake after a merger, a change of provider or a migration, when nobody really knows what is online any more;
• checking your external surface, alongside a deeper audit of your applications.

It is a poor choice for a SaaS product with customer accounts, an internal application for staff only, or any platform where several customers share the same servers. There you need grey box, with test accounts.

As for duration, a black-box external audit usually takes one to four days of testing, depending on the number of domains and exposed services. Our external pentest follows this approach, and the External Discovery audit is the short, one-day version.

More on the other approaches: grey-box pentesting and white-box pentesting. To price your own case, the quote form asks precisely which approach you have in mind.