What a test can and cannot tell you

"We had a pen test" is one of the least informative sentences in security, because the phrase covers work that answers completely different questions.

Three kinds of work

Vulnerability scanning enumerates known weaknesses across many systems, continuously and cheaply. It answers what known problems do we have, and where? It finds breadth, produces false positives, and never chains anything together.

Penetration testing is an authorised, time-boxed attempt to reach an objective using the means an attacker might. It answers can someone get from here to there, today? It finds depth, including the logic flaws and chains no scanner will ever see.

Red teaming tests the organisation rather than the system: can an objective be achieved without the defenders noticing and responding in time? It answers do our detection and response actually work? It is the most expensive and the least useful if you already know the answer is no.

What a clean report means

Very little on its own. It means nobody found a way in within the time, scope and skill applied. Before accepting it, ask: what was in scope, what was explicitly excluded, how many days were spent, and what did the tester run out of time to examine? Those four answers tell you what the report is worth.

Choosing

• You do not know what you have exposed → discovery and scanning first. A test of an incomplete inventory tests the wrong things.
• You know your estate and want to know whether a specific route exists → penetration test with a stated objective.
• You have detection and response, and want to know whether they work → purple or red team.
• You already have unfixed findings from last time → fix those. Another test will find them again and charge you for the privilege.

The most common mistake is buying depth before breadth: a beautiful test of one application while nobody owns the forgotten server that will actually be the way in.