Annual penetration tests vs continuous attack surface monitoring: what the scope agreement actually covers
Julian ThorneApril 29, 2026
Key takeaways
A penetration test is accurate against its agreed scope. The scope is bounded by what the organization knows about its attack surface before the test begins. Assets outside that knowledge are outside that scope — and outside that scope is where attackers operate.
Vulnox digital footprint assessments show organizations expose an average of 1.7 new unintended externally-facing assets per week, meaning a 12-month gap between assessments produces approximately 88 assets that have never been reviewed (Vulnox assessment data, 2024).
The most consequential finding gap is not new vulnerabilities on known assets — it is new assets entirely: subdomain takeover candidates, cloud storage provisioned by business units outside central IT, and development environments promoted to production without security review.
Continuous monitoring and annual penetration testing address different threat models. Continuous monitoring catches surface area drift. Penetration testing finds chained exploitation paths. Organizations that treat them as substitutes for each other are missing the function of both.
The practical argument for continuous monitoring is not frequency — it is scope completeness: monitoring starts from what an attacker can enumerate, not from an internal asset register that may be incomplete.
TL;DR
The problem with annual penetration tests is not that they happen once a year. It is that the scope agreement that defines them is drawn from the organization's internal knowledge of its own attack surface, and that knowledge is consistently incomplete. Continuous attack surface monitoring starts from external enumeration — what an attacker can see — rather than from an internal register. Those two starting points produce different pictures of the same environment, and the gap between them is where breaches happen.
What the scope agreement does not cover
A financial services company completed a penetration test in Q2. The scope was agreed from their asset register: primary application, API gateway, four ancillary subdomains, two cloud environments managed by central IT. The test was thorough within that scope. The report was clean.
Vulnox conducted a digital footprint assessment of the same organization eight months later following a separate engagement. External enumeration returned 23 subdomains not in the penetration test scope. Of those, six pointed to active services. Two were development environments provisioned by the product team during a sprint in Q3 — after the penetration test — with no authentication on their admin interfaces. One was a subdomain from a marketing campaign that had been decommissioned from the campaign side but whose DNS record still pointed to a cloud application instance that remained active, populated with customer contact data from the campaign, accessible without authentication.
The penetration test found nothing wrong because it tested what was in scope and the scope was accurate. The problem was not the test. The problem was that the scope agreement is a document produced from internal knowledge, and internal knowledge of the external attack surface is systematically incomplete. No one in the organization knew those three environments existed in the state they were in. They were not in the asset register. They could not have been in the scope agreement.
Why scope agreements structurally miss what attackers find
A penetration test scope agreement is produced through a process where the client lists what they want tested and the testing firm confirms it is feasible within the engagement timeline and budget. The list comes from the client's internal asset register, from the security team's knowledge of the infrastructure, and from conversations with IT and development teams. That process produces an accurate scope for the assets that exist in organizational awareness.
An attacker does not start from organizational awareness. They start from what is enumerable from the internet: DNS records, certificate transparency logs, WHOIS data, web crawlers, port scanning of IP ranges associated with the organization's ASN. The difference between those two starting points is the gap that matters.
In organizations with active development, the gap widens continuously. A developer provisions a cloud environment for a feature sprint. The environment gets a subdomain. The sprint completes and the environment is nominally decommissioned — but the DNS record persists, the cloud instance remains running, and neither appears in the asset register because they were created outside the central IT provisioning process. From the organization's internal view, the environment does not exist. From an external enumeration, it is discoverable in under an hour.
The 1.7 new unintended assets per week figure from Vulnox assessment data represents this specific class of asset — not new assets that were properly provisioned and registered, but assets that appeared on the external attack surface without appearing in any internal inventory. Across a 12-month period between penetration tests, that accumulates to approximately 88 assets that have never been reviewed by any security process.
Example
A SaaS company conducted an annual penetration test covering their production application and API layer. External enumeration run as part of a digital footprint assessment seven months later found a Grafana instance exposed on a subdomain that had been created by a DevOps engineer to provide external dashboard access to a client. The client relationship had ended. The engineer had left the company. The Grafana instance was running version 8.2.x, which had a known path traversal vulnerability allowing unauthenticated file read. The instance had credentials for the production database connection string stored in its configuration. None of this appeared in the penetration test because the subdomain did not exist when the scope was agreed.
Certificate transparency logs are the most reliable source for discovering subdomains that have appeared since a previous assessment. Every certificate issued for a subdomain is publicly logged at the time of issuance. Querying crt.sh for an organization's domain returns every subdomain that has had a certificate issued, including historical ones, with timestamps. The gap between the most recent penetration test date and current issuances is the discovery period — the window during which new subdomains appeared that no security review has covered.
What external enumeration finds between assessments
Assessment base: Findings from Vulnox digital footprint assessments across 47 client environments, 18-month period.
Development and staging environments created during the inter-assessment window
The most consistent finding in digital footprint assessments conducted after a recent penetration test is externally accessible development and staging environments created after the test scope was agreed. These environments are not the result of security failures in the deployment process — they are environments that were provisioned correctly for their intended purpose and never reviewed for external exposure. The development team created them. The security team did not know they existed. The penetration test did not cover them because they did not exist when the scope was agreed.
The client assumption is that penetration testing plus secure development practices equals covered attack surface. It does not, because secure development practices govern the security of code, not the discoverability of infrastructure. A development environment with secure code and no authentication requirement is accessible to anyone who enumerates the subdomain.
Subdomain takeover candidates accumulating after asset decommissioning
DNS records for decommissioned assets persist after the underlying services are removed. A subdomain pointing to a cloud provider endpoint that no longer exists under the organization's account is a subdomain takeover candidate: an attacker can provision a new service at that endpoint and claim the subdomain. The condition is created by correct decommissioning of the service without corresponding cleanup of the DNS record. In organizations with active development and marketing operations, subdomain takeover candidates accumulate at a rate that makes point-in-time DNS audits insufficient.
Subdomain takeover allows an attacker to serve content from a trusted domain — the organization's own subdomain — without any access to the organization's infrastructure. Phishing from a trusted domain, credential harvesting pages that browsers do not flag, malicious downloads served from a URL the target organization has previously used in legitimate communications. The attack does not require exploiting any vulnerability in the organization's systems.
Cloud storage provisioned by business units outside central IT inventory
Business unit cloud storage — S3 buckets, Azure Blob containers, Google Cloud Storage buckets — provisioned directly by teams for operational purposes appears on the external attack surface without appearing in any central IT or security inventory. These are not shadow IT in the sense of unauthorized use of cloud services — they are authorized cloud accounts used in ways that do not route through asset registration. The security configuration applied to them is whatever the provisioning engineer set at creation, which is frequently a default that permits broader access than intended.
Cloud storage enumeration against an organization's known cloud account ranges is a standard recon technique. Buckets with predictable names, buckets associated with the organization's domain, and buckets identifiable through certificate transparency logs or web application source code are all findable without any access to the organization's systems. The assets are on the external attack surface whether or not they are in the internal inventory.
The finding that makes annual pentests look better than they are
Common belief
Organizations that conduct annual penetration tests and remediate findings are incrementally more secure year over year — each test produces a smaller finding set as previous vulnerabilities are addressed.
What we found
An e-commerce organization had conducted annual penetration tests for four consecutive years and could demonstrate year-over-year improvement in finding severity and remediation rates. When Vulnox conducted an external digital footprint assessment, enumeration identified 34 externally accessible hosts outside the scope of any of the four previous penetration tests. Nine of those hosts had critical or high severity findings. The security program was genuinely improving against its tested scope. The tested scope covered approximately 60% of the actual external attack surface.
This is true for the agreed scope. The agreed scope gets better tested each year, and the population of known vulnerabilities within that scope decreases. What this metric does not measure is the attack surface outside the scope, which is growing.
The practical result is an organization that can demonstrate improving security posture against its known attack surface while accumulating unreviewed exposure in the unknown surface. The penetration test finding count drops. The external asset count that has never been reviewed increases. These two trends happen simultaneously and the first one makes the second one invisible — a declining finding count looks like security improvement, but it may be measuring an increasingly well-tested fraction of a growing total.
The counterintuitive outcome: a high remediation rate on annual penetration tests is not evidence that the security program is reducing total risk. It is evidence that the security program is reducing risk within the scope it tests, which is a smaller fraction of the total attack surface than most security teams realize.
What stays outside penetration test scope by structural necessity
Assets created after scope agreement
Penetration test scope is agreed before the engagement begins. Any asset created after that agreement — including during the engagement period — is outside scope by definition. In organizations with active deployment pipelines, new externally accessible assets can appear on the same day the scope agreement is signed. The test cannot cover what did not exist when the rules of engagement were set. This is not a failure of the testing methodology; it is a structural characteristic of point-in-time assessment.
Acquired entity infrastructure
Acquisitions create external attack surface immediately and the acquiring organization's security program scope often lags the transaction by months. The acquired entity's subdomains, cloud accounts, and externally accessible services become part of the acquirer's attack surface on the day the deal closes. They are not in the acquirer's asset register. They are not in the scope of any existing penetration test engagement. They are enumerable by any attacker who looks up the acquisition and searches for infrastructure associated with the acquired entity's domain.
Third-party and partner integrations
Partner portals, integration endpoints, and shared infrastructure provisioned for specific business relationships appear on the external attack surface without being associated with the security team's view of the primary application. These assets are often managed by business teams with limited security oversight, built on timelines that did not include security review, and sometimes forgotten when the business relationship they supported ends — leaving active infrastructure with no owner and no maintenance.
What each approach actually measures
Annual penetration test
Tests agreed scope thoroughly. Finds chained exploitation paths, business logic vulnerabilities, and vulnerabilities requiring authenticated access or deep application knowledge. Scope is bounded by organizational awareness of assets. Point-in-time: accurate at assessment date, degrading throughout the year as the environment changes. Attacker perspective is constrained by scope agreement.
Essential for finding complex vulnerabilities in known assets. Not a substitute for knowing what assets exist. The value degrades faster in organizations with high deployment velocity, active M&A activity, or large business unit cloud usage outside central IT.
Continuous external attack surface monitoring
Starts from external enumeration — what an attacker can discover — rather than from an internal asset register. Finds new assets as they appear. Does not find application-layer vulnerabilities requiring authenticated access or business logic understanding. Provides current-state visibility rather than depth assessment.
Catches surface area drift that penetration tests cannot cover by design. Does not replace penetration testing for complex vulnerability discovery. Most valuable in organizations where the gap between internal asset knowledge and actual external footprint is large — which is most organizations with active development, cloud adoption, or recent acquisitions.
Both in combination
External monitoring informs penetration test scope: assets discovered through continuous monitoring feed into the scope agreement for the annual test, closing the gap between organizational awareness and actual attack surface. Penetration test findings inform monitoring priorities: vulnerability classes found in the test suggest what to watch for as new assets appear.
The combination produces a security program that is complete in the way that neither approach alone achieves. The practical barrier is that most organizations treat them as alternatives rather than complements, typically because budget is framed as 'monitoring or testing' rather than 'monitoring and testing with appropriate allocation between them.'
Where the pentest-vs-monitoring debate is heading
Within 24 months, regulatory frameworks covering financial services and healthcare will begin specifying that penetration test scope must be derived from external enumeration rather than internal asset registers, effectively mandating a continuous monitoring component as a precondition for compliant testing.
Regulators in both sectors have observed the pattern of organizations passing penetration tests and experiencing breaches on assets outside test scope. The technical mechanism is well understood and the fix — scope from external enumeration — is straightforward to specify. PCI DSS 4.0 already moved in this direction by requiring that penetration test scope include all system components. The next step in regulatory evolution is specifying how scope is determined, not just what it covers. Financial services regulators in the EU and UK have indicated interest in attack surface management as a component of third-party risk assessment requirements.
Confidence: mediumIf no major financial services or healthcare regulatory framework has specified enumeration-derived scope requirements by Q2 2027, the prediction is early rather than wrong — the structural driver remains.AI-driven infrastructure provisioning will make continuous attack surface monitoring non-optional within 18 months for organizations using AI-assisted cloud management, because human-driven asset registration cannot keep pace with the rate at which AI tooling creates externally accessible infrastructure.
AI infrastructure management tools provision resources in response to load, configuration changes, and operational triggers without human approval for each action. Each provisioned resource is a potential external asset. The 1.7 new unintended assets per week figure from current assessment data reflects human-driven provisioning. AI-driven provisioning at scale will accelerate that rate significantly. An organization cannot produce an accurate penetration test scope agreement for infrastructure that is being created and destroyed faster than any registration process can track.
Confidence: highIf AI infrastructure management tools implement mandatory asset registration hooks that update security inventories in real time as infrastructure is provisioned, the prediction does not hold. No major platform currently implements this by default.
The real argument is about scope, not frequency
The framing of 'annual penetration test versus continuous monitoring' misses the actual problem. The argument is not about how often assessment happens. It is about where assessment starts.
A penetration test that starts from external enumeration — that runs subdomain discovery, certificate transparency queries, and port scanning of the organization's IP ranges before agreeing a scope, then tests everything that enumeration finds — is a more complete assessment than a continuous monitoring program that only watches known assets. Frequency is not the variable that determines coverage. Enumeration methodology is.
The practical implication is that organizations arguing about annual versus continuous are usually arguing about cost and operational overhead when they should be arguing about scope methodology. An annual test with enumeration-derived scope covers the actual attack surface once a year. An annual test with register-derived scope may cover 60% of the actual attack surface twelve times more thoroughly than needed. Neither is the right answer. The right answer is scope completeness first, then decide on frequency based on how fast the surface area changes.
Counterargument
The counterargument is that external enumeration as the basis for penetration test scope is operationally impractical — it requires the testing firm to conduct reconnaissance before the engagement can be scoped, which adds time and cost, and it produces a scope that clients cannot validate against their internal understanding of their infrastructure. This is a real operational constraint. It is also exactly the constraint that makes the gap between known and actual attack surface so consistent: the operationally convenient approach to scoping is the approach that produces incomplete coverage.
One concrete action this week
Before your next penetration test scope agreement is finalized, run subdomain enumeration against your primary domain and any subsidiary or acquired company domains — using crt.sh for certificate transparency logs and a tool like Subfinder or Amass for active enumeration. Compare the output against your current asset register. Any host in the enumeration output that is absent from your register is a candidate for inclusion in the penetration test scope. If the enumeration returns significantly more hosts than your register contains, that gap is worth understanding before you agree a scope that excludes it.
Further Reading
Digital Footprint
digital footprint analysisVulnerability Assessment
vulnerability assessment workflowsExternal Attack Surface Management
external attack surface managementNIST Vulnerability Assessment Definition
NIST's vulnerability assessment definitionNIST Cybersecurity Framework 2.0
NIST Cybersecurity Framework
Frequently Asked Questions
What does a penetration test miss that continuous attack surface monitoring finds?
A penetration test is bounded by its agreed scope, which is drawn from the organization's internal asset register. Continuous attack surface monitoring starts from external enumeration — what an attacker can discover. The gap between those two starting points contains: assets created after the scope agreement was signed, subdomains provisioned by development and marketing teams outside central IT registration, cloud storage created by business units, acquired entity infrastructure not yet integrated into the parent organization's inventory, and DNS records pointing to active services that nobody currently in the organization knows about. These assets are outside penetration test scope by structural necessity, not by oversight.
How quickly does an attack surface change between annual penetration tests?
Vulnox digital footprint assessment data across 47 environments shows an average of 1.7 new unintended externally-facing assets per week. Over a 12-month inter-assessment period, that accumulates to approximately 88 assets that have appeared on the external attack surface and received no security review. The rate varies significantly by organization type: companies with active deployment pipelines, recent acquisitions, or significant business unit cloud usage outside central IT see higher rates. The figure specifically counts unintended assets — infrastructure that appeared on the external attack surface without being registered in any internal inventory.
Can continuous monitoring replace annual penetration testing?
No. They address different threat models and neither substitutes for the other. Continuous monitoring catches surface area drift — new assets appearing on the external attack surface, DNS records pointing to decommissioned services, cloud storage with misconfigured permissions. Penetration testing finds chained exploitation paths, business logic vulnerabilities, and vulnerabilities that require authenticated access or deep application knowledge to discover. The combination produces complete coverage: monitoring ensures scope completeness, testing provides depth. Organizations that replace one with the other are missing the function that the replaced approach provides.
Why does penetration test scope miss newly created assets?
Penetration test scope is agreed before the engagement begins, based on what the organization knows about its infrastructure at that point in time. Any asset created after scope agreement — including assets created during the engagement period — is outside scope by definition. In organizations with active deployment pipelines, new externally accessible assets can appear daily. There is no mechanism in standard penetration test engagements to expand scope dynamically as new assets appear. The test methodology is not at fault; the constraint is structural to how point-in-time assessments work.
What is subdomain takeover and how does it relate to attack surface monitoring?
Subdomain takeover occurs when a DNS record for a subdomain points to a cloud service endpoint that no longer exists under the organization's account. An attacker can provision a new service at that endpoint and effectively claim the subdomain, serving content from the organization's trusted domain without any access to the organization's infrastructure. The condition is created by decommissioning a service without removing its DNS record — a common outcome of normal asset lifecycle without a corresponding DNS cleanup process. Attack surface monitoring detects these conditions by continuously checking whether DNS records resolve to endpoints the organization actually controls.
How should penetration test scope be determined to cover the actual attack surface?
Scope derived from external enumeration covers the actual attack surface more completely than scope derived from internal asset registers. The process: run subdomain enumeration against the organization's primary domain and any subsidiary or acquired domains using certificate transparency logs (crt.sh) and active enumeration tools, then compare the output against the internal register. Assets in the enumeration output that are absent from the register are candidates for scope inclusion. Any host reachable from the public internet is accessible to an attacker regardless of whether it appears in the internal register, and scope that excludes it leaves that exposure outside the assessment.
Related Articles
Basic digital footprint check: what free tools find and what they miss
A basic digital footprint check using free tools takes less than an hour and reliably surfaces subdomains, certificate history, and indexed content. What it misses — assets with no DNS record, credential material in archived repositories, and vendor-created infrastructure — is where the regulatory exposure lives.
SaaS digital footprint: what customer-facing infrastructure exposes before anyone attacks
In Vulnox assessments, the average SaaS company has 11 externally reachable assets its own team cannot name. That gap between what you think you expose and what is actually visible is where most breaches begin.
Digital footprint exposure and data protection risk: what regulators find that internal teams miss
Forgotten subdomains, undocumented SaaS integrations, and exposed API keys are data protection violations waiting to be discovered -- by a regulator, not your internal team. Vulnox assessments found at least one forgotten subdomain in 71% of environments reviewed. Here is what that exposure looks like through a regulatory lens.
Ready to Secure Your Digital Assets?
Get a comprehensive vulnerability assessment for your website today.