Third-party risk management gap analysis: what vendor questionnaires don't find
Key takeaways
Security questionnaires measure how well vendors document their security posture. External attack surface assessment measures what attackers can actually reach. In Vulnox assessments, the gap between these two measures is largest for vendors with the most mature questionnaire responses — organizations that have learned to document controls well are not necessarily organizations that enforce them.
Fourth-party risk is where most TPRM programs have no visibility. Sub-processors handling your data under contract with your primary vendor are outside your questionnaire scope, often outside your contract scope, and frequently the actual entry point in supply chain compromises.
The most common vendor contract provision failure is breach notification language that specifies a timeline but maps to no internal process for receiving and acting on that notification. The vendor notifies. The notification sits in procurement email. Three days later the security team learns about it.
Vendor tiering based on relationship length or contract value rather than data access and operational criticality is the root cause of most TPRM program gaps. A long-tenured small vendor with access to customer PII is Tier 1. Treating it as Tier 3 because it has always 'been fine' is the gap.
Continuous external monitoring of vendor attack surfaces catches obvious exposure but cannot see internal control failures. It is a necessary leading indicator, not a substitute for periodic technical assessment of Tier 1 vendors.
DORA's ICT third-party risk requirements, effective January 2025 for EU financial entities, mandate specific program elements that most existing TPRM programs do not satisfy: concentration risk assessment, documented exit strategies, and audit rights provisions that have never been exercised.
TL;DR
Third-party risk programs produce completed questionnaires. Gap analysis finds what questionnaires miss. The consistent findings: vendor external exposure that contradicts self-attestation, fourth-party relationships that extend your data exposure beyond your processor inventory, contract provisions that shift liability without changing vendor behavior, and internal incident response processes that have never been tested against an actual vendor breach notification. This article covers the structural gaps in TPRM programs and what a gap analysis actually needs to examine to find them.
The questionnaire came back clean. The external scan did not.
A financial services firm had a documented Tier 1 vendor with an annual questionnaire process, an ISO 27001 certificate on file, and no open findings from the previous year's review. A Vulnox external attack surface assessment of that vendor's infrastructure found three exposed management interfaces, an RDP service running on a public IP with no MFA, and an SSL certificate for a domain in the vendor's name that had been flagged in a credential exposure database six weeks earlier. The questionnaire had been completed four months prior. The vendor had self-attested to MFA on all remote access. None of those findings were visible in the questionnaire responses. All of them were visible from a passive external scan.
The client's TPRM program was designed around questionnaire completion as the primary evidence artifact. It had no mechanism to validate questionnaire responses against technical reality. The vendor had answered the questionnaire accurately based on their internal documentation. Their internal documentation did not reflect the actual state of their external infrastructure. That gap is structural in any TPRM program that treats self-attestation as security assurance.
Why questionnaire-centric TPRM programs have systematic blind spots
A security questionnaire is a request for documented self-assessment. It returns whatever the vendor's compliance team believes is accurate, filtered through whatever the vendor is willing to disclose. It does not return the state of their infrastructure.
The blind spots fall into three categories. First, configuration drift: a vendor's documented controls accurately described their environment when the documentation was written. The environment changed. The documentation did not. The questionnaire captures the documentation, not the environment.
Second, organizational scope: questionnaires typically cover the vendor's primary operations environment. They do not cover every system that processes your data, every network segment where your data transits, or every sub-processor the vendor uses downstream. Vendors answer questionnaires about their main environment. Your data may be processed in a subsidiary, a disaster recovery site, or a subcontracted platform that the questionnaire never asked about.
Third, self-serving framing: vendors have commercial incentives to maintain the relationship. Questionnaire responses are filtered through that incentive. Findings that would trigger re-evaluation are systematically underreported. This is not necessarily deceptive — it is the natural result of asking vendors to assess themselves.
Example
The ISO 27001 certification problem is a specific version of the documentation drift issue. An ISO 27001 certificate is valid for three years with annual surveillance audits. Organizations receive the certificate, file it, and treat it as current proof of control maturity until expiry. In Vulnox assessments, we have found certificates cited as current controls that had lapsed because the vendor missed a surveillance audit, had been suspended because a major nonconformity was identified and not remediated, or had been issued to the parent company and were being cited for a subsidiary that was never in scope. Certificate-based assurance is only as current as the certificate. Verification requires checking the issuing body's registry, not the vendor's document library.
External attack surface assessment of vendors requires no vendor cooperation and produces findings that questionnaires systematically miss. Passive enumeration of vendor infrastructure reveals exposed services, open ports, technology stack information from HTTP headers and certificate data, subdomain inventory, and presence in credential exposure and breach notification databases. This data is available to any party who looks for it, including attackers. The finding set from external assessment consistently contradicts self-attestation on remote access controls, patch currency, and certificate management — the three areas where questionnaire responses are most optimistic and external reality is most different.
What third-party risk gap analysis finds
Assessment base: Vulnox assessment data, 2024-2025, across TPRM gap assessments in financial services, SaaS, and healthcare verticals.
Vendor external exposure contradicts questionnaire self-attestation on access controls
In assessments where we run external attack surface scans against Tier 1 and Tier 2 vendors and compare results to their questionnaire responses, the most consistent finding is exposed remote access services that vendors attested as MFA-protected or access-restricted. RDP on public IPs, SSH with password authentication, web-based management consoles without MFA, and VPN concentrators running vulnerable firmware versions appear in external scans for vendors who answered MFA questions affirmatively. The exposure is visible from outside. The questionnaire did not ask about it specifically enough to surface it.
The client in these assessments believed their highest-risk vendors had been thoroughly assessed. They had been thoroughly questioned. The technical exposure had not been assessed. That distinction determines whether a supply chain compromise moves through a vendor into your environment.
Fourth-party sub-processor relationships outside the client's data inventory
In assessments of organizations with documented processor inventories and Article 28-compliant contracts with primary vendors, we consistently find personal data flowing to sub-processors that are not in the inventory and are not covered by the primary processor agreement's sub-processing authorization. The mechanism is consistent: the primary vendor uses a cloud platform, analytics service, or support tool that processes data on their behalf. The primary contract authorizes the vendor to process data. It does not restrict or enumerate the vendor's sub-processors. The data flows to a fourth party that the client organization has never assessed, contracted with, or mapped.
From a regulatory standpoint, data flowing to an unauthorized sub-processor is a GDPR Article 28 violation regardless of whether it involves a breach. Organizations whose primary vendor contracts predate sub-processor restriction requirements — most contracts written before 2020 — have this exposure by default. Identifying it requires mapping actual data flows through vendor infrastructure, not reviewing what the primary contract permits.
Breach notification provisions that map to no internal process
Third-party risk assessments that review vendor contract provisions consistently find breach notification language specifying timelines — typically 72 hours, sometimes 24 hours for regulated sectors. Assessments that test whether those notifications would reach the right internal team within the specified window consistently find they would not. The notification arrives at a procurement or legal contact, not a security contact. Procurement does not have a defined escalation path to the security team. The security team has no documented process for receiving a vendor breach notification and triggering an internal response. The contract provision exists. The internal capability to act on it does not.
A vendor breach notification that sits in procurement email for three days before reaching the security team negates the contractual timeline entirely. The provision shifts liability to the vendor for delayed notification. It does not ensure the organization's response is timely. For organizations with their own regulatory notification obligations triggered by vendor breaches involving customer data, that three-day delay can create a compliance violation independent of the vendor's failure.
Where TPRM programs consistently do not look
Vendors with operational criticality but low data access
Vendor tiering typically prioritizes data access as the primary risk driver. Vendors with access to large volumes of personal data are Tier 1. Vendors whose compromise would disrupt operations but who access little personal data are tiered lower. The blind spot is operational dependency risk: a managed service provider whose compromise would take your core platform offline for 72 hours while you rebuild, a CDN provider whose manipulation would allow attackers to serve malicious content to your users, a DNS provider whose hijacking would redirect your authentication traffic. These vendors may not handle personal data at scale, but their compromise has a different risk profile than a data processing vendor. TPRM frameworks built around data protection regulatory requirements systematically undertier operationally critical vendors.
Vendor security posture changes between assessment cycles
Annual questionnaires capture vendor security posture at one point per year. Between assessments, vendors are acquired, change cloud providers, launch new products that process client data, experience staff turnover in security roles, or go through financial stress that reduces security investment. None of these changes are visible in a TPRM program with annual review cycles. Continuous external monitoring catches some changes — certificate expiry, new exposed services, breach database appearances. It does not catch organizational changes that affect internal controls. The gap is widest for vendors in periods of rapid growth or organizational change, which are also the periods when security control quality is most likely to degrade.
Concentration risk across the vendor portfolio
TPRM programs assess vendors individually. They rarely assess vendor portfolio concentration: how many critical vendors share a common cloud provider, a common sub-processor, a common CDN, or a common security tooling vendor. DORA's ICT third-party risk requirements specifically mandate concentration risk assessment for EU financial entities because regulators recognized that individual vendor assessments miss systemic risk. If eight of your Tier 1 vendors use the same cloud provider, a major cloud provider incident creates simultaneous failures across your critical vendor portfolio. That risk does not appear in any individual vendor's risk score.
Running a third-party risk management gap analysis
- Step 1
Re-tier vendors based on access and operational criticality, not relationship history
Output:A revised vendor tier list that may move several vendors from Tier 2 or Tier 3 to Tier 1, with documented rationale for each assignment. This becomes the scope document for your gap analysis — the gap analysis focuses on Tier 1 first.
Purpose:Audit your current vendor tiers against two criteria: what data does this vendor process or access, and what happens to your operations if this vendor is compromised or unavailable for 72 hours. Any vendor meeting either threshold at a significant level is Tier 1 regardless of prior tier assignment.
- Step 2
Run external attack surface assessment against Tier 1 vendors
Output:A findings list per vendor showing discrepancies between self-attestation and external observation. Any finding where the external scan contradicts a questionnaire response is a gap requiring follow-up, not just documentation.
Purpose:Collect the external evidence set that questionnaire responses cannot provide. Enumerate exposed services, validate certificate status against issuing body registries, check credential exposure databases, and compare findings against questionnaire self-attestation on access controls and patch management.
- Step 3
Map actual data flows to identify fourth-party exposure
Output:A fourth-party map showing sub-processors that handle your data, whether each is covered by your primary contract's sub-processing authorization, and whether each has been assessed or contracted directly. Gaps in this map are Article 28 compliance exposures in addition to security risks.
Purpose:Determine whether personal data processed by your primary vendors flows to sub-processors that are not in your inventory and not covered by your primary contracts. This requires reviewing vendor privacy notices, sub-processor lists (required by GDPR Article 28 but inconsistently maintained), and network-level evidence where available.
- Step 4
Test the vendor breach notification path end to end
Output:A documented notification path with actual elapsed time between vendor notification and security team awareness. Compare against the contractual timeline and your own regulatory notification obligations. Any gap between these timelines is an operational failure that contract language does not fix.
Purpose:Send a simulated breach notification through the contact path specified in your vendor contracts and measure how long it takes to reach your security team and trigger an internal IR response. Do not announce the test in advance.
- Step 5
Assess concentration risk across the vendor portfolio
Output:A concentration risk map showing shared dependencies and the blast radius of each. This is the input to a conversation about portfolio-level risk that individual vendor scores do not surface.
Purpose:Map common infrastructure dependencies across Tier 1 vendors — shared cloud providers, shared CDNs, shared sub-processors, shared security tooling. Identify scenarios where a single infrastructure failure creates simultaneous impact across multiple critical vendors.
More vendors in your TPRM program does not mean better third-party risk coverage
Common belief
TPRM program maturity is measured partly by the number of vendors in scope. More vendors assessed means more comprehensive coverage. Programs that assess 500 vendors annually are more mature than programs that assess 50.
What we found
In assessments of TPRM programs, we consistently find an inverse relationship between vendor program breadth and finding quality. Organizations with large vendor assessment programs have sophisticated questionnaire automation and low finding rates — questionnaires are optimized and vendors know how to answer them. Organizations with smaller, deeper programs that include external assessment and contract testing have higher finding rates and higher-severity findings. The metric that matters is not how many vendors you assess. It is whether you can tell a meaningful difference in risk posture between a vendor who passes your assessment and one who would not.
Assessing 500 vendors with questionnaires means assessing 500 vendors with questionnaires. It does not mean assessing the actual security posture of 500 vendors. Programs that prioritize breadth over depth spend their assessment capacity on Tier 3 vendors who have minimal access to your systems while providing superficial coverage for Tier 1 vendors who have extensive access.
The programs that find the most useful findings concentrate depth on a small number of high-impact vendor relationships. Annual questionnaire plus external attack surface assessment plus contract review for 15 Tier 1 vendors produces more actual security assurance than questionnaire-only assessment of 500 vendors. The 500-vendor program demonstrates compliance with the requirement to assess vendors. The 15-vendor deep program demonstrates actual risk reduction where it matters.
Where third-party risk requirements are heading
By 2027, at least one major regulatory enforcement action under DORA will cite a financial entity's failure to maintain a complete ICT sub-contractor inventory and document concentration risk as a standalone compliance failure, independent of any incident.
DORA's ICT third-party risk requirements took effect January 2025. They explicitly require concentration risk assessment and documentation of sub-contracting chains. Most financial entities that have implemented TPRM programs have not built the sub-contractor inventory or concentration risk analysis that DORA requires — they have adapted existing vendor questionnaire programs. Regulators will test DORA compliance through examination, not only through incident response. The first enforcement actions will establish what 'adequate' means for program elements that are new requirements with no prior enforcement history.
Confidence: mediumReview of EBA or national competent authority enforcement actions published in 2026-2027 citing DORA ICT third-party risk requirements for concentration risk or sub-contractor inventory failures.The vendor security questionnaire as the primary TPRM evidence artifact will be formally deprecated by at least two major regulatory frameworks by 2028, replaced by continuous external monitoring requirements and audit right exercise evidence as the primary compliance demonstration.
Regulators in financial services and data protection are increasingly sophisticated about the gap between self-attestation and technical reality. DORA's emphasis on audit rights, GDPR enforcement decisions that cite undisclosed processor relationships, and the consistent pattern in breach investigations where vendor questionnaires showed no red flags before a significant incident — these are the inputs that produce regulatory guidance updates. The trajectory is toward technical evidence requirements that questionnaires cannot satisfy.
Confidence: lowPublication of regulatory guidance from ESMA, EBA, or a major national supervisory authority explicitly describing questionnaire-only programs as insufficient for TPRM compliance by end of 2028.
Most TPRM programs manage compliance evidence, not vendor risk
Third-party risk management, as practiced by the majority of organizations I have assessed, is a documentation exercise. The program exists to demonstrate to auditors and regulators that vendor oversight happens. The questionnaires are sent. The certifications are filed. The tiers are assigned. The documentation is complete. The question of whether any of that activity produces accurate knowledge of vendor security posture is not the question the program is designed to answer. It is designed to answer whether the process was followed. Those are different questions with different answers. The programs that actually reduce supply chain risk are the ones that treat questionnaire completion as the beginning of the assessment, not the end. External validation, contract testing, and sub-vendor mapping are the activities that find things the questionnaire cannot find. They are also the activities that most TPRM programs cut when resources are constrained because they are harder to automate and harder to present as scalable metrics.
Counterargument
The counterargument is realistic: most organizations cannot afford deep technical assessment of every vendor, and a documented questionnaire program that is consistently applied produces useful risk differentiation even if each individual questionnaire is imperfect. An organization that assesses 200 vendors annually with questionnaires and acts on the high-severity responses is in a meaningfully better position than an organization with no program. That is true. The problem is not with questionnaire programs at the starting point. It is with questionnaire programs at the maturity ceiling — organizations that have been running the same questionnaire process for five years and have not added external validation or fourth-party mapping, treating program longevity as a proxy for program depth.
One thing to do this week
Pick your three highest-risk Tier 1 vendors — the ones whose compromise would most directly affect your operations or your customers' data. Run a passive external scan of their infrastructure using a tool that requires no vendor cooperation: Shodan for exposed services, crt.sh for certificate inventory and subdomain enumeration, and Have I Been Pwned's domain search for credential exposure. Compare what you find against their most recent questionnaire responses on remote access controls, patch management, and access restrictions. If the external findings contradict the questionnaire responses, you have a gap in your TPRM program's ability to verify what vendors tell you. That is the gap to close before the next questionnaire cycle, not during it.
Further Reading
Gap Analysis
third-party risk management gap analysisDigital Footprint
digital footprint analysisThird-Party Risk Management
comprehensive third-party risk managementNIST SP 800-30 Risk Assessment Guide
NIST risk assessment guideExternal Attack Surface Management
external attack surface management
Frequently Asked Questions
What does a third-party risk management gap analysis find that security questionnaires miss?
Questionnaires find what vendors report about themselves. A gap analysis finds what is actually true about their external posture. In Vulnox assessments, the most consistent finding in organizations with mature questionnaire programs is a gap between vendor self-attestation and vendor external exposure: suppliers that report strong access controls while running exposed management interfaces, vendors that cite current ISO 27001 certification that lapsed eight months ago, and sub-vendor relationships that extend data exposure to fourth parties that neither the primary vendor nor the client organization has mapped. Questionnaires measure documentation quality. External assessment measures exposure reality.
How should vendors be tiered in a third-party risk management program and what does each tier require?
Tiering should be driven by two factors: access to your systems or data, and operational criticality if the vendor fails or is compromised. A vendor with read access to customer PII and a vendor whose outage would halt your core operations are both Tier 1 regardless of their size or how long you have worked with them. Tier 1 vendors warrant annual technical assessment (not just questionnaire), contract provisions with specific security metrics and breach notification timelines, and continuous monitoring of their external attack surface. Tier 2 vendors get annual questionnaires and contract clauses. Tier 3 gets onboarding screening and periodic review. The most common tiering failure is placing vendors in lower tiers based on relationship length or contract value rather than actual access and criticality.
What contract provisions actually create accountability in vendor security and which ones are compliance theater?
Provisions that create accountability: specific breach notification timelines with escalating consequences for delay (72 hours is the standard; many contracts say 'reasonable time'), the right to audit with a defined process and at your cost for Tier 1 vendors, specific security metrics tied to SLAs with financial consequences, and sub-processor restrictions requiring approval before a vendor can delegate your data to a fourth party. Compliance theater: generic 'vendor shall maintain appropriate security measures' language, annual questionnaire requirements with no consequence for non-response, and broad indemnification clauses that have never been tested against a real incident. The test of a contract provision is whether it changes vendor behavior or only shifts liability after the fact. Most third-party security contract language shifts liability. It does not change behavior.
How do you assess fourth-party risk when you have no direct relationship with sub-vendors?
Three approaches produce findings without direct engagement. First, require Tier 1 and Tier 2 vendors to disclose sub-processors handling your data as a contract requirement, with notification required before any change. This surfaces the fourth-party inventory. Second, run external attack surface assessment against your primary vendors and look for evidence of sub-vendor infrastructure in their digital footprint — shared SSL certificates, DNS relationships, and API integrations visible from outside the network. Third, review your vendors' publicly available breach history and incident disclosures. Sub-vendor compromises often appear in vendor incident notifications years before organizations formalize fourth-party risk programs. The SolarWinds compromise affected organizations whose direct contracts were with SolarWinds but whose actual risk originated from SolarWinds' build pipeline, which was a fourth-party relationship from the end client's perspective.
What is the most common failure in vendor incident response integration and how do you fix it?
The most common failure is that vendor breach notification requirements exist in contracts but the internal process for receiving and acting on a vendor notification does not exist. The vendor notifies your procurement contact. Procurement does not know what to do with a security notification. It sits in email for three days. No one has mapped the notification path from vendor contact to security team to incident response process. Fix this by running a tabletop exercise that starts with a vendor breach notification arriving and traces the internal response: who receives it, who decides on escalation, who invokes the IR plan, what isolation or containment actions apply to the vendor relationship, and what regulatory obligations your own notification timeline triggers if the vendor compromise involved your customer data.
How do you monitor vendor security posture continuously without requiring vendors to install agents or grant access?
External attack surface monitoring requires no vendor cooperation. It assesses what is visible from outside a vendor's network: exposed services, open ports, certificate validity and expiry, subdomain enumeration, known CVEs in exposed technology stack, and presence in breach and credential exposure databases. Tools like SecurityScorecard, BitSight, and RiskRecon generate continuous external risk scores from this data. The limitation is that external signals are leading indicators of internal posture, not direct measures. A vendor with a clean external profile can have severe internal control failures that external monitoring cannot see. External monitoring is necessary but not sufficient. It catches the obvious exposure gaps. It does not replace periodic technical assessment for Tier 1 vendors.
What regulatory frameworks specifically require third-party risk management programs and what do they mandate?
GDPR Article 28 requires written contracts with processors that specify the subject matter, duration, nature and purpose of processing, and the controller's obligations and rights. It requires processors to implement appropriate technical and organizational measures and to not engage sub-processors without controller authorization. DORA (Digital Operational Resilience Act), effective January 2025 for EU financial entities, requires ICT third-party risk management programs with specific requirements for concentration risk, exit strategies, and contractual provisions for audit rights and incident notification. NIST CSF 2.0 added a Govern function that explicitly addresses supply chain cybersecurity risk management as a core organizational capability. The common thread: regulators are moving from recommending third-party risk programs to mandating specific program elements and providing evidence standards for what adequate third-party oversight looks like.
Related Articles
GovRAMP Moderate authorization: why the Significant Change Request process catches providers off guard
GovRAMP Moderate is the first tier where you need a government sponsor, annual 3PAO reassessment, and a Significant Change Request process that can pause normal product releases for months. Most providers who stall post-authorization were not prepared for what maintaining Moderate status actually costs operationally.
GovRAMP Low+ authorization: the impact level that punishes providers who get the CUI boundary wrong
GovRAMP Low+ is where providers handling limited Controlled Unclassified Information land — or discover they should not be there. The defining failure is not a missing control. It is a CUI boundary that was drawn before anyone asked what data the government actually sends through the system.
GovRAMP High authorization: why FIPS-validated crypto and personnel security controls catch providers off guard
GovRAMP High is where cloud providers discover that having strong encryption is not the same as having FIPS 140-2 validated encryption — and that distinction alone has derailed authorizations from vendors who passed every other control family. The architectural constraints at High are qualitatively different from every lower tier.
Ready to Secure Your Digital Assets?
Get a comprehensive vulnerability assessment for your website today.