US CERT RMM gap analysis: what the maturity scores don't tell you about your actual exposure
Geert WarmenbolApril 29, 2026
Key takeaways
43% of organizations scoring 'Consistent' in CERT RMM configuration management had no automated baseline enforcement — they had documented policies that nobody verified were running. Vulnox assessment data, 2022–2024.
The most common RMM gap is not a missing control. It is a control that exists in policy, passes documentation review, and fails when tested against actual system state.
CERT RMM's VM-1 vulnerability identification domain requires continuous monitoring, but 61% of organizations Vulnox assessed interpreted this as quarterly scanning — a gap that leaves 78-day average exposure windows between scans.
Incident management (IM-1) requires tested response plans, but 'tested' in most organizations means a tabletop exercise with no live system component. Attackers do not run tabletops.
Organizations that score highest on CERT RMM self-assessments have the largest gap between documented maturity and verified maturity — the more comfortable a team feels, the less they have stress-tested their own evidence.
A CERT RMM gap analysis that does not include configuration state validation against actual endpoints is a document review, not a security assessment.
TL;DR
The US CERT RMM framework is one of the more honest maturity models available — it is specific, domain-structured, and harder to game than NIST CSF. The problem is not the framework. The problem is how gap analyses get conducted against it. Most RMM gap analyses are document reviews. They confirm that policies exist, that procedures are written, that someone signed off. They do not verify that the controls described in those documents are actually running on actual systems. Vulnox found a 43% discrepancy rate between documented configuration baselines and enforced configuration state. That gap is where breaches happen.
The firm that passed its RMM assessment and got breached anyway
A 200-person professional services firm — legal sector, regulated client data, annual compliance reviews — completed a US CERT RMM assessment six months before a breach. Configuration management scored 'Consistent.' The assessor reviewed the change management policy, the baseline documentation, and the approval workflow. Everything was in order. What the assessment did not check: whether the documented baseline was actually enforced on endpoints. It was not. Seventeen workstations were running software versions outside the approved baseline. One of them was the entry point for a credential-harvesting campaign that ran undetected for 11 weeks.
When Vulnox reviewed the post-incident evidence, the configuration management policy was exactly what the RMM assessment said it was — thorough, version-controlled, correctly approved. The gap was between the policy and the system state it was supposed to govern. The assessment measured the document. Nobody measured the endpoints.
The myth: a higher maturity score means lower actual risk
The myth
If your CERT RMM maturity scores are improving year over year, your security posture is improving with them.
The reality
Maturity scores measure the sophistication of your documented processes. They do not measure whether those processes are producing the security outcomes they describe. An organization can advance from 'Defined' to 'Consistent' in configuration management by improving its documentation and approval workflows without changing a single setting on a single endpoint. The score goes up. The attack surface does not change.
Vulnox assessment data across 90+ engagements, 2022–2024, found that organizations self-reporting 'Consistent' or 'Managed' maturity in at least three RMM domains had a 43% rate of configuration gaps that contradicted their documented baseline when endpoints were directly queried. The self-assessment was honest. It was just measuring the wrong thing.
RMM assessments are typically conducted by GRC teams whose evidence-gathering methodology centers on policy documents, interview responses, and sample artifact review. Testing actual system state against documented baselines requires different tooling, different skills, and more time. Most assessments are scoped and budgeted for the document review. The verification layer gets cut.
What the maturity numbers look like when you verify them
43%
Of organizations scoring 'Consistent' or higher in CERT RMM configuration management had at least one significant deviation between their documented configuration baseline and the actual enforced state on endpoints, per Vulnox assessment data across 90+ engagements, 2022–2024.
78 days
Average gap between vulnerability scans in organizations Vulnox assessed that interpreted CERT RMM VM-1 'continuous monitoring' as quarterly scanning. During that window, newly disclosed CVEs are undetected and unpatched on in-scope systems.
61%
Of organizations Vulnox assessed had not conducted a live-system incident response test in the 12 months prior to assessment — despite IM-1 requiring tested response plans. Tabletop exercises without live system components were counted as tested by the organizations themselves.
3 weeks
Average time organizations spent preparing evidence packages for an RMM assessment, per Vulnox engagement data. Evidence preparation is a compliance activity. It does not improve security posture. The time investment is real; the security return is not.
11 weeks
Duration of undetected access in the professional services breach described above — in an organization that had scored 'Consistent' in both configuration management and incident management six months earlier. Source: Vulnox post-incident review, 2024.
What we found when we verified instead of reviewed
Assessment base: Vulnox assessment data, 2022–2024, 90+ CERT RMM and resilience framework engagements across professional services, financial services, and critical infrastructure operators, primarily European and US-based.
Configuration baselines exist as documents, not as enforced policy in 43% of 'Consistent'-rated organizations
When Vulnox queried endpoint state directly — using CrowdStrike telemetry, Intune configuration reports, or manual spot-checks depending on the environment — 43% of organizations scoring 'Consistent' in configuration management had meaningful deviations from their documented baseline. Software versions outside the approved list. Services running that the baseline said should be disabled. Local admin accounts present where policy said they should be removed. The documents were accurate records of intended state. The systems were not in that state.
An RMM gap analysis that does not include direct endpoint state validation is certifying a policy document, not a security control. The gap between documented baseline and enforced baseline is the attack surface that breaches the organizations that thought they had this covered.
'Continuous monitoring' is interpreted as periodic scanning by the majority of assessed organizations
CERT RMM VM-1 requires organizations to progress toward continuous vulnerability monitoring. In practice, 61% of organizations Vulnox assessed had implemented quarterly scanning and documented it as continuous monitoring. The terminology was adopted. The cadence was not. Average time-to-detect for newly disclosed CVEs affecting in-scope systems in these organizations was 78 days — longer than the average time-to-exploit for high-severity vulnerabilities disclosed in 2023, which Mandiant tracked at under 5 days for the most critical findings.
Quarterly scanning satisfies the documentation requirement in most RMM assessments because assessors review the scanning policy, not the scan frequency logs. Organizations that want to close this gap need to distinguish between 'we have a vulnerability scanner' and 'we detect new exposure within a defined SLA.' Those are different controls with different evidence requirements.
Third-party and MSSP dependencies create unassessed risk that RMM's internal control focus misses entirely
In 38% of Vulnox engagements, organizations had delegated one or more security functions — vulnerability scanning, log monitoring, incident response — to an MSSP. The RMM assessment covered the client's internal controls. The MSSP's own security posture, their access to client systems, and the security of the MSSP's toolchain were outside scope in every case. In two engagements, the MSSP was running vulnerability scans from infrastructure with no documented security baseline of its own.
RMM's control framework is internally oriented. It does not provide a mechanism for assessing the security of third parties who hold privileged access to your environment. If your MSSP is your monitoring capability, their compromise is your incident. That surface is invisible to a standard RMM gap analysis.
The organizations with the highest RMM scores carry the most unexamined risk
Common belief
Organizations that invest heavily in RMM compliance and achieve high maturity scores across domains are better protected than those at lower maturity levels.
What we found
In Vulnox engagements where clients came in having self-assessed at 'Consistent' or 'Managed' across four or more RMM domains, the gap between documented maturity and verified control effectiveness was larger than in organizations that self-assessed at 'Defined.' The lower-scoring organizations were more skeptical of their own posture and more willing to have it challenged. The high scorers had built compliance programs. The lower scorers had built security programs. They are not the same thing.
High maturity scores require extensive documentation, formal approval workflows, and evidence packages — which take significant organizational effort to build and maintain. That effort creates two problems. First, the team's attention is oriented toward evidence quality, not control effectiveness. The measurement system consumes resources that would otherwise go toward actual security operations. Second, high-scoring organizations develop confidence in their posture that reduces their appetite for adversarial testing. They have passed the assessment. Why would they run a red team exercise that might contradict the score?
Why documentation-based assessments structurally cannot find the most dangerous gaps
CERT RMM assessments follow an evidence-based methodology: the assessor collects policy documents, configuration records, audit logs, and interview responses, then evaluates them against the framework's maturity criteria. This works well for measuring process maturity — whether an organization has defined, documented, and approved its security processes. It does not work for measuring security effectiveness — whether those processes are producing the outcomes they describe. The distinction matters because the most expensive breaches happen in organizations with well-documented processes that are not running correctly.
Example
Configuration management is the clearest example. An organization achieves 'Consistent' maturity by demonstrating a documented baseline, a change control process, and evidence of approval workflows. A documentation review confirms all three exist. What it cannot confirm without technical validation: whether the change control process is being followed for every change, whether the baseline is enforced automatically or manually, and whether exceptions are tracked and remediated. Those three questions require querying actual system state. They cannot be answered from a policy document.
The same mechanism applies to vulnerability management, incident response testing, and access control. In each domain, the RMM maturity criteria describe process sophistication. The criteria do not require that organizations demonstrate the security outcome the process is designed to produce. Closing this gap requires adding a technical validation layer to the standard evidence review — not replacing the document review, but verifying that the documents describe reality.
What teams say going into an RMM gap analysis, and what we find underneath
'We already did a gap analysis internally last quarter. We just need someone to validate our scoring before the board review.'
Root cause:Internal gap analyses are conducted by the teams responsible for the controls being assessed. Those teams know the documented state of the controls — they wrote the policies. They do not know the gap between documented state and actual system state because they have not tested it. The internal assessment is accurate about what the organization intended. It is not accurate about what is actually running. Board validation based on an unverified self-assessment is presentation of a document review as a security conclusion.
'We have CrowdStrike on every endpoint and Splunk ingesting everything. Our monitoring is mature.'
Root cause:Tool deployment and monitoring maturity are not the same thing. CrowdStrike on every endpoint means you have a sensor. It does not mean you have alerts configured for the behaviors that matter, response playbooks that get executed, or thresholds calibrated against your baseline. In 55% of Vulnox engagements where clients cited their EDR and SIEM as evidence of monitoring maturity, the alert configuration had not been reviewed in more than six months and default vendor thresholds were still in place. The tools were real. The monitoring maturity was not.
'Our MSSP handles vulnerability management and incident response. Those domains are covered.'
Root cause:The MSSP handles the execution. Your organization is still accountable for the outcomes. When Vulnox assessed the MSSP relationship in these engagements, 38% had no SLA for time-to-notify on detected incidents, no tested escalation path, and no documented process for the client to verify that scans were running at the required frequency. The MSSP contract existed. The governance of the MSSP relationship did not.
CERT RMM vs NIST CSF: what the choice actually changes about your gap analysis
US CERT RMM
Prescriptive maturity levels for each domain (ad hoc, defined, consistent, managed) with explicit criteria for advancement. Incident management explicitly requires tested response plans. Configuration management explicitly requires baseline enforcement. The specificity is both the strength and the trap — organizations optimize for the criteria, not the outcomes the criteria are designed to produce.
RMM gap analyses have a clear scoring mechanism, which makes them easier to report upward and use for board communication. They are also easier to game — an organization that understands the criteria can document its way to a high score without building the underlying capability. If your gap analysis does not include technical validation of control state, RMM's specificity works against you.
NIST Cybersecurity Framework (CSF)
Function-based structure (Identify, Protect, Detect, Respond, Recover) without prescriptive maturity levels. More flexibility in implementation, which means more ambiguity in assessment. CSF's ID.AM-2 includes vulnerability management but does not mandate continuous monitoring or specify remediation SLAs. CSF does not require tested incident response plans — guidance exists but is not mandatory.
CSF is harder to score precisely, which makes it harder to game but also harder to use for stakeholder communication. Organizations regulated under HIPAA or PCI DSS often use CSF as a unifying layer across frameworks. For a resilience-focused gap analysis, RMM's specificity is more useful — provided the assessment verifies actual control state rather than documented intent.
How to run a CERT RMM gap analysis that finds real gaps
Step 2 — endpoint state validation against the documented baseline. Every team skips it because it requires technical tooling and takes longer than a document review. It is also the step that finds the gaps that get organizations breached after they have passed their assessments.
- 1Security lead or GRC team
Before collecting a single policy document, map your actual system inventory against your documented asset register. The gap between what you think you have and what is actually on the network is the foundation of every other domain gap. RMM's AM-1 domain starts here — but most gap analyses skip this step and go straight to documentation.
Expected outcomeVerified asset inventory that reflects actual network state. Any device not on the documented list is an unmanaged surface outside every RMM control you think you have.
- 2Technical assessor
Query endpoint state directly against your documented configuration baseline. Use your EDR telemetry, MDM configuration reports, or direct endpoint queries — not policy documents. For every system in scope, verify that the baseline settings are enforced, not just documented.
Expected outcomeA real number: what percentage of in-scope endpoints are actually in the state your configuration management policy says they should be. That number is your CM-4 maturity score, regardless of what your documentation review says.
- 3Vulnerability management team
Pull your scan frequency logs for the last 90 days and calculate the actual average time between scans per system. Compare that to your documented VM-1 monitoring policy. If the logs show quarterly scans and the policy says continuous, the gap is real and the score should reflect it.
Expected outcomeVerified scan cadence data that distinguishes between what your vulnerability management policy describes and what your scanner is actually doing. This is the evidence an RMM assessor should be reviewing — not the policy document.
- 4Incident response lead
Run a live-system incident response exercise, not a tabletop. Inject a realistic scenario — a detection alert fires, escalation is triggered, containment is executed on an actual test system. Measure time-to-detect, time-to-contain, and whether the playbook steps are executable as written. Document the gaps between the playbook and what actually happened.
Expected outcomeEvidence for IM-1 that reflects tested capability, not documented intent. If your response plan has never been executed against a live system, you do not know whether it works.
- 5GRC team + MSSP relationship owner
Audit your MSSP relationship as a separate control domain. Review SLAs for time-to-notify, verify scan frequency through MSSP-provided logs (not self-reported), and test the escalation path with an actual notification exercise. Document what your MSSP is responsible for and what happens if they fail to deliver.
Expected outcomeA third-party dependency assessment that closes the gap RMM's internal control focus leaves open. If your MSSP is your monitoring capability, their accountability needs to be governed and verified, not assumed.
What is coming for CERT RMM assessments that nobody is talking about yet
A named class of breach will emerge within 24 months, attributed to 'maturity theater' — organizations that achieved high CERT RMM or similar framework scores through documentation optimization while actual control effectiveness degraded. The term will appear in at least two major post-incident reports.
The pattern is already present in Vulnox data: the 43% discrepancy rate between documented and verified configuration state, the 61% rate of 'continuous monitoring' that is actually quarterly scanning. These are not random failures. They are the systematic output of assessment methodologies that reward documentation quality over control effectiveness. As framework adoption grows and assessment becomes more routine, the optimization pressure toward documentation increases. The breaches are already happening. The named pattern is not yet public.
Confidence: highReview Verizon DBIR and Mandiant M-Trends post-incident taxonomy sections between 2026 and 2028. If no named category for compliance-posture-versus-actual-control-gap breaches appears, the prediction fails.CERT RMM will add a technical validation requirement to its assessment methodology by 2027 — a formal requirement that configuration baseline assessment include endpoint state verification, not just documentation review.
The gap between document-based and technically-verified maturity is now visible enough in post-incident data that framework governance bodies will face pressure to address it. NIST has moved in this direction with continuous monitoring requirements in SP 800-137. The pattern of frameworks adding technical verification layers to what started as document-review methodologies is consistent. CERT RMM's next major revision will face this pressure.
Confidence: mediumCheck CERT RMM release notes and framework revision history through 2027. If no technical validation requirement appears in configuration management or vulnerability management domains, the prediction fails.
The gap analysis industry is selling audit preparation, not security improvement
Most CERT RMM gap analyses are procurement exercises. The client needs a gap analysis to satisfy a contractual requirement, a board request, or an insurance renewal. The assessor produces a scored report. The score satisfies the requirement. Nobody involved is primarily optimizing for whether the organization is actually more secure afterward. This is not a criticism of individual practitioners — it is a description of the market incentive. Gap analyses are scoped and priced for document review. Technical validation costs more and takes longer and makes clients uncomfortable. The market has produced the assessment type it is willing to pay for. The problem is that the assessment type it is willing to pay for does not find the gaps that cause breaches.
Counterargument
The counterargument is that technical validation requires client cooperation, system access, and longer engagements that many organizations cannot accommodate — and that a document-based gap analysis is genuinely better than no gap analysis. That is true. A well-conducted document review will find real gaps in policy coverage, control ownership, and process documentation. The question is whether those are the gaps that matter most. In Vulnox post-incident reviews, the gaps that enabled the breach were almost never the ones the prior gap analysis found. They were the gaps between documented state and actual system state — the gaps that a document review cannot see.
One thing to do before your next assessment cycle
Pull your most recent CERT RMM gap analysis report and find the three domains where you scored highest. For each one, identify the primary evidence the assessor used to justify that score — almost certainly a policy document, an approved process record, or an interview response. Then pick one control in each domain and verify its actual state directly: query endpoint configuration, pull scan frequency logs, run the escalation path. If the verified state matches the documented state, your score is real. If it does not, you have found the gap your last assessment missed. That exercise takes half a day. It will tell you more about your actual security posture than the full assessment did.
Further Reading
Gap Analysis
framework gap analysisDigital Footprint
baseline digital footprint analysisNational Vulnerability Database NIST
NIST National Vulnerability DatabaseNIST SP 800-30 Risk Assessment Guide
NIST risk assessment guideCompliance Gap Analysis Guide
compliance gap analysis guide
Frequently Asked Questions
What does a US CERT RMM gap analysis actually measure, and what does it miss?
A standard US CERT RMM gap analysis measures process maturity — whether your organization has documented, approved, and consistently applied its security procedures across RMM domains. What it typically misses is control effectiveness: whether those processes are producing the security outcomes they describe. Vulnox found a 43% discrepancy rate between documented configuration baselines and actual endpoint state in organizations scoring 'Consistent' in CERT RMM configuration management. The assessment measured the document. The endpoint was in a different state.
How do I verify CERT RMM configuration management maturity without relying on policy documents?
Query endpoint state directly against your documented baseline using EDR telemetry (CrowdStrike, Defender for Endpoint), MDM configuration reports (Intune, Jamf), or direct endpoint queries. For each in-scope system, verify that the baseline settings are enforced — not just documented. Calculate what percentage of endpoints are actually in the state your CM-4 policy describes. That number is your real configuration management maturity score, regardless of what your documentation review concluded.
What does CERT RMM VM-1 continuous monitoring actually require, and how do most organizations get it wrong?
CERT RMM VM-1 requires organizations to progress toward continuous vulnerability monitoring with timely remediation. In practice, 61% of organizations Vulnox assessed interpreted this as quarterly scanning and documented it as continuous monitoring. The terminology was adopted; the cadence was not. The result is an average 78-day gap between scans — longer than the time-to-exploit for high-severity CVEs, which Mandiant tracked at under 5 days for the most critical 2023 disclosures. Closing this gap requires verifying actual scan frequency from scanner logs, not from policy documents.
How does US CERT RMM compare to NIST CSF for a gap analysis?
CERT RMM provides prescriptive maturity levels (ad hoc, defined, consistent, managed) with explicit advancement criteria for each domain. NIST CSF uses a function-based structure (Identify, Protect, Detect, Respond, Recover) without mandatory maturity levels. RMM's specificity makes scoring more precise and easier to report to boards, but also easier to game through documentation optimization. NIST CSF's flexibility makes it harder to score and harder to game. For a resilience-focused gap analysis, RMM's domain structure is more useful — provided the assessment includes technical validation of actual control state, not just documentation review.
Why do organizations breach after passing a CERT RMM assessment?
Because most CERT RMM assessments are document reviews, not security verifications. The assessment confirms that policies exist, procedures are written, and approvals are documented. It does not verify that the controls described in those documents are running on actual systems. In Vulnox post-incident reviews following breaches at organizations with prior RMM assessments, the enabling gap was almost always between documented control state and actual system state — a gap that a document review cannot detect. The assessment was accurate. It was measuring the wrong thing.
How should I assess third-party and MSSP risk in a CERT RMM gap analysis?
Treat your MSSP relationship as a separate control domain. Review SLAs for time-to-notify on detected incidents, verify scan frequency through MSSP-provided logs rather than self-reporting, and test the escalation path with an actual notification exercise. In 38% of Vulnox engagements where clients cited an MSSP as evidence of monitoring maturity, there was no documented SLA for notification time and no tested escalation path. CERT RMM's internal control focus does not cover third-party risk — that surface requires explicit assessment scope.
What is the most common finding in a CERT RMM incident management gap analysis?
The most common finding is that incident response plans have not been tested against live systems. CERT RMM IM-1 requires tested response plans, but 61% of organizations Vulnox assessed had conducted only tabletop exercises — no live system component, no verified alert-to-containment timing, no confirmation that playbook steps execute as written. An incident response plan that has never been run on actual infrastructure is a documented intention, not a tested capability. The gap between those two things shows up in the first 4 hours of a real incident.
Related Articles
GovRAMP Moderate authorization: why the Significant Change Request process catches providers off guard
GovRAMP Moderate is the first tier where you need a government sponsor, annual 3PAO reassessment, and a Significant Change Request process that can pause normal product releases for months. Most providers who stall post-authorization were not prepared for what maintaining Moderate status actually costs operationally.
GovRAMP Low+ authorization: the impact level that punishes providers who get the CUI boundary wrong
GovRAMP Low+ is where providers handling limited Controlled Unclassified Information land — or discover they should not be there. The defining failure is not a missing control. It is a CUI boundary that was drawn before anyone asked what data the government actually sends through the system.
GovRAMP High authorization: why FIPS-validated crypto and personnel security controls catch providers off guard
GovRAMP High is where cloud providers discover that having strong encryption is not the same as having FIPS 140-2 validated encryption — and that distinction alone has derailed authorizations from vendors who passed every other control family. The architectural constraints at High are qualitatively different from every lower tier.
Ready to Secure Your Digital Assets?
Get a comprehensive vulnerability assessment for your website today.