CMS Secret Shopper Program: How CMS Tests Your Network Adequacy in Practice

Beyond the Filing: What CMS Is Actually Testing

Network adequacy compliance has two distinct dimensions that Medicare Advantage plans sometimes treat as the same thing. The first is the paper adequacy of the network — the NPI counts, time-and-distance calculations, and HPMS filings that make up the annual submission process. The second is the functional adequacy of the network — whether members can actually get appointments with in-network providers in a reasonable timeframe, whether the provider directory is accurate when a member calls or goes online, and whether the plan can provide care in a member's preferred language.

CMS's secret shopper program exists specifically to test the second dimension. The program is authorized under 42 CFR 422.116 and the related compliance monitoring authority in 42 CFR 422.503, and it operates independently of the annual network adequacy filing review. A plan can have a clean HPMS submission and still receive a deficiency notice based on secret shopper findings.

Understanding how the program works — who conducts it, what they test, and how findings translate into compliance consequences — is essential for any MA plan compliance team that wants a complete picture of their regulatory exposure.

Who Conducts the Program and How Often

CMS contracts with third-party research organizations to conduct secret shopper testing on its behalf. These contractors operate under CMS oversight and follow standardized testing protocols developed and updated by CMS. The contractors are not disclosed publicly, and CMS does not announce in advance which plans will be tested in a given period.

The frequency of testing is not uniform across plans. CMS prioritizes plans for testing based on several factors, including prior-year performance on adequacy metrics, complaint volume from members and providers, Star Ratings performance in access-related measures, and geographic market characteristics (plans in rural or underserved markets tend to receive more frequent testing). Plans with a history of clean adequacy performance and low complaint volume are tested less frequently, though they are not immune from routine monitoring cycles.

CMS conducts secret shopper testing at multiple points in the benefit year, with particularly active testing periods in the late winter and spring months as CMS builds its picture of network performance for the current benefit year. Testing can also be triggered outside the routine cycle by specific complaint patterns or by referrals from the Medicare Drug Integrity Contractor (MEDIC) or state insurance department partners.

What the Secret Shopper Tests: The Four Core Dimensions

CMS's secret shopper protocol consistently tests across four dimensions of network performance. Each dimension maps to specific regulatory requirements, and findings in each dimension have defined escalation pathways.

Appointment availability and wait times: Callers contact in-network providers — drawn from the plan's submitted network data — and request appointments as new patients or as established patients seeking follow-up care. The testers document whether the provider answers, whether they confirm they are accepting new MA patients, and what the offered wait time is for an appointment. CMS benchmarks vary by specialty and appointment type. For primary care, CMS generally expects routine new patient appointments to be available within 30 days; for urgent care, within 24 hours; for specialists, within a specialty-specific window that ranges from 10 days for urgent referrals to 45 days for routine specialist visits.

Provider directory accuracy: Testers cross-reference the plan's online and print provider directories against their call results. When a directory lists a provider as accepting new MA patients, the tester calls and documents whether that claim is accurate. When a directory lists a provider's address or phone number, the tester verifies those details. Error rates are calculated at the plan level and compared against CMS thresholds. Under the 2024 CMS Final Rule implementing the 72-hour directory update requirement, CMS's expectations for directory accuracy have become more stringent, and the secret shopper program has updated its methodology to reflect the tighter standard.

Language access: CMS tests whether plans can connect members with providers who speak the member's preferred language, and whether the plan's member services line can connect callers with interpreter services within a reasonable timeframe. Under 42 CFR 422.112(b)(3), plans are required to ensure that members with limited English proficiency can access covered services without language barriers. Secret shopper testers calling with a stated language preference document whether the plan's member services staff offer interpreter services, how quickly they connect, and whether the connected interpreter is qualified for medical interpretation.

Behavioral health access: CMS has increased its focus on behavioral health access testing in recent benefit years, reflecting the broader policy emphasis on mental health parity and the behavioral health provider shortage. Testers specifically target psychiatric and therapy providers and document wait times, new patient acceptance rates, and whether providers distinguish between MA and commercial patients in their acceptance practices.

Targeted vs. Routine Secret Shopper Testing

CMS conducts two types of secret shopper activities: routine monitoring (sometimes called surveillance testing) and targeted audits. Compliance teams should understand the distinction because the triggers, scope, and consequences differ.

Routine monitoring is the background baseline. CMS's contractor tests a sample of plans across markets on an ongoing cycle, with results aggregated and reviewed against prior-period performance. Plans that perform within acceptable ranges on routine monitoring typically do not receive individual-level communications about those results — the data is folded into CMS's broader oversight picture. Plans that fall outside acceptable ranges on routine monitoring are flagged for follow-up, which may include a targeted audit or an informal inquiry letter.

Targeted audits are triggered by specific concerns: a pattern of member complaints about appointment access, a provider directory accuracy finding from a prior audit cycle, a referral from a state partner, or a tip from a provider alleging that a plan is misrepresenting network participation. Targeted audits are more intensive than routine monitoring — they involve more calls, cover a larger sample of providers, and may include testing of specific providers identified in the triggering complaint rather than a random sample. Targeted audits almost always result in a formal communication to the plan, even if the findings are not severe enough to trigger a deficiency notice.

How Findings Feed Into Deficiency Notices and Corrective Action

Secret shopper findings that exceed CMS's error rate thresholds are communicated to the plan through a formal finding letter. The finding letter describes the testing methodology, the sample tested, the error rates observed, and the specific standards that were not met. The letter does not always arrive quickly after the testing occurs — the data aggregation and analysis process means that findings from spring testing may not reach plans until late summer or early fall.

Findings are classified by severity. Provider directory accuracy findings with error rates above the threshold established in the 2024 Final Rule (which codified a specific error rate standard for the first time) are treated as compliance violations under 42 CFR 422.116(b) and typically result in a formal deficiency notice with a required corrective action plan. Appointment availability findings above threshold may result in a deficiency notice or, for first-time moderate findings, an informal corrective action request that does not require the full formal CAP process.

Plans that receive secret shopper-based deficiency notices face the same corrective action and monitoring framework as plans that receive adequacy deficiency notices based on HPMS filing review. The plan must submit a CAP within 30 days of the deficiency notice, implement the remediation steps within the CAP timeline, and demonstrate to CMS that the root cause has been addressed. CMS may conduct follow-up testing after the CAP period to verify that the issues identified have been resolved.

Uncorrected secret shopper findings — or patterns of repeated findings across multiple audit cycles — can escalate to civil monetary penalties under 42 CFR 422.752(a), intermediate sanctions, or, in the most severe cases, enrollment sanctions. While full enforcement action based solely on secret shopper findings is relatively rare, the findings are a meaningful input into CMS's overall assessment of the plan's compliance posture and can be a factor in more severe enforcement actions triggered by other issues.

The Relationship Between Secret Shopper Results and Star Ratings

Secret shopper findings are not published directly in Star Ratings, but the relationship between the two programs is closer than it may appear. Several of the measures that feed into the plan's Star Rating in the Access and Availability domain are constructed from data that overlaps substantially with what secret shopper testing evaluates.

The Getting Needed Care measure (CMS measure C01/D01 in recent rating years) and the Getting Appointments and Care Quickly measure draw on CAHPS survey data that asks members about their actual experience with appointment access and specialist referrals — the same dimensions that secret shopper testing evaluates from the supply side. A plan that performs poorly on secret shopper testing for appointment availability is likely experiencing the same underlying network performance problem that will eventually surface in member-reported CAHPS scores.

Additionally, CMS has indicated in its Star Ratings methodology documentation that compliance actions — including formal deficiency notices and corrective action requirements — can affect a plan's Part C Summary score through the compliance weighting factor applied to the overall Star Rating. A plan that accumulates formal compliance actions, including those arising from secret shopper findings, may see a measurable impact on its Star Rating beyond the direct CAHPS measure effects.

How Plans Can Prepare: A Practical Framework

The most effective preparation for the secret shopper program is not a specific audit-readiness exercise — it is building the operational infrastructure to maintain network performance continuously. Plans that perform well on secret shopper testing consistently share the following characteristics:

• Active provider directory monitoring: Quarterly outreach to all in-network providers to verify panel status, address, phone number, and MA patient acceptance status — not annual credentialing re-verification, which is too infrequent to catch the changes CMS is testing for.
• Appointment access data collection: Some plans conduct their own internal "secret shopper" exercises — typically quarterly — to sample appointment availability across key specialties and markets. This internal testing surfaces problems before CMS finds them and creates documentation of the plan's proactive monitoring program.
• Language access infrastructure: Ensuring the plan's member services line can connect callers with qualified medical interpreters within a defined service level — and documenting those service level results — provides both the operational capability and the evidence needed to respond to a language access finding.
• Provider communication protocols: Providers who are aware that their panel status and appointment availability may be verified by CMS (without disclosing the specific mechanism) are more likely to maintain accurate information with the plan. Some plans include directory accuracy compliance obligations in their provider contracts and follow up with providers who are repeatedly identified as inaccurate in directory verification exercises.

None of these measures guarantee that a plan will not receive secret shopper findings. But they substantially reduce the probability of findings above the threshold levels that trigger formal deficiency notices, and they provide the documentation infrastructure needed to respond effectively if findings do occur.