CMS Working on Predictive Modeling System to Combat Fraud

The Centers for Medicare & Medicaid Services (CMS) has is still working on creating a predictive-modeling system to combat Medicare and Medicaid fraud, according to a Government Accountability Office (GAO) report unveiled in December.  Although the agency rolled out a fraud prevention system in July 2011, it does not integrate with CMS’ payment-processing system, work that was supposed to begin last summer.  Now, CMS says the work likely will not begin until January. 

GAO was asked to (1) determine the status of the implementation and use of FPS, (2) describe how the agency uses FPS to identify and investigate potentially fraudulent payments, (3) assess how the agency’s use of FPS compares to private insurers’ and Medicaid programs’ practices, and (4) determine the extent to which CMS has defined and measured benefits and performance goals for the system. 

To do this, GAO reviewed program documentation, held discussions with state Medicaid officials and private insurers, and interviewed CMS officials and contractors.  GAO conducted this performance audit from October 2011 to October 2012. 

GAO reported that program officials blamed the lack of progress on not having enough time to develop system requirements.  According to the report, CMS has yet to define or measure quantifiable benefits or performance goals.  “Until program officials review the effectiveness of the system based on …measurable performance targets, they will not be able to determine the extent to which FPS is enhancing CMS’s ability to accomplish the goals of its fraud prevention program.” 

Sen. Tom Carper (D-Del.), one of three senators who said his questions about the system’s implementations had been ignored by CMS, called the system one with “enormous potential for fighting … waste and fraud,” but said in a statement that the agency “needs to do more … to get the system fully up and running.” 

Sen. Tom Coburn, M.D. (R-Okla.), meanwhile, pulled no punches, calling CMS’s lack of progress “troubling.”  “Unfortunately, GAO has found that even after spending $77 million on the program, CMS has no idea whether it is saving money or preventing fraud,” he said.  The system’s creation was authorized by the Small Business Jobs Act of 2010. 

In addition to the GAO report, the Office of the inspector General (OIG) for the Department of Health & Human Services (HHS) released a report on CMS’ FPS, finding that the program suffers from inconsistent data and flawed methodology that make it impossible to track inaccuracies.  OIG recommended CMS: 

• Require contractors to track money recovered from following FPS leads
• Coordinate with law enforcement to better report outcomes of cases referred from FPS
• Revise its methodology for calculating projected savings from improper payments to recognize that not all claims from a revoked provider are necessarily false and that some previously denied claims may eventually be paid
• Revise the methodology for calculating costs avoided to verify certain information and to include costs associated with the FPS 

In its report to Congress, CMS agreed to improve its savings methodology.  In addition, it said it already had: 

• Improved how it tracked recoveries of overpayment and better estimating law-enforcement recoveries
• Found ways to minimize or eliminate deficiencies in manual data reporting
• Evaluated a possible fix for systematically accounting for legitimate services and claims overturned on appeal
• Incorporated costs identified by OIG in calculating return on investment 

GAO Report 

CMS implemented its Fraud Prevention System (FPS) in July 2011, as required by the Small Business Jobs Act, and the system is being used by CMS and its program integrity contractors who conduct investigations of potentially fraudulent claims. Specifically, FPS analyzes Medicare claims data using models of fraudulent behavior, which results in automatic alerts on specific claims and providers, which are then prioritized for program integrity analysts to review and investigate as appropriate. 

However, while the system draws on a host of existing Medicare data sources and has been integrated with existing systems that process claims, it has not yet been integrated with the agency’s payment-processing system to allow for the prevention of payments until suspect claims can be determined to be valid.  Program officials stated that this functionality has been delayed due to the time required to develop system requirements; they estimated that it will be implemented by January 2013 but had not yet developed reliable schedules for completing this activity. 

FPS is intended by program integrity officials to help facilitate the agency’s shift from focusing on recovering large amounts of fraudulent payments after they have been made, to taking actions to prevent payments as soon as aberrant billing patterns are identified.  Specifically, CMS has directed its program integrity contractors to prioritize alerts generated by the system and to focus on administrative actions—such as revocations of suspect providers’ Medicare billing privileges—that can stop payment of fraudulent claims.  To this end, the system has been incorporated into the contractors’ existing investigative processes.  

CMS has also taken steps to address challenges contractors initially faced in using FPS, such as shifting priorities, workload challenges, and issues with system functionality.  Program integrity analysts’ use of FPS has generally been consistent with key practices for using predictive analytics identified by private insurers and state Medicaid programs.  These include using a variety of data sources; collaborating among system developers, investigative staff, and external stakeholders; and publicizing the use of predictive analytics to deter fraud. 

CMS, however, has not yet defined or measured quantifiable benefits, or established appropriate performance goals.  To ensure that investments in information technology deliver value, agencies should forecast expected financial benefits and measure benefits accrued.   

The Office of Management and Budget (OMB) requires agencies to define performance measures for systems that reflect program goals and to conduct post-implementation reviews to determine whether objectives are being met.  However, CMS had not defined an approach for quantifying benefits or measuring the performance of FPS.  Further, agency officials had not conducted a post-implementation review to determine whether FPS is effective in supporting efforts to prevent payment of fraudulent claims.  Until program officials review the effectiveness of the system based on quantifiable benefits and measurable performance targets, they will not be able to determine the extent to which FPS is enhancing CMS’s ability to accomplish the goals of its fraud prevention program. 

Accordingly, GAO recommended that CMS:  

• define quantifiable benefits expected as a result of using the system, along with mechanisms for measuring them, and
• describe outcome-based performance targets and milestones that can be measured to gauge improvements to the agency’s fraud prevention initiatives attributable to the implementation of FPS.
• develop schedules for completing plans to further integrate FPS with the claims payment processing systems that identify all resources and activities needed to complete tasks and that consider risks and obstacles to the program, and
• conduct a post-implementation review of the system to determine whether it is effective in providing the expected financial benefits and supporting CMS’s efforts to accomplish the goals of its fraud prevention program. 

In its comments, HHS agreed with and described actions CMS was taking to address the recommendations. 

FPS Background 

In April 2010, CMS established the Center for Program Integrity (CPI) to enable a strategic and coordinated approach to program integrity initiatives throughout the agency and to build on and strengthen existing program integrity efforts.  The center’s mission is to ensure that correct payments are made to legitimate providers for covered, appropriate, and reasonable services for eligible beneficiaries. To accomplish its mission, the center has undertaken a strategy to supplement the agency’s “pay and chase” approach, which focuses on the recovery of funds lost due to payments of fraudulent claims, with an approach that is directed toward the detection and prevention of fraud before claims are paid.  

The strategy has concurrent objectives to (1) enhance efforts to screen providers and suppliers enrolling in Medicare to prevent enrollment by entities that might attempt to defraud or abuse the Medicare program and (2) detect aberrant, improper, or potentially fraudulent billing patterns and take quick actions against providers suspected of fraud. In addressing the second objective, CPI intends to use predictive analytics technologies to detect potential fraud and prevent payments of claims that are based on fraudulent activities.  Accordingly, CPI is the focal point for all activities related to FPS. 

To advance the use of predictive analytics technologies to help prevent fraud in the Medicare program, the Small Business Jobs Act of 2010 appropriated $100 million to CMS, to remain available until expended, for the development and implementation of a predictive analytics system.  Enacted on September 27, 2010, the law required CMS to implement a system that could analyze provider billing and beneficiary utilization patterns in the Medicare fee-for-service program to identify potentially fraudulent claims before they were paid.  To do this, the system was to capture data on Medicare provider and beneficiary activities needed to provide a comprehensive view across all providers, beneficiaries, and geographies.  

It was also intended to identify and analyze Medicare provider networks, provider billing patterns, and beneficiary utilization patterns to identify and detect suspicious patterns or anomalies that represent a high risk of fraudulent activity.  The act further required the system to be integrated into Medicare’s existing systems and processes for analyzing and paying fee-for-service claims in order to prevent the payment of claims identified as high risk until such claims were verified to be valid. 

The act also specified when and how CMS should develop and implement the system. Specifically, it required that CMS select at least two contractors to complete the work and that the system be developed and implemented by July 1, 2011, in the 10 states identified by CMS as having the highest risk of fraud.  The act further required the Secretary of HHS to issue, no later than September 30, 2012, the first of three annual implementation reports that identify savings attributable to the use of predictive analytics, along with recommendations regarding the expanded use of predictive analytics to other CMS programs. 

The act stated that based on the results and recommendations of the first report, the use of the system was to be expanded to an additional 10 states at the next highest risk of fraud on October 1, 2012; similarly, based on the second report, the use would then be expanded to the remaining states, territories, and commonwealth on January 1, 2014. 

To meet the act’s requirements, CMS assigned officials within CPI responsibility for the development, implementation, and maintenance of FPS.  These officials included a business process owner, information technology program manager, information technology specialist, and contracting officer. In defining requirements for the system to address the mandate of the Small Business Jobs Act, these program officials planned to implement by July 1, 2011, system software for analyzing fee-for-service claims data, along with predictive analytic models that use historic Medicare claims and other data to identify high-risk claims and providers. 

Program officials further planned, by July 2012, to implement functionality into FPS to enable automatic notification to system users of potentially fraudulent claims and to prevent payments of those claims until program integrity analysts determined that they were valid.  In April 2011, CMS awarded almost $77 million to a development contractor to implement, operate, and maintain the system software and to design a first set of models for the initial implementation of FPS.  The agency awarded about $13 million to a second contractor in July 2011 to develop additional models that could be integrated into the system.   

FPS System 

FPS is a web-based system that is operated from a contractor’s data center and accessed by users via the agency’s secured private network.  The system is comprised of software that analyzes fee-for-service claims data as the claims are being processed for payment, along with hardware, such as servers that support connections between users’ facilities and CMS’s network, and devices that store the data used and generated by the system.  The system software and predictive models are designed to analyze the claims data and generate alerts to users when the results of analyses identify billing patterns or provider and beneficiary behavior that may be fraudulent and warrant administrative actions. CPI also engaged its internal program integrity analysts to help design the models and test the initial implementation of the system. 

In September 2011, CPI established a group that works with and provides training to the ZPICs on how to use FPS to initiate administrative actions more quickly against providers suspected of fraud.  According to CPI officials, they intend to continue to refine the system to provide analysts and investigators with data and statistical information useful in conducting investigations based on input provided during these training sessions.

In response to the Small Business Jobs Act, CMS implemented its initial release of FPS by July 1, 2011.  While the act called for CMS to first implement the system for use in the 10 states identified by CMS as having the highest risk of fraud, the agency chose to deploy the system to all the ZPIC geographic zones.  In addition, the system was integrated with existing data sources and systems that process claims, but it was not yet integrated with CMS’s claims payment systems.  As of May 2012, CMS had spent nearly $26 million on the implementation of FPS.  Of this amount, about $1 million was spent for internal CMS staff and $25 million for the development and modeling contractors. 

CMS’s initial release of the system consisted of system software for analyzing fee-for-service claims data and predictive analytic models that use historic Medicare claims and other data to identify high-risk claims and providers.  After the initial release, CMS implemented three more releases of software through July 1, 2012, that incorporated changes or enhancements to the system as well as additional models.  The four system releases yielded a total of 25 predictive analytic models in three different categories and with varying levels of complexity.  Specifically, these consisted of the following model types: 

• Rules-based models, which are to filter potentially fraudulent claims and behaviors, such as providers submitting claims for an unreasonable number of services.  These models also are intended to target fraud associated with specific services, including those that CMS has stated are at high risk for fraud, such as home health agency services and durable medical equipment suppliers.  These are the simplest types of models since the analysis conducted using them only involves counting or identifying types of claims and comparing the results to established thresholds. 
• Anomaly-detection models, which are to identify abnormal provider patterns relative to the patterns of peers, such as a pattern of filing claims for an unreasonable number of services.  These models generate analyses that are more complex because they require identification of patterns of behavior based on data collected over a period of time, and comparisons of those patterns to established behaviors that have been determined to be reasonable. 
• Predictive models, which are to use historical data to identify patterns associated with fraud, and then use these data to identify certain potentially fraudulent behaviors when applied to current claims data. 

These models are intended to help identify providers with billing patterns associated with known forms of fraud.  This is the most complex type of model implemented into FPS because it not only requires analysis of large amounts of data but may also require detection of several patterns of behavior that individually may not be suspicious but, when conducted together, can indicate fraudulent activity.  Of the 25 models that CMS had implemented by July 1, 2012, 14 were rules-based, 8 were anomaly-detection, and 3 were predictive. 

While FPS was integrated with existing data sources and systems that process claims, it had not been further integrated with CMS’s claims payment systems.  Specifically, FPS had not been integrated with the components of the shared systems that process the payment of claims.  However, this level of integration is required to enable FPS to prevent the payment of potentially fraudulent claims until they have been verified by program integrity analysts and investigators. 

While the act called for the implementation of FPS by July 1, 2011, including this capability, the agency’s program plans initially indicated that it was to be implemented by July 1, 2012.  However, the business process owner of FPS stated that planning for the development of this system functionality required extensive discussions regarding design and requirements with entities that maintain and use other systems, particularly the shared systems.  Consequently, FPS program officials did not complete requirements definition until May 2012.  CMS now intends to complete integration of the capability in January 2013. 

Although CMS has identified January 2013 as a target date for completing the development, testing, and integration of FPS with the claims payment systems, program officials had not yet defined detailed schedules for completing the associated tasks required to carry this out. 

Best practices, such as those described in GAO’s cost estimation guide, emphasize the importance of establishing reliable program schedules that include all activities to be performed; assign resources (labor, materials, etc.) to those activities; identify risks and their probability; and build appropriate reserve time into the schedule.  However, FPS program officials had not yet developed such schedules and did not indicate when they intend to do so.  Until it develops reliable schedules for completing associated tasks, the agency will be at risk of experiencing additional delays in further integrating FPS with the payment processing system, and CMS and its program integrity analysts may lack the capability needed to prevent payment of potentially fraudulent claims identified by FPS until they are determined by program integrity analysts to be valid. 

FPS and Zone Program Integrity Contractors (ZPICs) 

ZPICs are responsible for identifying and investigating potential fraud in the Medicare fee-for-service program.  CPI directs and monitors their activities. These contractors identify claims and provider billing patterns that may indicate fraud and investigate leads from a variety of sources, including complaints and tips lodged by beneficiaries.  ZPICs operate in seven geographical zones across the country. 

Officials from the ZPICs reported that FPS has not fundamentally changed the way in which they investigate fraud.  The system has not significantly sped up investigations or enabled quicker administrative actions in most instances.  Instead, officials reported that leads from the system were broad indicators that particular providers were suspect, but did not in all cases provide sufficient evidence of potentially fraudulent billing to allow for faster investigations or resolutions.  FPS investigations were similar to those from other sources in that they often required additional investigative steps, such as beneficiary and provider interviews.  On the other hand, ZPICs reported certain advantages as a result of using FPS.  For example: 

• analysts can query the system for specific data to support their analysis of leads and export data from FPS into other systems they use to conduct additional analysis of claim lines flagged by FPS.  
• Data generated by the system may also notify investigators of information available in other CMS databases, (e.g., national Fraud Investigation Database)  
• Using FPS’s near-real-time claims data, some investigators reported identifying and conducting interviews with beneficiaries shortly after they received services from providers under investigation, when beneficiaries can better recall details about their care.  
• Information in FPS has also helped substantiate leads from other sources. For example, one ZPIC noted that its investigators use information from the system to help verify tips and complaints about suspected fraud. 

False positives: ZPICs reported that certain FPS models identified and prioritized the investigation of a relatively high proportion of false positives—i.e., improper identification of suspect providers that were not engaged in fraud.  Some of these false positives related to the nationwide application of models, which did not take into account localized conditions that may help explain certain provider billing patterns.  For example, a physician in a rural area may provide care for beneficiaries dispersed across a large geographic range—something that would raise suspicion for a physician in an urban area.  

ZPICs also reported that the system sometimes prioritized leads that target forms of fraud that are not prevalent in their zone and that investigating such false positive leads has taken time away from other investigations.  In response to ZPIC feedback that certain models produced a high number of false positive leads, CMS changed the way the system generates leads and how it assigns risk scores to providers identified by those models. According to program integrity officials, CMS is also considering approaches to control for geographic variations in fraud. 

FPS functionality:  ZPICs cited challenges related to aspects of FPS’s functionality.  For example, when first implemented, the system only provided data directly relevant to the aberrant billing patterns associated with its leads.  ZPICs, however, said that determining whether a provider is potentially suspect requires contextual and background information, such as provider profile and billing history information. Because this information was not provided by FPS, the ZPICs had to use other sources to obtain this information.  Based on this feedback, CMS updated the system so that its leads now provide users with contextual and background information on providers identified by the system. 

FPS and Social Networks 

Social network analysis is emerging as an important tool to combat organized health care fraud since it can be used to demonstrate linkages among individuals involved in fraud schemes.  One official from a state Medicaid program noted that, since organized fraud operations often move from scheme to scheme, identifying the networks of individuals involved in fraud, rather than simply limiting their ability to perpetrate certain schemes, is increasingly important. 

While FPS does not yet include social network analysis, CMS program integrity officials were conducting a pilot to determine how to integrate social network analysis into future model development.  These officials stated that they intend to analyze and implement results of the study, as appropriate, by the end of September 2012. 

Expected Financial Benefits of FPS

The Clinger-Cohen Act of 1996 and OMB guidance emphasize the need for agencies to forecast expected financial benefits of major investments in information technology and measure actual benefits accrued through implementation.  OMB requires agencies to define and report progress against outcome-based performance measures that reflect goals and objectives of information technology programs.  

With regard to FPS, CMS had not yet defined an approach for quantifying the financial benefits expected from the use of the system.  CPI officials stated that they had not yet determined how to quantify and measure financial benefits from the system, but that they intend to do so in the future.  These officials stated their intention was to measure benefits based on savings resulting from the system’s contributions to the agency’s efforts to prevent payments of fraudulent claims.  However, while CMS could potentially quantify financial benefits resulting from the amount of suspended payments or other administrative actions based on the results of FPS, the capability of the system that could provide benefits through the suspension of payments had not yet been implemented.  

The officials further acknowledged the difficulty with determining benefits or return on the agency’s investment in FPS in part because fraudulent providers’ knowledge of CMS’s use of the system could likely have a deterrent effect and, as intended, prevent fraudulent activity from occurring.  In these cases, the amount of costs avoided would be unknown.  

In addition to the difficulties associated with the agency’s efforts to quantify financial benefits of implementing FPS, CMS has not established or reported to OMB outcome-based performance measures, targets, and milestones for gauging the system’s contribution to meeting its fraud prevention goals.   

Conclusion 

GAO concluded that as implemented, FPS provides functionality that supports program integrity analysts across the country in their efforts to identify and prevent payment of potentially fraudulent claims until they are determined to be valid.  CMS has also used FPS as a tool to better coordinate efforts with ZPICs, the contractors primarily responsible for investigating fraud.  Further, while the use of sophisticated predictive analytics to address health care fraud is relatively new, CMS’s use of FPS has generally been consistent with key practices identified by private insurers and state Medicaid programs we interviewed.  

Despite these efforts, agency officials have not yet implemented functionality in the system needed to suspend payment of high-risk claims until they are determined through further investigation to be valid, and have not yet developed detailed schedules for doing so.  Additionally, they have not yet determined ways to define and measure financial benefits of using the system, nor have they established outcome-based performance measures and milestones for meeting the performance targets that reflect the goals of the agency’s fraud prevention program.

Until such performance indicators are established, FPS officials will continue to lack the information needed to conduct a post-implementation review of the system to determine its benefits and effectiveness in supporting program integrity analysts’ efforts to identify potentially fraudulent claims and providers.  Furthermore, CMS officials, OMB, and Congress may lack important information needed to determine whether the use of the system contributes to the agency’s goal of predicting and preventing the payment of potentially fraudulent claims for Medicare services.  In this regard, the contribution of FPS to the agency’s effectiveness in preventing fraud will remain unknown.