SLI Global Solutions - Program and technical consulting that assists government agencies to improve service delivery

Test Management Services on North Carolina's Multi-Payer Medicaid System

The Situation

The North Carolina Department of Health and Human Services (DHHS) manages and operates the Medicaid Management Information System (MMIS) that pays benefits to providers that deliver medical services to low-income and other eligible families. The legacy system was built 35 years ago on antiquated technology, was inflexible to the need for ongoing modifications, and did not support the State’s vision for the first public multi-payer system in the US. At same the same time, the MMIS program had become more complex due to the need to conform to the regulatory framework of the Affordable Care Act (ACA).

To address these issues, DHHS initiated the NCTracks project to replace its aging MMIS and the related Reporting & Analytics (R&A) system. To accomplish these goals, DHHS established contracts with systems integration firms experienced in the implementation of the MMIS and R&A systems. Development work began in early 2009 and was completed in January 2016.

Due to the complexity and overall technical risk inherent in information technology projects of this magnitude, DHHS determined that outside independent test management support was necessary to assist the State in its obligations to thoroughly test the software throughout the project’s life and help ensure a system that met federal certification requirements.

The Solution

SLI Government Solutions delivered an approach to staff each development track with a dedicated Test Lead, and an overall Test Manager to coordinate all the team’s activities with DHHS and the systems integration vendors. It was determined that full-time, on-site support was required, so the SLI Test Management Team was co-located with the development teams at the client’s work location in Raleigh, NC.

  • The SLI Team provided multiple test management support services including the review and assessment of vendor-led test planning deliverables and test artifacts produced by the vendors for the two sub-projects. SLI monitored vendor-led testing through the analysis of actual test results to determine readiness for production of each release. SLI reviewed and refined defect management procedures to ensure proper defect classification and turnaround.
  • The State-led testing was managed by SLI through coordination of the agency participants, determination of test cases and data scenarios for User Acceptance Test (UAT), and final evaluation of the system for live production status. SLI Test Management Services included:
  • Planning, executing, and reporting of system/user testing, involving both internal and outsourced development
  • Coordinating and training of State-controlled resources and schedule for testing
  • Providing expertise in the chosen testing software, SilkCentral Test Manager; providing custom reports and analysis of testing velocity
  • Interfacing with System Integrators to assist and oversee its testing effort to ensure the success of the State’s testing effort. Interface with DHHS and non-DHHS organizations and individuals, as necessary, to enhance the testing program
  • Focusing efforts in a value-added fashion that achieved the best return on investment and overlaps the developers’ efforts only where prudent and necessary
  • Participating in project risk management programs to ensure that testing was used as an appropriate identification and management tool; and verification of non-system elements of the overall solution, such as manual processes, as directed by the State

SLI’s role was to act as a testing coach, expert, and quality partner. Throughout the Test Management effort, SLI facilitated the work of the NCTracks Project by developing constructive, collaborative relationships with the State and with the System Integration vendors, with the goal of ensuring comprehensive test coverage to maximize software quality. For example, SLI uncovered an issue with the scope, coverage and organizing model for the MMIS vendor's automated test suite. This finding enabled the State Team Subject Matter Experts to clarify the coverage 'holes' present within the System Integrator's regression testing approach and correct the gaps before they caused significant delays and rework.

The Outcome

SLI’s Test Management services were instrumental in assisting DHHS in a successful implementation. Test Plans, Scenarios and Scripts were submitted on time and were of a consistently high quality. The State testers were effectively trained on proper procedures to execute tests, record the results, and document defects. SLI was also effective in tracking defect resolution and conducting effective regression testing. Of particular importance was the work the SLI team conducted to help make valid go/no go implementation decisions. Joe Cooper, former CIO of DHHS, commented, “SLI was a valuable partner in assisting the agency in deploying a high-quality system that was comprehensively tested. Their team’s combination of testing expertise and MMIS knowledge gave me and our user community confidence that the system would help us deliver more timely and accurate results to our recipients and providers. SLI’s efforts were a significant factor in the agency receiving CMS certification of the MMIS on the first attempt, ahead of schedule.”

Ultimately, the project was completed with a high-quality deployment of cutting edge technology. The quality of NCTracks met the technical requirements and the business needs of DHHS were fulfilled. SLI’s strong local team, staffed with certified test management professionals and Medicaid subject matter experts resulted in a successful project and gave confidence in the deployment readiness of this groundbreaking MMIS implementation. In its first year, NCTracks processed 200 million claims and paid $10.3 billion to healthcare providers. Providers can now access multiple systems through a single sign-on portal for a near real-time view of claims that are being processed faster and more efficiently, effectively lowering the administrative burden and ultimately cost of delivering quality health care to eligible families in North Carolina.