The Bill and Melinda Gates Foundation contracted Khulisa to conduct 2 rounds of independent results verification reported by the CCS&PT partners for 2 reporting periods: (i) May 2013 to April 2014, and (ii) May 2014 to April 2015.
The overall purpose of the verification exercise was to: (i) confirm the partners’ annual reported results for the Foundation’s incentive payment decisions, whereby the “best performing partner” and the “best performing country” program receive incentive bonus payments; (ii) to improve the partners’ capacity around data management based on the findings from the verification exercise.
The main objectives of this exercise were to:
• Verify the quality of reported data for key indicators at selected sites;
• Assess the ability of data management systems to collect and report quality data; and
• Develop recommendations for strengthening the program’s M&E systems.
Approach
This project illustrates Khulisa’s experience in carrying out Data quality Audits in multiple African countries and across multiple implementation organizations. In order to meet the objectives of the verification exercise, the Khulisa team conducted data quality audits at the Programs partner offices, i.e. Marie Stopes International, Population Services International (PSI) and the International Planned Parenthood Federation (IPPF) plus the facilities that they support.
Over the two rounds of assessments, Khulisa carried out the following activities:
• Conducted 262 site visits: 120 and 142 visits during the first and second rounds respectively.
• Reviewed 5 indicators per round, and interviewed over 286 people (First round over 150, second round 136).
• Assessed the data management systems at site and partner country office levels.
• Verified program data for each indicator at each site.
• Calculated performance scores for each country and partner (round 1 only).
• Documented strengths and weaknesses of each indicator’s data management system.
• Provided recommendations on the improvement of data quality in the 4 countries by partner and indicator.
• Trained CCS&PT partners in conducting Data Verification (round 2 only)
Impact:
The the verification exercise allowed Khulisa to provide each country program with feedback on their Data Verification Factor (DVF) by channel and indicator. It was explained to each country program that any DVF below 1 represents “over-reporting” and any DVF above 1 represents “Under reporting.”
The Khulisa team was also able provide to country office teams was feedback on the Data Management Systems for the following five functional areas:
I. M&E Structures, Functions and Responsibilities
II. Indicator Definitions
III. Data Collection/Reporting Forms and Tools
IV. Data Management Processes
V. Links with National Reporting Systems
The team was also able to provide significant points requiring improvements or those that have been improved within each of the above five functional areas for each of the country teams
Overall, the CCS&PT partners found the data verification exercise very useful for strengthening their M&E systems and reporting on CCS&PT services delivered.