by Heather Dixon and Margaret Roper

When reviewing any of the Voluntary Organizations of Professional Evaluators (VOPE) or international organizations principles for research and evaluation, terms such as “ethics”, “confidentiality”, “informed consent” and “rights” are to be found.

But what does this mean when, as evaluators, we need to adhere to data privacy and protection regulations? Are we compliant if we only consider confidentiality in our design, data collection, analysis, and writing of reports? Are our informed consent forms (and processes) sufficient?

Given country-specific and regional regulations and the consequences of data breaches – data privacy, confidentiality, and security are now more important than ever. 

Below we will share a few of our lessons from our journey over the past 12 months to ensure that we uphold, and integrate data privacy and protection requirements in our evaluations and across our organizations’ functions.

Data Privacy Lesson #1: Understand the regulations but don’t drown in them!

While the many regulations and their jargon can seem overwhelming – following some simple rules will ensure compliance.  

Because we work internationally and across legislations and regulatory bodies, Khulisa M&E activities that involve data processing are reviewed on a case-by-case basis to ensure that Khulisa abides by the provisions of the South African Protection of Personal Information Act (POPI) No. 4 of 2013, the European Union General Data Protection Regulations (GDPR) (EU Regulation 2016/679) and other data protection legislation and policies that are country, region or client-specific.

Data processing includes the collection, storing, retrieving, amending, deleting, archiving, and sharing of data. Ultimately, all of these processes need to be secure, and confidential, and people need to be aware of what their data will be used for and how – at each of these stages.

The following links, which are not exhaustive, can be applied in the provision of evaluation services:

• USA: Foundations for Evidence-Based Policymaking Act (Evidence Act) (Enacted: January 2018
• USAID: Data will adhere to Automated Directives System (ADS)-508 Privacy Program, which details USAID’s internal policies and procedures for protecting programmatic data, and specifically personally identifiable information (PII)
• Europe:  General Data Protection Regulation (GDPR) 
• OECD Development Assistance Committee (DAC)
• South Africa: Protection of Personal Information Act (POPI) (No. 4 of 2013 and promulgated on 1 July 2020)

There are differences in the details of the requirements in each of the policies. However, in summary, for evaluators, focus on the following when reviewing the relevant one for your evaluation:

1. Focus on relevant sections and actions that are needed for your evaluation data and your purposes (for example, there are lots of requirements for human resource personal information and marketing which are interesting but not core to evaluation requirements)
2. Understand the principles as they complement VOPE and good evaluation principles for practice
3. Consider the extent your evaluation upholds the rights articulated in the policy applicable for your region, and think about these across the entire evaluation data collection process
4. Focus on the procedural sections of the policy – this will guide you on what personal data you are collecting, why it is necessary for the evaluation and how you will keep it confidential, private and secure (more of this in Lessons 2, 3, 4 and 5)
5. Read through the sections about data breaches and discuss with your team on how you will handle them and what the consequences are on the evaluation participants IF a breach happens (see more on Lesson # 6)

Data Privacy Lesson #2: Collaborate on developing and refining your policy and go public

Involve both your home office and technical teams to develop the policy. Data protection is not only about the tools and the analysis. It requires:

• Information Technology (IT) to confirm data storage;
• Human resources to ensure that contracts for subcontractors and fieldworkers include signing the data protection policy and state consequences if data is breached;
• Operations to make sure paper-based records are stored securely (and where they cannot be easily breached);
• Those accountable for Contracts ensure risks are managed
• All those collecting, and working with data to understand and follow these procedures.  

Remember to take precautions if you need to share data with third parties. Also, ensure you are familiar with software protection and privacy policies.  Could your data be breached through your data collection or transcription software?

If everyone is involved in the policy development process then it is more likely to be implemented across the organization and evaluations. Once the policy is drafted, ask an external expert to review it (it could be a lawyer, a stakeholder, etc.). The policy should be shared on your website – this means anyone can access it and refer to the details.  

Keep the policy simple!

Build on the experience of others but don’t just accept the policy as it may not be relevant for your organization or the specific evaluation.

Data Privacy Lesson #3: Specify what sensitive personal information is required

Sensitive personal information may include but is not limited to, racial or ethnic background, physical or mental health information, religious beliefs, genetic or biometric data, and sexual orientation.

Khulisa will only collect sensitive information in circumstances when it is reasonably necessary for one or more of the services that we provide, functions that we carry out and that the relevant individual consents to the collection of the information, or we are otherwise required or authorized by or under law or a court/tribunal order to collect the sensitive information.

It is important to note that data protection and privacy do not only apply to sensitive information collected. These principles apply to ALL personal data (including names, and contact details) – but that additional processes may need to be followed.

The sensitivity of the personal information required will also determine if you need additional permission or to put in place additional data collection privacy measures. For example, you may need Institutional Review Board (IRB) approval (see #EvalTuesdayTip When is Ethics Approval Necessary) for health information, wellbeing status, or sensitive lived experience information. In these cases, it is critical that the tools used uphold confidentiality. This relates to all software utilized as well; including file transfer services, file conversion services, and how the tools are stored. Recently, Khulisa had to use internal resources to transcribe victim interviews as an external transcription service did not meet our IRB ethics requirements, and many online transcription services store data that would not uphold our ethical standards.

Data Privacy Lesson #4: Opt-in clause must be integrated into informed consent

The collection of personal and other data is based on informed consent from the data subject. The “Informed Consent Form”  specific to the evaluation or contract will document the relevant data processing and consent given by the individual.

The following information should be provided to individuals prior to the collection of data through informed consent forms:

• Describe the project and purpose of the activity
• Explain the rights and roles of the researcher/evaluator and the individual interviewee
• Guarantee the confidentiality and anonymity of data provided and the identity of the interviewee
• Describe what data is being collected
• Delineate why, where, how the data will be processed and used
• Clarify who is responsible for management

Internally, ensure that you put in place mechanisms that cover:

• Volume and storage mechanism (digital, paper etc.)
• Data backups (location, frequency)
• Retention schedules (how long data should be kept)

All potential evaluation participants must provide informed consent AND opt-in to participate in the evaluation and to provide minimal personal data. Ideally, this is included in the Informed Consent Form.

Critically important is that if an individual DOES NOT agree to the informed consent, they must be excluded from the data collection process.

Under a contractual requirement of an evaluation, the data may need to be uploaded to an evaluation repository.  Typically such data would be anonymized.

Some regulations also indicate that this informed consent can be revoked – and that individuals have the right to request their data be deleted at any stage in the future.  

Data Privacy Lesson #5:  Implement and monitor across the evaluation cycle

Data privacy and security are not just limited to tool development and data collection. Evaluators need to implement these practices across, and throughout the evaluation cycle.

Khulisa uses a quality assurance surveillance plan (QASP); where we describe the systematic methods used to measure performance and identify reports and resources required can ensure that data privacy elements are incorporated into each step of the evaluation.

All staff and fieldworkers that will be involved in the collection of, and handling of, data are trained on the requirements needed to ensure data privacy and protection at all stages. While some companies might outsource some aspects of data handling (such as by using cloud-based storage, or data collection services), the accountability of appropriate data handling still remains with the evaluation company.

It is the responsibility of evaluators to monitor that these protocols are being adhered to, and maintain data privacy standards throughout the evaluation process.

Data Privacy Lesson #6: Store data safely and anonymize early!

Khulisa stores, processes, and transfers data and information in the USA, South Africa, and in other countries as we operate globally and may transfer personal information to third parties for the purposes described in Khulisa’s policy.

All data is secured safely and Khulisa has taken steps to prevent the loss, damage, and unauthorized destruction of the personal information, and to prevent unlawful processing of this personal information. 

Khulisa may keep records of data for historical, statistical or research purposes as safeguards have been established to prevent use of records for any other purposes. Data may also be kept and stored securely if required by law, in accordance with the client contract or in accordance with the agreement with the subject, or if the data is stored for a longer period it will be in anonymized form.

Once the personal information is no longer needed for the specific purpose, it will be disposed of in the appropriate manner or stored.

Reminder: Only collect data that is needed for the analysis and which will answer the evaluation questions. Set up your data collection tools and processes to collect once and then, ideally, use a unique identifier across data collection points. Make sure data collection instruments and your database is secure.

# Data Privacy Lesson #7 Be prepared for when things go wrong

Fortunately, Khulisa has never had a data breach. But if we do, we have a procedure in place, staff are aware of what to do, and we will immediately notify not only anyone who is affected but also engage with our clients.

In order to prevent data breaches, Khulisa has internal reviews of projects where all ethical practices are considered. We reflect on all aspects of an evaluation, and have all instruments and reports quality assured, prior to dissemination to clients and to the public.

Further reading and additional resources

• GDPR for researchers

https://www.ukri.org/about-us/policies-standards-and-data/gdpr-and-research-an-overview-for-researchers/

• POPIA Code of Conduct for research

https://sajs.co.za/article/view/10933

• South African Data Protection Overview

https://www.dataguidance.com/notes/south-africa-data-protection-overview

• GDPR and the information lifecycle

https://www.grcworldforums.com/knowledge/gdpr-and-the-information-lifecycle/22.article