Respect and care for our partners were top of mind as we conducted our Triennial Impact Study

Data plays a major role in Plant With Purpose’s overall operations, and that’s true for our monitoring and evaluation as well. It's not enough to go into communities with good ideas about how transformation can take place – it's about being able to verify the real impact as a result of our interventions. Meticulously collecting information is a form of listening to our partners and learning from their experiences. This allows us to see how their work is truly helping communities. This data allows them to learn, improve, and expand their reach. Imagine doubling the number of people they serve – that's the power of data guiding Plant With Purpose's mission.

Recently, Plant With Purpose completed its Triennial Impact Report, an in-depth study that compares the results in our target watershed over three years against comparable populations, to measure the impact of our program and your support. Conducted once every three years, it is an intense and rigorous process. We work with our local teams to partner with survey administrators, who meet with and collect data from our program participants. The effort is worth it, as it fuels the ongoing improvement of our program.

As we collect data, we also want to be sensitive to ethical considerations. If it is not done carefully, the way a person collects information can create unintended harm, alter the results of their study, or present a burden to the participant. This is why our Plant With Purpose Innovation Lab has carefully designed a process of collecting data that incorporates a wide variety of ethical considerations. Here is a summary of some of our principles behind ethical data collection:

1) Create psychological safety for respondents

Working with people, especially across cultures, requires fostering psychological safety during surveys. This means creating an environment where participants feel comfortable expressing honest opinions, even if sensitive or critical. Fear of judgment or social repercussions can lead to biased responses. By emphasizing confidentiality, and a non-judgmental approach, researchers build trust. This allows participants to provide valuable, nuanced data that reflects their true experiences and perspectives, leading to more accurate and culturally relevant research findings.

One of the ways we secure this psychological safety is through using external agents to conduct surveys. The relationships formed between participants and our local Plant With Purpose staff are some of the most important components of our entire program. However, it is more difficult to create an environment where a person can express any opinion when responding to somebody they have a personal relationship with. By hiring external survey takers for this process, we aim to create that space.

2) Formulate questions with care

Truthful answers are the backbone of strong surveys. Wording that triggers response bias, where people answer based on wanting to appear a certain way, can skew results. We write questions in the best way we can to avoid leading questions or using hyperbolic language. 

Clear, neutral wording that focuses on the specific information we seek is key to unlocking truthful responses and reliable data.

While this may seem straightforward enough, in practice, it can be difficult to word a question or phrase neutrally. The wordsmithing of questions is given special attention and effort and is done in a team setting to reduce individual bias.

3) We collect data from families and communities where we work, or where we intend to work in the future

Keeping our survey study confined to our program watersheds and their comparison counterparts is important for keeping our data collection focused on what is relevant to our program. Our awareness of the time and opportunity costs of conducting a survey means that we do not want to ask that of a population that does not ultimately have the opportunity to benefit from the knowledge gained.

We also establish data collection from watersheds where we do not yet have a programmatic presence. This is to establish a comparison population. When we can see the difference between a Plant With Purpose watershed and one where we haven’t worked, it allows us to be more confident that the difference was a result of your support. A comparison watershed is nearly identical in other factors: population, culture, economics, and environmental factors.

However, we hope that the comparison watershed also has the potential to benefit from the study and our work. Comparison watersheds are typically watersheds that we have also identified as areas to expand our work as a country program continues to grow. The data we will have already collected gives us insight as to the condition of the watershed at the time Purpose Groups are introduced.

4) Be conscious of people’s time

In high-poverty areas, time is a precious resource. As we survey populations, we are sensitive to this. Keep surveys concise and focused, minimizing the burden on participants. Respectful time management demonstrates empathy and increases participation, leading to more robust data collection that accurately reflects the experiences of hard-working communities.

While our survey aims for depth, it also seeks to keep its duration under 20 minutes. Each question has been carefully considered for its utility.

5) Make sure survey participants know how their answers will be used (Informed consent)

Ethical surveys hinge on clear informed consent, and when conducting surveys in cross-cultural settings informed consent must be handled with special care. 

Participants should know who's behind the study. Explaining the survey's purpose in clear, concise language, avoiding jargon, is crucial. Participants must understand how the data will be used and protected. This empowers participants to make informed decisions about contributing to valuable cross-cultural research.

6) Use participatory methods to encourage participants to provide feedback

Rigid quantitative surveys don’t always capture everything going on in a rural village. Language barriers, access to literacy, and social dynamics may affect participation or true expression. Participatory methods bridge this gap. 

This sometimes looks like villagers using maps, drawings, or even drama to show their needs. A common approach in Plant With Purpose communities is using lentils or coffee beans as tallies to identify areas of interest or concern. These techniques can help create more opportunities for marginalized voices to be heard. Open discussions and group activities create a safe space for honest feedback, fostering a sense of ownership. By incorporating these methods, surveys become a two-way street, capturing rich data that reflects the community's unique perspectives and priorities.

7) Use random sampling to select households for participation

Random sampling helps to gather results more reflective of the population as a whole. In rural surveys, random sampling in the data collection process is the key to unbiased results. It ensures every household has an equal chance of being chosen, regardless of location, wealth, or social standing. This prevents skewed data that might only reflect the experiences of a specific group. Imagine a survey focused on farming practices, but only interviewing households near the main road. Random sampling avoids this, capturing the diverse realities of the entire village, from those near the center to those on outlying farms. This unbiased data paints a clearer picture of the community's needs and challenges, leading to more effective solutions.

It is important to note that we do not financially incentivize people to participate in the study, and we allow people to decline participation. While financial incentives in surveys may boost participation, they can also introduce bias. Respondents motivated by money might rush through, giving less thoughtful answers. Worse, some may even fabricate responses to secure a reward, skewing the data. The incentive's size matters too. Offering too much can attract people who prioritize money over the survey's topic, leading to unrepresentative results. For rural surveys, even small cash rewards might disproportionately attract those with greater financial need, altering the data's accurate reflection of the community. Plant With Purpose avoids using financial rewards and instead promotes awareness of how positive actions can ultimately benefit the whole community.

By prioritizing informed consent, cultural sensitivity, and unbiased sampling, Plant With Purpose ensures that our data collection reflects the true impact of our work. This empowers us to refine our approach, maximize positive influence, and ultimately, grow a greener, more equitable future benefiting all our program communities.