Evaluation as Protest

Ahmaud Arbery. Sean Reed. Breonna Taylor. Tony McDade. George Floyd. When does this racial terror end?

For the past three months, our communities have been inundated with painful stories and images of Black people targeted, harassed, arrested, and killed by police and racist vigilantes. As Black people all across the country grapple with the aftermath of an unprecedented global pandemic, somehow Black people also have to figure how to stay alive while jogging, buying groceries, and yes — even while sleeping.

Color of Change – State of Emergency: End the War on Black people!

It should never have taken this amount of publicly shared violence to engage white people like me in active anti-racism. I appropriately sit with that discomfort thinking of moments across my life where I could have, and should have, said something or done something to show opposition.

I know I am not alone, and many of my dear readers are thinking the same. I’ve spent the last week signing petitions, following up with friends, donating to anti-racist causes, and calling out racist posts from extended family and childhood friends. Going to facebook for me is like visiting my hometown and feeling uncomfortable in the overwhelming whiteness. I hate it, but burying it deep within and pretending it’s not a part of me won’t make it go away.

<Someone’s comment on my comment> Calling someone or something racist, is the last resort of an idiot.”

<My response> in my defense, it was my first resort to call the string of racist memes racist.”

Excerpt from a recent comment conversation I had on Facebook. Basically I called a cousin out on a string of racist memes he was sharing, then had an opportunity to exchange pleasantries with him and his friends…

But as my wife and I sat discussing what we could do to be better allies and accomplices, I started to think about how we could embrace our evaluation skill-set to support anti-racism efforts in a systematic and practical way.

So that’s what this post is. It is a brainstorm on action-oriented contributions. It’s not complete, so if you would like to add your own thoughts, please do.

I offer these observations as one human clinging onto her hopes for humanity.

I invite each of us to remember and unleash our humanity in the work we need to do to make us worthy of this planet and each other.

My simmering rage both fuels and exhausts me. It keeps me alive with a deeper level of understanding in my body and in my soul not just in my head of what is at stake: Everything.

Jara Dean-Coffey from her post on the Equitable Evaluation Initiative website: Embodied Knowledge: Simmering Rage | A note from Luminare Group Founder & EEI Director

To my BIPOC readers, colleagues, and friends.

I pledge to be a supportive ally and accomplice.

Your life matters. Your family’s lives matters. Your friend’s lives matter.

This is no little problem that will away go away when things “return to normal.” The evidence of inequality between BIPOC and white members of this country is pervasive a deep.

I am nowhere near perfect, but I will always listen. Even if the conversation is hard.

Reminder: check on your Black friends, family, colleagues, and students. We are not ok. A few have asked me what could/should this look like? Be honest, be vulnerable, be compassionate. Here are screen shots of an email I sent to my students and colleagues of African descent. pic.twitter.com/KVVJayflDi

— Dr. Ayesha Boyce (@AyeshaBoyce) June 1, 2020

My digital office door is always open. I usually say that I offer one 30 minute consultation for free. But that’s pretty much bunk, we can chat longer and it could be more sessions.

In other words, if you ever want to chat, I’m here. And I’m not going to try to sell you on anything. But I can make a pretty good thought partner.

Things I’m particularly good at for thought partnering:

• Coming up with digital strategies.
• Designing digital training programs.
• Evaluating digital efforts.
• Actionable data reporting.

Breaking the Cycle

It’s not a new problem.

It’s just being caught on tape.

Breaking the cycle starts with understanding that a cycle exists.

What is James Baldwin thinking more than 50 years after this interview, which could be written today? (You will notice that nothing I post anywhere in my own words started with me–it is common knowledge among BIPOC thinkers and writers and artists. White investment in individualism means that white leadership always tries to individuate themselves from these repeated patterns of behavior–they are somehow different–even when they demonstrate the documented behavior.)

A post from Vidhya Shanker on LinkedIn.

Working within the Evaluation Guiding Principles

You want to know something neat about being evaluator. Most of the time we work for a client who is paying us to evaluate a program or activity. Or we work for an agency that is paying us to evaluate a whole set of programs or activities. The majority of the time, we receive compensation for our work.

But if you read our guiding principles, you’ll see very little specifically connected to how we serve our bill-paying clients. Our client is only one part of the broader stakeholder groups we serve.

You don’t need a client to evaluate a program and deliver professional services. You just need stakeholders.

So why not use our skills as evaluators to systematically evaluate programs as we strive to contribute to the common good, regardless of whether or not the targeted programs have requested our services.

We just need stakeholders.

Short Version of the AEA Guiding Principles

A. Systematic Inquiry: Evaluators conduct data-based inquiries that are thorough, methodical, and contextually relevant.

B. Competence: Evaluators provide skilled professional services to stakeholders.

C. Integrity: Evaluators behave with honesty and transparency in order to ensure the integrity of the evaluation.

D. Respect for People: Evaluators honor the dignity, well-being, and self-worth of individuals and acknowledge the influence of culture within and across groups.

E. Common Good and Equity: Evaluators strive to contribute to the common good and advancement of an equitable and just society.

Stakeholders – individuals, groups, or organizations served by, or with a legitimate interest in, an evaluation including those who might be affected by an evaluation.

Sharing Evidence to Guide Practice

I get asked a lot for best practices in evidence sharing. Organizations that do a good job of sharing data in an actionable format.

Campaign ZERO’s website is that (JoinCampaignZERO.org). This is information design at its best. If you have data and evidence to share that you hope could become actionable, I suggest treating this site as a blueprint for how it’s done.

My favorite page on the site is the solutions page. Here is how it’s structured.

Campaign ZERO was developed with contributions from activists, protesters and researchers across the nation. This data-informed platform presents comprehensive solutions to end police violence in America. It integrates community demands and policy recommendations from research organizations and President Obama’s Task Force on 21st Century Policing. Together, we will win.

The Campaign Zero Planning Team (a.k.a. amazing people you should really follow on Twitter):
@deray, 33, is a protestor, dedicated to ending police and state violence.
@samswey, 29, is data scientist who leads the development of research, digital tools and platforms to end police violence and systemic racism in America.
@MsPackyetti, 33, is a St. Louis native raised in a tradition of social justice.

Localizing Critical Data

I want to mention two more sites from the same team who brought us Campaign Zero.

The first is MappingPoliceViolence.org

This site uses secondary sources to map police violence across the country. The data combines products of crowd-sourced sources with original research to paint a picture not currently shared by law enforcement agencies. The methodology is fully transparent and the full database is available for download.

The second is PoliceScorecard.org

The Police Scorecard provides a deep dive into the actions and policies for police forces in California. The score card, which grades individual police departments, is systematic and transparent in its evaluative approach. The overall mission is to expand the work nationwide.

Crime data is collected and reported much in the same way as it has for the last two decades. Seriously, not an overstatement. In the year 2000 I was a criminology major in undergrad. Looking at the Uniform Crime Reporting data and the National Victimization Crime Survey data creates flashbacks to my college days.

Sites like Mapping Police Violence exist because our current national data systems do not answer important evaluative questions in a meaningful way (if at all).

Ask some questions and try to find the answers. You shouldn’t need a Masters and two decades of data experience to find them. We also can’t wait for the bureaucracy to get their data act together.

Start local. Find the answers. Share the answers.

Break down the cryptic national data and put it into local context. Then working with social justice organizations in your area, share the data in a digestible format. Here are some examples of questions that should be easy to answer, but often are not.

• What police departments exist in your area?
• Is the police department demographically representative of the local area?
• Has the police department been involved in a shooting?
• Is there evidence that suggests racial profiling in arrests?

Adapting Data for Social Sharing

Lots of potentially useful data gets buried in poorly designed public websites and warehouses.

But if you are the kind of person who can adapt this information into useful guidance, it’s worth time and effort to explore these public sources. Yes, it would be great if our public organizations prioritized useful dissemination efforts. But we can’t wait for that happen.

Here is one way you can take data you find and adapt it into a useful format.

• Think of an audience you would like to reach. For example, a local mayor’s office or county executive.
• Think of a finding you believe might influence or move them to action.
• Find a simple template in Canva for a social media post or presentation.
• Create an image that shares data for that specific audience. Annotate that data, mention the audience and others who you believe could help the chart reach its intended target.

Rubric Supported Digital Content Analysis

In the modern era, websites and social media streams are the digital manifestations of a police department’s strategy. You “should” be able to find quite a bit about the policies and overall composition of an agency through reviews of publicly available information.

A scroll through a website is not a bad place to start deciphering a department’s strategy. But as an evaluator, you can do it systematically.

Create a basic rubric. It could be as simple as a list of questions you have about the department’s policies. (Such as, does this police department employ the use of body cameras or does it train on de-escalation techniques?). Then systematically go question to question, sourcing the specific answers as you go. Also rate based on the completeness of the answers (such as answered, partially answered, incomprehensible, or no information provided).

You could then take a further step, sharing your findings with the police department and asking them to fill in the blanks. Or you could report as it is, since complete or not this is the public face of the department.

Wondering what you might want to review? Building off the work of others is always the best place to start when available. Again I will advocate for Campaign ZERO.

Modeling Systems of Oppression and White Supremacy

Evaluators love logic models.

But for the most part we tend to develop models based on program activities designed to bring positive change.

But what if you believe that the system is not broken and in need of a fix. It’s working exactly as designed (to help the rich grow richer and maintain white supremacy).

Therefore the working systems of oppression need to be broken.

Use the skills you have to model white supremacy. Instead of creating a theory of change, develop a theory of oppression. Understanding how our systems and societal structures perpetuate inequality and maintain white supremacy is a step towards breaking the harmful causal mechanisms.

Systematically Documenting Community Stakeholder Experience

The world is complex.

One way to jump into that is to develop super sophisticated, and often utterly confusing, diagrams or algorithms in an attempt to map out our analyze through the complexity.

But an easier way I find is to put a focus on the stakeholders of interest. Tell the story of a person showing how they navigate a system you are trying to change. Then do it over and over, with different people.

Be intentional, and systematic. Document your approach and your methods.

For instance, what if you started with a black male high school sophomore with college aspirations. Given his age and home life, start pulling together the descriptive data on outcomes and challenges.

Think through current events. What is the likelihood that he would join his peers in a local peaceful protest near a downtown church? Then maybe face tear gas and rubber bullets as the police attempt to clear the path for a presidential photo opportunity.

Human lives are complex. Stories are data.

Single stories can be powerful. Collections of stories built systematically over time integrating numbers, images, and video can be hard to ignore.

With cell phones everywhere, more individual stories are coming to light. Reporters report and move on, it’s the way their field is designed. But as an evaluator, you can collect and synthesize.

Evidence Supported Political Activism

You can, and should, be in touch with your political representatives.

Let them know what you think.

But if you want to strengthen your argument. Bring data to your calls, to your emails, to your tweets, and your comments.

What other ways do you think we can leverage our skills as evaluators to support anti-racist actions?

This was all just a brainstorm. There is so much more we can do. Would love to hear your thoughts.