Evaluation is a dangerous profession.

Evaluation is a dangerous profession.

Not dangerous as in it requires hard hats or tethers. But dangerous in that it questions realities most would rather ignore. Traditions that are so deeply entrenched within organizational structures that merely hinting at their instability can lead to alienation and professional isolation.

Our frameworks and methods exist not only to allow us to answer important questions and make proper evaluative judgements of merit, worth, and significance. They provide the important structural scaffolding we, as evaluators, require to be effective.

Society must accept that some things are real; but he must always know that visible reality hides a deeper one, and that all our action and achievement rest on things unseen. A society must assume that it is stable, but the artist must know, and he must let us know, that there is nothing stable under heaven. One cannot possibly build a school, teach a child, or drive a car without taking some things for granted. The artist cannot and must not take anything for granted, but must drive to the heart of every answer and expose the question the answer hides.

James Baldwin from The Price of the Ticket: Collected Nonfiction

The real difference between research and evaluation.

While researchers follow their intellectual curiosities in order to answer questions. Evaluators follow their own as they proceed to question answers.

For a non-profit executive director that has put their heart and soul into a mission that has become their life’s work, of course the program works. Because if it didn’t, what does mean?

For the civil servant who is sworn to protect the health and welfare of thousands, of course they are fulfilling their mission. If they do all they can and give it their best, how could they possibly be making things worse?

If the decisions made by a group of people around a table set into action teams of highly qualified professionals, the wrong decisions could waste not only the decisionmaker’s own money and time but also keep others from making a real difference elsewhere. So they just have to be the right answers. Even if they are not.

If a new political party assumes control of their government, changes to programs will occur. Because they are programs the other people put into place, therefore, they must not work.

Someone already knows the answer (that yes, the program works or no, the program does not work). Questioning those answers is the purpose of evaluation.

The entire purpose of society is to create a bulwark against the inner and outer chaos, in order to make life bearable and to keep the human race alive. And it is absolutely inevitable that when a tradition has been evolved, whatever the tradition is, the people, in general, will suppose it to have existed from before the beginning of time and will be most unwilling and indeed unable to conceive of any changes in it. They do not know how they will live without those traditions that have given them their identity. Their reaction, when it is suggested that they can or that they must, is panic… And a higher level of consciousness among the people is the only hope we have now or in the future, of minimizing damage.

James Baldwin from The Price of the Ticket: Collected Nonfiction

Uncovering inconvenient realities.

So as a data visualization designer I sometimes play a role in simplifying complexities.

But so often those complexities that evaluators want to simplify, are not the kind that need data visualization design. Because they are not complex in the way that they are hard to understand. They are complex in that they are inconvenient and call into question all sorts of other assumptions.

These types of complexities don’t need to be simplified. They just need to be communicated.

A chart can be a good medium for communicating things that are hard to hear. And it doesn’t even need to be a complex chart. It’s the reason why simple bar charts and line graphs have stood the test of time. Numbers have power, and charts can amplify that power, even when the source is suspect.

A cartoon can also be a good medium for communicating this type of complexity. The medium allows you to say things you would never put down with words on paper or even say out loud.

The crime of which you discover slowly you are guilty is not so much that you are aware, which is bad enough, but that other people see that you are and cannot bear to watch it, because it testifies to the fact that they are not. You’re bearing witness helplessly to something which everybody knows and nobody wants to face.

James Baldwin from The Cross of Redemption: Uncollected Writings

You are not alone.

To each his suff’rings: all are men,
Condemn’d alike to groan,
The tender for another’s pain;
Th’ unfeeling for his own.
Yet ah! why should they know their fate?
Since sorrow never comes too late,
And happiness too swiftly flies.
Thought would destroy their paradise.
No more; where ignorance is bliss,
‘Tis folly to be wise.

Ode on a Distant Prospect of Eton College by Thomas Gray

For a little while a few years back, I tried to pretend that I was no longer an evaluator.

I was disillusioned by the amount of money we as a society would spend collecting data and writing reports that would subsequently not be read. I began to doubt that even the best designed most useful reports would receive more than just a handful of skims.

But maybe I was expecting too much. Sometimes it only takes a few reads for a report to have an impact. Or maybe I was just working with the wrong clients. I just needed to work with the people who were open to change.

Either way, I can’t stop being an evaluator. I can’t stop questioning answers. Maybe what I say won’t be heard, but I have to try.

And from the feedback that I receive when I share my cartoons, my guess is that maybe you can relate.

Not everything that is faced can be changed. But nothing can be changed until it is faced.

James Baldwin from I Am Not Your Negro (currently streaming in the US on Amazon Prime).