Today we have a guest post from Director of Idaho’s Legislative Office of Performance Evaluations and all around great guy Rakesh Mohan. The post is his, I just added the cartoons.
Do you know what happens when I tell someone that I’m an evaluator? I get a blank look, because people do not know what an evaluator does.
So I say that I’m a program evaluator. And they think I’m a computer programmer. Of course, it does not help that I look like and talk like many computer programmers who are in this country from India.
How can we as evaluators influence change if people don’t even know who we are?
Teachers, accountants, physicians, engineers, and social workers do not need to explain to anyone what their professions are all about – but we evaluators do.
The reason people do not know who we are is because we evaluators generally talk only among ourselves at evaluation conferences. This is certainly true for AEA conference, which I have attended every year since 1994.
At AEA conferences, we generally have 4 missing Ps: discussion about politics, and the presence of policymakers and the press that can carry our message to the public.
Basically we don’t include folks who are likely to use our work (policymakers and the press) and those who would be affected by our work if it is used (the public).
Many public policies are made inside the US Capitol, state capitols, and parliaments around the world.
These buildings are buzzing with political activities involving policymakers, the press, and ordinary folks representing the public.
Many evaluators seemed to be afraid of getting close to the politics. They are afraid that they may lose their credibility as independent evaluators if they come in contact with policymakers and their politics.
But these evaluators forget that evaluation is inherently political because it involves making judgment about prioritization, distribution, and use of resources.
As my mother used to say, “Wherever there are two or more people, there will always be politics over resources.” So instead of avoiding politics or getting bogged down in politics, we have to manage it.
Guess what? Evaluators are rarely walking the halls of those buildings where public policies are being made and influenced. Instead, they are hanging out by themselves in big hotels attending evaluation conferences.
In other words, we evaluators are not in the same building, let alone in the same room, where there is action. We need to get the ring-side seats in order to influence public policymaking.
I believe many of you would love to see your evaluation reports used for improving public policies and programs.
If that’s true, then we all have to work together to build a road that connects the Capitol and the conference hotel.
And the road would have to be travelled in both directions.
Call to Action
- What kind of reaction do you get when you tell someone that you are an evaluator?
- Have you worked with a policymaker? If yes, what did you learn from that experience? If no, why not?
- Have you worked with the press? If yes, what kind of experience did you have? If no, why not?
- Do you think evaluations are inherently political? If yes, how do you understand and manage the politics of evaluation without getting involved in it?
- What should AEA and other evaluation organizations do to integrate the 4 Ps in evaluation discussions?
Great question! I tend not to use the word “evaluator” when describing what I do anymore, unless I’m talking with people who are in the world of evaluation who I know are familiar with our language. When I did tell people I was an evaluator, I got a similar reaction that you got — people would have no idea what I was talking about or would have a very different idea of what an “evaluator” is. Instead, I find it more effective to describe who we help and what problems we help them to solve. Something like, “I help non-profits to demonstrate the value and effectiveness of their programs and services, so they can use that information to increase their success in achieving their missions.” Then if the person asks how I do that, I’ll give more details.
Thank you for the nice comment, Bernadette. For my work, I do the same as you do. I tell them that I evaluate state policies and programs for efficiency, effectiveness, and compliance. In other words, I help policymakers and state officials improve public policies, programs, and services.
However, wouldn’t be nice for the general public to know who we are and what we do in one or two words like artist, musician, teacher, counselor, social worker, accountant, physician, nurse, engineer, and computer programmer? Best wishes.
Here is a cartoon response to your first question!
https://sandyontheside.wordpress.com/
I like it Sandy 🙂
Nice visual reply to Q. 1. I love the cartoon. Thank you, Sandy.
I know this is counter your point Rakesh. But I kind of like when people don’t know what evaluator means. It opens up a conversation and I get to help create their definition based on their circumstances.
Chris, this is an excellent way to deal with the situation. At the same time, I believe it would be useful for the development of evaluation as a profession if evaluation is a household name, or at least, decision-makers/policymakers know who we are.
Most people have done and are doing evaluations in their personal or in their working places without knowing. Look at some of the internal quarterly or annual reporting processes of some programs or companies. The challenge is that the word “evaluation” or “evaluator” has been complicated by evaluators themselves – intentionally or unintentionally.
Thanks for commenting on the post, Emmanuel. I would appreciate it if you could please elaborate on your last sentence. A couple of examples would help me understand how evaluators themselves have complicated the word “evaluation” or “evaluator.” Best wishes.
Rakesh, I’m sure you are familiar with Evaluators Visit Capitol Hill (EVCH), a Washington Evaluators (WE) initiative. WE, the DC-based AEA affiliate, with the support of AEA and the Evaluation Policy Task Force on which you now serve, developed this initiative for the 2013 AEA Conference. During the conference (and yes, during the federal government shutdown), WE, under the leadership of WE President Brian Yoder, hosted EVCH to promote the role of evaluation and evaluators to policymakers (one of the 4 Ps). AEA members visited members of their Senate and House delegations and staff to drop off EPTF-provided policy documents related to the role of evaluation in government with organizational support from WE. A total of 69 participants from 31 states and the District of Columbia participated in an initial conference call. Eighteen participants visited twenty-one different congressional offices, according to a post-meeting survey. One-third of the congressional offices visited by an AEA member said they were interested in receiving additional materials from EPTF (Source: Evaluators Visit Capitol Hill Report to the WE Membership). In an effort to stay neutral, the visits were informative, not blatantly political, so I would add a fifth P: professional. While never missing in the conversation among evaluators (see the AEA Guiding Principles, especially Principle B on Competence at http://www.eval.org/p/cm/ld/fid=51), we seldom take our professional message out quite so publicly. To promote the role that evaluation plays in being responsible for the “General and Public Welfare,” (see AEA Guiding Principle E), perhaps we need to be a bit more public. The AEA Conference will be in DC again in 2017 (and every 4 years, per AEA Board policy). With support of the EPTF, maybe we can do it again, and reach a broader audience of elected officials and Congressional staff.
Thank you so much for posting your comment, David. What Washington Evaluators, along with AEA’s Evaluation Policy Task Force (EPTF), did in DC in 2013 was absolutely amazing.
Also, I believe EPTF is a significant step toward engaging policymakers and their staff with what we do as evaluators.
Eventually I would love to see every AEA local affiliate follow the lead of Washington Evaluators and make efforts to engage their city, county, and state policymakers and their staff. That would be cool. Then, it would be pretty damn difficult to confuse a program evaluator with a computer programmer. 🙂