The Perils of Doing Evaluation

One of the things I love to do is program evaluation. I really like evaluation because through it I am helping community organizations clarify their goals and objectives and making the case for why their work is important. I always approach evaluation in this way--as a positive process that can translate into a better program and more funding.

But, it isn't always a positive experience for me. Oftentimes, evaluators are viewed as threatening--because evaluation is so often tied to funding. This is especially true when a funding agency hires me to "go in" and do an evaluation. I am often viewed in these cases as the enemy.

At other times, the process of collecting data for an evaluation goes very smoothly. It is only after a report is submitted that sparks fly. Example: I was hired to do an evaluation of a program, and had done so since the program was founded--about 5 years. The program got moved to a new department at the organization. The new supervisor of the program didn't like the content of one of the reports that I submitted. So, that person didn't renew my contract. As the person put it, "We terminated your contract because your report was unprofessional."

Here's the rub: this person has labeled me as unprofessional (whatever that means), is telling people that I don't know what I am doing, and that my work is so poor that the funding agency told them to hire a new evaluator! All of which is untrue.

I expect that people might not be happy about the results of an evaluation or might not want to cooperate because there is something they find threatening about the process. But I didn't think that this person would be so disgruntled that they would out and out lie about me!!

So, how do I deal with the fact that this person is sullying my reputation as an evaluator? How do you do a good job as an evaluator, but at the same time not piss people off? You probably can't. I probably can't--and that is the risk of doing a good job. But how am I supposed to deal with the fallout from this person's comments? This person is in a high level position. Even though they have a reputation for not working well with other people, this person has influence. How do I defend my record as an evaluator?

And what lessons do I take from this into the next job I get as an evaluator?

I suppose lesson one is that evaluation is a political process. I don't know if understanding this would have helped me in the current case, but it is something that I didn't really understand until this happened. People often want evaluators to only report what they want to hear--the good stuff. So, some negotiation of difficult terrain is often necessary.

The other lesson I've learned is that it is really important to document everything. I tend to do that anyway, but that has really hit home for me. I saved every e-mail that I sent this person and documented phone calls and conversations that I had with this person, even though they were very limited at the beginning of the year, and then non-existent as it got closer to the time when my report was due to the funder. This person did not return repeated requests for a meeting time to get their feedback on what other people involved with the program were telling me. Since this person claims that I didn't talk to the "right people," I have proof that the attempt was made to do so.

I think the other lesson is that I should use the weight of the funder when necessary to force cooperation. When my request for a meeting was repeatedly ignored, I should have talked to the funder and explained what was happening. At least in this case, the funder would know up front that there was a problem.

Finally, I really appreciate that I have mentors who do this kind of work, have lots more experience than me, and are willing to discuss issues like this with me. I hope to talk with them in the next few days about how to navigate the situation I've found myself in.

I post an update after that meeting.

AAMC Minority Faculty Career Development Seminar

I just got back from a meeting in San Juan: the Association of American Medical Colleges Minority Faculty Career Development Seminar. If you are in any area of academic medicine, and are underrepresented, you really need to attend this meeting! Check out their meeting site here.

I had three goals for the meeting. I think I met them all, so I am proud of that. I’ve listed them below, along with what I learned and what my next steps will be.

Goal 1: Getting More Specific in my Research Planning

Before the meeting, I had begun developing a research plan. It was really not concrete enough, though. I had ideas about things I was interested in, but hadn’t really thought enough about how to turn those ideas into projects and actually map them onto a 3-5 year plan.

The meeting did two things for me. First, it really drove home the need to make this a priority. In one of the sessions, Dr. Cora-Bramble shared her experience putting together her portfolio for promotion to full professor. Another speaker, Dr. South-Paul, shared her experiences as a chair of a department, working with faculty who wanted to go up for promotion. If I am going to get through the tenure and promotion process successfully, I need a concrete and specific plan so that I can “make the compelling case.” My portfolio needs to be organized in such a way that there is no question that I should be promoted.

The second thing I learned related to this goal is that my plan needs to be realistic, and include a consideration of my own “social context.” Duh!! I am a sociologist! I study social context! This should have been obvious, but it wasn’t! One speaker said, “What you can do now is limited by your social context—your family needs, your current training, your experiences.” So, my plan needs to include an examination of those things and make room for additional training, or time for major life events—like the fact that my daughter just started her senior year!!

Goal 2: Demystifying the Tenure Process

This goal is related to the fact that at my institution, the tenure and promotions process is currently being changed. In addition, we have a fairly new chair and a newly merged department. Even if that was not true, getting straight answers about the process is darn near impossible!

So, I needed some advice on how to get the answers I need. At the meeting, I got some really practical advice on the things to include in my portfolio, with the caveat that I needed to know what was required at my own institution. I was also told to assume that those reviewing my portfolio wouldn’t understand or be familiar with my work. So, it is my job to provide enough information that I demonstrate what I do and why it is important.

Dr. Corbie-Smith suggested asking the chair of the tenure and promotion committee to give me information on the average number of publications and average amount and sources of grant funding obtained by folks who were promoted to each rank. I should also ask to see successful portfolios.

Goal 3: Meet other URM Faculty

This goal was simple, but important, as I am one of 2 URM faculty at my institution. There weren’t as many opportunities to really talk with people, but did I meet a lot of people. We are in similar situations, dealing with similar issues. I really connected with a handful that I’d love to stay in contact with.

Next steps

• I am going to spend part of every day working on my research plan.
• I will schedule a meeting with the faculty professional development person at my institution to start working on my portfolio.
• I will take a colleague who is submitting her portfolio this fall out to lunch to talk about her experience and get advice on how to plan for my own review. I will also ask if she is willing to share her portfolio with me.
• I will ask the chair of my tenure development committee to help me stay up-to-date on the process changes that are being discussed.
• I will send an e-mail to the folks I connected with at the meeting and commit to staying in touch with them. I will also commit to attending the reunion meeting in 2011.

My sister, Dr. Locs, met me there, so we had some time together with no husbands, kids or whatever pressing on us. It was a great time of renewal! She also really helped me think through all the stuff I was learning. I think that is why I was able to meet my meeting goals. Next year, she will be registered as a participant and I will be her guest!!

Are you a "strong black woman?"

I have decided I am not. Or, rather, that I won't be any more. It is too hard.

I have had a rough couple of weeks on the faculty front. I am really struggling with whether all the stress is worth it. I feel like I can never stop working. That has been especially hard to deal with the last month or so because my daughter ended up in the psychiatric ward at the local children's hospital, followed by two weeks of a partial hospitalization program. Needless to say, I've been absent from my office a lot the past month and not working as much as I should be.

One of the things I am most interested in is mental health. Specifically, how being a parent, employee and partner/spouse affects our mental health. Black women have always done it all, but at what price? What is up with the "strong black woman" stereotype? I think that is what led to my daughter's hospitalization--she didn't want to share her feelings or cry or let people know how angry she was. The fact that she never sees me do those things didn't help.

We (black women) tend to wear our strength like a mantle. We wrap it around ourselves and hold on tight. We roll our eyes at the drama queens, wondering why they just can't learn to "handle their business." We don't talk about mental illness, even though too many of us know someone who has "bad nerves" or is "just crazy" or has committed suicide.

And by doing so, give ourselves the burden of not being able to expose the anger, pain, disillusionment, hurt without feeling like we are not being who we are at our core--black women who have struggled and overcome, continue to struggle and overcome.

At least that is how I feel.

But going through this experience with my daughter has shown me that she is the strong one. She was willing to ask for help when she needed it, instead of hurting herself. And she is learning that it is much harder to let all those feelings out than to act like nothing bothers you and everything is okay.

Now it is my turn.

March 2009 Re-Tightening

Here are some pics of my re-tightening, taken on Monday, March 2nd. I spent part of Sunday on the re-tightening after a good wash, then braided it while it was still wet. Same process as before, except I didn't need to wash with bar soap! Still growing and still fabulous!

On the faculty front: this week has been stressful, for a lot of reasons. Why do people assume new folks know the university policies and procedures??? Through my ignorance, I am finding out that the person who really runs the department is the chair's secretary (duh, Toki, you're just figuring that out??), who questions everything I do (behind my back) and goes running to my chair when "something is amiss."

But, e-mail is a wonderful thing, used correctly. I document my conversations with the chair with a follow-up e-mail, or make my requests for things via e-mail so I have a written response from her. So, the secretary's attempts at sabotage have failed. Still, it's an almost daily hassle that I wish I didn't have to mess with.

I've decided that I need to write a paper on the mental health of URM faculty, because mine is suffering.