External Validation

I’ve thought a lot about the role of external letters in the tenure evaluation process. You can find mine in the Library. Like so many things, there’s the theory and then the reality. Ideally, the external letters bring a larger perspective to the internal evaluation. Not only would they provide a broader view of the candidate’s contributions to the discipline and his/her subfield, but they would protect the process from potential bias within the candidate’s home institution. Presumably, the external writers take their task seriously, reviewing the candidate’s dossier thoroughly and commenting on the impact of the candidate’s work. To increase the likelihood of thoughtful and candid evaluations, the identity of the letter writers are kept anonymous and often the candidate doesn’t see the letters.

For obvious reasons, confidentiality is good policy. For other reasons, maybe it’s not. Much of the external review process is a black box for the candidate. I suspect at most universities (as it was at AU) there are some general guidelines about choosing respected, established individuals – usually full professors – with expertise to speak authoritatively about the candidate’s scholarship. The candidate usually has some input in compiling the list of potential letter writers.

I can remember from Kentucky, and then at AU, feeling a lot of pressure to choose the right people for this list. Don’t choose people with whom you’ve worked because they won’t qualify, but choose people who know you well enough to speak with some legitimacy about your work. Don’t choose your friends, don’t choose former grad school colleagues. Do choose people who will give honest and fair assessments, but don’t choose people who are reflexively supportive or who have an axe to grind. Know them, but don’t know them too well.

Right. As if. In the end, I provided a few names of people who I thought would have read my work, and who were well-regarded and had reputations for fairness in the discipline and in my subfield. I have no idea if any of them were selected to write for me. The Dean would have added some of his own candidates to the list, probably with the input of senior colleagues in my department (although I don’t know for sure), and then made invitations to a subset of those.

And then invitations are accepted or declined. I don’t know if anyone declined to write for me. If this works anything like letters of recommendation that we write (or don’t) for students, one possible reason for not writing is that we don’t think the candidate is suitable for the position. If so, then what does it mean if a candidate gets all – or mostly – positive letters? Is it like when all the kids on a team get trophies? The kids feel really good, and so do the coaches who give out the trophies, but how do we assess the value of the trophy when everyone gets them? How do we assess the value of positive letters, especially if we expect that there will only be positive letters?

Assuming that the dean or the chair (or other administrator) has done his/her job properly and chosen letter writers based on excellence, should all positive letters be taken at face value as honest evaluations of the candidate’s merit? Or, is this considered an indication of the pressure to provide supportive evaluations when called to pass judgment on a colleague?

And what if the letters are mixed? Is there a weight assigned to positive and negative letters? How is the determination made? At least in my discipline, we are social scientists accustomed to operationalizing and measuring our concepts. How do we do that in the context of external reviews for tenure?

I don’t know how many letters were solicited by the Dean, and I don’t know from whom. I don’t know what universities they were at or what standards they used in evaluating me. All I know is that five letters were sent back to the Dean with evaluations of my dossier. Four praised my contributions to the field and supported tenure. The other (Reviewer #2) was less sanguine, focusing largely on the quantitative indicators of my productivity (number and placement of peer reviewed articles). Still, the reviewer acknowledged the breadth of my full record and the different standards that different programs have for evaluating candidates for tenure. Overall, I thought the reviews were positive.

I did wonder about the impact of Reviewer #2’s evaluation. But my colleagues in the department and the school had evaluated my progress several times during the probationary period, and had been enthusiastic about my work and contributions to the program and the discipline.

So, with these letters, I submitted my dossier for consideration in early November 2009. Two weeks later, after a meeting of the senior faculty in the department, the chair darkened my doorway with the news that he’d decided I was unworthy of tenure at AU.