Friday, October 3, 2008

Faculty assessment

My institution is going through a big push to structur-ize and formalize many bits of language surrounding an overall "strategic plan." Since this is going on in layers at all levels of the organization, part of that means our department needs to participate. This is coming from the Dean, who got the directive from the President. So in other words, we gotta do it. We (with me coming in at the end of the process here) have been working to satisfy their requirements for a "Faculty Assessment System," which needs to include a portfolio-based reporting system for each individual's progress as well as a rating system of some kind to quantify whether people are meeting the "minimum" requirements. It makes for some long, contentious debates in faculty meetings. It also eventually comes down to a discussion around part of the fundamental philosophy of academia: how do you quantify the multidimensional achievements of someone who contributes to a huge range of essential activities in any meaningful way? What makes someone in a faculty position "a success" or at least "satisfying the minumum requirements" in a way that you can have in a spreadsheet when it comes to assess everyone at the end of the year?

Clearly it is nearly impossible to quantify. It's nearly impossible to even categorize thoroughly. There is a natural, instinctive resistance to the idea of quantifying or dissecting it from the more experienced faculty who know full well how differently each person brings their contributions to the table in a good department. On the other hand, those of us at the beginning (especially us supernoobs) kind of like, at some level, the idea of having a framework within which we are explicitly expected to operate. Rather than all the vague handwaving about having "enough" papers or "enough" grant money or "enough" teaching contribution that usually makes the tenure process so hard to navigate, we liked the concept of having some formalization to the assessment process... everybody's main problem (ours too) was with how to QUANTIFY that as a set of metrics.

So how could the professorial animal be metric-ified? We have a few computer science/biomodeling types in our department, maybe somebody could devise an extremely complex algorithm that could appropriately score and assign weight to all the various components of a successful faculty career, in some kind of Venn diagram shape so that if you contributed particularly strongly in one category you would balance your deficiencies in others... create the mathematical representation of ACADEMIC SUCCESS, the Plato "Idea Professor" that could be born in each of us as either a functional or dysfunctional Dr. Horse. That we could dutifully use to plug in all of our parameters, it could chug away on a Linux box somewhere for a few weeks, and spit back out to us a quantification of our own personal FIT to the model.

But it would still be just that: a model. Models never fully represent their living instances. They never take all the parameters into account. They sometimes show false positives, false negatives, and always come down to a bottom line that cannot fully describe the thing they are trying to assess because of emergent behaviors that no one was expecting when they created the model. So we would have to get to the point where we incorporate some kind of machine learning into the model, so it can adjust and change its mind and take into consideration other factors that were not known at the beginning and so on and so on...

And by then, we might as well just be a group of people with expertise in the matter thinking deeply, talking thoroughly and deciding upon it.

Which is great as long as you are in a department with people you trust, who you know have everyone's best interests in mind, and who have a track record of helping each other hold down the fort. That's the way my department is now. I just can't guarantee it will stay that way over the years, and so I still like the idea of having something formalized in a document (even if the complicated algorithm part would never take off).

No comments: