Why academic evaluation is wrong… although it isn’t
A brief look to any of the mainstream educational systems around the world allows us to acknowledge one undeniable fact: we have a strong tendency towards evaluation. We are in love with and strongly depend on it. We evaluate students, of course. But we also evaluate teachers, schools, school districts or regions, educational systems. We even evaluate whole countries. Take OCDE’s PISA evaluation, for example. Done world-wide, it determines the skill level a country has on abilities such as critical reading and math, and it ranks the country according to its results. Many countries use these annual results (among other indicators) to help measure the level of success or failure achieved by any educational reforms, and to draft short or long term plans to improve the nation’s educational system and outcome. It is obvious our societies grant an essential role to evaluation when analyzing our steps towards the future: students take grades to have an idea on how they can develop professionally, teachers take feedback to adapt their methods and optimize their skills, countries take de OCDE’s ranking to modify their educational policies, and so on.
Of course, it has to be said that evaluation does not only happen inside the school and education world. Academic evaluation is an adaptation of a much broader behavior found on every civilization, on every group, on every person. Evaluation is, indeed, something as natural as judging or criticizing. In his book “Thinking Fast and Slow”, Economic Sciences Nobel Prize Winner Daniel Kahneman talks about some of the behavioral foundations of evaluation. His main thesis states that we go through two systems of thought: System 1 is fast and automatic, almost reflexive towards external stimuli (reading a facial expression, driving or riding a bike or remembering our age), although it tends to make incorrect interpretations; whilst System 2 is slow and deliberate, it comes forward when solving math problems, when analyzing a text or when evaluating a situation. Kahneman argues we use System 1 for too many situations, and System 2 for too few. And even in our use of the slow and deliberate System 2, it’s spinning gears tend to be fixed on the instantaneous but erratic System 1. Meaning, that even when we believe we are evaluating rationally, the variables we use for assessment might come erratic and unreflexively. It is my opinion that many of these cognitive errors common on evaluation at a behavioral level, have transpired to the more complex, but still ineffective system of academic evaluation.
So, does this mean that evaluating is intrinsically wrong, and we should just forget about measuring our students’ progress? Not exactly. I believe evaluation in the academic system is as necessary and compulsory as evaluation at behavioral level. But, as evaluation at behavioral level, we haven’t been doing it correctly. Or at the least, we could do it a lot better. Let’s examine some of the various methods our current educational systems use to evaluate how much students have progressed.
First, and probably most common, is the scaled-grade system. The scale might be different in many countries, but the core principle stays the same: a student is assigned a letter or a number that can mean anything from excellent to poor performance. The letter or number is a translation of a series of achievements a student has obtained throughout a specific period of time: results from tests, participation in class, frequency and quality of homework delivery are some of the common variables used by teachers to assign a grade. This system tends to be very efficient, but mostly by economical principals: it is fairly easy to understand, easy to use and analyze, and when the variables are chosen wisely, it seems to be fair and clear for both the student and the teacher. Unfortunately, this system doesn’t take into account any attitudinal, circumstantial or behavioral context that might affect the score. In fact, it is rarely critical of itself. Many educators and teachers have argued that this system is incomplete, stating that numerous amounts of students have seen their grades affected by a wide variety of situations such as family problems, depression, unhinging mental illnesses like the Attention Deficit Disorder, irrational fears to tests or the teacher, social pressure from friends or family, to name a few. Indeed, a large range of studies have proven that many of these situations can influence anywhere from mildly affecting to being critical on the performance of the student in school, regardless of his or her age. And yet, this widely-used and accepted system has failed to come up with a convincing solution that can take all these into account at the moment of assigning a grade. I find rather unfair that an elementary with a happy, communicative and economically sustainable home might have a strong advantage over another elementary student living through its parents’ nasty divorce.
Other more innovative schools have adopted alternative systems of evaluation that work additionally (and, sometimes, in substitution) of the scaled-graded system. Other assessment tools such as performance feedback, descriptive rubrics and assessing attitudinal competencies have been used and tested by many teachers. These alternative methods tend to be better that the scale-grade system, but still fail to take into account a lot of the student’s personal context. This is because, even though they leave more space for the acknowledgement of the students’ personal situation and can sometimes lead to nurturing discourse between teacher and student, they are still reigned and guided by the same logic of categorization that is followed by the scale-grade system. Attitudinal competencies, for example, are mostly descriptive and rarely graded by numbers or letters, but still insist on accommodating the student on a branded category (Beginner to Expert, Poor to Excellent, Yes or No, among other common categories). I believe these alternative tools are step forward to achieving a more complete and truthful evaluation, but still have space for improvement. Additionally, many of these alternative systems rarely have a translatable value in the wider educational system. It seems we are yet to create a methodology that can welcome and adopt personal experience into the evaluation output.
It can be argued, of course that when in workforce, personal experience should rarely overcome professional performance, and that one cannot fail to reach professional expectancies because of purely personal reasons. To some extent, this is true. I do believe that one should learn to separate personal from professional life, and that even when they cannot be completely indifferent one towards the other, one should try to solve problems from both sides without much interference of the other. But, I also believe that school is not workforce, nor is a training towards it. Colleges and universities might well be the bridge between these two, but everything before higher education should be examined and approached from a perspective different from that of the production-oriented mentality predominant in the workforce world.
I arrive, thus, to the core of my thesis. At the beginning I stated that evaluation is seminally important for determining how are we going to act in the future. A student knows what he needs to study better, a teacher learns which practices work and which don’t, an educational system measures its new policies. This is because, evaluating is always done and understood towards and objective. Without an objective, without a purpose, evaluation cannot exist. How do we know if we are doing something correctly, if we do not know why are we doing it? Or, to put it more specifically, how important is it for a child to get high marks on Geography if the child hates Geography? The answer is none. It is not important. Other than satisfying the intrinsic logic of the educational system, for the child it has no significance or value. I believe that some of the core purposes of education are incorrect, and thus, education cannot be properly evaluated. This thesis becomes clear when I ask myself this question: why do we educate our children? Do we educate them to be honest and compassionate citizens? To be independent and capable of solving their own problems? To be happy with themselves and share and spread that happiness towards others? Or do we educate them to be productive members of the society, independently and above of everything else? Education has come a long way since it started working as a self-criticizing discipline, and many educators might argue that productivity should not be the main goal of our children’ education so, why do we still evaluate them under this premise? And why do we shape our whole educational system under this same premise? Evaluation and educational systems are both ruled and thought from a productivity-oriented mentality.
Our current system uses courses based on knowledge disciplines to diffract, analyze and understand our world. It uses grades to evaluate and measure the success we have in comprehending each of these subjects, and our introduction to the disciplines evolves gradually in its complexity. Subjects are seen separately from each other, and most of the time even from different specialized teachers. Each subject is presented as a body of knowledge unrelated to anything else. Rarely does Chemistry and Civism act jointly, or Sports and Language, or Psychology and Geography. But the world does seem to merge and mash them together and make them almost undistinguishable. Is it really possible to understand Biology without Chemistry? Sports without Physics? Language without Anthropology or Sociology or Psychology? Of course not! We might be able understand some things about each discipline, but eventually all disciplines touch each other.
One could argue that this is the main reason behind teaching all these disciplines in school. But the problem is not if the disciplines are there or not, but that they are presented in the form of subjects, taught separately and independently. This is yet another example on how we are approaching education from a productive perspective. We are addressing the world in a categorized manner, and we are teaching our children that they can be professional A or professional B, but not both. French philosopher Edgar Morin is an important figure opposing this antiquated mentality. He argues in favor of the amalgamation of disciplines, stating that we should eliminate subject categorization and start thinking in terms of problem-solving and handling complex situations. To exemplify this in a clearer way, I’d like to cite a quote I once read from Google Chief Education Evangelist Jaime Casap: “Do not ask a student what they want to be when they grow up. Ask them what problem they want to solve.” This allows us to shift our idea of what schools can and should do. Instead of providing an excruciatingly long and forgetful body of knowledge, schools and teachers can focus on guiding the student to self-discovery, personal growth, problem-solving skills, conflict resolution and the orientation of deep intrinsic motivation towards a professional goal. If a person is aided throughout his childhood and youth in developing all these abilities, he or she will be perfectly capable of searching and adapting knowledge to reach its own objectives.
I call for a change on the paradigm of academic evaluation. The change in how and why we educate has started, and it is a new paradigm slowly gaining momentum. Teachers, psychologist, sociologist and educators have learned that productivity is an outcome, and not a pathway or a goal. Many studies have been made asserting the strong impact that personal feelings of achievement have over productive outcome, but rarely does this relation has proven to work backwards. Our current educational system seems to be rooted in the needs of a categorized industrial world (although that is for an anthropologist to determine). This generation demands un-categorization, it demands an interdisciplinary perspective, it demands a focus on our human abilities over our professional capacities. And our educational system should have the plasticity to adapt to these new demands. New technologies will probably provide the complex evaluation tools we need to assess a complex educational system. But unless we succeed in changing the way we talk and understand education, we will always fail when trying to provide truthful and fair evaluation to students. As educators, we have a responsibility towards our students of providing them with truthful, valuable education; and towards society to guide good citizens. Good education can be the difference between a prosperous society and a miserable one. It all depends on what you evaluate as “good” education.