Academe's Dirty Little Secret
11:09 PMEd. Note: The image below titled "The Learning Pyramid" is NOT based on any verifiable research; perhaps, no research at all (see this and that). This pyramid is widely cited yet it is, as Christopher Harris shared in a recent email, a hoax. Nonetheless, I stand by the idea that the marriage of teaching and content creation is a powerful pedagogical practice. (see the comments)
A paradigm of instructional design that suggests we create learning experiences for our students where they create content that educates.
I had a conversation with a couple of friends who teach at university. We were talking about how much variation there is not only in the nature of assessment in different classes but also what work merits an A (the highest grade) or an F (failure). Some professors consider a mark of 85% an A, some require more others less. Some determine students entire grades from a single exam or paper; others require several short papers, a mid-term, final exam, and a major research paper. And these variations can occur when different instructors teach the same course and students receive the same credit (or not) for different work.
(It's amazing to me that student societies don't protest this sort of thing.)
We're no better at the K-12 level. I once attended a workshop where 30+ math teachers graded the same set of three problems. Each teacher decided how many marks each question would be worth and converted the marks to a percentage. Scores varied from percentages in the 30's to the 80's.
I guess this is academe's dirty little secret: there are no assessment standards.
To its credit the National Council for Teachers of Mathematics (NCTM) published a set of Principles and Standards for School Mathematics in 1995 (updated in 2000). The document includes The Assessment Principle. These are guidelines only. Not all math teachers are members of the NCTM, not all math teachers are aware of these standards, and adherence to the standards is voluntary not compulsory.
One of my friends shared that she is required to ensure her classes achieve a predetermined average grade. I think that's unethical. Sometimes you get a strong group of students, sometimes a weaker group, more often it's somewhere in between. There really is no way to "predetermine" a class average. We all agreed it should be possible for a class to all achieve A grades or F grades.
I don't believe a student's grade in any course is a measure of their intelligence. I do believe it is a measure of the time, energy, and effort invested in the class. But I digress, that's another issue.
As we talked some more about assessment and pedagogy the conversation got a little more intense as I challenged my friends by saying: "If all you do is lecture do your students really need you? Couldn't they just read your assigned text, show up on test days, write the final exam and be done with it?"
That started us on a bit of a heated discussion about instructional design and assessment. What became really clear to me is that most instructors in university teach and assess their students at the bottom of Bloom's Hierarchy.
At one point, one of my friends said: "Look, we're not going to throw out hundreds of years of tradition because of what we think sitting here discussing this."
"That's not a fair criticism. It's not like there isn't scads of research in teaching and learning that I can point to supporting what I'm saying. I'm not making this up as I go along. The fact is, if you just lecture to your students, on average they're going to retain maybe 5% of what you say. You can do a lot better to make their learning sticky."
"If you really want to ramp up your teaching have your students create content that educates. That will naturally engage them at a higher cognitive level."
"You can require your students to demonstrate their understanding of what they are learning by having them apply their knowledge analyzing and evaluating relevant novel situations or problems. Better yet, get them to create content that educates an interested learner and they will automatically incorporate all those levels of engagement while they make their learning sticky. I don't need to tell you that there's nothing like having to teach a thing to make you really learn it.
Whispering by saaam
Bloom's Revised Taxonomy by dkuropatwa
Learning Pyramid by dkuropatwa
Bloom's Revised Taxonomy (revised) by dkuropatwa
The Twin Triangles by dkuropatwa

8 comments
I do agree with your general point, but the numbers about what people remember seem to be similar to what's often associated with Dale's Cone of Experience. The research for that is not exactly solid, although similar numbers are repeated all over.
ReplyDeleteWhere did you get your set of numbers for remembering 5%, 10%, etc.? Is there research that we can see to look at that?
Will Thalmeier has written about this extensively previously.
One aspect of assessment that continues to be controversial is the idea of assessing effort. You say a grade is a reflection of time, energy and effort. Many would argue that's not really a great way to assess. Assessment should consider achievement regardless of effort. What this needs is a pre-assessment to measure understanding at the beginning of a course. If at mid-term, a student has achieved learning, it may not have included traditional types of "effort".
ReplyDeleteThere are 2 big gaps or variables here that continue to plague my thinking. First is the quality of the assessments themselves. Traditional paper and pencil testing likely still has a major place in the Mathematics world but in general is overused and still isn't providing us with accurate pictures of learning as your describing. In our efforts to be more accountable, I fear us stooping to the lowest common denominator in order to crank out data. Second, our perceptions of what is required to learn is also fairly narrow. We place high emphasis on attendance for example. That may or may not be a factor given the opportunity students have to learn outside the classroom.
These are simply some topics that for me, continue to cloud the ideas of assessment.
Darren,
ReplyDeleteYou said, "I don't believe a student's grade in any course is a measure of their intelligence. I do believe it is a measure of the time, energy, and effort invested in the class."
Where is the teacher's role in this? How do we factor in the teacher's willingness to differentiate their instruction to accommodate for their students' struggles with the material? How flexible is the teacher? Can they adapt their material even if only one student doesn't "get it?" Isn't it our responsibility as educators to help our students master or understand the material to then use it in meaningful ways?
I work with too many students who invest time, energy and effort and still struggle with understanding and mastery. Whose fault is it if they are failing the tests?
Or how about the math teachers who don't give partial credit when the student demonstrates understanding of the concept but makes a "careless" error? Do punitive grading methods have any place in our classrooms?
Is the issue tweaking grades or scrapping them entirely? (my preference)
ReplyDeleteI concur with Seymour Papert who gave two grades at MIT - A & Incomplete
Darren,
ReplyDeleteBut won't achievement suffer because they've missed out on those conversations? It seems that you're punishing them twice. Once for not attending, and once for not achieving.
On the other had if they attend and do well, they're rewarded twice. If they attend and don't achieve, perhaps the conversations and value of attendance was not reflected in the assesment so it was flawed to begin with. If they don't attend but do well...then again, the learning through conversations and attendance likely wasn't included in the assessment.
If on the other, attendance is important and valued because of ideas of social learning and contributing to the learning of others, then I can see its importance regardless of a direct link to achievement. That's what I'm trying to foster in my undergrad classes. You aren't only responsible for your own learning but others as well and they will be assessed specifically on the level of their contribution to others. Sometimes this occurs synchronously and other times and more likely it occurs asynchronously or at least away from large class gatherings.
Another of academe's dirty little secrets: grades don't really reflect the quantity or quality of what students have learned.
ReplyDeleteIn one of my classes today one of my most mathematically wounded students started participating in class. Standing next to her, at the back of the room while another student was sharing their work on the SMARTboard, she quietly started talking to me; sharing her understanding of what was being written on the board and correcting the work of the student at the front of the class in a way only I could hear. I can't tell you what a quantum leap this is in her learning, yet, it will never be reflected in any grades she receives and she'll most likely fail the course.
It sounds like you feel students are "unfairly" penalized when they do not attend class. Attending class results in all sorts of learning; some deep, some incidental. Students have a choice and there are consequences for the choices they make. I don't think that's unfair. I think that's reasonable, and fair. Students are free to choose not to learn. Any penalties they incur are a direct result of the choices they make. Aren't there consequences for you if you don't show up to work?
You said: "the learning through conversations and attendance likely wasn't included in the assessment."
Sure it is. All the common errors that were pointed out in class that the absent student didn't know about effects their test scores. How to deal with an ambiguous question (which often comes up in probability and combinatorics) we discuss in class. The absent student will do the best they can and gamble that they address the question in a way that maximizes the way they are assessed.
Attendance is important. As you say in your last paragraph, esp. as they come to rely on each other for their learning. I've seen that more and more as a direct result of class discussions that flow from each night's scribe post. The sense of ownership they have about their learning and their sense of responsibility towards helping each other learn are things I've only seen in my class since I established a blog for every class. Before the blog, this sort of thing just didn't happen. I suspect you found (and will find) the same with your classes at U. of Regina.
My point is simply not that attendance isn't important but it shouldn't be separated as a component of the assessment process.
ReplyDeleteThe student you mentioned that demonstrated her learning shouldn't be given credit because she showed up but maybe because she gained understanding and/or insight.
To me it's almost akin to giving kids credit for having a tidy desk or binder. Yes, those habits may assist in learning but that's not the issue.
Even in the work place, the bottom line is "did you do your job?"If I can do my job, however that's defined, in whatever way or space I choose, that should be good enough. If I'm not doing my job, perhaps attendance or other habits may be the issue and they should be addressed.
Unless attendance is spelled out as integral i.e. if you work in at McDonald's you have to there, it's simply a habit that may or may not benefit your achievement.
In the case of school, again, it might be important, in your case i can see why but I wouldn't assess it since their level of achievement will be reflected in other areas. There's no need to isolate it.
Attendance is my class won't be graded separately but their contributions will. That does mean that they will need to attend for the most part but I'm also going to ask them to assess their social learning experiences that occur asynchronously as well. My course has no final exam so that may make it different but their work and learning should reflect conversations, readings and viewings that occur in all kinds of ways. Attendance isn't the issue, social learning is.
I agree. Students shouldn't get marks for attending class; they don't in mine, nor, it seems, do they in yours.
ReplyDeleteYou said: "Attendance is my class won't be graded separately but their contributions will."
Grading their contributions in a fair and meaningful way is the difficult part. Imagine two different students who fufill the requirements of "making a contribution". One by following the guidelines you give them precisely the other by genuinely engaging with the material produces work that is qualitatively superior because it goes beyond the guidelines you've given them. To be fair, how can you give them the same mark? How can you not?
The nature of the course you are teaching is very different from what I teach so it's natural that the nature of the assessment should be different. The hard part, IMHO, in a course like yours, is that even if you articulate the assessment criteria in a way that everyone understands what is required to get an "A" all the "A" students will not be equal and it's very hard to reflect this in a single grade.
Note: Only a member of this blog may post a comment.