Crowdsourcing Authority in the Classroom
“A wacko holding forth on a soapbox. If Ms. Davidson just wants to yammer and lead discussions, she should resign her position and head for a park or subway platform, and pass a hat for donations.”
That is an example of some of the negative comments I received when I wrote a blog on grading in my “Cat in the Stack” column on a website for the Humanities, Arts, Science, and Technology Advanced Collaboratory( HASTAC). I titled the post How To Crowdsource Grading and its premise grew out of a course I taught last year at Duke called “This Is Your Brain on the Internet"
In the course, we looked at everything that had to do with individuals and groups thinking together, on ways of managing collaboration, and on what collective thinking is, in terms ranging from the cognitive to the sociological. However, when I taught this course, I ended up using quite conventional grading methods. It was the students themselves (and the A students, not the B or C students) who, in their course evaluations, wisely suggested that I had to rethink grading.
They were right.
So I posted a blog where I advocated a peer-to-peer system where the students leading our discussion for a class would also decide if the blog responses posted by the other students were satisfactory and, if not, would offer feedback on how to make them better. Every study of learning says that you learn best by teaching someone else. Why not make evaluation, standards, feedback, collaboration, and interaction yet another aspect of our brains on the internet, another area to explore together?
The point of “This Is Your Brain on the Internet” is to show how, in Internet culture, we are often judging, responding, offering feedback, and working together through crowdsourcing but our educational system rarely if ever does anything to prepare students for offering or receiving feedback. In fact, very little in our society prepares us for responsible and responsive exchange. Typically, we learn how to please a figure in power. We do not practice or learn principles for helping one another through an iterative, interactive process. In a digital age, everyone is offered an invitation to participate. But where in our schooling or even in our informal learning process do we teach young people how to take responsibility for representing themselves in public, for participating in public dialogue?
I’m fascinated that the blogosphere was so annoyed with me for wanting to teach responsible judgment practices as part of my pedagogy. I think it is because grading, in a curious way, exemplifies our deepest convictions about excellence and authority, and specifically about the right of those with authority to define what constitutes excellence. If we “crowdsource grading,” we are suggesting that those without authority can also determine excellence. That is what happens in the non-refereed world of the internet, that’s what digital thinking is, and it is quite revolutionary.
If my students are going to thrive in this new world, they need to understand more about this process. We all do. As newspapers, magazines, publishing houses, tv and radio ratings and all other forms of traditional communication and entertainment falter or even collapse, we all have to understand more about what “Do It Yourself” really means. In the workplace and in our communities, we have to learn more about how to make judgments, to offer feedback, and to take criticism from those who are not “the boss of us.”
Who teaches our kids the forms and mechanics of interactive judgment that they need if they are to succeed in the digital future? We all need this skill. That future is here.
A terrific resource is the "Community Based Learning" website based at Washington State University. It contains loads of data on peer evaluation and a smart discussion of the issues.
Comments
Thanks so much for letting me know about this. I haven't heard before from anyone in nursing so I'm especially pleased to hear from you. I would think nursing, of all professions, is one where peer-to-peer reliance and self-reliance, collaboration and evaluation, would be crucial. I'm so pleased my post was of use to you. Your colleagues are very fortunate!
Thanks so much, Nils. You are absolutely right that the idea of crowdsourcing authority in the classroom relates directly to the idea of democratizing knowledge that our HASTAC Scholars held such a popular and important forum on (http://www.hastac.org/forums/hastac-scholars-discussions/democratizing-knowledge-digital-humanities) Over 6000 people tuned into that forum and there are 110 comments that could be published in book form. That's amazing, and speaks to the urgency of the topic.
For those who don't know, the HASTAC Scholars are 130 undergraduate and graduate students around the country and some internationally too who are nominated by over 40 institutions that also pay them a small $300 scholarship. It recognizes them as scholars of the future and creates an amazing interactive community. And, Nils, you're right that their next forum is on this topic of grading and evaluation.
I'm in Korea now and have been part of an incredible Digital Natives workshop mostly based in the Pacific Rim: http://aoardlifesci.wordpress.com/ The issue here is distributed and online education as well as teaching new modes of learning and learning what new modes of interaction, mostly games, actually teaches--not only social behaviors but also content and cognitive enrichments. Interaction (positive and negative) is another outcome. How do we incorporate that into our classrooms? That's the issue here. Thanks for jumping over from www.hastac.org to the DML Central site. Spread the word. This is a new site, incredibly rich and deep. It's great that you and your colleagues are part of this community too. Thanks for all you do.
Cathy
A quick response. First, thanks for the kind mention above. We welcome feedback to those ideas by comment or trackback.
Second, I think this topic is closely connected with the Democratizing Knowledge topic recently discussed by HASTAC scholars. http://www.hastac.org/forums/hastac-scholars-discussions/democratizing-knowledge-digital-humanities
That discussion opens with thoughts about "fostering community partnerships and opening access to university resources and research." I think crowdsourcing authority pushes the idea one step further, suggesting that perhaps the academy needs to go out and work in the crowd, rather than assuming it only needs to provide more access to its assets and authority.
As our group has been talking about community-based learning, one of the ideas that seems to be misunderstood is that we are not rejecting faculty or the university in favor of community, but rather, suggesting that there needs to be a permeability and the university needs to work in community-based models and within communities of practice.
I understand there will be an upcoming HASTAC Scholar's discussion on Grading, Evaluation, and Assessment. I think that will also tie into this conversation about democratizing knowledge and crowdsourcing authority.
Finally, you may be interested in these personal musings about learning and collaboration: http://www.nilspeterson.com/2009/10/30/learning-in-a-community-of-angels/
Thanks for these comments. Given these ideas about the "good receiver of feedback," I'm thinking my first assignment in "This Is Your Brain on the Internet" will be to read all of the blogosphere's response to my idea of crowdsourcing grading so we can actually see how evaluation happens online. Many times, such as on Digg, people are responding to one another and not at all to my original blog. Are the comments performances or evaluations? Thanks again for your responses.
So true. The "disruption" piece that is so often used to shoot down tech in the classroom has nothing to do with the technology and everything to do with classroom management and setting clear expectations (that need to be revisited when they are not met!). Great article!
Cathy writes:
"The point of “This Is Your Brain on the Internet” is to show how, in Internet culture, we are often judging, responding, offering feedback, and working together through crowdsourcing but our educational system rarely if ever does anything to prepare students for offering or receiving feedback. In fact, very little in our society prepares us for responsible and responsive exchange."
...and this explains the feedback exemplified by the example given in the opening paragraph of this post. We want learners to engage with conversations, online and offline, in productive ways--which means we need to find ways to support them in learning how to offer and receive feedback effectively. Cathy, good receiver of feedback that she is, refuses to bite back at the negative commenters--and in this respect, she is a nice model for the kind of participant we want in our classrooms.
Cathy,
I would like to tell you how much I look forward to your future publication.
I first came across your thoughts on peer evaluation through Tomorrow's Professor. As a fairly new nursing instructor (in my 3rd year) in an undergraduate BSN program, I took your advice and shared the post with my dean, primarily to support a major change that I had made in two of my courses during the previous semester, the introduction of team-lead classes.
I think it helped to decrease some of her skepticism and she really liked the idea of choosing 2 students to "grade" the blogs on a weekly basis. This has been one of our major challenges with getting faculty to adopt the idea of blogs in various courses...there is always some instructor discussing the 200+ posts they had to evaluate and comment on each week and how time consuming it is.
Nearly every course I teach includes a peer evaluation component and two of my courses include, what I refer to as a team-lead class, where a learning team (group of students)is responsible for providing the instruction and developing activities to increase classroom participation for the assigned content on that date. Therefore, I'm sure that many members of the blogosphere would deduce my teaching strategies as exceptionally lazy instructional methods.
However, one comment that I have not found in any of the posts, is that in many ways, monitoring and facilitating peer evaluation is more work for the instructor than grading a paper or project. With a paper or project, the instructor has basically envisioned the content, when they wrote the grading rubric / criteria.
For example, in the evidence-based practice papers, I have a very specific pre-conceived notion of what needs to be in the significance to nursing section (based primarily on how I was taught in my undergraduate program, how I was evaluated in my graduate program, and what appears in most peer-reviewed nursing research journals) - so, it is fairly cut and dried and simple to grade, those key pieces are there or not. However, when you bring the peer evaluation piece into it, they present some ideas that you maybe had not previously considered, which are valid or you realize they are way off base and you now have and educational content area to cover that you didn't anticipate.
Furthermore, the peer evaluation piece requires a lot of coaching and support as many students are very uncomfortable with the idea of offering feedback / constructive criticism or receiving it, especially when asked to provide feedback to a colleague that they view on a higher rung of the clinical ladder. And, in the profession of nursing, this will occur many times in every shift. How effectively this occurs, is one of the key drivers of quality client care.
All of that being said, and a very long post, I would like to thank you for bringing to light the value of peer evaluation and how integral a role it plays in many professions, especially healthcare. And, I would like to remind my colleagues that this concept, especially in higher education, requires much more work on the instructor's part than is readily apparent.