The COMARKER™ Development Blog


As teachers of academic writing we wanted marking essays to be easier on ourselves and our students. COMARKER is our solution. COMARKER makes it easy for instructors to make comments on written documents. COMARKER remembers our comments and shares them with every marker using the system, so we never again have to type the same comment twice, and our wisdom grows through collaboration. Already we see the power of collaboration making our own marking easier. This blog records our challenges and our progress as we develop COMARKER for everyone.


Working Towards Solving the Problem of Marking Essays

Since February this year I have been on an Ed-Tech conference whirlwind tour, taking Comarker to conferences in Ashland, VA; Portland, OR; Seattle, WA; and next month, Victoria, BC. The best part of all this mileage has been the meeting teacher and administrators. These educators have inspired us to keep working to improve our essay marking technology with their openness to new ideas and technologies, and their willingness change the ways they teach when confronted with a changing world.

One aspect of these conferences that has been disappointing has been the vendor areas. These halls are the carnival midways of the conferences, and attendees are lured in with the promise of prizes and raffles, and once inside, they realize why the lures were necessary. Unfortunately, these rooms and halls are filled with tools that are, for the most part, no better than the 20th century tools they are meant to replace, yet they are, universally, much more expensive. Emblematic of this disturbing reality is the glut of interactive writeboards in the marketplace (I counted more than ten IWB vendors at the NCCE conference in Seattle).

The problem with interactive writeboards (IWB) in that they don’t seem to help teachers teach anything that you couldn’t teach with a projector and a whiteboard. For a few thousand dollars (from about $1000 to $7000) a classroom can have an IWB, that solves none of the problems that students have when trying to learn things that weren’t already solved by 20th century technology:

  • erasable marker
  • overhead projector
  • tape player
  • television

The old white boards cost $40, last for 25 years, and, when coupled with a netbook ($300) and projector ($400), replicate 90% of the function of the interactive writeboard. It seems to me that that would be $740 well spent because none of those systems lock teachers into a pedagogical style that doesn’t suit the ways they already teach. Moreover, there is no training required ($0). And according to the educational resource trainers I have met in my travels, training recalcitrant teachers is the biggest problem schools face when trying to update their technology.

In my classroom, this simple set up is where I stop: projector, laptop, and whiteboard. If I wanted to add interactivity to my whiteboard it would be as simple as game of Wii Tennis or Bowling. With a wiiMote ($50) and a copy of the Smoothboard software ($30), anyone can create their own IWB. This garragiste IWB comprised of a laptop, projector, and WiiMote duplicates 100% of the functionality of an off the shelf  interactive writeboard. The home made solution is eight-and-a-half times less expensive than many of the proprietary IWBs, replicates all of their functionality, and is just as easy to install and train teachers to use. Does it seem crazy to anyone else that a technology that shows no impact on student learning (according to a study published in 2007 by the School of Educational Foundations and Policy Studies, Institute of Education, University of London) should occupy so much of the elearning marketplace and be so grossly overpriced?

Obviously this rant has little to do with our little formative assessment web app. We think that Comarker solves real problems for teachers, students, and administrators. But, what I have learned on the road is that educators need a rubric to assess the technology that is being sold to them. Without a good rubric, schools will continue to exhaust their budgets on “21st Century Teaching Tools” like interactive writeboards, and thinking that they are doing right by their students. I hope to flesh out that rubric on the blog over the next few weeks.

  • Share/Bookmark
Bad news for those who want to apply completely automated, industrial processes to writing assessment. A recent UK experiment with computerized marking progrmas has given them a failing grade. The Chartered Institute of Educational Assessors (http://www.ciea.org.uk/) recently put the computer marker through its paces by feeding it samples of essays written by the likes of Hemmingway and Churchill. The machine did not take kindly to the test and gave poor marks to some of the best writers of the 20th century. Not that this news is surprising, for computers are still fairly clumsy at natural language processing – just ask your grammar checker in MS Word. To imagine that highschool essays like those required in the British A-levels and CGSEs do not fall into the category of natural language is foolish. Foolish because in many of the rubrics designed to help teachers assess student writing at the highschool level – six plus traits, for instance – to demonstrate mastery, the student must break out of the mold of rigid formula. They must show spark by using metaphor, descriptive language, unconventional sentence structure, and all of the other things that set great writing apart from merely competent writing. How can a parser understand whether a metaphor works, or falls flat? How can a machine see the difference between stylish concision and lack of vocabulary?
Comarker skates by this problem because from the beginning we understood that putting teachers and markers out of work was antithetical to the educational enterprise. Academic writing tests our ability to generate meaning from the disparate and the abstract and that is preceisely what the computer cannot yet generate. So to generate feedback and authentic assessments and evaluations for students COMARKER relies on a more creative and intelligent system : human teachers. Revolutionary isn’t it.

A: to get to the “does not compute.”

Bad news for those who want to apply completely automated, industrial processes to writing assessment! A recent UK experiment with computerized marking progrmas has given them a failing grade. The Chartered Institute of Educational Assessors recently put the computer marker through its paces by feeding it samples of essays written by the likes of Hemmingway and Churchill. The machine did not take kindly to the test and gave poor marks to some of the best writers of the 20th century. Not that this news is surprising, for computers are still fairly clumsy at natural language processing – just ask your grammar checker in MS Word. To imagine that highschool essays like those required in the British A-levels and CGSEs do not fall into the category of natural language is foolish. Foolish because in many of the rubrics designed to help teachers assess student writing at the highschool level – six plus traits, for instance – to demonstrate mastery, the student must break out of the mold of rigid formula. They must show spark by using metaphor, descriptive language, unconventional sentence structure, and all of the other things that set great writing apart from merely competent writing. How can a parser understand whether a metaphor works, or falls flat? How can a machine see the difference between stylish concision and lack of vocabulary?

COMARKER™ skates by this problem because from the beginning we understood that putting teachers and markers out of work was antithetical to the educational enterprise. Academic writing tests our ability to generate meaning from the disparate and the abstract and that is preceisely what the computer cannot yet generate or assess. So to generate feedback and authentic assessments and evaluations for students COMARKER™ relies on a more creative and intelligent system : human teachers.

We still need beta testers, so contact us if you want to help us make a system that improves on the shortcomings of full automation.

  • Share/Bookmark

Hooray for Press Release

Open call for teachers to Beta-test Collaborative Essay Marking Software.

Teachers find themselves making the same comments on different essays all the time. Until now there has been no way to avoid this repetition. However, at Acovan Inc we have been hard at work trying to solve this problem for nearly two years now. Our solution is COMARKER ™, a web-based tool for evaluating essays. The application remembers all the comments you make on student essays and returns them to you efficiently and automatically on demand, so with COMARKER™ you will give great feedback and save time.

Autocomplete is a great step forward already, but just the beginning for COMARKER™. COMARKER™ also returns relevant results from other markers, making COMARKER™ the first mass collaboration among evaluators of text.

All the research shows that formative assessment improves learning outcomes, so we built COMARKER™ with those principles in mind. COMARKER™ handles multiple drafts of each document, giving students a chance to revise and edit their work before the marker evaluates it using our simple, interactive rubric. Students learn throughout assignments rather than just at the end when they receive their grade. You don’t have to mark multiple drafts, but COMARKER™ makes it easy if you do.

We built COMARKER™ for ourselves, but along the way we realized that we could all benefit from synergy in a wider collaboration. We have been testing COMARKER™ in our own private tutorial academy for one year now, making big improvements along the way.

What we need now are critical users to test the program and show us how we can improve it. Our mission is to make learning richer for students and marking easier for teachers. We need 150 educators for a two-semester trial of the program. We will set your classs up in the system, and as the trial progresses, we will ask you now and then for feedback (surveys, polls, etc.). Will you help us end the pain of marking for teachers everywhere? For additional information please visit our site, watch the videos, read the FAQ, then fill in the contact form to get started.

The COMARKER™ team.

  • Share/Bookmark

Rubric Builder

Last week Brian Panulla, one of our Flex developers, flew up to Vancouver from Portland to meet with us about something we have been talking about since the summer: creating a a document standard for rubrics and a program that allows teachers to create, edit, and share those rubrics with their colleagues.  The meetings were a resounding success, one of the several that have come out of the OpenEd 2009 conference in Vancouver last summer. In two days of meetings we hammered out the requirements, some user stories, and the basic architecture. Brian will begin iterating later this month, and if all goes well, we will have a working prototype in about two or three months.

More sofisticated prototypes are to follow

More sophisticated prototypes are to follow

The rubric module will not just be for users of COMARKER™, but for teachers everywhere to create and share their rubrics. Of course, rubrics built with the new system will be fully interactive in COMARKER™, but since we are all about mass collaboration, everyone will be invited to share.

  • Share/Bookmark

Demo Videos 2.0

All finished with the new demo videos.  I like the audio quality compared with the last ones.  Check them out at our site, or watch them here (from uTube). Let us know if there is anything you want to see in the videos that we didn’t think to include.

Admin

Marker

Student

  • Share/Bookmark

Last week Tom and I finished the scripts for our new screen capture videos. We elected to use my voice again and I am much happier with the way they sound this time round.  I am presently editing the screen capture videos I took of Comarker™ and stitching the audio in.  If anyone has any feedback or criticism I would be greatful to hear it before I publish the videos to our sales site.

Student Reading

Marker Reading

Admin Reading

  • Share/Bookmark

1st Ever Wp Upgrade

Thank god almighty I didn’t kill my blog. Phew! Looking at a wp-config.php file makes me thirsty.

A magnum reward for my efforts.

A magnum reward for my efforts.

Don’t worry, i won’t drink it all myself.  I’ll share some with my two cats.

  • Share/Bookmark

Fallout From OpenEd09

When minds meet (like at conferences) the share reading lists.  One of the books that was generously reccommended (thanks @jonmott and @leighblackall) by conference goers is called The Cluetrain Manifesto. It is relevant to us at COMARKER™ because it is a marketing book, and that is what we are attempting to do, but it doesn’t offer prescriptions.  Instead, it causes nervous yet excited beads of sweat to form on my forehead.  I am not media illiterate; I have read Mcluhan, but I feel late to the party reading Cluetrain ten years after its first publication.

One section that hits hard today, just as the Canadian Government is deciding how closely it wants to follow America’s DMCA, is in the chapter “In Defense of Optimism”:
Continue reading…

  • Share/Bookmark

Gold From Opened09 pt. 1







  • Share/Bookmark

Open Ed 2009

I am sitting in the foyer at the UBC Downtown campus, drinking luke-warm coffee, and waiting for the keynote. Follow along at the conference website if you can’t attend, or wait until one of the many ed-techy commentators make some remarks about what they have seen.
More to follow.

  • Share/Bookmark