Quality Control in MOOCs

Like traditional education institutions, identity and reputation are important in MOOCs. For providers such as Udacity, Coursera, and edX, it means that the end user experience is vital in perceptions of overall quality. If students encounter a poor course (design, video, layout), that experience casts a reputation on the overall course provider. If they can’t offer quality courses, how do we know the assessments will be good quality? Or that plagiarism is being taken seriously?

The first open course that I offered had a big impact on how I have since viewed courses. When we opened CCK08, I thought “yay us, course is now open, people will learn, it will be awesome”. While we had that experience, we also had numerous concerns and complaints about course format, design, layout, and so on. Students had a different view of what a “free course” is all about than I had at the time. I discovered that students who participated in the course added enormous value for others. While they were taking the course without fee, they were offering their personal views, opinions, and time. The fee for taking a MOOC is active participation, which in turn increases the quality of the learning experience for others. Sometimes the language around MOOCs is almost “look at our wonderful MOOC overlords and the knowledge they bestow on the masses”. MOOCs and learners need each other. The gift of free courses by MOOC providers is promptly returned by learners in the form of their contributions. The MOOC platforms are nothing without learners. The gift of our participation is as valuable as the gift of an open course.

In mid-2012, I thought Udacity was the most vulnerable MOOC provider. Coursera was signing up universities every month or so and was offering hundreds of courses. edX was very well funded, $60m, and represented two of the top universities in the world: Harvard and MIT. Udacity, in contrast, was rolling out a few courses every semester. While Sebastian Thrun was very capable of drawing media attention, the optics of momentum were with Coursera. In August, 2012, Udacity cancelled a course before it started. This was an interesting decision and one that revealed, in my opinion, that Udacity was willing to take short term pain for long term reputation as a quality provider. Udacity owns its content, Coursera is more of a platform. That means that Udacity has a better range of monetization strategies – content, teaching, platform, recognition. Coursera as a brand will rely heavily on their university partners.

OT: edX is the most exciting MOOC provider – their content is outstanding, the platform is the best of all three providers. They are not profit-driven which means they have a different range of decisions and different criteria for making them. edX doesn’t get the media love of Udacity/Coursera (there is a story in there somewhere about the self-enforcing media plays around for-profit ventures and ability to feed into the public media through those connections). It has also managed to escape many of the concerns of plagiarism and quality.

Today, Coursera faced a quality crisis as its Fundamentals of Online Education course suddenly went dark. UPDATE: Keith Devlin provides clarification in the comments about how Coursera works – i.e. it was likely the instructors decision to close the course.

There had been some course rumblings: “The course so far is a disaster, ‘a mess’ as numerous students have called it”.

Interestingly, the communication with students has been horrible:

We are in the early stages of MOOCs and quality processes in Coursera are still being developed. An important question to consider relates to the quality and reputation of not only the institute and Coursera:

I’d add to that: who is advocating for learners who devoted a week to working through the content and suddenly had the course cancelled. How does a course make it into a system like Coursera without a quality check of technology and learning activities?

Update: email from Coursera to students: http://pastebin.com/RGxvuF7b

22 thoughts on “Quality Control in MOOCs

  1. There’s so much to think about in this troubling moment. What would help is a clear statement of Coursera’s quality control principles, that explain both the responsibilities and the rights of the individuals involved in delivering a course on such a large stage, as well as the rights to which learners might at least nominally be entitled. Otherwise there’s such a risk that innovation will be continually corralled by panic and short-term thinking, masked as timely intervention, particularly in relation to negative social media.

    Like you, I’m in sympathy with all those who have invested time in the first week of the course, especially as there evidently many who were enjoying it.

    But really, for me the issue is the reputational welfare of the individual academic who is delivering this course for Coursera in good faith. That’s the part of the contract that I’d like to see: who has some kind of institutional responsibility to take this factor into account in a serious way? Is it Coursera, or Georgia Tech? Institutional accountability is far too often sheeted home to the individual who took a risk, and happens to be standing in the spotlight when the crowd turns. So what happens next is really critical for anyone currently thinking of taking the helm of a massive course on a platfom they don’t control for themselves.

    • Great point – the faculty in these courses sit in a bit of a middle space. I would imagine that this could be very damaging to an academics self-worth, even career….

  2. Pingback: Quality Control in MOOCs | xED Book « Things I grab, motley collection

  3. I’m one of the participants, was posting and it all went “poooof”.

    Got an email some time later.

    Wish they could at least have left the forums open for a limited period so we could have grabbed our group members, found a new tribe home elsewhere to keep talking and working together and been able to share pur whereabouts with other groups.

    I have never worked so hard in my life. Up till 4am every night. It wasn’t the course so much as the buzz from being with people in the same field with the same passion and need to know “stuff”.

    I’m lucky, my group have a facebook page, we are collaborating with another refugee group on facebook and we are going to try building pur own mooc, learning by doing (:

    The spark was lit and the fire still burning. But why did Coursera have to try and extinquish the networks and shared work we created by pulling the plug on the forums with no notice?

    I get why the course was pulled, but the choice to just turn off the lights and walk put leaving us in the dark with our connections dashed, that was …I dunno, felt like we were a real afterthought instead of being the whole point of the exercise.

    Sarah. Brit in Italy.

    • Great question, Sarah: “But why did Coursera have to try and extinquish the networks and shared work we created by pulling the plug on the forums with no notice?” There is a real issue here about “who owns what students contributed”. Coursera (or member university/faculty) owns the content and can obviously limit access to that. However, the social networks that you create and the content that you generate should be yours.

      • I am taking a wild guess…not us?

        So keep the stuff we made, probably signed away the rights to that in the small print of enrolment.

        But don’t ram home the message that the students are just an *inconvenient necessity* by grabbing your ball and walking off with it, while we were in mid game and didn’t know how to contact our teammates once the pitch went “pooof”.

        Cos that is just………not cricket.

        The people I was working with, talking to, bouncing ideas around with, (including so many who WEREN’T in my group and now I can’t find) they were the real “gold nuggets” on the course.

        • Some lessons to learn from the experience…

          Massive – Check. The course had that.
          Open – Hmm… This is where I think it went awry. The course was conditionally open, until they closed and locked the doors.
          Online – Again, conditionally. It was online until the backbone was taken offline.
          Course – If the definition of course is an experience with a start and an end… It just ended earlier than originally anticipated:)

          We need to examine the meaning and intent of open in the context of MOOC. To me that means anytime, anywhere. The resources and discussions need to remain open and persistent for the benefit of this type of thing to work. Whether it’s a cMOOC, an xMOOC or something in between… if content or conversations are shuttered up at any point, it’s no longer open. When these elements cease to be open, the main benefit of the MOOC concept is fractured.

  4. I think a lot of elearning “Professionals” really need to ask themselves some questions after this. Some people did a lot of work and where looking forward to learning – a few people made a lot of noise – yes the google doc thing didn’t work AND? – but if you don’t like it, you can leave if you want to? It is open to join and open to leave.

    Now a lot of people have lost a lot of work, and the chance to learn many wanted to take, on a course, which apparently was a disaster? So if it was a disaster, how could the latter be true?


    • Hi Pat – it is not open to join and leave in the sense that you indicate. When you join an open course, you contribute your content, thoughts, and ideas. As a learner, I might not pay a fee, but I pay in time and sharing of ideas/concepts. The active involvement of learners significantly extends the value of the course. So, yes, it is open to sign up. But once you’ve contributed content, you share in the space. Technically, Facebook is also open to join/leave. But there is an informal contract in place between the company and the users. Users could go elsewhere, but after posting images, creating networks, joining groups, the space becomes personal. The attitude of “if you don’t like it, leave” is not the same as, let’s say, getting a free car “if you don’t like it, return it”. It’s more along the lines of going into a garage and helping to build the car with parts that you provide, effort that you put in. After a certain period of time and contribution, you have claim to (part of) the car.

  5. Having given a MOOC on Coursera last fall, and being right now in the throes of revising it to run again in March, I am pretty sure that the decision to cut off the course website in this case was the instructor’s. Coursera likely had no idea that was happening. Coursera provides the platform, but leaves it up to the instructor to design the course, to build the website, and to make it live or not. The instructor controls the site. They recommend to instructors that when a course ends, the site remain open indefinitely for registered students. Since all their courses are provided by recognized universities, it is those universities that are responsible for the course content. This is very different from Udacity, which is far more than a platform — indeed, it is essentially an online university, with all that entails. In contrast, I don’t think you can blame Coursera for what happens on their site than you can blame YouTube for the content you find on its site. I should add that I have no connection to Coursera, other than having used them as the platform for my course, and am in no way a spokesperson for them.

    • Thanks Keith – I appreciate the comments and the clarification. I’ll update the post to point to your comment. Question: does Coursera have standards that instructors must follow? i.e. “don’t kill a course and toast the forums mid-course?”

    • But, if Coursera is in a “just a platform” role, where does that leave their signed certificate option?


      If the quality of the Coursera brand gets tainted by poor quality courses, how can the certificates maintain their value to the students to the extent that they are prepared to pay for them ?

      Wouldn’t they become rather useless to the students if they were perceived by other institutions and employers as bits of paid for paper rather than evidence of achievement earned ?

      The above is why I assumed that Coursera had a hand in pulling the course. As an act of brand protection.

      Our forums are back! (big fat cheesy grin)

      • The certificate of completion is awarded by the instructor. The degree to which that certificate is endorsed by the instructor’s institution varies. Stanford allows the use of it’s name but makes it clear it is not a Stanford certificate in any official way.

  6. Pingback: bookmarks for February 3rd, 2013 | Morgan's Log

  7. Pingback: Lessons to be learned via ‘MOOC Mess’ at Coursera? « GlobalHigherEd

  8. Please try to negotiate with Google Admin for a Stable Long-term Future – this MOOC deserves a Future – just as all evolving consequent MOOCs do – this is a roles and standards model, and a ground breaker – it can’t die

  9. Pingback: Two Thoughts on the crash of the “Fundamentals of Online Education” MOOC | stevendkrause.com

  10. Pingback: MOOCs: Fail better « Doug Clow's Imaginatively-Titled Blog

  11. Pingback: Abschlussbericht der Woche 3. Eine ganz subjektive Zwischenbilanz - #MMC13 - der Open MOOC-Maker Course 2013

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>