[frv103]

I'm not sure that course completion percentage is necessarily the best metric to be drawing conclusions from. I will often sign up for MOOCs and courses for a single talk, chapter, or topic that I'm interested in with no intention of actually completing the course. I wonder I'm an outlier in this regard or if this approach is much more common than the author of this article realizes.

[Chathamization]

Indeed. Also some platforms (I believe Coursera and EdX were like this last time I used them) only let you access old content if you signed up with a course when it was active. So you are incentivized to sign up for any course if you think there's a possibility that you might want to access some of the content at some point in the future.

People shouldn't confuse completion rate with a particular course with a student reaching proficiency in a particular subject. There have been several courses I've started were I quickly got annoyed by the copious amount of filler in the lectures and dropped the course in favor of a book or web tutorial.

In general I think MOOCs are doing themselves a disservice by trying to copy college courses.

[alexgmcm]

>In general I think MOOCs are doing themselves a disservice by trying to copy college courses.

I agree completely - it seems almost 'cargo cult' - they are just copying colleges assuming it will be successful rather than examining what it was that made that work for colleges and how they can adapt it to the MOOC model.

[the_watcher]

I've worked in this industry a bit, and MOOCs rarely use "course completion" as their primary success metric. One reason is precisely what you mentioned - that many people simply take the course for as long as they need to get their desired value out of it, then leave. Another was that the team I worked with found that, by far, the single biggest factor in terms of course completion was that a third party (usually their boss) incentivized them to complete it, and of those people, they were more likely to find ways to skip sections, get the minimum scores required to pass, etc.

[gmiller123456]

So what ways do they measure success? I can attest that I have never started a course just for a few lectures, and I've always planned on taking the whole course. I've quit the majority of courses just because they weren't any good (usually poorly designed homework). I have quite some because they moved on to topics that I didn't find interesting, but I never looked at them from the beginning and said "That's where I'm going to stop". The ones I have completed were ones I thoroughly enjoyed, and felt like I learned a lot.

So just from a "course completion" point of view, I'd think it'd be hard to tell the difference between courses I liked, but moved on to topics I didn't like, vs. the ones I just thought weren't any good. But the fact that I stuck with some courses all the way through should at least be good for something.

[the_watcher]

They'll usually _track_ course completion, but it's not necessary for a course to have a high, or even an average completion rate to be a successful course. From what I've seen, they'll track active time spent in course, how much of the course is completed, use quizzes to reinforce learnings while also getting a check in on how actively the student has been taking the course, etc. They also rely on ratings and qualitative feedback. It's not as precise as something like "completions", but from what I've seen it's reasonably effective at assessing course value.

[bcheung]

Agreed. I notice that many MOOCs have this academic mentality that something must be complete and a certificate earned. I could really give a damn about a certificate. Gaining the knowledge I need is all I care about. Udacity really makes me mad when they remove courses that I paid for if I don't do every little exercise (even the ones that are so horribly designed and have little learning value in them).

[ska]

Gaining the knowledge may be all you care about, so long as you never have to demonstrate that gain.

Certificates/exams/etc. are all used as proxies for demonstrating that in a complex and expensive way.

Perhaps Udacity et. al. need an equivalent to "audit" - sounds like that would work for you.

[JamesBarney]

This seems like a valid criticism to removing a certification but not access to a course.

[gaius]

EdX has this - courses are free to audit, you only pay if you want a certificate (but you can always screenshot your progress page)

[bcheung]

It was more the removing access to a course that I paid for if I didn't follow their schedule that really upset me.

[ska]

ah, that's fair enough. annoying.

[anonymous5133]

Course completion metric is one of the worst metric to use to evaluate online courses/MOOCs. It is quite obvious that for other forms of traditional there is an investment that is being made by the student so of course they are encouraged to keep going. People spending money on something is a signal that they are highly motivated to take the course.

For MOOCs, there are very few entry requirements so of course you can going to get low completion rate because lots of people are just "window shopping" so to speak.

[dv_dt]

Yup, concluding that MOOC formats are more effective is similar to the charter school problem. If you filter for committed people then you end up with better looking success stats. But all it might be doing is finding and filtering those motivated enough to go through a school change process, or willing to put up with some hassle factor of something "online" but still requiring scheduled times, or other commitments. Similar for attendance being a factor for success in the article - it sounded like going in person was the key way to access the extra learning resources.

Filtering for the subset of people committed in a certain way isn't really a proof of a good learning format. And to me, not necessarily the best way to bring education to a wider range of students. Filtering for commitment might be argued that it's effective on some level as an efficient use of teaching resources - but I think that needs deeper examination too.

[MisterOctober]

I agree that there's a lot of shaky assertions made about this pedagogical method. From my own perspective, though, on-demand courses like on udemy essentially solve all the problems I hated about classes for my brick-and-mortar university degree. To wit :

1) No distracting interruptions from self-absorbed students asking 'questions' [more often than not, really just statements] that would have better have been reserved for after class / office hours

2) Prof / TA doesn't show up drunk and disorganized

3) If you miss something while taking notes, no sweat -- just rewind and listen again! As a copious note-taker, this is huge for me.

4) Flexible class times keep me much more healthy [can get sleep when needed, etc]

5) No travel time, finding a seat, wanting to talk to a girl instead of focus on lecture, etc etc with all the frustrations / inconveniences that are just not there in online courses

I could rattle on a lot more, but ca suffit for now

[lifeisstillgood]

These are aspirational course sign ups, and are of course similar to aspirational book purchases (where I will learn a new language through psychic connection between the desk the book is on and the my hands on the desk) and aspirational gym membership (where calories will be burnt simply because my body knows it might go to the gym and so gets all worried and sweaty anyway)

[komali2]

Yea, gym membership is a good comparison, actually. I know tons of people who signed up for a gym and never went, which means we could use the same argument the article does to insinuate that "signing up for a gym on your own is an ineffective way to get fit."

Clearly there are more effective (and more expensive) ways to get fit, but some people manage on their own with a gym membership and access to /r/fitness or whatever, so it's not exactly ineffective.

[InitialLastName]

> some people manage on their own with a gym membership and access to /r/fitness or whatever, so it's not exactly ineffective.

Some people manage to get fit on their own without a gym membership or any equipment at all, so it's not exactly effective either.

[realharo]

There is also the meme of having a lot of never-played games in one's Steam collection, that were purchased during sales. So it affects entertainment too.

[MisterOctober]

That hits where it hurts! I'm the worst offender when it comes to aspirational book purchases. To the topic: On udemy, I see tons of folks leaving reviews who have purchased hundreds of courses [why udemy displays this info is obscure to me - maybe to generate some kind of courseload-envy?]

[thinkmassive]

I’m taking two courses on Coursera right now but only paid for a certificate in one. I plan to view all content from the other course, however Coursera will not grade the end-of-section quiz for non-paying students.

It seems the only way to “complete” a Coursera course is to pay $50 so the assignments will be graded. At least this is the case for: https://www.coursera.org/learn/battery-management-systems

[scarface74]

What’s the purpose of the actual certificate? As someone who has done dozens of interviews - I’m not a manager, but for about two years I was responsible for hiring and I’ve sat as part of a panel- most people don’t take most certificates seriously. Some, including myself, take it as a slightly negative signal. I’ve seen too many paper tigers.

But then again, except for entry level positions, few hiring managers care about any formal education for IT positions.

From what I can gather, the few certificates that companies care about are RedHat, Cisco, and AWS certs. I’m sure there are a few more that are outside of my area of expertise.

I went through the Microsoft Architect Certification track as a commitment device to force me to study, but I didn’t put it on my resume and never told my employer. But I was trying to transition from a C/C++ bit twiddler to an “Enterprise Developer”.

[thinkmassive]

If someone has completed a certification or course, it becomes a potential topic for discussion. It doesn't matter much to me whether they paid for it, if there's proof it was completed (a code repo or documentation could be an alternative to paying for a certificate).

I have certifications that people well below my skill level were also able to achieve, but I know I can speak to the topics much more authoritatively. It's not a binary signal.

I agree that certain industry certifications are more legitimate than most, although I think continuing education falls in a different but partially overlapping area. Nand2Tetris is the first MOOC I have paid for, because it seemed like a very interesting course that's recognized as legitimate by a substantial portion of people with whom I might enjoy working. Paying is also a good motivator to see it to completion.

[scarface74]

If someone has completed a certification or course, it becomes a potential topic for discussion. It doesn't matter much to me whether they paid for it, if there's proof it was completed (a code repo or documentation could be an alternative to paying for a certificate).

Any technology that I’m able to discuss intelligently is usually referenced as part of what I did on a job. I leave off any technology that I know but don’t want to be asked about or come up in a recruiter’s keyword search.

For instance, I don’t mention C/C++ even though I did it for 12 years or PHP.

I want my interviews to be focused on my strong areas and technologies that I want to use- which are usually the ones that I have real world experience with.

[thinkmassive]

Do you not consider hobbies to be real world experience?

Personally I appreciate the opportunity to not be stuck taking on new work that depends entirely on my previous employment. I find that people who pursue interests outside the scope of their day job tend to be good at thinking through problems and maintaining a healthy attitude, so I’m glad to discuss hobbies with people.

[scarface74]

Two thoughts:

Every job has “must have” requirements and “nice haves but be willing to learn”. I focus on jobs that will hire me based on the must haves where I can learn the nice to haves.

A lot of times that means the line between “work” and self study gets blurry. I might be “working” 60 hour weeks but producing 40 hours worth of work product and spending the rest of the time learning. That also means now my resume has work experience with the new-to-me technology/framework. It helps to work for small companies. You get more opportunities to learn and do low priority side projects on the job.

If that opportunity doesn’t avail itself, then do a side project and throw it up on GitHub and then we can discuss it.

[bcheung]

Nand2Tetris looks really cool. Just signed up to audit the course. Thanks for mentioning it.

[gaius]

Some, including myself, take it as a slightly negative signal. I’ve seen too many paper tigers.

I take almost any MOOC as a positive signal that this candidate invests their own time in ongoing professional development, but I have encountered some who apparently learned nothing, maybe got someone else to take the tests or used a braindump site.

[scarface74]

So if a MOOC doesn’t necessarily show any competence in the skillset that they were studying and you still have to do the same technical screening, then having a MOOC certificate is neither positive or negative.

Also, if you do a proper technical screen, you should be able to find out whether they are keeping up with technology. In that case, why does it matter if they get it through a MOOC, Pluralsight, YouTube, etc.?

[gaius]

Well, it’s a pre-interview signal, like anything on a CV. Previous employers, alma mater, extracurriculars, yadda yadda. If I’m compiling a shortlist to interview, completion of a relevant MOOC from a quality provider such as EdX, the Andrew Ng course, etc, will increase the probability of them being on it. It’s not just the technical skill either, it’s the good attitude of starting and finishing something above and beyond normal duties.

[eksemplar]

We take certifications very serious in enterprise, but it depends on what type of certification and who issues it.

I mean, I get that a swarm of developers are brushing up on sites like coursea, Udemy, threehouse or whatever, but I’ve seen a few of those courses, and a lot of the instructors really shouldn’t be teaching anyone. I’m not sure a coursea certificate would be an advantage, because the quality of coursea isn’t recognized, but on the other hand, I really don’t see how it would be a disadvantage either. If it’s all you got, I probably wouldn’t hire you, but if you’ve got a CS degree and a coursea certificate, I really don’t see why that would ever be negative.

A Microsoft or Cisco certificate on the other hand is valuable and will get you a long way though. On the organizational site, prince2 is also really valuable even though the prince2 way of doing pm is somewhat dated.

To show you just how valuable they are, we wouldn’t hire a network engineer who isn’t Cisco certified and we wouldn’t hire anyone for operations who hasn’t completed a bunch of Microsoft and Azure certificates. Unless we absolutely had to, but then we would enroll you in the courses and get you certified.

[scarface74]

I’m not in operations so I don’t have context. But operations seems to value certs more than development. Why is that? No one really cares if a developer has a certain cert but the infrastructure guys do.

I’m assuming by “Microsoft Cert” you are referring to operations certs and not developer certs.

[stephengillie]

Like everything on the resume, the point of certificates is to get a first interview. And some people use them to convince others to take them seriously in their daily lives.

[boojums]

If you don't mind me asking, why did you want to change from bit twiddling to enterprise?

[scarface74]

Money and optionality.

10 years ago after staying at one company for 9 years and before that at another for three doing mostly C and C++ with a smattering of VB6, Perl, and JavaScript, I looked at where most of the jobs were in my local market - a major metropolitan area with a lot of Enterprise Java/.Net and web jobs.

I had two offers one for $20K more than I was making as a C++ developer and one as a high entry level .Net developer paying only $7K more (yeah wage compression is real). I took the second offer. I knew in 3 years my options were going to be limited for C jobs.

10 years and 4 additional jobs later and making $70K more, it was the right decision. There are very few jobs making what I make now in my local area for C developers, plenty for “Enterprise” developers/architects.

Going back on topic about certificates, I’m again at the same crossroads. I got to where I am being mostly a backend developer/architect with very little modern front end experience. Also, in my local market, “full stack web developers” are becoming a commodity and make less than I do now.

So along with learning $frontend_framework_of_the_week just to check off the box, I’m working on AWS certs, since they are still marketable and the only way I can make the next jump without going into management is by being an overpriced “implementation consultant”.

[toyg]

Aws won’t be marketable for long. It’s the sort of skillset that is a prime candidate for offshoring, and the bar to enter is very low.

[scarface74]

There are basically a few parts to dealing with AWS:

Net ops: traditional networking, patching, security groups, manually provisioning resources, web based load balancer, and the kinds of things you do on prem. This part is easily outsourced. Also companies that just do a lift and shift of an on premise mindset usually would be better off with a cheaper VPS solution or even colo.

Devops: CLoudFormation (Infrastructure as Code), Code Deploy, CodePipeline, OpsWork (Chef?). Most of the outsourced labor and honestly most of the AWS support companies don’t have a clue about this.

Development: Databass optimizations with SQL and NoSql data stores (DynamoDB, ElasticSearch), Autoscaling, SQS, SNS, lambda, etc. Again, most outsourced labor don’t have a clue about this either.

Most “AWS Architects” come from a traditional networking background and that’s all they know. They take their knowledge and map it very badly to AWS.

Someone who has done full stack software architecture and knows AWS inside out from a development, devops, and net ops perspective will be competitive.

Just like all other outsourcing, companies keep the “architects” in house or local and outsource the commoditized development.

Straight “AWS Architects” make less than “full stack developers” and wages for both seem to be stagnating in most of the US. Architects/Team Leads are making more but that’s stagnating too. But good “Implementation Consultants” who can combine both are making more. If you can market your consultants as “Cloud transformation consultants” (Yes I died a little saying that) you can make a lot more.

[gaius]

Coursera will not grade the end-of-section quiz for non-paying students.

EdX grades exercises for audit students, on every course I’ve done there the free/paid experience is identical, the only difference is whether they issue a “verified certificate” at the end. Often people audit courses and only upgrade to paid once they have a passing grade.

[ItsMe000001]

I just found the first couple of courses on edX where graded exercises are only for those who pay for a "certificate". It's all courses in a UC San Diego "MicroMasters" program around data structures and algorithms. That's all the programming exercises, more than 80% of the value of those courses for most people. Unfortunately I found the quality not worth the price, and I would say that is true even for beginners (I have a CS degree from 20 years ago and only watched to remember a few things I forgot, just for fun).

Anyway, I think this might be a sign for the things to come. A lot(!) of edX courses, despite being okay or even good, have very low participation judging by forum activity. I think the course makers at the universities are going to feel more and more pressure from money-counting management about cost/benefit for the institution.

It's sad that to me to see that everyone does their own (often inadequate) little course instead of working together to create something bigger, better. I would like to see platforms like edX and Coursera not as market places like Ebay for lots of still very traditional little courses, but as platforms that encourage jointly working on something better (in addition to the current model, not as total replacement). Of course, 90% of why they don't (can't) do or become that is that those making the courses won/t or can't do it. Everybody works in isolation on their own small course(s).

[whiddershins]

I think that’s a great point.

A ways back I took 1/3 of an MIT OCW course and it was incredibly impactful.

I guess some would say I should have completed it but that can be arbitrary. I got what I felt I needed.

[rhizome]

Not to mention that if you watch anything at Khan Academy, they jiggle the page and nag you every 30sec to log in to save your progress (even when you dismiss the notification).

[MisterOctober]

Taking this opportunity to gripe about KA : I took their intro to chemistry, and the production quality of the videos was so erratic that I thought it might be a joke -- e.g., first video disjointed, second coherent but sounded like it was recorded over a '50s telephone, etc etc.

Hate to look a gift horse in the mouth, but this one needs a housecall from the vet

[ItsMe000001]

Would you have a link? I found those videos that I did watch pretty good, definitely not lacking in quality. That was mostly math, just a few chemistry videos (some 20?).

[MisterOctober]

Yep! The first few videos here --

https://www.khanacademy.org/science/chemistry/atomic-structu...

[just_myles]

You're not. I only ever need specific portions that I can't get on youtube tbh. I don't agree with the assertion that attendance makes a better student. We all learn on the job at some point and that is where the real learning begins.

[swaggyBoatswain]

I do this all the time as well, I audit the entire course quickly to see what gaps of information I don't know. Or search out a specific topic only, on something I struggle with

[blihp]

I'm the same way but I think people like us tend to be heavily over-represented on HN and probably represent a small fraction of the total audience. Similarly, I like to watch YouTube videos for specific technical content where the vast majority seems to prefer pop culture and cat videos.

A couple of other personality types / situations to consider:

1) Some people treat learning the way they treat a workout... procrastination is the path of least resistance. So metrics like course completion % 'gamify' it for them to give them the push they need to keep motivated.

2) In the business world, think of the times someone has said 'I'm not trained in that' (or something along those lines, often to try to get out of doing something) and the boss says 'well, go get trained in that.' Course completion is a thing that can be shown to said boss to 'prove' they learned something. While self-learners would tend to keep their mouths shut about not knowing it and say 'sure, no problem' and then go find whatever training materials were needed to learn said thing.

[jjeaff]

I was thinking about this the other day. The idea that most people are just watching garbage on YouTube 'might' be true, and that viewing educational or technical content is an outlier, but I'm not so sure.

Sure, there are countless massively popular channels with garbage pop content, but that may be due to the "general" nature of the content. The broad audience keeps it at the top of the most watched lists.

I think if you aggregate all the technical content, the "long tail" might be more numerous than the hump of the curve.

There are channels about programming, woodworking, plumbing, HVAC, construction, 3d printing, PC building, electrical engineering, mechanical engineering, knitting, photography, cinematography,graphic design ... it just goes on and on. Sure, most of those channels may only have a hundred thousand or fewer subscribers (with some exceptions in the millions) but I wouldn't doubt that in aggregate they surpass junk like moviemojo and cat videos in time watched.

Youtube is an amazing phenomenon that I think doesn't get enough credit because of all the junk on there, but the fact that I can search for how to remove the abs module on a 2007 BMW 750li and get multiple videos showing how to do it just blows me away and definitely makes it a lot easier to get things done.

[InitialLastName]

> Youtube is an amazing phenomenon that I think doesn't get > enough credit because of all the junk on there, but the > fact that I can search for how to remove the abs module on > a 2007 BMW 750li and get multiple videos showing how to do > it just blows me away and definitely makes it a lot easier > to get things done.

One of the garbage things about YT, though, is that the third thing on that video's "watch next" list will be some dude telling you that ABS part of a global conspiracy to get mind-controlling computers into every car.

[jjeaff]

It doesn't take long to recognize and avoid that type of video. But man, a single click on one of those videos will take you down a rabbit hole so fast.

[stephengillie]

Consuming irrelevant information is its own form of procrastination.

[jjeaff]

How do you define irrelevant? I consume useful information very frequently.

[MaxBarraclough]

Why else would we be here?

[Alex3917]

Right, signing up for a MOOC is basically just a way of bookmarking it. It’s not really any different than how most people don’t buy all the stuff they add to their Pinterest boards.

[nicodjimenez]

Great point, courses are way too long. If a course could be completed in a day you'd have high completion rates.

[vezycash]

For me, it's the length of individual videos that matter.

I prefer those with 4-7 minutes minutes per video. I've noticed that many long videos can be much shorter with minimal editing to remove unnecessary pauses, uhmms...

[MisterOctober]

Definitely. Especially for courses in the 20+ hour range, long lectures [say, more than ten minutes] can be a real drain.

Though I think most of the online courses I've taken have delivered exceptional value, as you say, the "one-take-and-done" editing style is ubiquitous and leaves one to sit through a lot of ummmming, digression, and jabbering.

[Come to think of it, so did most of my courses at university!]

[philipov]

I'm the opposite. When searching for lectures on youtube, I filter to remove anything shorter than 20 minutes and then ignore anything that's still under an hour.

[vezycash]

On platforms like udemy, the good courses range from 1.5 hours to as much as 97 hours. For this length, the shorter the better for individual videos.

For YouTube playlists, the same. However, for one-off youtube tutorial videos, I do long videos as well.

[ItsMe000001]

Not for me. I've taken and completed well over 50 courses by now (took health-related time off work for a few years but when I felt better wanted to do something), and the most frustrating courses where Udacity courses (and a few of the edX courses) where it felt like I had to click Next Next Next Next continuously, could not take my hand off the keyboard and lean back in my chair because there was the next silly easy question to answer and "Next" to click for the next short soundbite video. I definitely prefer well-made(!) longer videos.

The worst is when they start each and every 3.5 minute video with a ten second introduction, for those who already forgot what course they are in and what they just heard last only three minutes ago.

When I see things like this, or the doctor that I saw yesterday (who didn't know me) immediately switching to a "baby talk" style (for all conversations with her patients, I later observed) I wonder what kind of people (students, patients) they have to deal with in their jobs that they switch to by default assuming the worst.

[soneca]

I agree that course completion might be a very flawed metric.

I made a career transition to software development and the resource that had the most impact, by far, in my learning was freeCodeCamp.

At the time FCC had only 3 certificates (now is 6 I guess), but I only completed the first one - Frontend.

[SonnyWortzik]

quite common. Many of my colleagues including myself will sign up for a course online only because we saw an interesting topic in one of their modules. We go straight for the module, learn what we want and move on, never complete the course.

[ndnxhs]

I have signed up for online corses just to see what the content actually is before deciding if I want to do it.