[restofus]

tldr; The experts make it look impossible but the masters make it look easy and actually teach you

Just ranting out here so feel free to ignore :-)

This is not really aimed at the author but towards the "elite" group. There was another elite commentator in one of the other thread who said he dropped out of ML class because This course included gems such as "if you don't know what a derivative is, that is fine" and he thought math was important in ML. Before the ML class I could not even argue with these guys because I did not know squat about AI and talking to these experts their advice was to take a year off and learn math and then start learning AI which in my case was not possible. Today after a couple of months of online classes I am actually using ML in my daily work and its not magic that only the elite with deep profound math knowledge can use. Another programmer who is working in khan academy actually had a blog post about how he implemented ML by learning from Prof Andrew's class now that is real world impact. I may be missing something but can one of you experts please explain why you need deep math knowledge when the professor who has been doing a lot of research in this field a lot more than you does not think so ?. The professor in his classes keeps reassuring that even after using it for so many years he has difficulty in the subject but I'm guessing these experts know it all :-).

This is the reason in my opinion even though wall street is full of smart people they do not care about the rest of the population or the general masses the attitude is we are smart and we can do what we want you guys are dumb and deserve what you get and if someone outside of their elite group starts talking their language they do not like it.

On a similar note when you look at the people complaining about khan academy most of them are these so called smart people.

Let me talk about my background I have been working as a programmer for around 11 years , no math background though thought myself math by using Khan academy and before my layoff (now am working on my own startup ) used to make 90K (in a southern state).

So guys you are not the center of the world we are crashing into your fraternity you are no longer the only experts who can talk about ML , the guys at stanford are smarter than you and know what they are doing and FYI they don't need you its the other way around. Another interesting thing is that mostly the current students seem to agree with the author, If you are smart you should probably take the effort to learn more rather than asking them to tailor the classes to what you think matters . Also ask yourself this question if you were the Professor what do you think is more satisfying teaching 40 full time students or 20000 who are in the field already and make more impact in the field ?.

[Dn_Ab]

I am not one of the elite. I had to drop out due to a not simple life. So I think I can offer an outside perspective. I did teach myself a lot of math to be able to understand machine learning. I did not read ahead, that is boring. But I found that as I went along, the more I wanted to tune the algorithm beyond the standard fare, the more power I wanted, well then the more math I had to read and understand. I simply was not satisfied to write decision tree code without understanding how. Or if you read a paper and you want to implement something you will need to be able to understand the math hiding the ideas because the pseudo code is often rubbish and bugged - assuming its even there at all. I did this to implement matrix factorization, to extend logistic regression beyond binary and with sparser weights, SVM etc.

And then something began to happen the more I learned. I no longer thought I need naive bayes this, Decision tree that, random forest there or whatever. I thought I need this concept from statistics or that idea from information theory, i just need to group and count there and that loss function is useful here. So I could come up or modify something to my need. As I go long I am finding that while before I looked for an excuse to use something fancy sounding now I prefer to go as simple as possible - but without having gone through the hard stage I could not appreciate where the simpler solution is better.

I also learned a great deal of differential calculus when implementing an automatic differentiator (a backpropogating Neural net is basically just a special case of reverse auto diff). Its fast can work with decent sized vectors (10^5 - 10^6 entries I tested) and can do gradients, hessians and jacobians of arbitrary functions. I also expect that I can easily extend it to be able to work with tensors although I haven't needed them yet. Using it I wrote a stochastic gradient descent algorithm and can plug in arbitrary loss functions and a whole bunch of algorithms just merge. I could also easily write say L-BFGS for it. Neural networks, logistic, linear regression, support vector were basically just swapping out one line.

This flexibility is what you gain.

===========================

In the below fn is an arbitrary mathematical function such as

  let eq1 (x:float[]) = 1. - 4. * x.[0] + 2. * x.[0] ** 2.- 2. * x.[1] **3.

  let newtonOpt prec iters fn (guess:float[]) = 
    let rec iterate delta iter cguess = 
     match delta with
      | _ when delta < prec || iter > iters -> cguess
      | _ ->    let h = hessian cguess fn 
                let _, g, _ = grad_ cguess fn
                let gs =  cguess - m.Inverse() * g
                let cdel = (gs - cguess).Norm(2.)
                iterate cdel (iter + 1) gs
    iterate Double.MaxValue 0 guess

example of a loss function

  [<ReflectedDefinition>]
  let llog (cx:float[]) _XdotW y  = y * log (1./(1. + exp(_XdotW))) + (1. - y) * log(1. - (1./(1. + exp(_XdotW))))

[trunnell]

Sigh. I disagree.

It's not about you. Math does not exist to exclude you. People who use math are not trying to exclude you, they are just trying to make their jobs easier. Math can be just as accessible as this ML material when taught properly.

If your point is that more effort needs to be made to teach advanced topics in accessible ways, well, duh. That's a straw man.

I'm sorry that you feel so alienated from institutions like Stanford that you feel the need to compare their students to "wall street elite" that "do not care about the rest of the population" who "deserve what they get" for "being dumb." I think this rhetoric is out of line on HN.

If the author wrote, "CS 229A lets in the riff raff, I don't like those people" then you'd have a point. But that's not what he wrote. He's simply looking to learn more and get his money's worth. As others here pointed out, he probably should have taken the full CS 229 instead.

[romaniv]

Agree.

A lot of the things I've learned in AI class are pretty darn simple, once explained properly. So far all this knowledge has been hidden behind jargon and badly defined notations. IMO, the guys at Stanford are liberating it.

Moreover, this is the first time in my life I'm actually seeing a practical use for things like derivatives in programming I can relate to. Sure, I know there are, theoretically, lots of different applications, but I've never seen them since college. Seeing an example of a practical application makes me more interested and more likely to deepen the knowledge of the underlying math.

[sausagefeet]

This could just be an expectation problem. I think most of these free courses have that 'Applied' stuck infront of them, ok, fine. But for my education I want rigor. I don't want to just know how to use something, I want to know how the guy who came up with it figured it out and I want to be bale to prove things about it. Not having to know what a derivative is does not fit this. I don't think it's a matter of the elite thinking only elite people can grok something like ML, it's a matter of that they expect the dirty details and are annoyed when they don't get it.

[noblethrasher]

I know ML and have a math degree; I don't see why you'd need to know what a derivative is. If anything, I think exposure to ML and functional programming before differential calculus could be beneficial since you'd better appreciate that differentiation is just one special application (no pun intended) of the concept of higher order functions.

[flatline]

If I recall, solving back propagation in multiple-layer perceptrons was an unsolved problem for some time, and the solution relies pretty much solely on partial differentiation. I don't know much about ML but things like neural networks were pure mathematical constructs before they were CS topics. I agree with the GP, though, you don't need to know the actual math for most of this stuff.

[disgruntledphd2]

On the point regarding the necessary knowledge of maths for ML (or indeed statistics which is the same material but a slightly different focus), I'm conflicted.

Coming at it from my perspective (learned a lot of math in high school, forgot most of it until I started a PhD), i would agree that a lot of the time, you don't need to understand the mathematical underpinnings of this stuff. That being said, as I've learned and remembered more of the math, my capability to understand (and debug errors) of all of this has increased tremendously.

I do think, if you intend to use ML every day, then you need to commit to understanding everything you use within a certain time frame of you beginning to use it (ideally immediately but that's often not possible). Anyway, derivatives are cool, and transform the way you look at the world, so you should definitely learn some of those.

[volodia]

Well for example take the least squares formula \theta = (X^T X)^{-1} X y. You derive that by setting a derivative to zero, and solving for \theta. If you don't know what a derivative is, then you're just using an equation that came out of nowhere and that you don't understand.

[restofus]

If you want to know how the guy came up with it , do you think someone taught him that or he delved into the topic deeper by himself and figured it out .The professor makes it clear that you can do the derivates if you know so I do not understand how the course would be better if that was given as a assignment. Also if the professor thought the same way why would he want to teach a class instead of working for a bigco or a research firm ?

[sausagefeet]

Of course the guy who came up with it taught himself, there was nobody to teach him. The point of a college is to speed a lot of this up, though, and tell you the results these people got and how and the implications. I think you're drawing a bigger contrast between what we said than I intended. I'm not saying that Applied courses are wrong or should not exist. I have taken several in my day because I just don't have the mental discipline to learn a lot of theory all the time. What I am saying is that I don't think really smart people complaining about not getting enough out of the course is not a matter of them thinking they are elite but that they just want more and don't see the point in wasting time on something that won't give it to them. My favorite classes in college were always the ones that required a lot of theory and a lot of rigor because it brought so much more together for me (even though I often didn't really understand it), I think a the elite you speak of probably have similar feelings.

[clyfe]

> khan academy actually had a blog post about how he implemented ML

I belive that this might be the article in question:

http://david-hu.com/2011/11/02/how-khan-academy-is-using-mac...