[cs702]

Often I find I spend days or weeks deciphering mathematics in compsci papers only to find the underlying concept is intuitive and plain, but you're forced to learn it bottom up, constructing the authors original genius from the cryptic scrawlings they left in their paper... and you realise a couple of block diagrams and a few short paragraphs could have made the process a lot less frustrating.

This is SO TRUE.

The same thing happens to me regularly, and not just with "computer science" but with other technical fields, hard sciences, and mathematics. The purpose of most academic papers is not to explain (let alone teach!) ideas in an intuitive manner, but rather to express them in formal, correct, unambiguous terms -- that is, to make them as accurate and critique-proof as possible for publication in some journal.

[pygy_]

> The purpose of most academic papers is not to explain (let alone teach!) ideas in an intuitive manner, but rather to express them in formal, correct, unambiguous terms -- that is, to make them as accurate and critique-proof as possible for publication in some journal. Their intended audience is subject matter experts.

Their purpose really is to let the authors show off how smart they are, impress their peers, and advance their careers. The other properties derive from that.

;-)

edit: I don't want to disparage research in general, BTW, but specifically, the scientific paper redaction process.

[ska]

I assume you mean the "reduction" process. The "redaction" process is an important part of the scientific process!

I know you were being sarcastic, but I think you are also being a bit unfair. Publication is necessary these days at perhaps an unfortunate rate, I agree.

However at least most research mathematicians publish largely to share ideas. After all, that's the best part of the job. Certainly this is true of the vast majority of them in my experience, and the ones weren't motivated by this tended to be very focused on teaching and very good at it.

[cmsd2]

Could be mr pygy_ isn't english..

e.g. redaction in german means editing

[coldpie]

Just as a specific example, I had this experience with Bayes' Theorem <http://en.wikipedia.org/wiki/Bayes%27_theorem>. As an informal paper for my computer security class, we used Bayes' theorem to implement aimbot detection in a simple FPS. It sounds like a big, complicated theorem with a special name that some genius had to come up with and has complicated notation involving probabilities and logic symbols.

And then it's basically (paraphrasing with reckless abandon) just the probably of your event divided by the total probability space. Lots of words and jargon and theory given in countless papers and articles, and it pretty much just boils down to intuitive addition, multiplication, and division.

And our aimbot detector actually worked pretty damn well! Just gather some data points to determine probabilities, plug them into the simple formula, and it was always correct in our test cases.

[ska]

"and it pretty much just boils down to intuitive addition, multiplication, and division."

Or does it? After all, probability is one of the very first mathematical tools to be divised, but a rigorous theoretical underpinning for probability and statistics had to wait until measure theory, millenia later. And this was not for want of trying.

Part of what makes things "boil down" to the simple and intuitive is years of hard work. Reading Newtons original work on the calculus is painful and convoluted, it took many hands to polish it to the point you might have seen it in.

If we've done our jobs well as mathematicians, eventually the essence of an idea will be easy to understand an apply. If you really want to understand it though, you may have to dig into some much deeper work. And often, as with many thing is in mathematics, human intuition will just tend to be wrong about it (e.g. the Monty Hall problem).

[nly]

It's the same with the crypto we rely on. There are some quite complex mathematical underpinnings for the primitives, but once understand their properties all the common protocols are actually incredibly trivial. DSA and various other iconic crypto protocols are just linear algebra using these new mathematical primitives. The complexity is about the same as using hash functions to build hash tables. Making protocols secure is actually mostly an implementation challenge (as it is with hash tables), and keeping systems secure, pragmatically, is about key management and social challenges.

I was incredibly proud when I noticed, just through algebraic manipulation, without reading it anywhere first, that you should be able to extract the public key used to create an an ECDSA signature. Schnorr signatures don't have this property. This is kind of sad in a way becaue it's trivial, but you have no know how the primitive functions and the difference between a field and a group.

[applesinjuice]

Fixed link for the lazy: http://en.wikipedia.org/wiki/Bayes_theorem

[coldpie]

Thanks. I reported this bug nearly two years ago :)

https://news.ycombinator.com/item?id=4112327

[abecedarius]

formal, correct, unambiguous terms

Unambiguous? I wish! The notation in most fields is way more ambiguous than code; you're expected to resolve the ambiguity by the norms of whatever research community it's for. Code isn't necessarily clear, but at least all the information is there to be decoded.

An example of trying to do better: http://groups.csail.mit.edu/mac/users/gjs/6946/sicm-html/boo...

[tipsy]

And what's sad is that this "formal, correct, unambiguous term" pattern doesn't only apply in academic papers, but also in textbooks and learning and education (e.g. have you ever read a statistics textbook?) -- when we are trying to teach students concepts, we end up getting the opposite of what we wanted to achieve.

[jseliger]

The purpose of most academic papers is not to explain (let alone teach!) ideas in an intuitive manner, but rather to express them in formal, correct, unambiguous terms -- that is, to make them as accurate and critique-proof as possible for publication in some journal.

In other words, the purpose is primarily tenure and advancement.

This may be a problem.

[logicallee]

The issue is that a lot of intuitive stuff is wrong. When you formalize, you remove the simple, intuitive explanation - but you make it much harder for you to remain wrong, if you are wrong - or to become wrong, if you started off right.

As a simple explanation, consider the difference between explaining the Monty Hall problem - which might seem to be philosophical, open to interpretation - and coding it up. The moment you code it up --- go ahead, code up a monte carlo simulation that compares the two alternatives of switching doors or maintaining the same choices, and spits out a running count of which is what percentage correct. I'll wait while you code. ---

the moment you do that, you see two lines in your code that make the explanation 100% irrefutable and completely obvious.

That is why papers are written this way. Intuition can go both ways.

[SilasX]

That would be more convincing if the scientific papers were written in a way that make the point as clearly as the coded-up version of Monty Hall problem. In practice, it's more like they publish the assembly code and when you ask why they didn't do it in Python or something, they lecture you about the need for formal rigor.

[tel]

I think there's generally selection bias about what part of an exposition makes the "a-ha" hit in two ways. First, your a-ha moment may not be the same as someone else's, but you're less likely to observe theirs. Second, your own a-ha is likely the product of a larger production than the moment itself of which you're most attuned to.

A good mathematical author must be guarding against both of these selection biases.

[johnbm]

Nah, Monty Hall is trivial to demonstrate. Just do it with 100 doors instead of 3. Problem solved, intuition remains.

[Crito]

Such that the contestant chooses 1 door, the host then opens 98, and they are given the opportunity to switch to the last remaining door?

That's actually a pretty brilliant way of explaining it. With numbers like that the answer becomes much more intuitive.

[logicallee]

Why 100? Why not just 5? Some people would 'get' it at five, some people at 100, and some people at a million. If you have to choose out of a million doors, and no matter what the host opens all but one of them, so that your prize is either behind the door you picked, or behind the other one -- then should you switch your choice?

Well, obviously, you should - with a million doors, it becomes obvious that you have just a 1 in 1,000,000 chance of having picked it.

But thing is - that "obvious" thing 'should' be just as obvious with 1000 doors, 100, 20, 5, or...3....

It's a matter of degree - not kind.

So appealing to a way of intuiting it that is a lot more 'obvious' - while in fact having the exact same format of question, just goes to underscore how fickle intuition can be.

That said, taking individual variables to ridiculous extremes is a great way to thought experiment and an awesome way to get intuition to work better.

[Crito]

Yeah, I'm not wed to a particular number... I'm just observing that larger numbers seem to make it more intuitive than, say, 3 doors.

[ColinWright]

I never understood this assertion. Most people I explain it to in this way still think it's 50:50 because you only have two doors left.

[Shamanmuni]

That's odd, I found the 100 doors example is the most efective way of explaining it.

It's easy, pick a door, then the host discards 98 doors in which the car isn't. Do they still think that the probability of the other door left is the same than the one they picked? I want to play gambling games with them!

[ColinWright]

So people I ask say this:

    Pick a door, the host discards 98 doors
    where the car isn't.  There are now two
    doors left, so it's 50:50.

Actually, I'm with them (except for the 50:50 bit). I don't find the 100 door version any more convincing than the 3 door version. Under the usual assumptions the reveal of the other doors gives no information about the one you picked, so that will always remain 1/N. The remaining door will therefore be (N-1)/N, which is bigger if N>2. So switch.

[johnbm]

Because it is near impossible to open 98 doors at random and not reveal the car.

[ColinWright]

To the people who have trouble with this, that's neither relevant nor helpful. They just say "two doors left, hence 50:50."