[vagabund]

The comments of every ML paper posted on this site are dominated by people either baselessly discounting the results as a party trick or illusion, or shoehorning in their conjecture about what approach the field is overlooking.

As someone just trying to learn more about the implications of new research, I find myself resorting to /r/machinelearning, or even twitter threads, to get timely and informed discussions. That's a shame, given what HN sets out to be.

[mountainriver]

As an ML engineer I found the comment insightful. I agree HN takes a critical approach to list ML but that’s largely because there’s been so much snake oil with it

[nynx]

I’m certainly not discounting the results and I don’t see anything wrong with suggesting what I think would generally be a good path to look at in the future.

[vagabund]

It's not wrong per se, and I'm obviously in no place to police the discussion, but it's only tangentially related to the post and often clouds out what would be a more pointed deliberation over this research.

Maybe I'm expecting too much of HN, but I've seen these same two top level comments under myriad ML posts.

Sorry for the meta-discussion that's gotten us further away from this really remarkable paper.

[nl]

I agree.

It's completely speculative. There is no evidence at all that Spiking NNs really work better is any circumstances.

Speaking as someone who has worked in the ML field, it feels to me like advocates for them are caught up in the biological plausibility argument. That's an interesting branch of research, but has very little to do with how AI should be implemented using transistors. In some ways the "neural networks" name has done a great disservice because people keep getting caught in the trap of comparing them to how the human brain works.

[robbedpeter]

Spiking comes with persistence baked in, so anything done with them has an implicit sequence and temporal context. Like LSTM, it automatically means the architecture is going to handle some problems better than a naive perceptron.

Transformers have a sequence context, but it constructs its own context dependent notion of orderliness with attention.

Persistent or recurrent activation states can extend the context window past the current tokenizing limitations. Better still would be dynamic construction where new knowledge can be carefully grafted into a network without training, and updates over the recurrent states feeding back into modifying learned structures.

Spiking networks might provide a clear architecture to achieve some of those goals, but it's really just recurrence shuffled around different stages of processing.

[nl]

> it's really just recurrence shuffled around different stages of processing

Interesting. I hadn't really thought about this. Although I wonder if there is a more direct way of achieving this.

[nynx]

Point taken, I do agree with you that it’s probably best to stay on topic in these kinds of posts.

[ceeplusplus]

As a community grows it attracts people who don't have the same background that drew the original members of the community together, so it becomes inevitable to see this kind of layman commentary. I've seen it happen to r/hardware which has been taken over by gamers with no CS background and AMD shareholders when it used to have a lot of knowledgable people commenting.

[nynx]

I don't claim to be an expert, but I actually do undergraduate neuromorphic computing research. So, I don't know much, but I do know a little about what I'm talking about.

[gwern]

Don't forget /r/mlscaling!