|
|
|
|
|
|
by tysam_and
1182 days ago
|
|
|
Like jimsimmons said below, I believe it traditionally refers to 'hidden'. which was in vogue at the time for both feedforward nets and RNNs as well as any other other neural networks in the 90's or so and on. This trend actually continued for a while and I learned it in one of Hintons' main online classes which was made somewhere between 2012-2015 or so IIRC (though I opted to switch to reading and trying to implement raw papers instead as my brain works intuitively strangely, on the whole). You can think of it as everything the RNN knows about what you're doing and a thing that evolves from place to place as you go. Because it is iterated on itself as a map, it abides by some very interesting properties that let it represent some very difficult functions, though actually attaining a representation of those functions is rather difficult indeed in my experience from what I've seen. There are one or two rather successful projects trying to keep RNNs both alive and competitive with transformers. I think they do very well on the whole, though the transformers seem to have slightly improved parameter efficiency, generally speaking. I hope this helps you with your question, please do let me know if you have any other follow up questions on this topic/matter. (: (: :) :) |
|
|