Hacker News new | ask | show | jobs
by mingyeow 1164 days ago
Noob question here - what’s the best tutorials to get started in mixing LLM models and building on top of one another, assuming very good programming background but little AI background? I asked chatGPT this question, and it was helpful but not comprehensive, but I figure intelligent humans on this forum will give the best answers.
2 comments
extasia 1164 days ago
My answer would be quite specific to what exactly you're trying to achieve.

Id be wary of just hacking away without understanding at least the fundamentals of ML + NLP or you'll find yourself lost pretty quick.

I'm a former SWE turned NLP researcher, so i was recently in your position:)

link
gmreads 1162 days ago
Curious why and how you did the transition? Rapid progress in this space I don't think SWE is a viable career for next 20 years.
link
extasia 1160 days ago
I felt that my software work in the gambling industry wasn't aligned with my ethical beliefs (also family member with gamba addiction..)

I really enjoyed the research I did in my undergraduate degree (Physics) and so when my partner suggested I apply to the doctoral training program (it's called a CDT in England) it was sort of a no brainer to shoot my shot - the project sounded interesting and the course was designed for those coming from industry.

tl;dr opportunity came up and I jumped ship from SWE to ML research.

link
RockyMcNuts 1162 days ago
not sure what 'mixing LLM models' entails but these are maybe some good starting points

- karpathy - https://www.youtube.com/watch?v=kCc8FmEb1nY

- https://towardsdatascience.com/beautifully-illustrated-nlp-m...

- https://dzone.com/articles/a-deep-dive-into-the-transformer-...

- https://peterbloem.nl/blog/transformers

- http://nlp.seas.harvard.edu/2018/04/03/attention.html

- https://lilianweng.github.io/posts/2023-01-27-the-transforme...

- https://blog.quickchat.ai/post/tokens-entropy-question/

- https://dugas.ch/artificial_curiosity/GPT_architecture.html

- https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

- https://d4mucfpksywv.cloudfront.net/better-language-models/l...

- https://arxiv.org/pdf/2005.14165.pdf

- https://arxiv.org/pdf/2303.08774.pdf

- https://arxiv.org/pdf/2303.17564.pdf

link