[richardles]

I've also noticed that LLMs are really good at speeding up onboarding. New hires basically have a friendly, never tired mentor available. It gives them more confidence in the first drafted code changes / design docs. But I don't think the horse analogy works.

It's really changing cultural expectations. Don't ping a human when an LLM can answer the question probably better and faster. Do ping a human for meaningful questions related to product directions / historical context.

What LLMs are killing is:

- noisy Slacks with junior folks questions. Those are now your Gemini / chat gpt sessions.

- tedious implementation sessions.

The vast majority of the work is still human led from what I can tell.

[candiddevmike]

That sounds like a horrible onboarding experience. Human mentors provide a lot more than just answering questions, like providing context, comraderie or social skills, or even coping mechanisms. Starting a new job can be terrifying for juniors, and if their only friend is faceless chat bot...

[richardles]

You're right. We need to keep tabs on the culture for new hires for the reasons you mentioned. LLMs are really good at many onboarding tasks, but the social ones.

I think done right it is a superior onboarding experience. As a new hire, you no longer have to wait for your mentor to be available to learn some badly documented tech things. This is really empowering some of them. The lack of building human context / connections etc is real, and I don't think LLMs can meaningfully help there. Hence my skepticism for the horse analogy.

[skywhopper]

If they’re badly documented, how does the LLM help?

[lbreakjai]

This sounds horrible. Onboarding should ideally be marginally about the "what". After all, we already have a very precise and non ambiguous system to tell what the system does: the code.

What I want to know when I join a company is "why" the system does what it does. Sure, give me pointers, some overview of how the code is structured, that always helps, but if you don't tell me why how am I supposed to work?

$currentCompany has the best documentation I've seen in my career. It's been spun off from a larger company, from people collaborating asynchronously and remotely whenever they had some capacity.

No matter how diligent we've been, as soon as the company started in earnest and we got people fully dedicated to it, there's been a ton of small decisions that happened during a quick call, or on a slack thread, or as a comment on a figma design.

This is the sort of "you had to be there" context the onboarding should aim to explain, and I don't see how LLMs help with that.

[8note]

you still lose a bit from not having those juniors' questions around - where is your documentation sucking or your code is confusing?

[NitpickLawyer]

We are now at a point where the tech can help with both of those, today. You can have a cc session "in a loop" going through your docs / code and try to do x and y, and if it gets stuck, that's a pretty good signal that something sucks there. At least you can get a heatmap of what works ootb, and what needs more eyes.

[baq]

Both questions are getting scary good answers from the latest models. Yes, I tried, on a large proprietary code base which shouldn’t be included in any training set.