| I strongly believe technical interviews should try to mirror a pair programming session on a problem as if it was work, rather than a quiz or interrogation type format. If someone asked during such a session (where it's cameras, screenshare) that they wanted to do something like google some documentation, I wouldn't see that as a problem. Obviously it's a problem if someone just googles for a solution and pastes it in. I see LLMs in the same way. No issue with them using it do something like take the pseudo code they wrote in front of me and turned it into an implementation. Especially if they could talk through this code and make suggestions about further changes and so on, clearly showing they understand what's going on. The real concern is going to be when sophisticated agents can impersonate (clone voice and video) in a convincing way, as well as the capabilities to see the screen and type away as if it was a real person, and they're responding to you in real time. If the software is based on the models made by large companies, they'll be happy to give you recipes. They would refuse if the coding request mentioned something about cracking passwords or dumping credit cards. And all of them will have a meltdown if you try to ask them to say something politically incorrect (what a bizarre world that would be if that became the new captcha system for humans trying to figure out if they're wasting their time with a fake human). That said, this is going to be a cat and mouse game. There will be nothing to stop people from fine tuning models to get around being "jailbreaked" to reveal themselves as LLMs. Perhaps the best means is taking the time to research problems that causes "vibe coding" to completely fall down. And that is likely going to be things that are novel and haven't been littered all over the internet. That has a knock on effect of making such interviews a bit more interesting for the people doing them too. |