[mekael]

I had an heavy ai user on my team say that “those who learn how ti use the tools wont get fired, those who dont are gone”. I used it to generate a bunch of cfn and it worked fine from an example and a couple line prompt, doesnt seem that hard to learn to me.

Now reviewing the 1k lines it generated and making sure its secure, thats going to take me longer than writing it by hand.

[stavros]

Yeah, I think this is it. If you don't learn to use them, you'll be much slower than people who do, but also they're not really that hard to learn, so it's not super urgent.

[mekael]

I'm still confused about the things I'll be slower in though, and I'm being sincere with that confusion. If it's "boilerplate", then I haven't done enough research or pick a library which has little to none of that, or I'm not using the template(s) built into whatever framework I am using.

For example, in one of the projects I'm working on, I'm using the VSA pattern. I have the list of 50 to 75 features I need to implement and what "categories" they slot into, I have all of the frameworks and libraries picked out, and I have built out "feature templates" with all of the boilerplate setup (I'm reusing these over multiple projects going forward). for each of the features all I need to do is

'ftr new {FEATURE_TYPE} {FEATURE_NAME} {OUTPUT_FOLDER}'

and then plug int the domain specific business logic.

I'll most likely use Claude/Codex/Whatever to write out some of my tests, but the majority of the 'boilerplate' is already done and I'm just sorting out the pieces that matter / can't be automated.

Am I missing something huge with these tools?

Don't get me wrong, for doing reverse engineering they're great helpers and I've made a tonne of progress on projects that had been languishing.

[stavros]

I find that can write features 5-10x faster with these tools than by hand, at a comparable level of quality (though it hasn't been long enough for me to judge what'll happen in a year).

[mekael]

Would you be able to give an example of a feature? For my example, I need to query an ancient undocumented database , pull back a pile of data, do some validations on it, and then show it to the user or pass it along with another processing step. The human part is researching the database and the data living in it, and implementing the validation(s) while talking to a business user, everything else can be templated.

[stavros]

Oh yes, this is what LLMs excel at. Introspecting a database, either the schema or the live data, running a few checks to see whether all the data had the same shape (or how many different shapes it has), writing validations to catch edge cases, they do this extremely quickly and pretty accurately, whereas it would have taken me hours of trawling.

Then I can look at the output and say things like "what if the data is lowercase?" or anything else I suspect they may have missed. A few rounds of these and I have a pretty good feel for the quality of the resulting checks, while taking a few minutes of my attention/tens of minutes of wallclock time to do.

I have a more detailed example here: https://www.stavros.io/posts/how-i-write-software-with-llms/

I'd share all my plans but I once found that the LLM used my actual phone number as test data, so I don't share those any more, just in case.

[logicchains]

>Now reviewing the 1k lines it generated and making sure its secure, thats going to take me longer than writing it by hand.

Then you still need to learn how to use the tools to speed up reviewing the code.

[archagon]

You're not actually doing engineering if you're just vibe-coding, reviewing, and testing all the way down. What the hell is that? Just a weird simulacrum of software development that will break apart in unpredictable ways. Security consultants are going to have very lucrative careers in the coming years.

[mekael]

If I don't have experience with the underlying framework/language/thing being modified, it makes it quite difficult to trust the actual review. In this example, I haven't worked heavily with Cloudformation, so I can't call b.s if it leaves a database instance exposed to the wider public internet rather than in my company's private VPC.

[logicchains]

You can ask the agent to check that it doesn't leave a database instance exposed to the public, and present you with proof for you to check (references to the code and the relevant Cloudformation documentation). Then repeat this for all the things you'd normally want to check for in a code review.

[mekael]

In that case I'm just moving the reading of the documentation from reading it as I'm writing the yaml to when I'm doing a code review. Not saying it isn't helpful to have a pair researcher, just seems like I'm moving things around .