[ahamilton454]

I like that you point out that the cost to build software is still not 0. And in my expirence it’s further from 0 than I would expect. I often find myself thinking I can rebuild a project (or usually improve upon an existing one) in just a few days. And yet when it comes down to making anything well, it still takes time and iteration.

It’s a bit funny because I felt this way before coding agents as well, like you could clone something in just a few weeks. But in practice my expectations are rarely accurate.

[galaxyLogic]

I find myself fixing the spec of my software often, and that makes lots of existing code obsolete. Creating working code is getting cheaper with AI, but creating great specs which make the software easy and intuitive to use seems to be more difficult for AI.

Why? Because you have to actually use the product to discover what is wrong, or sub-optimal, with it.

[ahamilton454]

Yeah exactly, I literally don’t know how to change my spec until I’ve gathered more data.

I was building a transaction classifier recently and I initially thought it would be a trivial “solved” problem. Throw transactions into a tiny local LLM, let it classify. But that approach was too slow, and not accurate enough. I didn’t know that though until I tried and then needed to change the spec.

[woadwarrior01]

You'd probably get much further along by fine tuning a small BERT style encoder model based classifier for it. IMO, even something as simple as training a linear classifier on the CLS token embeddings from a frozen encoder might work.

[ahamilton454]

Yeah, Ive tried a bi-encoder, cross encoder and some small LLMs so far. I think I’ll do BERT soon too

[dijksterhuis]

age old machine learning wisdom: start with the simplest model, then try complex ones later

[spwa4]

So wait ... you're not even going to train based on what you want, just "throw into"? Did you actually put in work on a very clear and accurate prompt with a full manual on what to do?

[ahamilton454]

Throwing a tiny little LLM at it helped me assess that it was far too slow for me to reasonably use at the scale I needed. So it didn’t really matter how accurate the prompt was. I was more just pointing out that I didn’t know if would be too slow without trying it. I maybe could have done some simple math in retrospect, but trying it out was easy enough

[Exoristos]

Not every lottery winner has a detailed strategy.