[selridge]

No the same question CANNOT be asked of source code because it can execute.

You might as well ask for a record of the conversations between two engineers while code was being written. That's what the chat is. I have a pre-pre-alpha project which already has potentially hundreds of "prompts"--really turns in continuing conversations. Some of them with 1 kind of embedded agent, some with another. Some with agents on the web with no project access.

Sometimes I would have conversations about plans that I drop. do I include those, if no code came out of them but my perspective changed or the agent's context changed so that later work was possible?

I don't mean to be dismissive, but maybe you don't have the necessary perspective to understand what you're asking for.

[dang]

> maybe you don't have the necessary perspective to understand what you're asking for

Please don't cross into personal attack. You're making fine points, and that's enough.

https://news.ycombinator.com/newsguidelines.html

Btw, I think this is a particularly good point: "You might as well ask for a record of the conversations between two engineers while code was being written. That's what the chat is."

That's a good reframing. I can see why it might be impractical to share all of that, hard to make sense of as a reader, and too onerous to demand of submitters.

Since you have experience in this area, I'd like to hear your view on what we could reasonably require submitters to share, given that the flood of generated Github repos is creating a lot of low-quality submissions that don't gratify curiosity and thus don't fit the spirit of either Show HN or HN in general.

Some people would say "just ban them", but I'd rather find a way to adapt to this wave, since it is the largest technical development in a long time, and the price of opposing it is obsolescence.

[selridge]

"maybe you don't have the necessary perspective to understand what you're asking for"

this is in no way a personal attack. It's just a statement that's true. I didn't imply anything about them or their character or limitations, but they might not have the necessary perspective if that's the question they are asking.

I think it's critically important people figure out what they want to learn from what's being shared.

What do you need from submitters here? Even setting aside the burden of supplying it, what do you hope to learn?

[dang]

> I think it's critically important people figure out what they want to learn from what's being shared.

> What do you need from submitters here? Even setting aside the burden of supplying it, what do you hope to learn?

I appreciate your comments on this - they are the most interesting responses I've seen so far about this question (so I hope the meta stuff doesn't get too much in the way).

The hope is to make the submissions of AI-generated Show HNs more interesting than they are when someone submits just a repo with generated code and a generated README.

The question is what could, at least in principle, be supplied that could have this desired effect.

[dang]

(I thought I'd fork my reply to keep the meta stuff separate from the interesting stuff)

I believe you that it wasn't your intention, but when you address someone in the second person while commenting negatively on their perspective and understanding, it's going to land with a lot of readers (as it did with me) as personally pejorative. It's common for commenters (me too of course) not to perceive the provocations in their own posts, while being extra sensitive to the provocations in others' posts. If the skew is 10x both ways, that's quite a combination. It's necessary to remember and compensate for the skew, a la "objects in the mirror are closer than they appear".

Edit: total coincidence but I just noticed https://news.ycombinator.com/item?id=47115097 and made a similar reply there. I thought you might find this amusing, as I did.

[btrettel]

> maybe you don't have the necessary perspective to understand what you're asking for.

I disagree. Thinking about this more, I can give an example from my time working as a patent examiner at the USPTO. We were required to include detailed search logs, which were primarily autogenerated using the USPTO's internal search tools. Basically, every query I made was listed. Often this was hundreds of queries for a particular application. You could also add manual entries. Looking at other examiners' search logs was absolutely useful to learn good queries, and I believe primary examiners checked the search logs to evaluate the quality of the search before posting office actions (primary examiners had to review the work of junior examiners like myself). With the right tools, this is useful and not burdensome, I think. Like prompts, this doesn't include the full story (the search results are obviously important too but excluded from the logs), but that doesn't stop the search logs from being useful.

> You might as well ask for a record of the conversations between two engineers while code was being written.

No, that's not typically logged, so it would be very burdensome. LLM prompts and responses, if not automatically logged, can easily be automatically logged.

[selridge]

> LLM prompts and responses, if not automatically logged, can easily be automatically logged.

What will you do with what you’ve logged? Where is “the prompt” when the chat is a chat? What prompt “made” the software?

If you’re assuming that it is prompt > generation > release, that’s not a correct model at all. The model is *much* closer to conversations between engineers which you’ve indicated would be burdensome to log and noisy to review.

[btrettel]

> What will you do with what you’ve logged?

Could be a wide variety of things. I'd be interested in how rigorously a software was developed, or if I can learn any prompting tricks.

> Where is “the prompt” when the chat is a chat?

> The model is much closer to conversations between engineers which you’ve indicated would be burdensome to log and noisy to review.

I disagree. Yes, prompts build on responses to past prompts, and prompts alone are not the full story. But exactly the same thing is true at the USPTO if you replace "prompts" with "search queries" and no one is claiming that their autogenerated search logs are burdensome.

Also, the burden in actual conversations would come from the fact that such conversations are often not recorded in the first place. And now that I think about it, some organizations do record many meetings, so it might be easier than I'm thinking.

> What prompt “made” the software?

All of them.