[CHY872]

'My customer has given me the documentation to arcane antiquated format X (insert pdf, but it includes 24-bit integers, hex encoded data, semantically signficant whitespace). Here is a sample of the message format, and this struct should represent the contents. Please write me a Python parser which takes an input file in the format and provides the output. In particular, given input X, the output should be equivalent to Y. There should be a set of unit tests for important functionality, which should explain to a reader what is complicated about the format'. is something that ChatGPT4 will just drop out a solution to that works in a few seconds.

If your job is to make an accurate parser for it, probably you want to hand code it. If your job is to make sense of the data the customer has provided you with, this is merely an impediment to your actual job, and ChatGPT has you covered. Yes, there'll be mistakes. But ChatGPT can do in a few seconds what'd take you hours.

[probably_wrong]

Would you apply the same criteria to code instead of file formats? As in "my customer has given me this code for architecture X and I need it working on architecture Y"?

For those who answer "yes" to the above I'd encourage them to read the story of the Therac-25 [1], a machine where hardware mechanisms in model A where replaced by software mechanisms in model B leading to a race condition that would dose patients with massive doses of radiation.

> Yes, there'll be mistakes.

"Over the following weeks the patient experienced paralysis of the left arm, nausea, vomiting, and ended up being hospitalized for radiation-induced myelitis of the spinal cord. His legs, mid-diaphragm and vocal cords ended up paralyzed. He also had recurrent herpes simplex skin infections. He died five months after the overdose." [1]

"But your honor, I specifically wrote 'add unit tests for important functionality'!"

[1] https://en.m.wikipedia.org/wiki/Therac-25

[smugma]

The story of the Therac25 is an important one. I first read it in college and re-read it every 5-10 years.

That said, I don’t think it’s a strong rebuttal to the arguments here. Context matters. Earlier poster mentioned they’d use it for a React front end but not a backend deletion.

Knowing the tool and the appropriateness of the tool in context is one of the lessons to be learned from the Therac 25 tragedy.

[CHY872]

'Please translate this AVX-256 algorithm to Arm NEON' is absolutely a prompt I'd give ChatGPT, and similar prompts have revealed new and useful intrinsics. Of course, results to be checked.

Assurance is a complex topic, and any safety critical device should have a carefully thought through architecture and rigorous testing program which minimises the risk of incidents. It therefore seems scarcely relevant here, beyond the fact that a well defined delivery system should be able to handle multiple human errors during implementation without leading to crucial failure modes occuring.

[nine_k]

This is a great example of LLM usefulness. It also illustrates how it's half of the solution. Certainly the sense-making part works reasonably well. What it lacks is the formal rigor part. With them both in place, the computer could generate a perfectly correct solution, not just approximately correct.

Computers are very good at formal rigor, and we have quite some rigorous methods of program synthesis.

Whoever manages to connect these dots will own much of the industry.

[matheusmoreira]

I tried to get it to show me how to implement a feature I wanted in GCC once. It seemed like the code would get worse with every iteration. It did provide a very useful high level overview of the codebase and abstractions involved though. I don't think I'd ever have even gotten started without that help.

[bluefirebrand]

If you don't understand the format yourself how can you ever trust that GPT is actually giving you an accurate result?

The problem with using an ML model to parse stuff you don't understand is that you then have no way to verify the accuracy

If there are no stakes to the results then that's fine to trust blindly but if this is something you need for your job, that's risky and franky stupid to trust to ML

[CHY872]

Yes, if you take a problem you don't understand, ask a GPT to write a solution, do nothing to check the solution, trust it blindly, and then use the solution for some safety critical problem, you're playing with fire. But there's a spectrum, and please don't assume I'm at that end of it.

Lots of the time you understand the problem, but the problem is repetitive. Parsing a weird file format might well be that. Beyond that, you have solutions that are easily checked. For example, if I ask ChatGPT to optimise an algorithm for a certain CPU cache, I can easily read whether it did that. And then, there are parts of a software job that are crucial and subtle, and parts that are not.

As a practitioner, traditionally that leads to a shift in the focus and speed with which you approach a task - some pieces of code are 100 lines that took you 2 weeks to get to and were hard fought, some are 2000 lines which you wrote in a day.

Lastly, so much of solid software is being able to understand a probably unfamiliar domain, and ChatGPT can be a great buddy in terms of gaining problem context, finding the limits of your own understanding.

I don't use co-pilot like things, but I've found ChatGPT to be a massive enabler in terms of being able to be productive in unfamiliar problem-spaces.