[TheOnly92]

It definitely depends on the task at hand, but when you're teaching programming you don't teach stuff with boilerplate. Using ChatGPT for reference to replace Google/Stackoverflow was definitely one of the ways I'd expected the students to utilize it, but it probably wasn't providing answers in ways a beginner/novice could understand.

I'd expect simple tasks like calculating RMSE to definitely be within the abilities of LLM, you might combine things like actually reading the vectors from a CSV file (or a custom format) and calculating RMSE then sorting them etc to see the limitations of LLMs. Most students have no issues with calculating RMSE, they have issues with trying to do all the other stuff that leads to it, and then the combination of sorting and other tasks.

Regarding the restrictions, most of them are just don't use itoa/strtod or strcpy or some other standard library functions.

[cfn]

Thanks, yes, RMSE is a simple task, I was focusing on its ability to translate the name (raiz quadrada do erro quadrático médio) correctly. It is funny that the Portuguese code has Portuguese comments but the name of the function is calculateRMSE even though I don't mention RMSE in the prompt.

I agree with you, in my experience, ChatGPT is a better search engine but it is not capable of composing the various parts of an application in a cogent manner. I also think that the current UI is not appropriate for software development and I am sure there are efforts going on to create something closer to Jupyter notebooks for programming. That may be a game changer for your students (and you).

[TheOnly92]

True that based on my experience the variable names and function names remains in English despite the prompt, maybe its just the convention overall in the programming world, or maybe ChatGPT is finetuned to do so.

I don't think Jupyter notebooks or like similar REPL interfaces will help too much for my course, at least in the current syllabus. I'm aiming to teach about pointers, memory management etc, the more fundamental parts of how to interact with computers instead of a high level language. Though I would agree that the current UI is suboptimal, some improvements in allowing students to visualize memory layouts and see how their code manipulates memory will help a lot.

[HappMacDonald]

I'd hazard to guess that REPL of a simple virtual machine would work wonders for teaching about pointers and memory management.

I can't recall exactly but I think https://godbolt.org/ might do that for example?

[TheOnly92]

A simple virtual machine might be nice, but imagine the pains of trying to guide students to install something across different environments.

Godbolt is a compiler explorer, it shows disassembly of a code but there's nothing to visualize each step in the process.