[testing1235]

I also tutor students in the entry level C++ and Python courses (which are taken during your first two semesters as a CS student), and I must agree that a large cohort of my class is only able to program if they have ChatGPT/Claude open on one half of their screen. I'm not sure how to solve this either, unless we want to start doing in person "interview" styled questions as an exam on a locked down computer.

I honestly think that doing an in person fake technical interview with a few easy Leetcode questions at the end of your education would be a good way to weed out those that have failed to even learn the basics of the trade.

[bruce511]

I'm so old I remember when calculators started appearing in general people's hands. Schools first banned them (what da ya mean you can't add a column of numbers by eye?) But gradually we switched over. We had a flirting interaction with log tables, and never did get to use a slide rule. I've no doubt old-school businesses were aghast at our ineptitude.

I'm so old we learned to program with giant C reference books. There was no internet, much less Google. We didn't have no fancy auto-complete, crumbs a text editor was considered advanced. Them youngsters coming to us couldn't program without Googling syntax, or using an IDE.

So yeah, sure, AI is changing the game. It's hard to evaluate students because the tools they are using are different to our experience. For decades we "make them code" as a measure of ability. In 3 years (their college experience) the toolset has changed.

Good students, good employees, are those who understand the problem and can adapt to a solution. AI is a tool that can be wielded well, or badly. Our approach to hiring will need to adapt as well. But good people are still out there, and good people make good workers.

To be honest I never was much in love with the leet code measure of hiring. Past a certain coding skill level I was more interested in the person than their ability to memorize an algorithm. Today that necessary skill level is lower, or at least harder to evaluate, but the problem-solving-mind is still the thing we're looking for.

So be careful of seeing the use of new tools as a weakness. The history of the world is littered with obsolete technologies. (Here's a sextant, where are we?) Rather see people who use tools for what they are, tools. Look for people who are curious, who see patterns, who get things done.

And to students I say, mastery of tools is a necessary step, but ultimately an uninteresting one. See beyond them. Be curious. Look under the hood. Ask questions like "is this code good enough to be running 30 years from now?" Because a huge amount of what you see now has foundations in code written a long time ago, and written well enough to stand for decades.

College is not "learning to program". College is learning how to adapt to an ever changing world, that will require your adapting many times over your career.

[bluefirebrand]

> College is not "learning to program". College is learning how to adapt to an ever changing world, that will require your adapting many times over your career.

You're gonna have to do a lot of work to convince me that people who only know how to drive an LLM are learning how to adapt to sweet fuck all

At least with a calculator, people still had to know the difference between addition and multiplication, in order to use the calculator correctly

[arvinsim]

> You're gonna have to do a lot of work to convince me that people who only know how to drive an LLM are learning how to adapt to sweet fuck all

Driving an LLM properly requires knowing to evaluate if the results are correct. People can certainly try to pass generated code over for PR. But even just one code feedback or debugging should uncover if the person understood what they were doing.

[bruce511]

What if driving an LLM well is actually a desirable skill?

What if changing from a "write code" based idea of programming changes to a "remove technical debt from code" skill?

What if the next generation of programmers is not focused on the creation of new code, but rather the improvement of existing code?

What if the current crop of programmers has to literally adapt from a world that has valued code quantity to a world that values code quality (something we dont especially prioritize at the moment?)

I'd argue that we're asking the current generation to be massively adaptable in terms of what was expected of us 10 (or 30) years ago, as to what will be required of them 5 years from now.

And to be clear, I'm not suggesting that LLMs will teach them to be adaptable. I'm suggesting that a world that contains LLMs will require them to be adaptable.

[bluefirebrand]

> What if changing from a "write code" based idea of programming changes to a "remove technical debt from code" skill

I don't believe you can do this if you can't write code, but sure. Maybe

> What if the current crop of programmers has to literally adapt from a world that has valued code quantity to a world that values code quality

LLMs seem more likely to increase the value of quantity and decrease the value of quality. That's playing out in front of us right now, with people "vibecoding"

> I'm suggesting that a world that contains LLMs will require them to be adaptable.

And ones who can't adapt will be ground to mulch to fuel the LLM power plants no doubt

[s1mplicissimus]

Your response just triggered a deja-vu from back when scaffolding tools were the new hot thing, now everyone and their dog was able to spin up that todo application within one CLI command. Except the generated code was mostly boilerplate that had to be heavily adapted for any real life use case, unveiling all the ignorance that could be covered up to that point. It's the same with vibe code. Looks fun until you throw it into reality - and then you're on your own and better know how to deal with stuff.

[mrheosuper]

i don't think you can compare Calculator to LLM.

A calculator will always give you correct result as long as you give it correct input. This is not the case with LLM. No matter how good your prompt is, there always a chance the output is completely garbage.

[typewithrhythm]

One big problem from the hiring side is the time to evaluate someone once complex tools are involved.

[AI_beffr]

did you ever consider the idea that AI is not the same as a calculator? or consider the fact that there is no reason why there couldnt be another quantum leap next year? and another one after that?

[linguae]

I teach computer science at a community college in Silicon Valley. Even before generative AI became available to the general public, cheating has been an issue with CS programming assignments.

One way I try to disincentivize cheating on projects is by having in-class paper exams, including weekly quizzes, as well as in-class paper assignments, and making sure that these in-class assessments are weighted significantly (roughly 60% of the overall grade). No electronic devices are allowed for these assignments. This forces my students to be able to write code without being able look up things online or consult an AI tool.

I still assign take-home programming projects that take 1-2 weeks to complete; students submit compilable source code. Practical hands-on programming experience is still vital, and even though cheating is possible, the vast majority of my students want to learn and are honest.

Still, for in-person assessments, if I had the budget, I’d prefer to hand out laptops with no Internet connection and a spartan selection of software, just a text editor and the relevant compiler/interpreter. It would making grading in-class submissions easier. But since we don’t have this budget, in-class exams and exercises are the next best solution I could think of.

[bruce511]

This reply will likely sound disrespectful, but I post it not to be so, but rather to perhaps spark an alternate path.

As the world changes, education can be slowest to adapt. My father did his math on a slide rule. I was in high school as we transitioned to using calculators.

My personal take on your approach is that you're seeing this from the wrong side. Creating an artificial environment for testing suggests to me you're testing the wrong thing.

Of course most school, and college, classes devolve to testing memory. "Here's the stuff to learn, remember it enough to pass the exam." And I get it, this is the way it's always been, regardless of the uselessness of the information. Who can remember when Charles 1st was beheaded? Who can't Google it in an instant?

Programing on paper without online reference tools isn't a measure of anything, because in the real world those tools exist.

Indeed, the very notion that we should even be testing "ability to write code" is outdated. That the student can create code should be a given.

Rather an exam should test understanding, not facts. Here's 2 blocks of code, which is better and why? Here's some code, what are the things about it that concern you?

Instead of treating the use of AI (or Google, or online help, or that giant C reference book I had) as "cheating", perhaps teach and assess in a world where AI exists.

I truly do get it. Testing comprehension is hard. Testing understanding is hard. Testing to sift wheat from chaff is hard. But, and I'm being harsh here i know, testing memory as a proxy for intelligence or testing hand-code-output as a proxy for understanding code is borderline meaningless.

Perhaps in the age of AI the focus switches from 'writing code' to 'reading code'. From the ability to write to the ability to prompt, review, evaluate and so on.

Perhaps the skill that needs to be taught (to the degree that community college seeks to teach skills) needs to be programing with AI, not against it.

I say all this with respect for how hard your job is, and with my thanks that you do it at all. I also say it understanding that it's a huge burden on you that you didn't necessarily sign up for.

[const_cast]

The problem is that tools like AI are useful if and only if you have the prerequisite knowledge, otherwise they are self-destructive.

It's similar to a calculator. We give student graphing calculators, but ONLY after they have already graphed by-hand hundreds of times. Why? Because education does not work like other things.

Efficiency, in education, is bad. We don't want to solve problems as fast as possible, we want to form the best understanding of problems possible. When I, say, want to book an airplane ticket, I want to do that in the fastest way possible. The most efficient manner. I care not about how an airport works, or how flight numbers are decided, or how planes work.

But efficient education is bad education. We can skip 99% of education, if we wanted. We can have, say, the SAT - and spend 1 year studying only for the SAT. Don't bother with the other 12 years of schooling.

Will you get an acceptable score on the SAT this way? Maybe. Will you be intelligent? No, you will be functionally illiterate.

If we use AI for programming before we can program, then we will be bad programmers. Yes, we can pass a test. Yes, we can pass a quiz. But we don't know what we're doing, because education is cumulative. If we skip steps, we lose. If we cut corners, we lose. It's like trying to put a roof on a house when the foundation isn't even poured.

[tekla]

I wish I could have gone to these schools where testing is just memorization. Everything would have been so easy

[CamperBob2]

I'm not sure how to solve this either, unless we want to start doing in person "interview" styled questions as an exam on a locked down computer.

Don't lock down the computer unless you are hiring people to work in a SCIF. Instead, give candidates a brutally hard/weird problem and tell them to use any resources they can get their hands on, by fair means or foul. (They will do that anyway if you hire them.) Then watch how they deal with it.

Do they just give up and stalk off in a huff?

If they Google for answers, do they use sensible queries?

If they use AI, do their prompts show skill at getting ideas, avoiding blind alleys, and creating effective tests?

If they call their friends, see how effective they are at communicating the requirements and turning the answers into a solution. Might be management material.