[aaaronic]

I'm trying something similar this semester with my course via AGENTS.md. I think this one is overly verbose and probably falls out of context windows pretty quickly, based on my experience (for me, a very terse but clear set of 30 lines performed better than providing examples and more nuanced explanations during my testing with a few models).

I have included the basic "I am a student -- help me learn, don't just do everything for me," but I also am trying out telling it to generate a .history folder with a markdown history of every prompt and a summary of the action take in response.

I _know_ there are some tools that offer the prompt history automatically, but I've told students they can use _whatever_ tool they want, but should let me know if the folder isn't showing up as they work.

The .history folder is required if they used AI and I intend to review it and try to give specific feedback to the students using it as too much of a crutch.

I just started this last Friday, so wish me luck!

[wrs]

As a general rule with LLMs, don't just tell it to do something if you actually need to make sure it gets done. Use a hook script to make it do that, or use the history that's already there (transcripts of all sessions are retained in ~/.claude, for example). There are innumerable scripts out there to parse these, or your agent will whip one up for you in 5 minutes.

[aaaronic]

I was hoping to get them access to a specific tool like GitHub Copilot via GitHub Education, but when I looked, sign-ups were paused, so I went the tool-agnostic approach. Even during installation a lot of them were telling me how "chat" told them to fix their installation issues (but some were clearly using an alternative to ChatGPT, specifically).

However, I see from other comments on this post that I may need to include a CLAUDE.md as a copy (and could maybe just leave the .history part out of that version?).

[mswphd]

worth clarifying "chat" is actually (linguistically) completely separate from a shorthand for ChatGPT. Livestreamers (e.g. on twitch/youtube) often talk to "chat", the people watching. Visually, they're just narrating actions etc to a 3rd party who is not present.

This has leaked into (some) younger people's vocabulary. A particular example is saying "Chat is this real?"

https://knowyourmeme.com/memes/chat-is-this-real

but some people use it more freely.

[aaaronic]

Ha, I have a 16 year old who yells "chat" often enough when online gaming to be familiar with that use.

I'm positive these students did use an LLM to get the help instead of crowdsourcing, but it is an interesting linguistic overlap.

And while I'm on my "old man" soapbox -- "look it up" and "search for it" somehow became "search it up" with the young people. I corrected my son for years before I started hearing college students also saying it that way...

[thaumasiotes]

> "look it up" and "search for it" somehow became "search it up" with the young people

It wouldn't actually be necessary for the phrase "look it up" to exist for this to happen. You're free to apply the particle "up" to pretty much any English verb if you want the semantics that it provides. Compare rustle up, turn up, etc.

You might also want to take note of the episode of Kim Possible where Ron is unsatisfied with the performance of an actor studying to play him, and tells the actor to "Ron it up".

[brabel]

> You're free to apply the particle "up" to pretty much any English verb if you want the semantics that it provides.

I have been speaking English for 20 years but it's my second language. I don't think the semantics of "up" matters when I try to understand phrasal verbs like "turn up". I don't see anything about "up" (as in a direction) in "turn up" or "show up" when it means "to appear" or "to be discovered"... where is the semantic connection?? I think native English speakers just think "up" intrinsically relates to "appear" or "be found" but there's no such connection in other languages I know of.

Similarly with things like "fed up" (as in 'tired of'). Where is the "upness" here?

[noelsusman]

Not entirely sure what your point is, but if you're implying that kids don't use "chat" to refer to any LLM (usually ChatGPT) then that's very wrong. It doesn't have anything to do with the usage of "chat" you described.

[pastel8739]

I don’t think this is right. Certainly one usage of “chat” is the livestream version, as you describe, but both Gen Z and Boomers use “chat” to refer to AI tools specifically

[FistfulOfHaws]

I’ve done something similar for myself to learn Django. Claude Code has a built in Learning Mode, I extended that with a Coaching Mode. Where it is instructed on how to coach me, how to help stub out features, how to give feedback in code review, etc. With the main instruction being never to write code for me when in that mode. It can write basic logic examples/pseudo code and discuss different approaches to the problem. I have found it to be really effective, it is my go to for learning new things. I’m using it now to learn Elixir.

[bob1029]

I like to think of the initial system prompt as a fuse or bootloader. The user prompt and feedback from tool execution are where most of the alignment comes from in multi-turn use. You want to point the agent in the right direction and then get the hell out of its way.

If your agent isn't performing as expected but can otherwise see and describe the tools as you expect, your mental model of what the tools should be is probably wrong. Adjusting the system prompt can address this, but it quickly bloats and starts to turn into a game of whack-a-mole.

I've got an agent that talks to a very large data warehouse and the system prompt is somewhere around 100 tokens. Most of the important information lives in the user's request and in the environment.

[j_french]

I wish you good luck! And add that I'd be interested to hear how you get on. I intend to adopt a similar approach with my classes in September. The . history folder is a great idea.

How do you intend to assess your students?

[aaaronic]

I told them up front this was new territory and we're all learning from it. If they're overusing it I assured them they won't be dealing with a student integrity issue, but I will be giving them feedback if they're clearly over-reliant and if they continue to do so, it could eventually impact the grade on later assignments.

I'm hoping they learn to use it as a tool instead of trying to offload all cognition to it.

This is a CS course targeted at non-majors, so thankfully the "fundamentals" aren't as critical as the overall themes and general skills.

[knollimar]

"if they're clearly over-reliant and if they continue to do so, it could eventually impact the grade on later assignments."

Not sure if this is fair. Consider some ofher poor study tool. Would you dock their grade for using externally curated flash cards too much?

[allthetime]

“ I think this one is overly verbose and probably falls out of context windows pretty quickly”

It likely will. Half way through a session I routinely watch the agent append my rules to the top of its thinking only to do exactly what it said it wasn’t going to do after another minute of thinking.

It will then apologize profusely right before doing it again.

As others have said, use hooks.

[lucamark]

Love it! I think the power of LLMs to acquire new skills and deepen the knowdledge is underestimated.

When used correctly, they offer a huge advantage over those who don't use them and think they understand but remain superficial. I encourage you to ask even the most obvious questions.

[weird-eye-issue]

This is not long at all compared to what is actually being used in production by my agents and others like Claude Code...