I'm still not sure if specifying a role made a difference or not in terms of performance. Different but similar instance, when I tried to create an agent in Claude to play a specific role (frontend / design engineer expert), I found that this seemed to perform worse vs. just using default Claude, but this is all very anecdotal.
Maybe it is cargo culting at this point, idk. When I first started experimenting with this, about two generations of models back, the role play prompt made a noticeable difference.
Example: with early Claude (pre-Claude Code) if you asked for a Rust program you’d get something that only resembled Rust syntax but was a mix of different languages. “You are a senior software engineer that develops solely with the Rust programming language” or something like this made it generate syntactically correct Rust.
Similar prompts led to better, more focused tests. I find that such prompts are not as necessary anymore, but anecdotally I’ve still felt a difference.