It's not. The diff, which is the sole input to GPT-3 here, does not carry the causal context - that is, why the change was made. Nor does it carry the broader understanding necessary to summarize the what succinctly and accurately - things like high-level design terms that mostly exist on diagrams or in issue trackers, but not in the code itself. By adding those details in a commit message, the author can add extra information.
And yes, technically they could do it in comments, which would allow GPT-3 to process it. Except, I think it's highly unlikely for a coder using ChatGPT to write commit messages for them to actually author useful comments on their own. If they write any at all, they'll probably use ChatGPT (or Copilot) for that too.
You know what? You do that. I'm gonna start a SaaS business providing AI generators for code, commit messages and documentation. Models that take into account all the context to deliver more accurate, better results. All I need is a live feed from your JIRA, Confluence, Teams/Slack, Zoom, Exchange, SharePoint or whatever alternatives you use. This will guarantee you the biggest bang for your buck!
Elsewhere in this thread someone was wondering if sending a change diff to a fly-by-night third party SaaS could be leaking company IP. They were thinking too small.
But cynicism aside - giving the model access to all that contextual info would definitely increase the chances it would generate useful commit summaries. It would also increase chances it would generate much more convincing bullshit, full of just the right phrases to confuse your programmers, PMs and architects alike.
And yes, technically they could do it in comments, which would allow GPT-3 to process it. Except, I think it's highly unlikely for a coder using ChatGPT to write commit messages for them to actually author useful comments on their own. If they write any at all, they'll probably use ChatGPT (or Copilot) for that too.