|
|
|
|
|
|
by omoikane
1154 days ago
|
|
|
On this page: https://github.com/minosvasilias/godot-dodo/tree/main/models It seems that some of the GPT syntax errors appear to be because the models were trained for Godot 3, but the tests were conducted against Godot 4, hence the error messages like "KinematicBody2D does not exist in 4.x (should be CharacterBody2D)". |
|
|
However, it is not quite as clear cut as OpenAI's models simply being trained on Godot 3.x projects only. Not only do they sometimes produce valid 4.x syntax (gpt-4 more often than 3.5-turbo), indicating there were at least some 4.x projects in the training data, they also hallucinate other invalid syntax, such as Python-specific functionality, or simply non-existent methods.
I do think evaluating against Godot 3.x would increase their scores somewhat, but i have not had time to do so yet.