[karsuren]

What is the perf of GPT4 vs GPT3.5 in the 'reasoning', 'reflection', 'critisism' and 'resolver' tasks like in your project? I see that you have commented out gpt3.5 and replaced with GPT4 in config yaml. Was GPT3.5 perf too bad? I don't think many people have GPT4 API access. If this requires atleast GPT4 to be effective, it might take a while before anyone else in the community can take it up.

[mr_kotan]

GPT3.5 works not bad for generating, but it is not very good at self-reflection and self-critisism. Or maybe my prompts are not so good.