OpenAI doesn't have some secret technical knowledge either. All of these models are just based on transformers