| I’m not sure the training date cutoff or prompt weighting says anything about whether this is hallucinated or not. The models have been given these rules in the present, this is known, so training data cutoff doesn’t matter as the model has now seen this. Zero shot learning in gpt4 is not new. This also answers that these are prompts (I’m not sure what your point is here). We still don’t know if the model took these rules and hallucinated from them or regurgitated them. Only the people with access know that. We also don’t know if there’s been some fine tuning. Some of the rules being posted are a bit off though. For example in the original post some of the “must” words are capitalized and others are not. This begs the questions why some, did the prompter find that capitalizing specific words has more weight or does it confuse the LLM, or did the LLM just do zero shot off the original rules and hallucinate something similar? I’d bet these are hallucinated but similar to the real rules. Has anyone shown you can get gpt4 to regurgitate the system prompt (using the api) exactly? Using a system prompt similar that dictates no sharing the prompt etc. That would give a better indication than this imo. |