| Convert your vector embeddings into a set of questions and their ideal responses. Use this dataset to test your LLM and catch failures caused by prompt or RAG updates. Get started in 3 lines of code: ``` pip3 install fiddlecube ``` ``` from fiddlecube import FiddleCube fc = FiddleCube(api_key="<api-key>")
dataset = fc.generate(
[
"The cat did not want to be petted.",
"The cat was not happy with the owner's behavior.",
],
10,
)
dataset ``` Generate your API key: https://dashboard.fiddlecube.ai/api-key # Ideal QnA datasets for testing, eval and training LLMs Testing, evaluation or training LLMs requires an ideal QnA dataset aka the golden dataset. This dataset needs to be diverse, covering a wide range of queries with accurate responses. Creating such a dataset takes significant manual effort. As the prompt or RAG contexts are updated, which is nearly all the time for early applications, the dataset needs to be updated to match. # FiddleCube generates ideal QnA from vector embeddings - The questions cover the entire RAG knowledge corpus. - Complex reasoning, safety alignment and 5 other question types are generated. - Filtered for correctness, context relevance and style. - Auto-updated with prompt and RAG updates. |
If you have your own LLM, you may have sensitive/private data "in" it from your training. You may not be allowed to use this service from a legal point of view.