Thanks for the feedback. There's an example in the linked GH discussion but let me provide a quick ETE example here, centered around parsing structured information from a hypothetical shipping company email.
1. In the app, write a TypeScript schema for this, one good example could be
interface DeliveryInformation {
/* Tracking number for the delivery */
tracking_number: string;
/* Status of the delivery, one of "preparing", "out-for-delivery", or "delivered" */
status: string;
/* Weight of the package, e.g. "2oz" or "3lb" */
weight: string;
/* submission date time representation */
submitted_ts: string;
}
If you click Generate you'll see it generate a context-free grammar looking text, which is what llama.cpp reads. Click the download file to save it as grammar.gbnf
You might want to add an appropriate modifier before “grammar” on your top page to indicate your domain.
Being a natural-language person, I thought your tool might be something that, say, generates rules for verb conjugations from a corpus of a human language.
1. In the app, write a TypeScript schema for this, one good example could be
If you click Generate you'll see it generate a context-free grammar looking text, which is what llama.cpp reads. Click the download file to save it as grammar.gbnf2. Grab a model for llama.cpp, e.g. this quantized Llama2 chat model: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/blob/ma...
3. Grab the prompt in this gist (https://gist.github.com/a10y/d926039eee63cc2bcaf6345f9a419e3...) and save as prompt.txt
4. Compile a recent commit of llama.cpp and run the following command
EDIT: fixed some typos