[simonw]

This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here.

For example, given this code from https://github.com/microsoft/guidance/blob/main/notebooks/ch...

    create_plan = guidance('''{{#system~}}
    You are a helpful assistant.
    {{~/system}}
    {{#block hidden=True}}
    {{#user~}}
    I want to {{goal}}.
    {{~! generate potential options ~}}
    Can you please generate one option for how to accomplish this?
    Please make the option very short, at most one line.
    {{~/user}}
    {{#assistant~}}
    {{gen 'options' n=5 temperature=1.0 max_tokens=500}}
    {{~/assistant}}
    {{/block}}
    {{~! generate pros and cons and select the best option ~}}
    {{#block hidden=True}}
    {{#user~}}
    I want to {{goal}}.
    ''')

How about something like this instead?

    create_plan = guidance([
        system("You are a helpful assistant."),
        hidden([
            user("I want to {{goal}}."),
            comment("generate potential options"),
            user([
                "Can you please generate one option for how to accomplish this?",
                "Please make the option very short, at most one line."
            ]),
            assistant(gen('options', n=5, temperature=1.0, max_tokens=500)),
        ]),
        comment("generate pros and cons and select the best option"),
        hidden(
            user("I want to {{goal}}"),
        )
    ])

[marcotcr]

I think the DSL is nice when you want to take part of the generation and use it later in the prompt, e.g. this (in the same notebook).

---

prompt = guidance('''{{#system~}}

You are a helpful assistant.

{{~/system}}

{{#user~}}

From now on, whenever your response depends on any factual information, please search the web by using the function <search>query</search> before responding. I will then paste web results in, and you can respond.

{{~/user}}

{{#assistant~}}

Ok, I will do that. Let's do a practice round

{{~/assistant}}

{{>practice_round}}

{{#user~}}

That was great, now let's do another one.

{{~/user}}

{{#assistant~}}

Ok, I'm ready.

{{~/assistant}}

{{#user~}}

{{user_query}}

{{~/user}}

{{#assistant~}}

{{gen "query" stop="</search>"}}{{#if (is_search query)}}</search>{{/if}}

{{~/assistant}}

{{#if (is_search query)}}

{{#user~}}

Search results: {{#each (search query)}}

<result>

{{this.title}}

{{this.snippet}}

</result>{{/each}}

{{~/user}}

{{#assistant~}}

{{gen "answer"}}

{{~/assistant}}

{{/if}}''')

---

You could still write it without a DSL, but I think it would be harder to read.

[slundberg]

You can serialize and ship the DSL to a remote server for high speed execution. (without trusting raw Python code)

[crooked-v]

Why not just use JSON instead, though? Then you can just rely on all the preexisting JSON tooling out there for most stuff to do with it.

[foota]

There's prior art for pythonic DSLs that aren't actual python code.

[PeterisP]

Your example assumes a nested, hierarchical structure while the former example is strictly linear. IMHO that's the key difference there, as the former can (and AFAIK is) be directly encoded and passed to the LLM, which inherently receives only a flat list of tokens.

Your example might be nicer to edit, but then it would still have to be translated to the actual 'guidance language' which would have to look (and be) flat.

[itake]

My guess is you can store the DLS as a file (or in a db). With your example, you have to execute the code stored in your db.

[emehex]

We could write a python package that could? A codegen tool that generates codegen that will then generate code? <insert xzibit meme here>

[netdur]

I think chatgpt4 can easily write the python code... wait a second!

[quickthrower2]

Would love to hear your opinion on guidance, in the context of prompt injection attacks :-)