how to print the output?

[aus70]

I followed the instructions to install and demo guidance:

from guidance import models, gen
llama2 = models.LlamaCpp(path) 
llama2 + f'Do you want a joke or a poem? ' + gen(stop='.')

but the output is <guidance.models._llama_cpp.LlamaCpp at 0x10301da30> instead of the expected text. What am I missing?

Thanks!

[nchammas]

What do you get if you try this?

llama2 += 'Do you want a joke or a poem? ' + gen(stop='.')
print(llama2)

Alternately, you can try:

llama2 += 'Do you want a joke or a poem? ' + gen(name='response', stop='.')
print(llama2['response'])

[aus70]

print(llama2) prints a newline

print(llama2['response']) ends up with an error
KeyError: "Model does not contain the variable 'response'"

[nchammas]

Are you sure you are running += as in my code and not just + as it was in your original code?

[aus70]

I'm sure I was using just + :) , becasue I copied & pasted the code on the Guidance GitHub page

Using += works fine... maybe it'd be worthwhile to fix the example code.

Thanks!

[nchammas]

I think the example code from your screenshot is correct as-is when run within a Jupyter Notebook.

I assume you were running things in a normal Python terminal. Is that correct?

This is definitely something the docs should clarify, as it confused me at first as well.

[hudson-ai]

Thank you both for the discussion, and @nchammas for jumping in to help!

I agree -- we should clarify this in the docs.

lm + xyz() will "run" xyz (hence the notebook visuals updating), but you need new_lm = lm + xyz() or lm += xyz() to "get" the output.

Think of adding stuff to a model as "extending" its state. += does that in-place, while + simply computes the result (but discards it if you don't assign it to some variable).

[aus70]

I assume you were running things in a normal Python terminal. Is that correct?

I was using ipython