Gemini context caching

[jobryan205]

Hi all,

I am trying to use Context Caching with my calls to Gemini via Instructor (https://ai.google.dev/gemini-api/docs/caching?lang=python). I can't seem to make it work, however. Does Instructor support Gemini's Context Caching abilities? If not, are there plans to support it in the future?

The following is a call to Gemini via the Gemini SDK that successfully uses context caching:

response = client.models.generate_content(
                    model=model.model_name,
                    contents=prompt,
                    config={
                        "max_output_tokens": max_tokens,
                        "cached_content": cache.name
                    }
                )

The following is my attempt to use context caching via Instructor:

response = instructor_client.messages.create(
                    messages=[{"role": "user", "content": prompt}],
                    response_model=schema,
                    model=model.model_name,
                    validation_context=validation_context,
                    generation_config={
                        "max_output_tokens": max_tokens,
                        "cached_content": cache.name
                    }
                )

The above doesn't work, however.

[jonbuffington]

I am just starting to explore this also. Perhaps, the approach outlined by #940 (reply in thread) may help us.