Gemini LLMs via VertexAI?

[schnee]

I'm attempting to use Gemini models (e.g. gemini-1.5-flash-001) and I'd like to ultimately deploy in GCP. At the moment, local dev is fine. Does anyone have an example that shows how to create an Agent using a model hosted via VertexAI?

Initialization / authentication might look like:

       key_file = "path_to/.config/gcloud/application_default_credentials.json"
        if os.path.exists(key_file):
            os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = key_file
        else:
            print("No key file: assume running in GCP.")

        # initialize the vertex AI client
        vertexai.init(project="project name", location="us-central1")
        self.llm = LLM(
             model = "vertex_ai/gemini-1.5-flash-001",
             vertex_credentials_json = json.dumps(key_file)
        )

This does not work, and I'm not sure how to make it work. The connection between LiteLLM and the CrewAI LLM class is a little opaque to me here.

[schnee]

It seems to work if I simply do not set vertex_credentials_json in the LLM constructor. E.g.

        self.llm = LLM(
             model = "gemini-1.5-flash-001",
        )

and then refer to self.llm inside the Agent.

I'll leave this open for a day or two and then close it (because "seems to work" might not capture the whole concept).