Unable to use custom local model (pipeline) with Agent: AttributeError 'split'

[QDung210]

Question

Hi Pydantic-AI team,

I have finetuned a Qwen/Qwen2.5-7B-Instruct model into my own model on HuggingFace:

Model: Dqdung205/qwen-function-calling-model

Since my model cannot be used through the HuggingFace API (it is local only), I am trying to load it locally using Unsloth + Transformers and integrate it with Pydantic-AI's Agent.

Here is my minimal code:

from unsloth import FastLanguageModel
from transformers import pipeline
from pydantic_ai import Agent, RunContext
from pydantic_ai.models import Model
from pydantic import BaseModel
from typing import Any
import json

# -----------------------------
# Custom local runner
# -----------------------------
class LocalQwenModel(Model):
    """Adapter for local Qwen model (Unsloth pipeline)."""
    def __init__(self, pipeline):
        self.pipeline = pipeline

    async def run(self, prompt: str, **kwargs: Any) -> str:
        result = self.pipeline(prompt, max_new_tokens=256)
        text = result[0]["generated_text"]
        if prompt in text:
            text = text.replace(prompt, "").strip()
        return text

# -----------------------------
# Load finetuned model locally
# -----------------------------
router_model, router_tokenizer = FastLanguageModel.from_pretrained(
    model_name="Dqdung205/qwen-function-calling-model",
    max_seq_length=1024,
    dtype=None,
    load_in_4bit=True,
)

ROUTER_PIPE = pipeline(
    "text-generation",
    model=router_model,
    tokenizer=router_tokenizer,
    device_map="auto"
)

local_runner = LocalQwenModel(ROUTER_PIPE)

# -----------------------------
# Attempt to create Agent
# -----------------------------
class BookingState(BaseModel):
    name: str = ""
    date: str = ""
    time: str = ""

agent = Agent(
    model=local_runner,
    description="Agent đặt lịch khám theo từng bước."
)

Traceback

Device set to use cuda:0
AttributeError Traceback (most recent call last)
/usr/local/lib/python3.11/dist-packages/pydantic_ai/agent/init.py in init(self, model, ...)
288 self._model = model
289 else:
--> 290 self._model = models.infer_model(model)
291
292 self._name = name

/usr/local/lib/python3.11/dist-packages/pydantic_ai/models/init.py in infer_model(model)
644 try:
--> 645 provider, model_name = model.split(':', maxsplit=1)
646 except ValueError:
647 provider = None

AttributeError: 'LocalQwenRunner' object has no attribute 'split'

Question / Issue

I want to use my local finetuned Qwen model with pydantic-ai's Agent for function calling and multi-step workflows, without relying on a HuggingFace API provider.

What is the correct way to load and wrap a local pipeline (like Unsloth/Transformers) so that Agent accepts it?

Do I need to implement a specific interface, or is there a recommended pattern for custom local models?

Thank you for your help!

Additional Context

pydantic-ai : 1.0.15