Skip to content

Worker creation fails when top_k is passed as a float #101

@a-priestley

Description

@a-priestley

Using the Home Assistant integration "Local LLMs", and configuring rkllama as an ollama backend for the voice assistant conversation agent (so stt and tts not relevant here).

When the model has not been loaded manually, use the "assist" functionality to send a prompt to the backend triggering the worker creation in rkllama, but creation fails.

Client exception message:

Sorry, there was a problem talking to the backend: HomeAssistantError("Failed to communicate with the API! Failed to load model 'home-llama-3.2:3b': Unexpected Error loading the model home-llama-3.2:3b into memory. Check the file .rkllm is not corrupted, properties in Modelfile (like Context Length allowed by the model) and resources available in the server (status code: 500)")

Rkllama exception message:

rkllama.worker - ERROR - Failed creating the worker for model 'home-llama-3.2:3b': invalid literal for int() with base 10: '20.0'

Even though the value being passed has no decimal value, it is not being parsed as an integer. As a simple workaround I have just changed the type to float:
a-priestley@56700d7

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions