Skip to content
This repository was archived by the owner on Sep 11, 2025. It is now read-only.

Conversation

@kevinmingtarja
Copy link
Contributor

@kevinmingtarja kevinmingtarja commented Nov 26, 2024

Description

This PR would allow users to create their own message with custom role. I encountered this use case when trying to use Llama 3.1 (served using an OpenAI compatible server), which has a different set of supported roles: [system, assistant, user, ipython].

Basically, i want to be able to do:

package main

import (
	"github.com/hypermodeinc/modus/sdk/go/pkg/models"
	"github.com/hypermodeinc/modus/sdk/go/pkg/models/openai"
)

type customMessage struct {
	openai.MessageBase
}

func generate(prompt string) (string, error) {
        ...
        input, err := model.CreateInput(
		...
		&customMessage{
			MessageBase: openai.MessageBase{
				Role:    "custom",
				Content: "...",
			},
		},
	)
        ...
}

Given the increasing number of inference providers implementing the OpenAI API (to some degree), I think we might need this for other models too eventually.

Checklist

  • Code compiles correctly and linting passes locally
  • For all code changes, an entry added to the CHANGELOG.md file describing and linking to this PR

@kevinmingtarja kevinmingtarja requested a review from a team November 26, 2024 01:52
@kevinmingtarja kevinmingtarja enabled auto-merge (squash) November 26, 2024 01:58
@kevinmingtarja kevinmingtarja merged commit e5abb35 into main Nov 26, 2024
86 checks passed
@kevinmingtarja kevinmingtarja deleted the kevinm/hyp-2650-support-messages-with-custom-role-in-openai-model-interface branch November 26, 2024 03:10
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants