Custom Fint-tuned models on AWS to spacy-llm

[innocent-charles]

Hello ! @rmitsch and others, I was asking if it is possible to write a custom task with spacy llm that integrate custom fine tuned model deployed on the AWS sagemaker or EC2, or even AWS Lambda....

[rmitsch]

Hi @innocent-charles, if you want to connect to a provider that is not natively supported yet, you have to options:

• Write your own custom model
• Most likely easier: use the langchain integration

In the case of models deployed on AWS, we currently have a user-contributed PR for the native support for Amazon Bedrock models in the works that might be useful for you.

Meanwhile you can use the langchain integration, as langchain already supports what you're asking for (if I understand it correctly, haven't worked with AWS-hosted models myself yet).

The task is orthogonal to the model. I. e. you can use any task with any model, and you can also combine a custom task with a custom model. It's not necessary however to write a custom task just because you use a custom model.