Issues with Bedrock Prompt Management

[alexdemers]

I'm using Prompt Management for Bedrock and this library is almost compatible with it.

First, when using Prompt Management, the request cannot contains some fields:

And when testing, I can't getting that error. So after some figuring out, I found out that inferenceConfig was being set because of the default maxTokens value of 2048 set in the ConfiguresTrait.

So, I overrode maxTokens using: ->withMaxTokens(null).

Next, I get errors because of "Unknown named parameter $responseMessages" when I finally get a response. Looking at the code, in ConverseTextHandler::prepareTempResponse it initiates responseMessages while that property does not exist in Prism\Prism\Text\Response (same with ConverseStructuredHandler).

Looking at the commit history, it seems that this commit was not applied to the Bedrock provider: prism-php/prism@b8fac4d

[andrew-alchemy]

I encountered the same issue with the restricted fields and solved for my specific requirements by defining a custom handler and provider. Maybe a bit of a hack but it might help you as a contingency.
I love Prism but feel the Bedrock support is being left by the wayside as the Prism folks chase every new and evolving quirk with the spectrum of providers.

        $this->app['prism-manager']->extend('aws-prompt-provider', function ($app, $config) {
            return new \App\PrismLLM\Provider\AWSPromptAwareProvider(
                credentials: \Prism\Bedrock\BedrockServiceProvider::getCredentials($config),
                region: $config['region']
            );
        });

class AWSPromptAwareHandler extends ConverseTextHandler
{
    /**
     * @return array<string,mixed>
     */
    public static function buildPayload(Request $request, int $stepCount = 0): array
    {
        return array_filter([
            'messages'                          => MessageMap::map($request->messages()),
            'additionalModelRequestFields'      => $request->providerOptions('additionalModelRequestFields'),
            'additionalModelResponseFieldPaths' => $request->providerOptions('additionalModelResponseFieldPaths'),
            'promptVariables'                   => $request->providerOptions('promptVariables'),
            'requestMetadata'                   => $request->providerOptions('requestMetadata'),
            'performanceConfig'                 => $request->providerOptions('performanceConfig'),
        ]);
    }
}

class AWSPromptAwareProvider extends Bedrock implements Provider
{
    public function text(TextRequest $request): TextResponse
    {
        $handler = AWSPromptAwareHandler::class;

        $client = $this->client(
            $request,
            $request->clientOptions(),
            $request->clientRetry()
        );

        $handler = new $handler($this, $client);

        return $handler->handle($request);
    }
}

[alexdemers]

It seems that the issue with responseMessages was submitted as a PR yesterday: #34

[ChrisB-TL]

I love Prism but feel the Bedrock support is being left by the wayside as the Prism folks chase every new and evolving quirk with the spectrum of providers.

@andrew-alchemy It is the same team maintaining both. We as open to helping with issues and PRs here as the main package.

@alexdemers will take a look at this. That is indeed a regression from a new Prism version. Issues like this should reduce or disappear once prism 1.0 is tagged, which I believe will be soon!

[ChrisB-TL]

Fixed in #34