AWS Bedrock support

[csheaff]

Hello, it would be grand to be able to use AWS models from Amazon Bedrock, such as Anthropic Claude Sonnet 3.5.

[karthink]

Can you provide a link to their API documentation?

[csheaff]

...looking for a way to use just http requests but i'm not sure it's possible.

[karthink]

I'm not familiar with AWS Bedrock. How do you access models (or other computation) running there?

[csheaff]

the easiest approaches are to use the aws cli on the command line or a python sdk. But i'm guessing what would be most convenient here is being able to send https requests with lisp.

relevant? pokepay/aws-sdk-lisp#35

[csheaff]

This is as close as i can find to the payload structure:

https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html

On the command line one would would do:

aws bedrock-runtime converse \
--model-id amazon.titan-text-express-v1 \
--messages '[{"role": "user", "content": [{"text": "Describe the purpose of a \"hello world\" program in one line."}]}]' \
--inference-config '{"maxTokens": 512, "temperature": 0.5, "topP": 0.9}'

[karthink]

This is as close as i can find to the payload structure:

https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html

This makes it seem like you can make http requests? Sorry, I'm not understanding how this service is structured.

If you can make http requests I can add support for it to gptel.

[csheaff]

yes i think so. one could authenticate using environmental variables

AWS_SESSION_TOKEN, AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID, AWS_REGION

...or just have the user enter them in during config. In my case i have to renew my credentials often for security reasons, so being able to update them without closing emacs and re-exporting the variables in the terminal would be a plus.

[karthink]

Sorry, I don't follow how environment variables are relevant to making a http request.

Is there a curl command you can run to receive model responses from AWS Bedrock?