Build Generative AI Applications with Foundation Models
What is Amazon Bedrock?
Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. Using Amazon Bedrock, you can easily experiment with and evaluate top FMs for your use case, privately customize them with your data using techniques such as fine-tuning and Retrieval Augmented Generation (RAG), and build agents that execute tasks using your enterprise systems and data sources. Since Amazon Bedrock is serverless, you don’t have to manage any infrastructure, and you can securely integrate and deploy generative AI capabilities into your applications using the AWS services you are already familiar with.
One of Amazon Web Services provides Generative AI functionality in AWS infrastructures.
Amazon Bedrock endpoints
Amazon Bedrock provides the following service endpoints. -
bedrock- Contains control plane APIs for managing, training, and deploying models. For more information, see Amazon Bedrock Actions and Amazon Bedrock Data Types. -bedrock-runtime- Contains data plane APIs for making inference requests for models hosted in Amazon Bedrock. For more information, see Amazon Bedrock Runtime Actions and Amazon Bedrock Runtime Data Types. -bedrock-agent- Contains control plane APIs for creating and managing agents, knowledge bases, prompt management, and prompt flows. For more information, see Amazon Bedrock Agents Actions and Amazon Bedrock Agents Data Types. -bedrock-agent-runtime- Contains data plane APIs for invoking agents and flows, and querying knowledge bases. For more information, see Amazon Bedrock Agents Runtime Actions and Amazon Bedrock Agents Runtime Data Types.
To interact with a pre-trained model such as Claude, I only need to
consider interacting with bedrock-runtime for now.
BedrockRuntime - Boto3 1.35.62 documentation
boto3 is a Python SDK for AWS, and to access Bedrock Runtime API,
import boto3
client = boto3.client("bedrock-runtime")These are the available methods:
- apply_guardrail
- can_paginate
- close # Closes underlyling endpoint connections
- converse
- converse_stream
- get_paginator
- get_waiter
- invoke_model # run inference
- invoke_model_with_response_stream
To get conversational responses from LLM hosted on AWS Bedrock, I must call converse() method.
converse - Boto3 1.35.62 documentation
Sends messages to the specified Amazon Bedrock model.
Converseprovides a consistent interface that works with all models that support messages. This allows you to write code once and use it with different models. If a model has unique inference parameters, you can also pass those unique parameters to the model.
Amazon Bedrock doesn’t store any text, images, or documents that you provide as content. The data is only used to generate the response.
Amazon Bedrock Runtime examples using SDK for Python (Boto3) - Anthropic Claude - AWS
import boto3
client = boto3.client("bedrock-runtime", region_name="us-east-1")
model_id = "anthropic.claude-3-haiku-20240307-v1:0"
user_message = "Describe the purpose of a 'hello world' program in one line."
conversation = [
{
"role": "user",
"content": [{"text": user_message}],
}
]
response = client.converse(
modelId=model_id,
messages=conversation,
inferenceConfig={"maxTokens": 512, "temperature": 0.5, "topP": 0.9},
)
# Extract and print the response text.
response_text = response["output"]["message"]["content"][0]["text"]
print(response_text)157.0 lb
Protein shake 300 Mac & Cheese 400 Tuna Salad 500 Sushi 400 Berries 200 Protein bar 200
Total 2000 kcal
4 mi run, push-ups
MUST:
TODO: