AWS Bedrock provider support #542

inercia · 2025-10-17T10:58:54Z

AWS Bedrock provider, supporting multiple model families (Anthropic Claude, Amazon Titan, Meta Llama, and Mistral).

pkg/model/provider/bedrock/client.go

rumpl · 2025-10-20T22:18:04Z

pkg/model/provider/bedrock/adapter.go

+// processClaudeChunk processes Claude model chunks
+func (a *StreamAdapter) processClaudeChunk(chunk []byte) (chat.MessageStreamResponse, error) {
+	// Claude chunks follow the Messages API streaming format
+	var claudeChunk struct {


This is more of a question in general about the code, I see there's a lot of hand-made code for calling different models, doesn't AWS provide an SDK that can help with these? I'm not really a pro in aws and their sdks

You are right. This was based on some previous code of mine, but I have updated the code for using more modern features of the AWS Bedrock SDK.

rumpl · 2025-10-20T22:19:18Z

@inercia small update, sorry this is taking this long, I'm trying to get access to Bedrock to try this out.

Had a small question, see above.

This is a lot of code, can we ping you if we have some issues in the future with this part of the code? :)

inercia · 2025-10-21T09:48:20Z

Thanks for reviewing this. Just for the record, I have been testing this with the following code:

#!/bin/bash

# get the AWS Bedrock creds
set -a
source .env.bedrock

export OPENAI_API_KEY="empty"
export ANTHROPIC_API_KEY="empty"

echo "************************************************"

./bin/cagent run --model="amazon-bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0" \
    --debug --tui=false --yolo \
    ./examples/haiku.yaml \
    "Write a haiku about the weather"

echo "************************************************"

./bin/cagent run --model="amazon-bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0" \
    --debug --tui=false --yolo \
    ./examples/script_shell.yaml \
    "Show me all the GitHub repositories for GitHub user 'inercia'"

echo "************************************************"

./bin/cagent run --model="amazon-bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0" \
    --debug --tui=false --yolo \
    ./examples/pythoninst.yaml \
    "Create a Python program for calculating the Fibonacci sequence. Find the optimal algorithm for the task. Check the code works by running it"

Copilot

Pull request overview

This PR adds comprehensive AWS Bedrock provider support to cagent, enabling access to various AI models hosted on AWS Bedrock including Anthropic Claude, Amazon Titan, Meta Llama, and Mistral. The implementation follows the established provider pattern in the codebase and includes support for both AWS credential chain and bearer token authentication methods.

Key changes:

New Bedrock provider implementation using AWS SDK v2 with Converse API for streaming
Flexible authentication supporting AWS credentials and custom bearer tokens (for proxy scenarios)
Region configuration via environment variables or explicit provider options
Comprehensive documentation and example configuration files

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
pkg/model/provider/provider.go	Registers the new "amazon-bedrock" provider in the factory function
pkg/model/provider/bedrock/client.go	Core Bedrock client implementation with authentication, region handling, message/tool conversion
pkg/model/provider/bedrock/adapter.go	Stream adapter to convert Bedrock Converse API events to cagent's MessageStream interface
pkg/model/provider/bedrock/client_test.go	Unit tests for client initialization, configuration, and region handling
docs/USAGE.md	Documentation for authentication methods, configuration options, and usage examples
examples/bedrock.yaml	Example configuration demonstrating Bedrock provider usage
go.mod, go.sum	AWS SDK v2 dependencies for Bedrock Runtime API

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pkg/model/provider/bedrock/client.go

Copilot · 2025-12-11T12:02:28Z

pkg/model/provider/bedrock/adapter.go

+					ID:   *start.Value.ToolUseId,
+					Type: "function",
+					Function: tools.FunctionCall{
+						Name: *start.Value.Name,
+					},
+				}
+				response.Choices[0].Delta.ToolCalls = []tools.ToolCall{toolCall}
+
+				// Store tool call info for delta events
+				if e.Value.ContentBlockIndex != nil {
+					a.toolCallData[int(*e.Value.ContentBlockIndex)] = &toolCallInfo{
+						ID:   *start.Value.ToolUseId,
+						Name: *start.Value.Name,


Potential nil pointer dereference. The code dereferences start.Value.ToolUseId and start.Value.Name without checking if they are nil first. While these values should typically be present in the AWS SDK response, defensive nil checks should be added to prevent potential panics if the API returns unexpected data.

Added defensive nil checks for start.Value.ToolUseId and start.Value.Name before dereferencing them, with appropriate warning logging.

docs/USAGE.md

Copilot · 2025-12-11T12:02:29Z

pkg/model/provider/bedrock/client_test.go

@@ -0,0 +1,143 @@
+package bedrock


The StreamAdapter implementation in adapter.go lacks test coverage. The adapter handles critical streaming logic including tool calls, content deltas, and error handling. Similar providers like Gemini have adapter_test.go files that test the stream processing logic. Consider adding tests for the StreamAdapter to cover scenarios like tool call streaming, text streaming, error handling, and finish reasons.

The adapter tests would require mocking the AWS Bedrock stream events which is complex. I'd recommend addressing this in a follow-up PR if we want to proceed with it.

pkg/model/provider/bedrock/client.go

pkg/model/provider/bedrock/client_test.go

Copilot

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated 7 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-11T12:43:09Z

pkg/model/provider/bedrock/adapter.go

+// processConverseEvent processes a Converse API stream event
+func (a *StreamAdapter) processConverseEvent(event types.ConverseStreamOutput) (chat.MessageStreamResponse, error) {
+	response := chat.MessageStreamResponse{
+		Model: a.model,
+		Choices: []chat.MessageStreamChoice{
+			{
+				Index: 0,
+				Delta: chat.MessageDelta{
+					Role: string(chat.MessageRoleAssistant),
+				},
+			},
+		},
+	}
+
+	switch e := event.(type) {
+	case *types.ConverseStreamOutputMemberMessageStart:
+		// Message start event - provides role
+		slog.Debug("Converse MessageStart event", "role", e.Value.Role)
+		return response, nil
+
+	case *types.ConverseStreamOutputMemberContentBlockStart:
+		// Content block start - may be text or tool use
+		if e.Value.Start != nil {
+			switch start := e.Value.Start.(type) {
+			case *types.ContentBlockStartMemberToolUse:
+				// Tool use started - check for nil values before dereferencing
+				if start.Value.ToolUseId == nil || start.Value.Name == nil {
+					slog.Warn("Converse ContentBlockStart (ToolUse) missing required fields",
+						"tool_use_id_nil", start.Value.ToolUseId == nil,
+						"name_nil", start.Value.Name == nil)
+					return response, nil
+				}
+
+				toolCall := tools.ToolCall{
+					ID:   *start.Value.ToolUseId,
+					Type: "function",
+					Function: tools.FunctionCall{
+						Name: *start.Value.Name,
+					},
+				}
+				response.Choices[0].Delta.ToolCalls = []tools.ToolCall{toolCall}
+
+				// Store tool call info for delta events
+				if e.Value.ContentBlockIndex != nil {
+					a.toolCallData[int(*e.Value.ContentBlockIndex)] = &toolCallInfo{
+						ID:   *start.Value.ToolUseId,
+						Name: *start.Value.Name,
+					}
+				}
+
+				slog.Debug("Converse ContentBlockStart (ToolUse)",
+					"tool_id", *start.Value.ToolUseId,
+					"tool_name", *start.Value.Name,
+					"index", e.Value.ContentBlockIndex)
+			}
+		}
+		return response, nil
+
+	case *types.ConverseStreamOutputMemberContentBlockDelta:
+		// Content block delta - streaming content
+		if e.Value.Delta != nil {
+			switch delta := e.Value.Delta.(type) {
+			case *types.ContentBlockDeltaMemberText:
+				// Text content delta
+				response.Choices[0].Delta.Content = delta.Value
+				slog.Debug("Converse ContentBlockDelta (Text)", "length", len(delta.Value))
+
+			case *types.ContentBlockDeltaMemberToolUse:
+				// Tool use input delta (streaming JSON arguments)
+				// Accumulate but DON'T send to runtime until complete
+				if e.Value.ContentBlockIndex != nil && delta.Value.Input != nil {
+					if toolInfo, ok := a.toolCallData[int(*e.Value.ContentBlockIndex)]; ok {
+						toolInfo.Arguments += *delta.Value.Input
+						slog.Debug("Converse ContentBlockDelta (ToolUse) accumulated",
+							"chunk_length", len(*delta.Value.Input),
+							"total_length", len(toolInfo.Arguments))
+					}
+				}
+			}
+		}
+		return response, nil
+
+	case *types.ConverseStreamOutputMemberContentBlockStop:
+		// Content block stopped - now send complete tool call if this was a tool use block
+		if e.Value.ContentBlockIndex != nil {
+			if toolInfo, ok := a.toolCallData[int(*e.Value.ContentBlockIndex)]; ok {
+				// Ensure arguments is valid JSON - if empty, use empty object
+				args := toolInfo.Arguments
+				if args == "" {
+					args = "{}"
+				}
+
+				// Send the complete tool call now
+				toolCall := tools.ToolCall{
+					ID:   toolInfo.ID,
+					Type: "function",
+					Function: tools.FunctionCall{
+						Name:      toolInfo.Name,
+						Arguments: args,
+					},
+				}
+				response.Choices[0].Delta.ToolCalls = []tools.ToolCall{toolCall}
+				slog.Debug("Converse ContentBlockStop - sending complete tool call",
+					"tool_id", toolInfo.ID,
+					"tool_name", toolInfo.Name,
+					"args_length", len(args))
+			}
+		}
+		slog.Debug("Converse ContentBlockStop", "index", e.Value.ContentBlockIndex)
+		return response, nil
+
+	case *types.ConverseStreamOutputMemberMessageStop:
+		// Message stopped - provides stop reason
+		if e.Value.StopReason != "" {
+			response.Choices[0].FinishReason = mapConverseStopReason(e.Value.StopReason)
+			slog.Debug("Converse MessageStop", "stop_reason", e.Value.StopReason)
+		}
+		return response, nil
+
+	case *types.ConverseStreamOutputMemberMetadata:
+		// Metadata event - provides token usage
+		if e.Value.Usage != nil {
+			usage := &chat.Usage{}
+			if e.Value.Usage.InputTokens != nil {
+				usage.InputTokens = int64(*e.Value.Usage.InputTokens)
+			}
+			if e.Value.Usage.OutputTokens != nil {
+				usage.OutputTokens = int64(*e.Value.Usage.OutputTokens)
+			}
+			response.Usage = usage
+			slog.Debug("Converse Metadata", "input_tokens", usage.InputTokens, "output_tokens", usage.OutputTokens)
+		}
+		return response, nil
+
+	default:
+		slog.Warn("Unexpected Converse stream event", "type", fmt.Sprintf("%T", event))
+		return chat.MessageStreamResponse{}, fmt.Errorf("unexpected stream event: %T", event)
+	}
+}


The processConverseEvent function and the StreamAdapter lack test coverage. This is a complex event processing function that handles multiple event types, tool calls, text streaming, and usage tracking. Consider adding unit tests for each event type (MessageStart, ContentBlockStart, ContentBlockDelta, ContentBlockStop, MessageStop, Metadata) and verify tool call accumulation logic works correctly.

pkg/model/provider/bedrock/client.go

examples/bedrock.yaml

docs/USAGE.md

Copilot · 2025-12-11T12:43:11Z

pkg/model/provider/bedrock/client.go

+// convertToConverseMessages converts cagent messages to Converse API format
+func convertToConverseMessages(messages []chat.Message) ([]types.Message, []types.SystemContentBlock, error) {
+	var converseMessages []types.Message
+	var systemBlocks []types.SystemContentBlock
+
+	for i := 0; i < len(messages); i++ {
+		msg := messages[i]
+
+		// System messages are handled separately
+		if msg.Role == chat.MessageRoleSystem {
+			systemBlocks = append(systemBlocks, &types.SystemContentBlockMemberText{
+				Value: msg.Content,
+			})
+			continue
+		}
+
+		// Convert role
+		var role types.ConversationRole
+		switch msg.Role {
+		case chat.MessageRoleUser:
+			role = types.ConversationRoleUser
+		case chat.MessageRoleAssistant:
+			role = types.ConversationRoleAssistant
+		case chat.MessageRoleTool:
+			// Tool results are sent as user messages with tool result blocks
+			role = types.ConversationRoleUser
+		default:
+			return nil, nil, fmt.Errorf("unsupported message role: %s", msg.Role)
+		}
+
+		// Build content blocks
+		var contentBlocks []types.ContentBlock
+
+		// Handle tool results - group consecutive tool results into one user message
+		if msg.Role == chat.MessageRoleTool && msg.ToolCallID != "" {
+			// Collect all consecutive tool results
+			toolResults := []chat.Message{msg}
+			j := i + 1
+			for j < len(messages) && messages[j].Role == chat.MessageRoleTool {
+				toolResults = append(toolResults, messages[j])
+				j++
+			}
+
+			// Convert all tool results into content blocks
+			for _, tr := range toolResults {
+				var toolResultContent []types.ToolResultContentBlock
+				toolResultContent = append(toolResultContent, &types.ToolResultContentBlockMemberText{
+					Value: tr.Content,
+				})
+
+				contentBlocks = append(contentBlocks, &types.ContentBlockMemberToolResult{
+					Value: types.ToolResultBlock{
+						ToolUseId: aws.String(tr.ToolCallID),
+						Content:   toolResultContent,
+					},
+				})
+			}
+
+			// Skip the tool results we already processed
+			i = j - 1
+		} else if msg.Role == chat.MessageRoleAssistant && len(msg.ToolCalls) > 0 {
+			// Assistant message with tool calls
+			if msg.Content != "" {
+				contentBlocks = append(contentBlocks, &types.ContentBlockMemberText{
+					Value: msg.Content,
+				})
+			}
+
+			// Add tool use blocks
+			for _, tc := range msg.ToolCalls {
+				// Parse tool arguments
+				var input map[string]any
+				if tc.Function.Arguments != "" {
+					if err := json.Unmarshal([]byte(tc.Function.Arguments), &input); err != nil {
+						slog.Warn("Failed to unmarshal tool arguments", "tool", tc.Function.Name, "error", err)
+						input = make(map[string]any)
+					}
+				} else {
+					input = make(map[string]any)
+				}
+
+				// Convert to document type
+				inputDoc, err := convertToDocument(input)
+				if err != nil {
+					return nil, nil, fmt.Errorf("failed to convert tool input: %w", err)
+				}
+
+				contentBlocks = append(contentBlocks, &types.ContentBlockMemberToolUse{
+					Value: types.ToolUseBlock{
+						ToolUseId: aws.String(tc.ID),
+						Name:      aws.String(tc.Function.Name),
+						Input:     inputDoc,
+					},
+				})
+			}
+		} else {
+			// Regular text message
+			if msg.Content != "" {
+				contentBlocks = append(contentBlocks, &types.ContentBlockMemberText{
+					Value: msg.Content,
+				})
+			}
+		}
+
+		if len(contentBlocks) > 0 {
+			converseMessages = append(converseMessages, types.Message{
+				Role:    role,
+				Content: contentBlocks,
+			})
+		}
+	}
+
+	return converseMessages, systemBlocks, nil
+}


The convertToConverseMessages function lacks test coverage. This is a critical function that handles message conversion including system messages, tool calls, and tool results. Consider adding unit tests to verify correct handling of different message types, edge cases like empty tool results, consecutive tool messages, and invalid message roles. Other providers like Anthropic have extensive test coverage for their message conversion functions (see pkg/model/provider/anthropic/client_test.go).

This is a valid suggestion, but adding comprehensive unit tests for this function is a larger effort that would be better addressed in a follow-up PR. The function is already exercised through integration paths.

Copilot · 2025-12-11T12:43:12Z

pkg/model/provider/bedrock/client.go

+// convertToConverseTools converts cagent tools to Converse API format
+func convertToConverseTools(tools []cagentTools.Tool) ([]types.Tool, error) {
+	var converseTools []types.Tool
+
+	for _, tool := range tools {
+		// Convert tool parameters schema
+		schemaMap, err := cagentTools.SchemaToMap(tool.Parameters)
+		if err != nil {
+			return nil, fmt.Errorf("failed to convert tool %s schema: %w", tool.Name, err)
+		}
+
+		// Convert schema to document
+		inputSchema, err := convertToDocument(schemaMap)
+		if err != nil {
+			return nil, fmt.Errorf("failed to convert tool %s input schema: %w", tool.Name, err)
+		}
+
+		converseTools = append(converseTools, &types.ToolMemberToolSpec{
+			Value: types.ToolSpecification{
+				Name:        aws.String(tool.Name),
+				Description: aws.String(tool.Description),
+				InputSchema: &types.ToolInputSchemaMemberJson{
+					Value: inputSchema,
+				},
+			},
+		})
+	}
+
+	return converseTools, nil
+}


The convertToConverseTools function lacks test coverage. This function converts tool schemas to the Bedrock Converse API format. Consider adding tests to verify correct schema conversion, handling of tool descriptions, and proper error handling when schema conversion fails. This is especially important since different model families on Bedrock might have different schema requirements.

Same as above - valid but would be better as follow-up work.

krissetto · 2025-12-11T16:07:20Z

pkg/model/provider/bedrock/adapter.go

+		}
+		return response, nil
+
+	case *types.ConverseStreamOutputMemberMetadata:


I haven't gone deep into the code yet, but testing this branch with global.anthropic.claude-haiku-4-5-20251001-v1:0 I don't see any token usage info in the TUI. Could you double check that?

examples/bedrock.yaml

- AWS Bedrock provider, supporting multiple model families (Anthropic Claude, Amazon Titan, Meta Llama, and Mistral). Signed-off-by: Alvaro Saurin <[email protected]>

Signed-off-by: Alvaro Saurin <[email protected]>

inercia requested a review from a team as a code owner October 17, 2025 10:58

rumpl added the enhancement label Oct 17, 2025

rumpl reviewed Oct 17, 2025

View reviewed changes

pkg/model/provider/bedrock/client.go Outdated Show resolved Hide resolved

inercia force-pushed the inercia/aws-bedrock-provider branch from d2e7d92 to 93b0ac6 Compare October 17, 2025 11:51

inercia requested a review from rumpl October 17, 2025 11:52

inercia force-pushed the inercia/aws-bedrock-provider branch 2 times, most recently from 780d7fd to f5a38ac Compare October 20, 2025 08:19

rumpl reviewed Oct 20, 2025

View reviewed changes

inercia force-pushed the inercia/aws-bedrock-provider branch from f5a38ac to 24e64ba Compare October 21, 2025 09:42

krissetto added kind/enhancement New feature or request and removed enhancement labels Oct 29, 2025

mfenderov mentioned this pull request Dec 6, 2025

Add AWS Bedrock provider support #1045

Open

Copilot AI review requested due to automatic review settings December 11, 2025 11:50

inercia force-pushed the inercia/aws-bedrock-provider branch from 24e64ba to ff4d630 Compare December 11, 2025 11:50

Copilot started reviewing on behalf of inercia December 11, 2025 11:50 View session

Copilot AI reviewed Dec 11, 2025

View reviewed changes

inercia force-pushed the inercia/aws-bedrock-provider branch 2 times, most recently from 284d323 to a3006e7 Compare December 11, 2025 12:33

inercia requested a review from Copilot December 11, 2025 12:35

Copilot started reviewing on behalf of inercia December 11, 2025 12:35 View session

Copilot AI reviewed Dec 11, 2025

View reviewed changes

inercia force-pushed the inercia/aws-bedrock-provider branch from a3006e7 to 28474dd Compare December 11, 2025 13:19

krissetto reviewed Dec 11, 2025

View reviewed changes

inercia force-pushed the inercia/aws-bedrock-provider branch 2 times, most recently from 3a4ec8d to 24801ef Compare December 11, 2025 16:19

inercia added 2 commits December 15, 2025 16:09

AWS Bedrock provider support

8c7e481

- AWS Bedrock provider, supporting multiple model families (Anthropic Claude, Amazon Titan, Meta Llama, and Mistral). Signed-off-by: Alvaro Saurin <[email protected]>

Do not crash if the env is nil

4554e6b

Signed-off-by: Alvaro Saurin <[email protected]>

inercia force-pushed the inercia/aws-bedrock-provider branch from 24801ef to 4554e6b Compare December 15, 2025 15:09

AWS Bedrock provider support #542

Are you sure you want to change the base?

AWS Bedrock provider support #542

Uh oh!

Conversation

inercia commented Oct 17, 2025

Uh oh!

Uh oh!

rumpl Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

inercia Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

rumpl commented Oct 20, 2025

Uh oh!

inercia commented Oct 21, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

inercia Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

inercia Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

inercia Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

inercia Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

krissetto Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants