A practical guide to building AI agents on AWS Bedrock

Most teams can get a demo agent running in an afternoon. The hard part is the next nine months: turning it into something a business depends on. This guide is the version we wish we had when we started building agents on AWS Bedrock for customers in financial services, industry, and sport.

What an AI agent on AWS Bedrock actually is

An AI agent is a foundation model with a job to do, a set of tools it can call, and a loop that decides what to do next. On AWS, Bedrock provides the models (Anthropic Claude, Amazon Nova, Meta Llama, Mistral, and others), a managed agent runtime, Knowledge Bases for retrieval, and Guardrails for safety, all behind a single IAM and VPC boundary.

You don't have to use the managed runtime. The Strands Agents SDK, LangGraph, and custom orchestrators all run perfectly well against the Bedrock InvokeModel and Converse APIs. The choice is about how much control you need over the planning loop.

A reference architecture for AI agents on Bedrock

A production agent on Bedrock usually has six moving parts:

• Model layer. One or more Bedrock models, routed by task complexity. Cheap models for classification and extraction, frontier models for planning.
• Orchestrator. Bedrock Agents, Strands, or a custom loop that owns planning, tool calls, and memory.
• Tools (action groups). Lambda or ECS functions exposed through OpenAPI schemas. This is where the agent actually does work.
• Knowledge. A Bedrock Knowledge Base on top of S3 with OpenSearch Serverless or Aurora pgvector as the vector store.
• Guardrails. Bedrock Guardrails for denied topics, PII redaction, content filters, and contextual grounding.
• Observability. CloudWatch, X-Ray traces, and an evaluation harness. Without this you are flying blind.

Picking a model for your AI agent

We default to Anthropic Claude for planning-heavy agents and Amazon Nova for high-volume, latency-sensitive paths. The right answer is almost always a mix: route within the agent, measure on your own evals, and revisit quarterly as new models land in Bedrock. Cross-region inference profiles dramatically improve throughput; turn them on early.

Tools are the product

The model is interchangeable. Your tools are not. They encode what the agent can actually do in your business: read an order, raise a ticket, run a query, update a record. Three rules we keep coming back to:

• Narrow and named. One tool, one verb. Models pick the right tool when names and descriptions are unambiguous.
• Idempotent. Agents retry. Tools must tolerate it.
• Typed inputs and outputs. JSON schemas catch hallucinated arguments before they reach your systems.

Retrieval that actually grounds answers

Bedrock Knowledge Bases hide a lot of plumbing: chunking, embeddings, the vector store, and re-ranking. Use it as a starting point, then tune chunk size, hybrid search, and metadata filters for your corpus. Pair it with Guardrails' contextual grounding check to refuse answers that are not supported by retrieved sources.

Guardrails, security, and data residency

Bedrock runs inside your AWS account, models do not train on your data, and you can pin inference to specific regions, including eu-north-1 and eu-central-1 for European workloads. Apply Bedrock Guardrails to every agent invocation, scope IAM roles per tool, and put PrivateLink in front of Bedrock when the agent runs inside a VPC.

Evaluation: the only thing that tells you it works

The single biggest difference between a demo and a production agent is an evaluation harness. Build a small, opinionated test set of real tasks the agent must complete, and score every change against it. Bedrock's model evaluation jobs help with model-level comparisons; for agent-level quality you almost always need your own scorer, often LLM-as-judge plus a handful of deterministic checks.

What it costs

Token spend dominates. Prompt caching, model routing, and aggressive context trimming usually cut cost by 40–70% versus a naïve implementation. Knowledge Base storage, Guardrails policy units, and the Lambda or ECS hosting your tools are typically rounding errors next to inference.

Common mistakes

• Treating the agent as a chatbot instead of a workflow. The interesting agents do not talk; they act.
• Skipping evaluations and trusting vibes. Every prompt change is a regression risk.
• Letting the agent see too much context. More tokens is not more intelligence.
• One giant tool that does everything. Models pick badly when tools overlap.
• No human in the loop for irreversible actions. Agents are confident. Confidence is not correctness.

Where to start building your AI agent

Pick one workflow that is well-defined, measurable, and currently consumes real human time. Build the smallest possible agent against it. Wire up Guardrails and evals from day one. Ship it to a small group of real users for two weeks, then decide whether to scale, rework, or kill it. That is the loop. Everything else is theatre.

We have written about one such build in our case study on Rubberduck, an AI assistant for developers at Scania PDM IT built on Bedrock and the Strands SDK.