Adventures with AWS MCP and Claude

Introduction

I recently installed the AWS MCP Server for Claude. It’s been a very cool product, and I of course, decided to use it in readonly mode because while I may be dumb, I am certainly cautious. While Claude has not, to date, suggested anything destructive to me, either via the AWS CLI, or anything else it has recommended, it can and does hallucinate about things regularly, and has suggested inefficient ways of getting things done to me. I would not suggest giving it write access to an AWS environment that you care about, at least not without reviewing every single thing it intends to do (More on that in another blog post). And there are enough stories on the internet of people learning that lesson the hard way.

It has offered me helpful information, and has assisted me with writing scripts to provision environments, including writing JSON for IAM roles needed for a project I’ve been working on. I want to make it clear that I find it useful, and I’m not a luddite, at least not in that particular way when it comes to these tools. But, as with my powertools in my woodworking hobby (Want to buy a table?), wearing proper PPE (Limited permissions, not blindly accepting Claude’s output and suggested tool usage), is needed.

The actual problem

When you install a tool in Claude, if you don’t provide baseline guidance for when and how to use it, Claude may use it in ways that aren’t particularly productive to what you’re trying to get done. It often tends to lean towards wanting to check assumptions if it has a tool that allows it to do so. While that sounds great on paper, asking it in a fresh session to “ssh into this dev box and check the test migration we’re doing” since adding the AWS MCP server has resulted in Claude checking:

1. Is my current IP in the security group
2. What is the public IP of that EC2 instance

Each of those calls takes several seconds, and a few hundred tokens. Which adds up in wasted time and additional unneeded tokens against your quota.

Now this example is relatively trivial, but it’s one I recently ran into and solved, so it’s fresh in my mind.

Solving this problem

Claude provides some tools that you can use to control its behavior, both in the context of your specific project, and globally.

In this case I chose to solve it globally. If you’re not already familiar, in your home directory on Linux and Mac OS, there is a directory called .claude. You can access it by doing cd ~/.claude. This directory contains a variety of files that pertain to Claude’s configuration and governance on your machine. Like your local context per project, .claude can contain a CLAUDE.md.

CLAUDE.md is a markdown file that has instructions for the agent, which can include how and when to use a particular tool. Your preferences (Like don’t write those crazy verbose readmes that no one wants to read), and fun things like that. Amazon provides a starter rules file that you can include On their github. The existing guidance includes some great guardrails, such as “When uncertain about specific AWS details (API parameters, permissions, limits, error codes), verify against documentation rather than guessing. State uncertainty explicitly if you cannot confirm.” which can be a real problem with AI agents, as confident incorrectness can lead you to agree to things you shouldn’t.

However, these guardrails don’t really stop it from wasting your time and tokens. Which is probably what you’re here for.

- Do not jump to conclusions that you need to use the AWS tooling for problems. Ask if you should, if not prompted to do so. 
- Don't assume that infrastructure is the problem first when troubleshooting. Identify infrastructure theories explicitly and wait for guidance before pursuing. 
- For write operations using the AWS provide commands in a script or inline for review, 
  and for me to run instead of you. READONLY operations are fine though. 

While these 3 instructions probably aren’t perfect, and may be a little bit heavy handed, they helped right away. Rather than Claude immediately jumping to use its new toy (I know it doesn’t actually think and thus does not have that kind of motivation), the tool is now available when I want/need, so it adds convenience in that way, but I can still say “Hey Claude! How many production web application servers are currently running in our AWS environment” and get an answer without a question like “I’m seeing this exception in our production logs” we don’t go down a multi-minute rabbit hole of it looking at my Aurora RDS configuration before being told to look at code.

What we’ve learned for next time

Get explicit with your governance instructions when you’re introducing new tools. Tell Claude how you’re using that particular tool, how/when/why you want it used. Create explicit agents for using that tool when needed that have more specific prompting.