Five VPCs connected by peering need ten peering connections. Ten VPCs need forty-five. Every new VPC means going back into the route tables of every existing VPC to add the new relationship, and every peering connection is a distinct thing that can be misconfigured, forgotten, or left open longer than it should be. That math is the whole argument for hub-and-spoke, and it’s why almost nobody designs a multi-account AWS network as a full peering mesh past the first few accounts.
AWS Landing Zone – Part 3
Part 1 and Part 2 covered the account structure and the guardrails that govern it. This post is about the network underneath: Transit Gateway as the hub almost everyone reaches for, centralized egress and what it costs against the alternative, IPAM (IP Address Management) as the CIDR governance layer, and where VPC Lattice fits without pretending it replaces Transit Gateway wholesale.
Transit Gateway as the hub
AWS Transit Gateway is a managed, regional routing hub. Spokes, VPCs, VPN connections, Direct Connect, attach to it once, and it handles routing between them using its own route tables instead of point-to-point relationships. One Transit Gateway per Region is typically enough since it’s highly available by design, though there’s a legitimate case for more than one where you want to limit the blast radius of a routing misconfiguration or separate control-plane operations between teams. Cross-region connectivity happens through Transit Gateway peering; hybrid connectivity, Direct Connect or VPN, attaches the same way a VPC would.
The account this lives in matters. Transit Gateway, IPAM, and centralized egress infrastructure all belong in the Infrastructure OU’s networking account, sometimes literally called Network or Network Hub, not scattered across workload accounts and not in the Control Tower management account. That account becomes the place a network engineer actually works day to day, and it’s worth treating its own access and change process with the same care as the Security OU from Part 2.
Laid out, the topology looks like this:
Transit Gateway topology!
Every spoke VPC gets one attachment to the hub instead of a direct relationship with every other spoke. Whether Prod and Test can actually reach each other is a routing table decision inside the Transit Gateway, not something the topology forces one way or the other, which is exactly the control you lose with a flat peering mesh.
Centralized egress and what it actually costs
The other big reason to centralize on a hub isn’t just routing hygiene, it’s cost. Every VPC that needs internet access wants its own NAT gateway per Availability Zone for resilience, and NAT Gateway data processing charges add up fast once you’re running dozens of VPCs, each with its own pair or trio of gateways running independently. Centralizing egress means routing outbound traffic from every spoke through the Transit Gateway into a single egress VPC in the network account, where it exits through one set of NAT gateways instead of dozens. One practitioner audit found organizations spending around $15,000 a month on NAT Gateway data processing alone, spread thin across VPC after VPC, and cut that by 40 to 70 percent by consolidating.
Adding traffic inspection to that same choke point is a natural next step. AWS Network Firewall sits in the egress VPC and inspects traffic before it reaches the NAT gateway, at a real but bounded cost, roughly $0.40 an hour per endpoint plus a per-gigabyte processing charge, which is a fair trade in a regulated environment and a harder sell if you’re mostly optimizing for spend. One genuine gotcha here: DNS doesn’t follow this path by default. Route 53 Resolver and DNS Firewall are a separate egress route entirely, so centralizing your data-plane egress through Transit Gateway and Network Firewall doesn’t automatically mean your DNS queries are inspected the same way. If DNS-based filtering matters to your threat model, it needs its own explicit design, not an assumption that it rides along with everything else.
IPAM: get the CIDR plan right before account one
IP address planning is one of those things that’s cheap to fix before it exists and expensive to fix after. Amazon VPC IPAM gives you a hierarchical pool structure, a top-level pool subdivided into regional pools, then further into business-unit or environment pools, so that a new VPC in the Workloads_Test OU pulls its CIDR from a pool already guaranteed not to overlap with Prod, Sandbox, or anything else in the organization. IPAM should be delegated to the network account rather than run from the Control Tower management account, the same separation-of-duties instinct as everything else in this series.
The failure mode IPAM prevents is duller than a security incident but just as disruptive: two teams independently pick 10.0.0.0/16 for their VPCs, everything works fine in isolation, and then someone needs to connect the two networks and discovers the ranges collide. Fixing that after the fact means readdressing a live VPC, which is exactly the kind of maintenance window nobody wants to schedule. Getting the CIDR plan right before the first account exists costs an afternoon. Getting it wrong costs a migration.
Route 53 Profiles solve the equivalent problem for DNS. Instead of manually associating private hosted zones, resolver rules, and DNS firewall rule groups to every VPC individually, you bundle them into a single profile in the network account and share it across accounts through AWS RAM. New VPCs associate with the profile once and inherit the whole DNS configuration, rather than someone remembering to wire up each piece by hand every time an account gets vended.
Where VPC Lattice actually fits
VPC Lattice gets pitched sometimes as a Transit Gateway replacement, and that framing oversells it. Transit Gateway operates at Layer 3, it moves packets between IP addresses and doesn’t know or care which identity sent them; security has to come from route tables, security groups, and NACLs. VPC Lattice operates at Layer 7, HTTP, HTTPS, and gRPC specifically, and it’s service-centric rather than network-centric: services register into a service network, consumers discover them by DNS name, and access is governed by IAM policy rather than which subnet you happen to be in. It’s also currently single-region, so it isn’t a drop-in for anything that needs to span regions the way Transit Gateway peering does.
In practice these coexist rather than compete. Transit Gateway keeps doing the job of moving bulk traffic, hybrid connectivity, and anything that isn’t a clean HTTP service call. VPC Lattice picks up new service-to-service communication where IAM-based authorization is a better fit than managing security group rules across account boundaries. I’d reach for Lattice for a new internal API a team wants to expose across accounts without punching new holes in the network layer, not as a project to migrate an existing Transit Gateway backbone onto.
As a decision, it collapses to one real question about the shape of the traffic:
Shaping traffic!
Bulk data, non-HTTP protocols, or anything crossing regions stays on Transit Gateway. A specific, well-defined service boundary within one region is where Lattice earns its keep, and it’s rarely an either-or choice at the level of the whole network.
What’s next
The network is the part of a landing zone that’s genuinely painful to change once workloads depend on it, which is exactly why it deserves the same up-front discipline as the OU structure in Part 1. The last post in this series moves from design to operations: how account vending actually scales once you’re provisioning dozens of accounts a month, why Control Tower can tell you about drift automatically but won’t fix it for you, and the CI/CD pipeline that should sit in front of every change to the guardrails from Part 2 before it reaches a real account.
There’s a specific error message in AWS Control Tower that ruins a Tuesday. Someone edits or detaches a managed SCP (Service Control Policy) on the Security OU (Organization Unit), and the console locks you out with a warning that the shared accounts may no longer be working, and that you shouldn’t provision new accounts until it’s fixed. AWS documents this exact failure mode in its own knowledge base, which tells you it happens often enough to need a canonical writeup. The lesson isn’t “don’t touch SCPs.” It’s that the guardrails Control Tower manages for you and the guardrails you write yourself live in the same policy type, and mixing them up on the wrong OU is how you find out which one was load-bearing.
AWS Landing Zone – Part 2
This is Part 2 of the landing zone series. Part 1 covered what a landing zone is and the build-vs-buy decision behind Control Tower, LZA, and AFT. This one is about the layer that actually does the enforcing: service control policies and their newer sibling, resource control policies, the logging architecture that has to hold up when an auditor asks for it, and IAM Identity Center as the one door every human uses to get into any of this.
SCPs and RCPs aren’t the same guardrail
It’s worth being precise here because the names invite confusion. A service control policy caps what your own identities, users and roles inside your organization, are allowed to do, regardless of what their IAM policy says. A resource control policy caps who can touch a given resource, an S3 bucket, a KMS key, an SQS queue, and a growing list of others, regardless of what that resource’s own policy allows. AWS puts it simply: use an SCP to limit your own principals, use an RCP to restrict access to your resources from principals outside your organization. Neither one grants anything. Both are ceilings, not floors, and the permission a principal ends up with is whatever’s left after intersecting the SCP, the RCP, and the identity or resource policy underneath.
RCPs are the newer of the two, and they’re moving fast. They launched in November 2024 covering five services: S3, STS, KMS, SQS, and Secrets Manager. Since then AWS has kept extending the list. Cognito and CloudWatch Logs picked up support in January 2026, DynamoDB followed a few weeks later, and the per-organization quota doubled to 2,000 RCPs this past July. That pace tells you something: RCPs are still filling in gaps, not yet the mature, complete tool that SCPs are. Check the current supported-service list before designing a control around a service it doesn’t cover yet.
SCPs got their own significant upgrade in September 2025, when AWS gave them full IAM policy language support: conditions inside Allow statements, individual resource ARNs, NotAction with Allow, wildcards in the middle of an Action string. Before that update, SCPs were noticeably blunter than a regular IAM policy, which pushed a lot of teams toward broad deny-everything-except statements because anything more surgical wasn’t expressible. Existing SCPs kept working unchanged after the update, but if yours predate September 2025, there’s a real case for revisiting them now that more precise allow-with-conditions patterns are possible.
Where these guardrails get scoped
Tie this back to the OU structure from Part 1. In practice, almost every SCP and RCP you write gets attached at the Security OU, the Infrastructure OU, the Workloads OU, or somewhere in between, and the Security OU is the one to treat with real caution. That’s where Control Tower’s own managed SCPs live, the ones protecting the Log Archive and Audit accounts, and it’s exactly the OU where the lockout scenario above happens. If you need custom guardrails for security tooling, write them, but write them as additions at a level below where Control Tower’s own policies sit, not as edits to the managed ones.
Put visually, the two guardrail types gate different paths to the same resource:
SCPs,RCPs
The distinction matters operationally, not just semantically. An SCP written to block a risky action only stops your own users and roles from doing it. If the same S3 bucket is reachable by a principal from another AWS account entirely, only an RCP, or the bucket policy underneath it, actually stops that. Teams that treat SCPs as a complete security boundary and skip RCPs are leaving exactly this gap open, usually without realizing it until an access review turns it up.
Centralized logging that holds up under audit
Control Tower sets up an organization-level CloudTrail trail automatically, which, since landing zone version 3.0, replaced the older model of a separate trail per account. One trail logs everything, management account and every member account, and delivers into an S3 bucket that lives in the Log Archive account, the one nobody logs into day to day. AWS Config runs alongside it, aggregating configuration history into the Audit account rather than Log Archive, a distinction worth remembering when someone asks where a specific piece of evidence actually lives.
The pattern that makes this scale past a handful of accounts is delegated administration. Instead of every security service being manageable only from the management account, you designate a member account, almost always the Audit account, as the delegated admin for GuardDuty, Security Hub, Config, and similar services, and manage all of them centrally from there without ever touching the management account for day-to-day work. This has expanded well beyond security services specifically; one recent count put the number of AWS services supporting delegated administration at 37, up from roughly a dozen when the pattern first appeared. GuardDuty is regional, so this has to be repeated per Region you actually monitor, which is easy to miss if you’ve only ever tested in one.
Here’s what that centralization looks like end to end for a single finding:
Sequence of tracking
None of this is exotic engineering. It’s mostly turning on the right delegation and pointing things at the Audit account instead of leaving them scattered. The payoff shows up specifically during an audit, when “show me every API call across every account for the last year” is a single query against one bucket instead of a scavenger hunt across forty.
IAM Identity Center as the only door in
Every human touching any of these accounts should be going through IAM Identity Center, not IAM users with long-lived access keys. The mechanics are straightforward: permission sets define what a person can do, assignments connect a permission set to a group and an account or set of accounts, and Identity Center issues short-lived credentials rather than anything standing. Assign to groups, not individuals. When someone changes teams you move the group membership and every downstream permission follows automatically.
Identity Center federates cleanly with an external identity provider over SAML, Okta, Entra ID, whatever your organization already runs, with SCIM (System for Cross-domain Identity Management) handling user and group provisioning so you’re not managing a second identity store by hand. The one thing every landing zone still needs underneath all of this is break-glass access: a small number of IAM roles or users, outside Identity Center entirely, that work even if federation itself is broken or misconfigured. It’s not a contradiction to have SSO for everything and also keep a locked-down emergency path around it. It’s the same reason a building keeps a physical key next to the electronic badge reader.
One detail worth knowing if you’re serious about treating this whole layer as code: permission sets and their assignments can be managed through a CI/CD pipeline just like the SCPs and RCPs above, JSON templates in a repository, a pipeline reacting to changes and pushing updates to Identity Center. It’s a small thing, but it means access changes go through the same review process as everything else instead of being a console click nobody remembers making six months later.
What’s next
Guardrails and identity get most of the attention in landing zone design because they’re where the compliance conversations happen, but the networking layer underneath all of this has its own decisions and its own ways to quietly overspend. Part 3 covers Transit Gateway and the hub-and-spoke pattern almost everyone converges on, centralized egress and what it actually costs against a pile of per-VPC NAT gateways, IPAM as the CIDR governance layer you want in place before account number one, and where VPC Lattice fits next to Transit Gateway instead of replacing it.
Somewhere around account number fifteen, most AWS environments hit the same wall. Nobody’s quite sure which of the last few accounts got the CloudTrail baseline. Root MFA is enforced in some accounts and quietly skipped in others. Somebody wrote an SCP (Service Control Policy) blocking public S3 buckets during an incident eight months ago and never got around to rolling it out anywhere except the one OU (Organisation Unit) where the incident happened. None of this shows up in a design review. It shows up as a GuardDuty finding, usually the kind that makes someone ask out loud, in a meeting, “wait, how many AWS accounts do we actually have?”
AWS Landing Zone!
That question is usually the moment a landing zone stops being a nice-to-have and starts being infrastructure. This is the first post in a series on building one properly. I want to use it to settle three things: what a landing zone actually is, the account structure that almost everyone converges on regardless of tooling, and the real decision sitting in front of you once you accept you need one: AWS Control Tower on its own, Control Tower plus the Landing Zone Accelerator, Control Tower plus Account Factory for Terraform, or something fully custom.
What a landing zone actually is
A landing zone isn’t a product, even though AWS sells something with the name attached to it. It’s a pattern: a pre-configured multi-account AWS environment with a consistent security baseline, a defined account structure, and a governance layer that stops new accounts from drifting away from that baseline the moment someone spins one up. AWS’s own framing is close to this: a recommended starting point that includes default accounts, account structure, and network and security layouts, from which you then deploy actual workloads.
Worth being direct about scope here. If you’re running two or three accounts total, none of this is worth the overhead yet. Plain AWS Organizations with a couple of hand-written SCPs covers you fine. The inflection point tends to show up once the account count outgrows what one person can hold in their head, which I’d put somewhere around ten to twenty accounts, earlier still if more than one team can create accounts independently.
It’s also worth clearing up a naming collision. Search “AWS landing zone” today and you’ll still find plenty of material about the original AWS Landing Zone solution, a CloudFormation toolkit from around 2018 built around an account vending machine and StackSets-based baselining. AWS has been steering customers off that solution toward Control Tower for years now. If you’ve inherited one, migrating off it is its own project and outside what I’m covering here. Everything below assumes Control Tower as the starting point, because that’s what AWS actively builds and recommends today.
The account structure nobody really argues about
Whatever tooling sits on top, the account and OU shape converges on roughly the same layout everywhere, because it’s solving a genuinely small number of problems: keeping security tooling separate from workloads, keeping environments that shouldn’t share a blast radius apart, and giving an auditor, and future you, a clean story about what lives where.
Control Tower creates a Security OU automatically, along with two accounts inside it: a Log Archive account, where CloudTrail and Config data lands and stays untouched, and an Audit account, which holds the read-only tooling and access your security team actually uses to look at that data. From there, the recommended shape adds an Infrastructure OU for shared services and networking accounts (Control Tower won’t create this one for you), a Workloads OU for accounts running real applications, usually split into Workloads_Prod and Workloads_Test once you outgrow a single account, and a Sandbox OU where people can break things without touching anything that matters.
That basic shape looks like this laid out as a tree:
Account Structure
The two OUs that matter most for day-one governance are Security and Infrastructure, since almost every guardrail worth writing gets scoped against one or the other. This part of the design is boring, and it’s also the part most likely to cause real pain if you get it wrong early. Moving an account between OUs later means re-checking every SCP and every piece of automation that assumed the old placement. I’d rather spend an extra day on the OU layout before account number one exists than spend a quarter untangling it after account number sixty.
The actual decision: Control Tower, LZA, AFT, or custom
Once the shape is settled, the real question is how much of the plumbing you build yourself versus let AWS build for you. There are effectively four paths, and I’ve ended up recommending each of them at different times depending on what the organization already had in place.
Plain AWS Control Tower. This is where I’d start almost every new environment. Control Tower is a managed service that orchestrates AWS Organizations, Service Catalog, and IAM Identity Center to stand up a landing zone in under an hour, then keeps working after that: it applies controls (AWS’s current term for what most people still call guardrails, split into preventive, detective, and proactive types), it ships an Account Factory that vends new accounts against a template instead of by hand, and its dashboard flags accounts that have drifted from baseline. If you already have an established multi-account environment you built by hand and don’t want a rip-and-replace, AWS added a Controls Dedicated experience in late 2025 specifically for this case: you get the managed control catalog on top of your existing Organizations setup without adopting the full prescribed account structure first. For most organizations, plain Control Tower, in one of these two modes, covers the job completely.
Control Tower plus the Landing Zone Accelerator (LZA). LZA is AWS’s own CDK-based solution that layers additional orchestration on top of Control Tower rather than replacing it. AWS is explicit that Control Tower should be the foundation and LZA the enhancement, not the other way around. Where LZA earns its complexity is highly regulated environments carrying multiple overlapping compliance frameworks at once, say FedRAMP alongside PCI alongside a customer-specific security addendum, where controls need to land identically across every region from day one rather than getting bolted on region by region as you expand. The honest trade-off: LZA is config-file driven on the surface, but underneath it’s CDK stacks and a genuinely large number of Lambda functions doing the orchestration, and that becomes its own thing to reason about the day a deployment doesn’t behave the way the docs implied it would. I wouldn’t reach for LZA because it sounds more capable. I’d reach for it when a compliance requirement is actively dictating structure that Control Tower’s controls can’t give you on their own.
Control Tower plus Account Factory for Terraform (AFT). If the rest of the org already runs on Terraform, and account provisioning through a separate tool, console clicks for Control Tower or CDK for LZA, feels like an unnecessary second language, AFT is the fit. It follows a GitOps model: you write an account request as a Terraform file, push it, and a pipeline handles provisioning and customization end to end, with a Step Functions trace token so you can actually follow a request through the workflow instead of guessing why an account isn’t ready yet. AWS keeps sanding down the rough edges too. As of a recent update, AFT can automatically re-apply an account’s customizations when that account moves between OUs, which used to be a manual step and a common, quiet source of drift. The catch with AFT generally: it’s a second pipeline sitting on top of Control Tower, with its own state and its own failure modes, so you’re not avoiding operational surface area, you’re relocating it somewhere your team already knows how to debug. Good trade if your Terraform muscle is already strong. Worse trade if you’d be adopting Terraform for the first time just to get AFT.
Fully custom, no Control Tower. I’ve reached for this exactly when an organization had a specific, named requirement that Control Tower’s opinionated model genuinely couldn’t accommodate, and that’s rare. Control Tower is extensible enough that you can work directly in Organizations alongside it and have it reflect changes made outside its own console. Going fully custom means rebuilding drift detection, guardrail enforcement, and account vending from primitives yourself, which is a real, permanent amount of undifferentiated engineering. I’d want a concrete reason before signing up for that, not a general preference for owning the whole stack.
Put as a decision tree, the four paths collapse to three questions worth asking in order:
Making a decision!
Notice that “fully custom” only shows up after two other doors have already closed. That’s deliberate. It shouldn’t be the default answer to either of the first two questions, and in most environments it never gets reached at all.
What I’d actually do, and what’s next
Asked cold, with no other context: plain Control Tower, the standard OU shape above, and hold off on LZA or AFT until something concrete forces the question, either a compliance framework you can name or an existing Terraform investment you’d rather extend than fork. Landing zones are one of the few places in cloud architecture where the boring default is usually the right call, and the genuinely interesting decisions show up later, in the guardrails you write yourself and the exceptions you inevitably have to carve into them.
That’s where the next post picks up: service control policies and the newer resource control policies doing the actual enforcing, centralized logging that holds up under an audit, and IAM Identity Center as the front door every human uses to touch any of this. Part 3 covers the networking foundation, Transit Gateway against a flatter hub-and-spoke, and where IPAM actually earns its keep. Part 4 gets into running this thing once it’s live: account vending at scale, drift you find before someone else does, and the CI/CD pipeline for the landing zone code itself.
Ask anyone who has wired Prometheus metrics into CloudWatch what the real bottleneck was, and it was never Prometheus. It was the collector. You’d stand up an OpenTelemetry Collector as a Deployment or a DaemonSet, hand-tune its scrape config, guess at memory limits for cardinality you hadn’t measured yet, and then spend the next year treating it like one more piece of cluster infrastructure that needed patching, right-sizing, and on-call attention of its own.
AWS Managed Prometheus Collectors
On July 31, AWS announced managed Prometheus collectors for Amazon CloudWatch, a fully managed, agentless scraper for Amazon EKS, Amazon EC2, Amazon ECS, Amazon MSK, and Amazon OpenSearch Service. You supply a scrape configuration and a way to reach your resources, and CloudWatch takes it from there: provisioning the collector, scaling it, and keeping it running. What it scrapes arrives in OpenTelemetry format, queryable with PromQL right next to your AWS vended metrics.
That’s the announcement. The more useful question, if you’ve spent any time in AWS’s observability stack already, is what’s actually new here versus what’s just wearing a new badge.
The new part is the destination, not the collector
This isn’t new collector technology. AWS shipped this same agentless scraper (automatic target discovery, no in-cluster agent, multi-AZ, metrics that never leave your VPC) as the Amazon Managed Service for Prometheus collector back in November 2023, initially scoped to EKS and writing into an AMP workspace. That lineage is visible directly in the API today: you still create one of these with the CreateScraper operation, under the aws amp CLI namespace, not some new CloudWatch-specific command. What’s new in this announcement is the destination and the source list. Alongside an AMP workspace, the same managed collector can now write into a CloudWatch dataset, and the supported sources have grown from EKS-only to include EC2, ECS, MSK, and OpenSearch.
Swap that final --destination flag for an ampConfiguration with a workspace ARN and you’re back to the exact collector AWS shipped in 2023. If you’re already running AMP collectors for Grafana dashboards, that’s the useful takeaway: this is an incremental destination option on infrastructure you may already trust, not a new product to evaluate from a cold start.
It also builds on something CloudWatch shipped only six weeks earlier: native OTLP ingestion with PromQL querying, launched June 9. That release is what made CloudWatch a legitimate destination for Prometheus-shaped metrics at all, with per-GB pricing and curated Container Insights dashboards for EKS. Managed Prometheus collectors are the agentless front door to that same pipeline, extended past what Container Insights auto-instruments.
How a scrape actually reaches CloudWatch
The mechanics will feel familiar if you’ve used any managed AWS scraper before. You hand the collector a set of subnets, and it creates an Elastic Network Interface in each one. It scrapes your targets over those ENIs using OTLP, then delivers the result to your CloudWatch dataset through a VPC endpoint. Nothing crosses the public internet.
What isn’t obvious from the diagram is that target discovery differs meaningfully by source, and that’s what determines how much of an existing scrape config actually survives the move:
EKS uses Kubernetes service discovery against the cluster API, plus control-plane metrics (kube-scheduler and kube-controller-manager, exposed starting at Kubernetes 1.28).
ECS uses DNS-based discovery through AWS Cloud Map, so tasks register once and get picked up without a static IP list.
EC2 is static_configs against private IPs, no different from pointing a self-managed setup at a fixed target list.
MSK uses DNS-based discovery against the cluster’s own bootstrap DNS name, which resolves to every broker and survives broker replacement without reconfiguration.
MSK is worth a closer look because it needs a prerequisite step: enabling Open Monitoring on the cluster, which exposes a JMX Exporter on port 11001 and a Node Exporter on port 11002. A scrape config covering both looks like this:
That gets you broker-level Kafka metrics (topic throughput, consumer lag, under-replicated partitions) from the JMX exporter, and host-level CPU, memory, and disk metrics from Node Exporter, both queryable with PromQL once they land. Two things to check before you plan around this: MSK Serverless and MSK Express aren’t supported, and public access combined with KRaft metadata mode is excluded too.
The scrape config is Prometheus-compatible, not Prometheus
This is worth reading closely before you migrate anything, because “Prometheus-compatible YAML” undersells how much is actually missing. The supported configuration surface covers global settings, scrape_configs with static_configs and dns_sd_configs, and relabeling. What isn’t there matters more than what is:
Minimum scrape interval is 30 seconds. An SLO built on 10 or 15-second scrapes doesn’t fit here.
No file_sd_configs, and no Consul, Eureka, or other external service discovery. If your target list comes from anywhere other than Kubernetes, Cloud Map, a static list, or MSK’s broker DNS, this collector can’t reach it.
remote_write and remote_read don’t apply, because delivering to CloudWatch is the whole job. A fan-out to a second backend doesn’t carry over.
Configuration blobs cap out at 256 KB, base64-encoded.
None of this is really a flaw. It’s the standard trade-off of a managed service. AWS narrowed the surface to what it can operate reliably at scale, and the narrowing happens to remove exactly the parts of Prometheus configuration that are hardest to run safely inside someone else’s control plane. But “copy your existing prometheus.yml over” is rarely literally true. Diff your current scrape configs against this list before you commit to a cutover date, not after.
What ships automatically, and what doesn’t
EKS and MSK both get a curated dashboard the moment metrics start flowing — EKS OTel and MSK OTel in the CloudWatch console — plus attribute enrichment on every metric: account, region, an inferred unit, and for EKS specifically the cluster name and ARN. That enrichment is what makes PromQL filtering pleasant instead of an exercise in label archaeology. The EC2/ECS path doesn’t come with an automatic dashboard, which is a reasonable trade given how varied EC2/ECS metric shapes are in practice, but it’s worth knowing before cutover rather than after.
Operationally, a managed collector vends its own logs to CloudWatch Logs covering target discovery, scrape successes and failures, and configuration errors, and you manage the scraper itself with aws amp list-scrapers, describe-scraper, and delete-scraper. Deleting one tears down its ENIs, and that cleanup takes a few minutes, so don’t script an immediate re-create into the same subnet.
Cross-account guidance has shifted too. Instead of the role-chaining pattern AMP has historically used for cross-account scrapers, AWS’s current recommendation for both EKS and MSK is CloudWatch metric centralization: replicate the metrics to a monitoring account rather than granting the scraper cross-account reach in the first place.
What this actually costs
Pricing has two components that scale independently, and it’s worth keeping them separate. The collector itself is billed hourly, similar in shape to an ENI or a NAT gateway, though the specific per-hour rate isn’t broken out yet on AWS’s public pricing page as of this writing. What is confirmed is that everything the collector delivers rides on standard CloudWatch OpenTelemetry ingestion pricing: currently $0.50 per GB for general OTel metric publishing, a flat rate that bundles in fifteen months of storage with no separate per-metric or per-API charge. A typical data point with ten to fifteen attributes runs 300 to 600 bytes, for a sense of scale. Console PromQL queries (Query Studio, dashboards) are free; programmatic PromQL queries cost $0.01 per million samples scanned.
That per-GB model is a real shift in how you’d think about cost here. Classic CloudWatch metrics billed per unique metric-dimension combination, so cardinality was the thing to fear. Under OTel ingestion, cardinality by itself is free; what costs money is the serialized size of each data point, which grows with the number and length of the labels attached to it. Fifteen short labels on a metric barely move the needle regardless of series count. A handful of long, high-cardinality values move it fast: unbounded request IDs, verbose ARNs repeated on every point. If you’re porting scrape configs over from a self-managed setup, that’s the moment to prune labels you’ve been carrying out of habit rather than actual query need.
One more line item: scraping itself can rack up VPC data transfer charges between the collector and your targets, separate from CloudWatch ingestion. AWS’s own suggestion is to enable gzip on your /metrics endpoints, which cuts transfer cost without changing what actually gets ingested.
Where I’d actually reach for this
If your team is already CloudWatch-centric and has been putting off a real Prometheus setup mainly because nobody wanted to own collector infrastructure, this closes that gap cleanly. For EKS and MSK metrics specifically, it’s a solid default: alarms and dashboards in the same place as everything else, without standing up Grafana and an AMP workspace just to get there.
I’d still keep a self-managed collector, or the AMP-workspace version of this same scraper, in a handful of specific situations: sub-30-second scrape requirements, target discovery through Consul, Eureka, or file-based service discovery, or a remote_write fan-out to more than one backend because you’re mid-migration or running a deliberately multi-vendor setup on purpose. None of those are rare for a mature platform team, so this doesn’t replace the self-managed path outright. It’s a genuinely good default for the common case, which happens to be most of it.
Availability is broad but not universal — it’s live everywhere CloudWatch’s OTLP endpoint already exists, minus Asia Pacific (New Zealand) for now. Worth checking before you write the migration runbook, not after you’ve scheduled it.
Somewhere in your notes app, or maybe just muscle memory, there’s a paragraph you retype into Claude at the start of almost every work session. The AWS tags that are actually mandatory versus the ones nobody enforces. The five things a pull request review always has to check before it counts as “done.” Which environments tolerate an open security group and which ones get you a message from the security team five minutes later. You paste it in, the model gets up to speed, and you move on to the actual task. Next week you paste it in again, because the model has no memory of last Tuesday, and the teammate who just joined your channel has never seen that paragraph at all.
Skill.md!
That’s the exact problem Anthropic built Agent Skills to solve, and the fix is almost embarrassingly low-tech: a folder with a markdown file in it, named SKILL.md.
What a SKILL.md file actually is
A Skill is a directory. At minimum it holds one file, SKILL.md, which opens with a short YAML header (called frontmatter) and is followed by ordinary markdown instructions. That’s the entire format. No special runtime, no proprietary config language, nothing to compile.
If you’ve ever asked Claude to build you a PowerPoint deck or clean up an Excel sheet, you’ve already used one without realizing it. The document tools Claude ships with for PowerPoint, Excel, Word, and PDF are, under the hood, ordinary Skills, the same folder-and-markdown format you’d use to build your own.
Anthropic frames building a Skill as putting together an onboarding packet for a new hire: here’s what you need to know, here’s how we do things, here’s where to look if you get stuck further in. The engineering post announcing the feature, published on October 16, 2025, walks through this using a PDF-handling Skill as its running example, and the team signs off the post with a joke about their shared fondness for folders. Fair enough — a Skill really is just a folder that happens to be very good at its one job.
Beyond instructions, a Skill can bundle reference docs, templates, and actual executable scripts, all living in the same directory. Claude reads whatever it needs and leaves the rest sitting on disk untouched. That last part turns out to be the whole point of the design.
The problem this actually solves
Before Skills, there were two bad options for capturing “how we do things here.” Cram everything into one enormous system prompt, and Claude carries your entire team’s handbook in its head on every single request, most of it irrelevant to whatever you actually asked. Or build a separate, narrowly scoped agent for every recurring task, which stops scaling somewhere around the third or fourth workflow.
Skills get around both by loading information in stages, a pattern Anthropic calls progressive disclosure. None of a Skill’s full content sits in context until Claude decides it’s actually relevant to the request in front of it.
How the loading actually works
There are three levels, and the token cost changes sharply between them, according to Anthropic’s documentation:
Level
When it loads
Rough cost
What’s in it
1. Metadata
Always, at startup
~100 tokens per Skill
Just the name and description
2. Instructions
Only when the Skill is triggered
Aim under ~5k tokens (body under 500 lines)
The markdown body: workflow, rules, templates
3. Resources and scripts
Only as needed
Nothing, until read or run
Reference files, templates, executable scripts
Install thirty Skills and Claude carries thirty short descriptions around at all times, which is cheap. It reads the full body of exactly one of them: whichever matches your request. Anything that Skill bundles beyond its own SKILL.md, a reference doc or a Python script, stays untouched on the filesystem unless Claude specifically goes looking.
Here’s what that looks like end to end, using a Skill built to review Terraform plans before they’re applied:
Skills flow example
Notice the script step. Claude runs check_tags.py through bash and only the output, a pass or a list of violations, comes back into context. The script’s actual code never loads. It’s also why Anthropic’s guidance leans toward writing real utility scripts for anything deterministic, tag validation, checking for overlapping form fields, whatever the equivalent is in your domain, rather than asking Claude to regenerate that logic from scratch every single time. A script is more reliable, and it’s nearly free, token-wise.
When, and why, to reach for one
Skills earn their keep on anything you’d otherwise explain more than once. A few honest signals it’s worth the twenty minutes:
You’ve caught yourself pasting the same context, checklist, or gotcha into more than two or three conversations already
The task follows a procedure your team already agreed on, a review process, a report format, a migration sequence, and you want everyone’s output to actually look the same
You want to stack several of these specialized capabilities in one session; Skills compose, so a single request can trigger more than one at a time
Where beginners usually get tangled is the line between a Skill and MCP (Model Context Protocol). A Skill is packaged know-how: instructions, and optionally code, sitting on Claude’s filesystem. It is not a live connection to Jira, a production database, or your calendar. That’s exactly what MCP connectors exist for. The two aren’t rivals. A Skill’s instructions can absolutely tell Claude to call a specific MCP tool partway through a workflow, and Anthropic’s own guidance is to reference it by its fully qualified name, something like GitHub:create_issue, so Claude doesn’t confuse it with a similarly named tool from a different server. The Skill supplies the how and the when. MCP supplies the reach into a live external system.
Skills and MCP
Anatomy of a real one
Here’s a small but complete Skill: the same Terraform-review example from the diagram above.
---
name: reviewing-terraform-plans
description: Reviews Terraform plan output against infrastructure safety rules before apply. Flags unapproved destroys, missing required tags, and security groups or storage buckets opened to the public internet. Use whenever the user pastes terraform plan output, uploads a plan file, or asks whether a Terraform change is safe to apply.
---
# Reviewing Terraform Plans
## Quick workflow
1. Read the pasted plan output, or run `terraform show -json tfplan.binary` if a binary plan file was provided.
2. Flag every resource marked for **destroy** or **replace**. Check [RULES.md](RULES.md) for whether that resource type needs a second approver before it's safe to apply.
3. Verify tagging by running `python scripts/check_tags.py plan.json` instead of reading the JSON by eye. It lists every resource missing an `owner` or `environment` tag.
4. Flag any security group, storage bucket policy, or load balancer rule that widens access to `0.0.0.0/0`.
5. Write up the findings using the format below.
## Report format
Always structure the review like this:
```
## Plan Review: [environment]
### Blocking
### Needs a second look
### Looks safe
```
Only use "Looks safe" for resources that passed both the destroy check and the tag check.
## Rules for destroys and replaces
See [RULES.md](RULES.md) for the sign-off matrix: which resource types need a second approver, which are safe to auto-approve, and how to write a rollback note.
## Utility scripts
**scripts/check_tags.py** — validates required tags against a JSON plan export. Exits non-zero and lists offending resources if anything is missing `owner` or `environment`.
Now the walk-through. The frontmatter is the only mandatory part, and both of its fields do real work rather than just labeling the file.
name has firm rules:
Maximum 64 characters
Lowercase letters, numbers, and hyphens only
No XML tags
Can’t contain “anthropic” or “claude”
Anthropic’s naming guidance recommends a gerund form, like reviewing-terraform-plans or processing-pdfs, on the theory that it reads as an activity rather than a vague noun like helper or utils. That’s a recommendation, not a hard rule, but it’s a good one.
description matters more than it looks like it should, because it’s the only thing Claude sees before deciding whether to open the file at all:
Non-empty, maximum 1,024 characters
No XML tags
Written in third person, not “I can help you review…”
Must cover both what the Skill does and when to reach for it
Get this vague, “helps with infrastructure,” say, and the Skill just won’t fire reliably, because Claude is pattern-matching your actual request against this exact text before it ever reads the body.
Everything below the frontmatter is “Level 2,” read only once the description matches. The workflow section is a numbered procedure rather than loose prose, because this is what Anthropic calls a narrow-bridge task: there’s basically one correct order of operations, so it gets specific, low-freedom steps instead of general guidance. A code-review Skill, by contrast, is an open-field task. Several valid approaches exist depending on context, so the right move there is a looser set of principles Claude can apply with judgment.
The report format is a plain template. Claude copies the structure rather than inventing its own headers each time, which is what keeps five plan reviews from five different engineers from looking like five different documents.
RULES.md is the interesting one. It’s referenced by name but not pulled into SKILL.md itself, so it costs zero tokens unless a destroy actually shows up in the plan. That’s progressive disclosure paying off directly: most Terraform plans don’t touch anything destructive, so most reviews never load it at all.
And check_tags.py exists because validating tag presence is exactly the kind of deterministic, no-judgment-required task a script handles more reliably, and far more cheaply, than having Claude parse JSON by eye on every single run.
Where it’s not worth the effort
A few situations where reaching for a Skill is the wrong move.
One-off tasks. If you’re never going to ask for this again, just ask directly. Claude tends to only bother consulting a Skill for something it can’t already handle cleanly with its built-in tools, so a simple “summarize this PDF” often won’t trigger a Skill at all, matching description or not, because there was no real gap for the Skill to fill.
A single fact or preference. “I prefer metric units” isn’t a workflow. It’s a preference, and it belongs in your settings or a saved memory, not a folder with a YAML header.
Anything that needs a live connection to an external system. As covered above, that’s MCP’s job.
Content with a shelf life. Baking in “as of this month, use the v1 API” is a trap. Anthropic’s own best-practices guidance calls this out directly: keep only the current method in the main instructions, and tuck anything deprecated into a collapsed “old patterns” section instead of sprinkling date-conditional logic through the file. A Skill that says “if it’s before August, do X” is simply wrong the moment August ends.
It’s a standard now, not just a Claude feature
Skills launched on October 16, 2025 as an Anthropic-specific mechanism across Claude.ai, Claude Code, the Claude Developer Platform, and the Agent SDK. On December 18, 2025, Anthropic published the format as an open, cross-platform standard, meaning a well-written SKILL.md folder isn’t locked to one vendor. At the time of writing, the same format is supported, with varying degrees of completeness, by dozens of other agent tools, including Cursor, GitHub Copilot, VS Code, Gemini CLI, and OpenAI Codex.
Practically, that’s a reason to actually invest in writing these well. A Skill your team builds for Claude Code today is a reasonable bet to keep working if part of your toolchain moves somewhere else next year.
Guidelines worth following even though nothing enforces them
Beyond the hard constraints on name and description, Anthropic’s best-practices guide is mostly soft guidance. It’s worth taking seriously anyway.
Keep it concise. The default assumption should be that Claude already knows what a PDF is or how a REST API works, so don’t spend tokens re-explaining things it already knows. Match your level of specificity to how fragile the task actually is: loose, principle-based instructions for judgment calls, and exact scripts with no room for improvisation anywhere one wrong step corrupts data. Anthropic’s own comparison is a narrow bridge versus an open field, and it’s a genuinely useful way to decide how much rope to give.
Keep reference files exactly one level deep from SKILL.md. If Claude has to follow a chain of three linked files to find the actual instruction, it tends to skim with something like head -100 rather than read the whole thing, and you lose information you thought you’d included. And build a few test cases before writing extensive documentation, not after, so you’re solving problems Claude actually has rather than ones you imagined it might have.
Guardrails: treat a Skill like software, because it is one
This part is easy to skip past, and shouldn’t be. A Skill can execute code and invoke tools, which means a careless or malicious one can do real damage: read files it shouldn’t, reach out to an external URL with data picked up along the way, or quietly do something other than what its own description claims.
Anthropic’s security guidance is blunt about it: only install Skills from sources you trust, yourself or Anthropic directly, and if you’re using one from anywhere else, audit every file in the directory first, not just SKILL.md. Pay particular attention to anything that reaches out to an external URL, since fetched content can carry instructions of its own that Claude never asked for.
If you’re rolling Skills out across a team, Anthropic’s enterprise guidance is direct about treating this like any other software approval process rather than something looser: run a risk assessment before deployment, require an evaluation suite of representative queries covering cases where the Skill should trigger, shouldn’t, and sits ambiguously in between, and don’t let a Skill anywhere near production until it clears that bar. The general shape most teams land on beyond that: someone other than the author reviews the actual instructions and code, it gets tried out in an isolated environment first, and there’s a simple record somewhere of what’s approved and who owns it. “It’s just a markdown file” is exactly the assumption that causes problems once that markdown file starts running Python.
It’s also worth knowing the sandbox itself isn’t identical everywhere. Skills running through the Claude API get no network access and can’t install packages at runtime; whatever’s pre-installed is what you get. Skills in Claude Code have the same network access as anything else running on your machine. That difference alone should shape what you’re willing to let a given Skill actually do.
Actually building one
The fastest way in is almost too simple to feel like real advice: ask Claude to write it. Claude already understands the SKILL.md format natively, so work through a task normally, then say something like “turn this into a Skill,” and it’ll produce a properly structured file with sensible frontmatter on the first pass. From there, the loop that actually improves a Skill is boring but effective: use it on a real task, notice where Claude reached for the wrong file or skipped a rule, and go fix that specific gap instead of rewriting the whole thing speculatively.
For a more structured on-ramp, Anthropic’s quickstart and cookbook cover the API side end to end, and DeepLearning.AI has a short course built with Anthropic if you’d rather work through examples with someone walking alongside you.
Either way, the actual test for whether a Skill is worth keeping is the same test you’d apply to any piece of internal documentation: does the next person who hits this task get through it faster because the file exists? If yes, it’s earning its space on disk. If not, it’s just another file nobody reads, which, folder or not, is the exact problem you were trying to get away from in the first place.
You merge a pull request that Cursor wrote for you in ninety seconds. By hand, it would have taken the better part of an hour. For a moment you feel like you’ve found a genuine shortcut. In July 2025, the AI research nonprofit METR ran the most careful study anyone has done on that exact feeling, and found that experienced developers using AI tools on codebases they already knew well took 19% longer than developers doing the same work by hand. Afterward, the AI group still believed they’d been sped up by 20%. Same people, same tasks, two completely different numbers.
Price of an AI
That gap, between what AI feels like it’s saving you and what it’s actually costing you, is what this post is about. Almost all of the rigorous measurement on this so far is about software developers specifically, since developers leave a paper trail of commits and tickets that’s unusually easy to study. Swap “pull request” for “change ticket” or “runbook,” though, and the same shape shows up for sysadmins, SREs, and support desks too. The costs land in two currencies: the dollars on next quarter’s invoice, and the skill you quietly stop building because the model always answers first.
The productivity story is messier than either side wants it to be
METR’s result held up because the study design was unusually tight. Sixteen experienced developers, working in mature open-source projects they’d averaged five years on, were randomly told to use or not use AI tools on 246 real issues they’d have tackled anyway. This wasn’t a benchmark or a survey. It was a stopwatch on real work, in codebases these developers already understood cold, and that detail matters more than the headline number. The slowdown showed up specifically where the developer already had deep context.
METR tried to re-run the study with newer tools starting in August 2025, and by February 2026 they’d run into a problem worth mentioning on its own terms: a large share of the developers they invited, by some accounts as much as half, declined to take part at all unless they were guaranteed AI access. That’s not really a productivity statistic. That’s a dependency statistic. Among the smaller cohort who did participate, the slowdown shrank to something close to statistical noise, and METR now says, based on conversations with participants, that it believes AI is probably providing a genuine benefit in early 2026, while being upfront that its data is too thin to say how much.
None of this means AI coding tools are a bust. Google’s 2025 DORA report, drawing on nearly 5,000 technology professionals, calls AI “the great amplifier”: it doesn’t fix a struggling team or break a strong one, it magnifies whatever was already there. Unlike the year before, the 2025 data showed a positive relationship between AI adoption and both delivery throughput and product performance. Adoption still correlated with worse delivery stability, though, and DORA’s explanation is that acceleration without strong automated testing, clean version control, and fast feedback loops just means more change volume hitting a system that can’t absorb it safely.
Separately, Faros AI pulled telemetry from over 10,000 developers and told a version of the same story from another angle. On teams with heavy AI adoption, individual developers handled 9% more tasks and shipped 47% more pull requests per day, but none of that translated into their organizations delivering software noticeably faster overall. Doing more and delivering more turned out to be different things.
My read, watching this play out on real teams, is that the studies agree more than they disagree. AI’s payoff depends on how unfamiliar the terrain is and how strong the guardrails are around it. It’s least useful in exactly the place an experienced IT professional already has an advantage: their own well-understood system.
The dollar price tag
Uber’s leadership was unusually candid about what happened next. By April 2026, four months into the year, the company had already burned through its entire annual budget for AI coding tools. CEO Dara Khosrowshahi put it plainly on a podcast that June: “we blew through our AI budget in a quarter,” for what was meant to last the whole year. Uber’s fix was the one most FinOps teams eventually reach for anyway: a hard per-employee cap of $1,500 a month per agentic coding tool, visible on an internal dashboard so people can watch themselves approach the limit. Walmart quietly capped its own internal agent around the same time. Microsoft, according to reporting picked up by Fortune, began pulling back most of its direct Claude Code seats in favor of GitHub’s own Copilot CLI, for what looks like the identical underlying reason: nobody had modeled what happens once an agent, rather than a person, is the one deciding how many tokens to spend.
This isn’t really a story about one vendor mispricing its product. It’s a story about an entire category changing its unit economics mid-flight. GitHub moved Copilot off flat per-seat pricing and onto token-metered “AI Credits” in June 2026, because a one-line chat question and an hour-long autonomous coding session had stopped being remotely comparable in what they actually cost to serve. Cursor made a similar move the year before. Somewhere on Reddit, one developer described their monthly bill jumping by something like ten to twenty-seven times once metered billing kicked in, not because their habits changed, but because the same habits suddenly cost differently.
If you’re the one setting policy for a team, here’s the uncomfortable part: flat-fee subscriptions were hiding the real cost curve, which made it easy to justify sprinkling AI into everything on the assumption it was basically free at the margin. A February 2026 survey of 500 finance leaders found that 79% of enterprises had experienced an AI cost overrun in the prior twelve months, and the counterintuitive detail in that data is that overruns got worse, not better, as organizations’ cost-tracking practices matured. Mature FinOps teams aren’t actually worse at control. They’re just better at measuring, which means they’re the first ones to see the bill they’d already been running up in the dark. If nobody on your team can say what a specific agent run cost by tomorrow morning, that’s the gap worth closing before you expand usage, not after.
The skill price tag
The more interesting cost never shows up on an invoice at all. In June 2025, researchers at MIT Media Lab wired 54 people with EEG while they wrote essays over several months, some using ChatGPT, some using a search engine, some with no tools at all. The language here is deliberately careful, because the researchers themselves asked journalists not to describe the results as “brain rot” or “damage,” and the paper itself doesn’t use those words either. What they found instead was that the group using the LLM showed the weakest neural connectivity of the three, reported the lowest sense of ownership over what they’d written, and struggled afterward to accurately quote their own essays back to themselves. A smaller crossover group, who’d already spent months writing unaided before switching to AI help, didn’t show the same drop-off. They had something to fall back on.
That detail, that prior practice seems to protect you, turns up again in research Anthropic itself published this January. Researchers had developers learn a new asynchronous Python library called Trio, some with AI assistance and some without, then tested what they actually understood afterward. AI use measurably hurt conceptual understanding, code reading, and debugging ability, without buying most participants a real speed advantage. Only the people who fully delegated the work to the AI finished faster, and they paid for it in comprehension. It’s an unusual thing for a company that sells an AI coding agent to publish, which is exactly why it’s worth taking seriously.
There’s a pattern across both papers worth sitting with. The cost isn’t AI use in general. It’s AI use during the window before you’ve built a mental model of whatever you’re working on. A senior engineer running an agent against a service she’s operated for six years is drawing on schema she already has, and the tool can’t erode understanding that’s already load-bearing. A new hire pointed at that same service for the first time, with no schema yet, is in a completely different position, and whatever they don’t build now, they won’t have later. The tokens don’t refund the difference. For what it’s worth, the most useful finding in the Anthropic paper wasn’t the headline number. Researchers identified six distinct patterns in how people actually used the AI, and in the three where participants stayed cognitively engaged, asking the model to explain itself rather than just produce output, the learning outcomes held up even with AI in the loop.
METR’s dependency finding from the last section belongs in this pile too. Refusing to even attempt paid work without AI access isn’t a preference. It’s what skill erosion looks like from the inside, well before anyone gets around to measuring it.
The bill that shows up in the incident report instead
Security is where the dollar cost and the skill cost turn out to be the same cost wearing two hats. Veracode’s testing of over 100 language models across dozens of coding tasks in Java, Python, C#, and JavaScript found that 45% of the AI-generated samples introduced an OWASP Top 10 vulnerability, a pass rate that hasn’t meaningfully improved across testing cycles despite vendor claims to the contrary. Java came out worst, failing secure generation more than seventy percent of the time. Separate analysis from CodeRabbit put AI-authored pull requests at roughly 2.7 times the vulnerability density of human-written ones, and GitGuardian found 6.4% of repositories using GitHub Copilot leaking at least one secret, against a 4.6% baseline in repositories without it.
None of that should be surprising on its own. Models optimize for code that runs, not code that survives an attacker who’s read the same training data. What should worry an IT professional more is a pair of academic findings on Copilot that have held up since some of the earliest research on the tool. One of the first studies, by Pearce and colleagues, found that roughly 40% of Copilot’s suggested programs contained a vulnerability. A follow-up by Perry and colleagues went further and found that developers given AI assistance wrote measurably less secure code than developers working without it, and rated their own insecure solutions as secure more often. That’s the same false-confidence pattern MIT found in a totally different context, just wearing a security badge instead of an essay grade. The tool doesn’t only introduce the vulnerability. It quietly turns down your own alarm for noticing one.
That’s the plainest case for keeping a human genuinely in the loop rather than nominally in the loop. Not because AI-written code is always wrong, but because it’s wrong in ways that look right, to reviewers who’ve been trained by months of mostly-correct output to stop looking as hard.
So when do I actually reach for it
After all of that, here’s roughly where I’ve landed for my own team:
Reach for AI
Think twice
Boilerplate, scaffolding, first-draft tests
Code in a system you already know cold
A stack you’re deliberately exploring
A junior’s first unsupervised weeks on an unfamiliar service
Sandboxes, non-prod IaC, throwaway scripts
Anything touching auth, payments, or PII
Log summaries, docs, low-stakes ticket triage
Security-sensitive code with no review time budgeted
Translating between two things you already understand
A judgment call you’ll want to have made yourself, later
The shape underneath that table is closer to a flowchart than a fixed rule, so here’s the actual decision I run through before reaching for a coding agent:
Two questions do most of the work here: do you already have a mental model of what you’re touching, and what happens if the output is wrong and nobody catches it for a week. Everything else, like which model or which vendor, is implementation detail that will have changed again by the time you read this.
Treat it like any other total-cost-of-ownership call
Every cost in this post is a total-cost-of-ownership problem, and IT professionals already know how to run those. Nobody greenlights a new platform because the sticker price looked reasonable without asking what it costs to operate and support over the next three years. AI tooling deserves that same discipline, just split across two ledgers instead of one: what it costs in tokens, and what it costs in the understanding your team would have built anyway by doing the work themselves.
That pull request Cursor wrote in ninety seconds probably did save you time. Just budget for the chance that it didn’t, that it needs a harder look than you’re inclined to give it, and that the muscle you skipped today is the one you’ll need on the day the model is wrong and you’re the only person who’d have known.
What I’d actually do on Monday
Instrument both meters, because right now most organisations are flying blind on one and in denial about the other.
On spend: get per-team token attribution before you get a budget surprise, set caps at the gateway rather than in a policy document nobody reads, and stop assuming that a more capable model is a more expensive model’s replacement rather than its addition.
On cognition: pick one thing per quarter that you learn the slow way. Not everything — that’s martyrdom, not strategy. One thing you’ll need to own, that you deliberately struggle through without the assistant. Getting painfully stuck is not wasted time; it is how the debugging skill gets built, and the debugging skill is what makes you worth having in the loop at all.
The engineers who’ll be valuable in three years aren’t the ones who avoided these tools. They’re the ones who stayed able to tell when the tools were wrong.
If you’ve spent any time around technology in the last couple of years, you’ve heard the word “agent” thrown around a lot — usually with more excitement than precision. This post is an attempt to fix that. I’ll explain what an AI agent actually is, when you should (and shouldn’t) reach for one, how they work under the hood, the vocabulary you’ll keep bumping into, and the two things that decide whether an agent project succeeds in production: cost and safety.
AI agents!
I’ve written it to be readable if you’re brand new to AI, while still being concrete enough for an IT professional who has to make systems, design or architecture decisions.
First, what is an “agent”?
The idea is older than the current AI wave. In classic computer science, an intelligent agent is defined as “an entity that perceives its environment, takes actions autonomously to achieve goals, and may improve its performance through machine learning or by acquiring knowledge” (Wikipedia). That textbook framing — perceive, decide, act — still holds today.
The modern, cloud-vendor version says much the same thing in plainer words. AWS defines an AI agent as “a software program that can interact with its environment, collect data, and use that data to perform self-directed tasks that meet predetermined goals” (AWS). The key word is self-directed: you give it a goal, not a step-by-step script.
What’s new is the brain. In today’s agents, the control flow is “frequently driven by large language models” (Wikipedia). The LLM is what lets the agent understand a fuzzy instruction in plain English, decide what to do next, and adapt when things don’t go as planned.
The one distinction that clears up most confusion: workflows vs. agents
Here’s the single most useful mental model I’ve found, from Anthropic’s engineering team. They separate “agentic systems” into two categories:
Workflows are “systems where LLMs and tools are orchestrated through predefined code paths.”
Agents are “systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks” (Anthropic).
In a workflow, you wrote the flowchart and the LLM fills in the boxes. In an agent, the model decides the flowchart at runtime. That difference drives everything else — predictability, cost, and risk all shift when you hand the steering wheel to the model.
How an agent actually works
Strip away the hype and most LLM agents are, in Anthropic’s words, “just LLMs using tools based on environmental feedback in a loop” (Anthropic). That loop is the whole thing:
Walking through it the way AWS frames the process — determine the goal, acquire information, implement tasks, then evaluate outcomes and adjust (AWS):
Goal in. A person gives the agent a task in natural language. A good agent will pause to clarify an ambiguous request before charging ahead.
Reason and plan. The LLM decides what the next step should be.
Act via a tool. The agent does something in the real world — queries a database, calls an API, runs code, sends an email.
Observe. Critically, the agent must “gain ‘ground truth’ from the environment at each step (such as tool call results or code execution) to assess its progress” (Anthropic). This feedback is what separates an agent from a chatbot that just guesses.
Loop or stop. It repeats until the goal is met or a stopping condition trips.
The building block: an “augmented LLM”
The atom of every agent is what Anthropic calls the augmented LLM — a language model enhanced with three capabilities: retrieval, tools, and memory (Anthropic). Modern models can actively generate their own search queries, pick the right tool, and decide what information is worth keeping. AWS describes a similar architecture: a foundation model, a planning module, a memory module, tool integration, and a learning/reflection mechanism (AWS).
The vocabulary you’ll keep hearing
Tools. A tool is anything the agent can call to affect or observe the world — a search API, a calculator, a code interpreter, a database query. The quality of your tool descriptions matters more than people expect. Anthropic recommends “poka-yoke” design (a manufacturing term for mistake-proofing) — writing tool parameters and descriptions so clearly, with examples and edge cases, that it’s hard for the model to misuse them (Anthropic).
Skills. A newer concept worth knowing: Skills are “folders that include instructions, scripts, and resources that Claude can load when needed” (Anthropic). They work through progressive disclosure — the agent scans available skills, and “when one matches, it loads only the minimal information and files needed,” which keeps it fast while still having specialized expertise on tap. Skills are composable (they stack), portable (“build once, use across Claude apps, Claude Code, and API”), and efficient because they only load what’s needed, when it’s needed (Anthropic). Think of a skill as a reusable competency you can hand to an agent, rather than cramming everything into one giant prompt.
MCP (Model Context Protocol). As soon as you want an agent to talk to real systems, you hit an integration problem: every tool has its own API. MCP is “an open-source standard for connecting AI applications to external systems.” The docs use a nice analogy — “think of MCP like a USB-C port for AI applications,” a standardized way to plug an agent into data sources, tools, and workflows (MCP). For IT teams, the appeal is “build once and integrate everywhere” instead of writing a bespoke connector per tool.
Memory. Agents distinguish between short-term memory (the current task’s context) and longer-term memory (things worth keeping across sessions). It’s one of the augmented-LLM enhancements above, and one of AWS’s core architecture components (AWS).
Multi-agent systems. Instead of one agent doing everything, you can have several specialized agents collaborate. Both the classical taxonomy and AWS list multi-agent systems as a category (Wikipedia; AWS). Anthropic’s orchestrator-workers pattern is a concrete version: a central LLM breaks a task into subtasks, delegates them, and synthesizes the results (Anthropic).
A quick note on “types of agents”
If you read the academic material, you’ll see five classical classes: simple reflex, model-based reflex, goal-based, utility-based, and learning agents (Wikipedia). AWS extends the list with hierarchical and multi-agent systems (AWS). You don’t need to memorize these, but they’re a useful reminder that “agent” is a spectrum of autonomy, not a single thing.
When to use an agent — and when not to
This is the section I wish more people read first. The honest guidance from Anthropic is to resist building an agent until you actually need one: “find the simplest solution possible, and only increasing complexity when needed.” Agentic systems “typically increase latency and costs while improving task performance,” so it’s a genuine trade-off, not a free upgrade. In fact, for many applications, “optimizing single LLM calls with retrieval and in-context examples is usually enough” (Anthropic).
A practical way to decide:
Use a workflow when the task is well-defined and you value predictability. If you can draw the flowchart yourself, hard-code it.
Use an agent when flexibility and model-driven decision-making are essential — the path can’t be known in advance, and the number of possible steps is large (Anthropic).
Common patterns that sit between a single prompt and a full agent are worth knowing, because they often solve the problem more cheaply: prompt chaining (sequential steps with checkpoints), routing (classify an input and send it to a specialist), parallelization (split work up, or run it several times and vote), orchestrator-workers, and evaluator-optimizer (one model generates, another critiques, in a loop) (Anthropic). My advice: try to solve your problem with one of these before you reach for full autonomy.
Keeping costs under control
Because agents run the LLM in a loop and call tools repeatedly, cost and latency are design concerns from day one, not afterthoughts. The trade-off is baked in — more autonomy means more model calls (Anthropic). Grounded in the sources above, here’s how to keep the bill sane:
Don’t build an agent you don’t need. The cheapest agent is the one you avoided by using a single well-designed LLM call, a retrieval step, or a fixed workflow instead (Anthropic).
Set stopping conditions. Anthropic explicitly recommends limits like a maximum number of iterations so an agent can’t loop forever and quietly run up cost (Anthropic). This is the single most important cost guardrail.
Load only what’s needed. This is exactly why progressive disclosure in Skills matters — it “only loads what’s needed, when it’s needed” instead of stuffing every instruction into context on every call (Anthropic). Less context per call means lower token cost.
Match the pattern to the task. A routing step that sends easy requests down a cheaper path, and only hard ones to the expensive full-agent path, can cut cost dramatically (Anthropic).
A caveat, so I don’t overstate things: specific pricing, token rates, and model-tier costs change constantly and depend on your provider. Check the current pricing page for whatever model you use — I’m deliberately not quoting numbers the sources didn’t give me.
Security, guardrails, and human oversight
Handing an autonomous system access to your tools and data is a real risk surface, and the sources are consistent about how to manage it: test in isolation, constrain what the agent can do, and keep a human in the loop.
Test in a sandbox. Anthropic calls “extensive testing in sandboxed environments” essential before you let an agent loose (Anthropic). Give it a safe playground before it touches production.
Design tools to prevent mistakes. The poka-yoke principle again — make dangerous or ambiguous tool use structurally hard, not just discouraged (Anthropic).
Use platform guardrails. Managed services provide built-in safety layers; AWS points to “Amazon Bedrock Guardrails” as an example of built-in security for agents (AWS).
Keep humans in the loop. Agents should be able to pause for human feedback “at checkpoints or when blocked” (Anthropic), and human review remains a core safeguard against biased or inaccurate output (AWS). For anything irreversible — sending money, deleting data, emailing customers — a human approval step is not optional.
If you take one security principle away: an agent should never have a capability you wouldn’t hand to a brand-new employee on their first day without supervision.
Bringing it together
Here’s the whole picture in one view:
An AI agent takes a goal in plain language and pursues it self-directedly.
Under the hood it’s an augmented LLM (model + tools + memory + retrieval) running a perceive → reason → act → observe loop.
Skills package reusable expertise; MCP standardizes how the agent connects to the outside world; multi-agent setups split work across specialists.
Reach for an agent only when a simpler workflow won’t do — autonomy costs latency and money.
Stopping conditions, sandboxing, and human oversight are what make the whole thing safe to run.
The technology is genuinely useful, but the teams who succeed with it are the ones who stay skeptical: they start simple, add autonomy only where it earns its keep, and never let an agent act unsupervised on anything that’s hard to undo.
If you’ve started using Claude Code, you’ve probably hit the same small annoyance more than once: you explain your project, Claude helps, and then in the next session you’re explaining the very same things all over again. Which framework you use. Where the tests live. How you like your commits phrased. It’s like onboarding a new teammate every single morning.
Understanding claude.md
The CLAUDE.md file fixes exactly that. It’s one of the simplest features to set up and one of the highest-leverage, especially once you understand what it’s really doing behind the scenes. This guide walks a first-time user through what it is, how it works, and how to use it well.
What is a CLAUDE.md file?
A CLAUDE.md is a configuration file that gives Claude project-specific context. The key detail — and the reason it matters so much — is that Claude automatically incorporates it into every conversation. You don’t have to attach it, paste it, or remind Claude it exists.
Mechanically, the file becomes part of Claude’s system prompt. In plain terms: every conversation starts with that context already loaded. So instead of re-describing your project each time, you write it down once and Claude simply knows it going forward.
Think of it as a persistent briefing note for your project. Whatever you’d find yourself repeating to a new collaborator is a good candidate for the file.
Where the file lives
You can place a CLAUDE.md in a few different spots depending on how widely you want it to apply:
Your repository root — the most common choice. Put it here so it applies to the project and can be shared with your team.
A parent directory — useful for monorepo setups where you want shared context sitting above several projects.
Your home folder — this applies universally, across all your projects, for instructions you always want in play.
You’re not limited to one. A home-folder file for your personal preferences plus a repo-root file for the project is a natural combination.
There isn’t just one CLAUDE.md — there’s a small hierarchy, each level with a different scope. Here they are, from broadest to most specific:
As a beginner, you’ll care most about the project file (./CLAUDE.md at your repo root) and maybe your personal user file (~/.claude/CLAUDE.md). The managed-policy level is something an IT/DevOps team sets up centrally.
How the levels combine
Here’s the part that trips people up: these files don’t override each other — they’re all concatenated together into context. Claude also walks up the directory tree from wherever you launched it, picking up a CLAUDE.md in each parent folder along the way. Content is ordered from the filesystem root down to your working directory, so the instructions closest to where you’re working are read last.
One practical tip from the docs: to confirm which files actually loaded in a session, run /context and look under Memory files.
Getting started with /init
The easiest way to create your first one is to run the /init command inside Claude Code. It analyzes your codebase and generates a starter CLAUDE.md for you.
One honest caveat, straight from the source: /init “captures obvious patterns but may miss nuances specific to your workflow.” So treat what it produces as a first draft, not a finished document. Read it over, correct anything that’s off, and add the details it couldn’t have inferred. The generated file is a starting line, not a finish line.
How to structure your CLAUDE.md
You don’t need every section, but these are the kinds of things worth documenting:
A project summary and architecture overview — what the project is and how it’s put together.
Directory structure — the key folders and what lives where.
Coding standards and conventions — how you want code written.
Common commands, with examples — the build, run, and lint commands you use often.
Testing requirements and workflows — how tests are run and what’s expected.
Tool integration documentation — the tools your project relies on.
Development environment setup — what someone needs to get running.
The reference material includes a worked example built around a FastAPI project, showing the project structure, standards, and common commands all in one file, along with an example of a custom performance-optimization command. If you work in a different stack, the same shape applies — just fill it with your own project’s details.
Best practices worth adopting early
A few principles will keep your file useful rather than bloated:
Keep it concise and human-readable. This isn’t a formality. Because the file loads into context every single time, its length has a real cost. Conciseness is part of using Claude well, not just tidiness.
Start simple and expand based on friction. Don’t try to document everything up front. Add to the file when you notice yourself repeating an instruction or running into a recurring rough edge.
Split it up if it gets big. If the file grows unwieldy, one option is to break the information into separate markdown files and reference them from inside CLAUDE.md.
Document what your team actually does — the real workflows you follow, not the idealized version you wish you followed.
Never put secrets in it. Keep API keys, credentials, and database connection strings out. This matters especially because you’ll often commit the file to version control.
When should you add something to it?
The docs give a nice, concrete rule of thumb. Add to your CLAUDE.md when:
Claude makes the same mistake a second time.
A code review catches something Claude should have known about this codebase.
You type the same correction into chat that you typed last session.
A new teammate would need that same context to be productive.
Keep it to facts Claude should hold in every session: build commands, conventions, project layout, and “always do X” rules.
Two handy shortcuts: # and /clear
As you work, you can use the # key to quickly add an instruction you find yourself repeating. Over time, these additions accumulate into your CLAUDE.md — a low-effort way to grow the file organically instead of sitting down to write it all at once.
The /clear command is its companion. It resets the context window between distinct tasks while preserving your CLAUDE.md. So you can wipe the slate clean between unrelated jobs without losing the project context you’ve carefully set up.
Sharing with your team
Because a CLAUDE.md at the repo root travels with the project, you can commit it to version control so your team benefits from the same shared context. Everyone’s Claude starts from the same briefing. Just remember the earlier warning — since it’s now in your git history, keep sensitive information out of it.
There’s also room to grow beyond the file itself: you can create custom slash commands as markdown files in a .claude/commands/ directory, which is a natural next step once your CLAUDE.md is settled.
Start simple, expand deliberately
If you take one thing away, let it be this: begin small. Run /init, review what it produces, and let the file grow as your real workflow reveals what’s worth writing down. A short, accurate CLAUDE.md beats a sprawling one that nobody maintains — and it costs you less context on every conversation, too.
For a first-time Claude user, this single file is probably the fastest way to make Claude feel like it genuinely understands your project rather than projects in general. Set it up once, tend it occasionally, and you stop paying the “explain it all again” tax for good.
Picture a platform team running a few hundred Lambda functions across a handful of accounts. Every function has several published versions kept around for safe rollbacks. One morning a deploy fails with a quota error, and the culprit isn’t the new function at all. It’s the 75 GB account-level limit on function and layer code storage, quietly filled up by copies of packages you thought lived in your own S3 bucket. If you’ve ever had to explain to a security reviewer why your deployment artifacts also sit in an internal bucket you can’t see, encrypt, or tag, this one is for you.
S3 buckets for Lambda code!
AWS has added a way to point Lambda at your own S3 bucket and have it read your code from there directly, with no hidden second copy.
What actually changed
There’s a new function configuration setting called S3ObjectStorageMode. It has two values.
The default is COPY, which behaves exactly like Lambda always has. If you don’t set the field at all, you get COPY, so nothing about your existing functions changes on its own.
The new value is REFERENCE. Set it when you create or update a function, and Lambda stops copying your .zip package into its internal storage. Instead it keeps a reference to your S3 object and reads the code from your bucket when it needs it. Your object becomes the one canonical artifact.
The feature works in all AWS standard regions where Lambda is available, and there’s no extra charge for it beyond the normal S3 storage and request costs you’d pay anyway.
How it works, old way and new
Under the old model (now COPY), the flow looks like this: you upload a .zip to your bucket, call CreateFunction or UpdateFunctionCode with the bucket name and object key, and Lambda pulls that artifact into a service-managed bucket. It builds the optimized, runnable version of your function from that internal copy. The catch is that this copy counts against your 75 GB account quota, and you have no say over how it’s stored.
With REFERENCE, that copy step disappears. Lambda records where your object lives and reads it directly. Two things fall out of that. First, your package no longer counts toward the 75 GB limit, because there’s no Lambda-side copy to count. Second, creating and updating functions gets faster, since Lambda skips the copy-into-internal-bucket step. AWS describes this as a faster time to first invoke for new functions and after updates.
Architecture
The diagram below contrasts the two modes and sketches the multi-account pattern that, in my experience, is the real reason to adopt this.
Two modes for S3 buckets of Lambda code.
In a COPY deployment, the artifact exists twice: once in your bucket, once inside Lambda. In REFERENCE mode there’s a single object, and your function holds a pointer to it. Extend that to an organization and the shape gets interesting: put every artifact in one bucket in a central “artifact” or shared-services account, then grant s3:GetObject to each workload account’s Lambda execution role through the bucket policy. Now one bucket is the source of truth for what’s deployed everywhere, with one place to enforce encryption, versioning, and retention.
When to reach for it
CI/CD and artifact management. Your pipeline uploads a package once, and the function references that same object. One set of lifecycle rules and access controls covers everything, and a rollback becomes “point the function at the previous S3 object version.”
Multi-account, multi-team setups. Centralize artifacts in one account, hand out cross-account s3:GetObject via bucket policies, and keep a single inventory of what code runs where.
Disaster recovery. Because you own the bucket, you can turn on Cross-Region Replication (CRR) or Same-Region Replication (SRR), and pair it with S3 Versioning and Object Lock to keep a tamper-resistant archive that survives an accidental delete or a corrupted deploy. See the S3 replication docs and S3 lifecycle examples.
Quota and compliance pressure. If you’re brushing up against 75 GB, or you need your own encryption, access logging, Object Lock, or compliance tags on the artifact, this gives you that control.
When to leave it alone
For plenty of workloads, COPY is fine and simpler. If you aren’t near the quota, don’t need custom encryption or tagging on the artifact, and have no DR requirement on your deployment packages, there’s little reason to change anything. The default exists for a reason.
Security notes
The trade in REFERENCE mode is control for responsibility. You now own the bucket’s posture: its encryption, access policies, lifecycle transitions, and audit trail are yours to configure and to get right.
For cross-account access, you grant s3:GetObject to the calling function’s execution role in the bucket policy.
Versioning plus Object Lock is the combination worth setting up early if you care about a durable, tamper-proof code archive.
No additional charge for the feature. You pay the standard S3 storage and request costs for the object, which you were largely paying already if your artifacts lived in S3.
Limitations and open questions
It doesn’t state whether REFERENCE needs KMS decrypt permissions beyond s3:GetObject when you use customer-managed encryption (It should), exactly how a referenced object version is pinned for rollback, or what happens to a live function if the referenced object is later deleted or modified. Any latency effect from reading code directly at runtime isn’t discussed either. Confirm these against the Lambda console and the developer guide before you roll it out widely.
Bottom line
This is a small setting with an outsized effect for teams operating at scale. If the 75 GB quota or “we can’t audit that copy” has ever slowed you down, S3ObjectStorageMode: REFERENCE is worth a look. Start on a non-critical function, get your bucket policy and versioning right, and expand from there. Original announcement on the AWS Compute Blog.
Picture this: you’re building a browser-based notebook where data analysts paste in Python, load a 3 GB dataframe, generate a few charts, then wander off to a meeting. Ninety minutes later they come back and expect their kernel, their variables, and their half-finished plot to still be sitting there.
AWS Lambda MicroVMs
Now try to build that on what AWS gave you before June 2026. Lambda? Fifteen-minute execution ceiling, no persistent process between invocations, no way to hold that dataframe in memory across the analyst’s coffee break. ECS or EC2? Sure, but now you’re running per-user containers or VMs, paying for idle capacity, and writing your own scheduler to reap dead sessions. Fargate gets you closer, but you still own the isolation story when the code being run was generated by an LLM you can’t fully trust.
This is the gap Lambda MicroVMs is aimed at.
What actually launched
AWS Lambda MicroVMs is a new compute primitive that exposes the Firecracker virtualization layer (the same one that has always run underneath Lambda) as something you can address directly. You get per-instance hardware isolation, snapshot-based startup, and — this is the part that matters — the ability to keep a single execution environment alive for an entire working session rather than a single request.
Alongside it, AWS shipped a companion resource called the Lambda Network Connector (LNC), which is how you attach a MicroVM to a private VPC when you need it to reach a database or an internal API.
How it actually works
Two resource types, and once you understand these the rest falls into place:
MicroVM image: a versioned artifact you build from a Dockerfile. When you create one, the service runs your Dockerfile, boots your application inside a MicroVM, and takes a Firecracker snapshot of the memory and disk state. Think of it as a “warm” image — dependencies already imported, JIT already warmed, whatever init your app does already done.
MicroVM: an instance launched from that image. Because it’s restored from a snapshot rather than cold-booted, it comes up close to instantly.
Each MicroVM gets its own HTTPS endpoint, and that endpoint speaks to individual ports on the guest — plain HTTPS, WebSockets, and gRPC all work, so you connect to it the same way you’d connect to any container you were running yourself.
On sizing: the default baseline is 2 GB of memory and 1 vCPU. You can configure that up to 8 GB and 4 vCPUs at launch, with vCPU pinned to memory at a 2:1 ratio (memory in GB is double the vCPU count). From whatever baseline you pick, a MicroVM will auto-scale vertically up to 4x during peak demand. Horizontally, the service claims you can launch several hundred MicroVMs inside a minute.
Lambda MicroVMs Instance Sizes
Sessions can run from a few minutes up to eight hours. Egress to the public internet works out of the box; VPC access requires the LNC.
Architecture
See the diagram below. The pattern is the per-session model: user requests come into your control plane, the control plane looks up whether that user already has a live MicroVM, and either routes traffic to the existing HTTPS endpoint or launches a new instance from a MicroVM image. When the user goes idle, you decide the lifecycle — keep it warm, snapshot it, or let it go.
AWS Lambda MicroVMs Flow
The interesting design question isn’t really “how do I launch a MicroVM” — it’s “who owns the session-to-endpoint mapping, and what’s my policy when the user disconnects?” You will end up building a small state machine. Plan for it.
When this is the right tool
Browser-based IDEs, notebooks, and the current wave of vibe-coding platforms — anywhere users bring their own code and expect their environment to feel persistent.
Analytics platforms running user- or LLM-generated queries where the working set is large and the session is long.
AI coding assistants that iterate on generated code and want to hold context between iterations, including RL-style loops that spin environments up and down to compare execution paths.
Security and vulnerability scanning where you need real isolation between scans and sometimes elevated OS privileges inside the guest.
CI/CD build and test runners where each job wants a clean, isolated box that starts fast.
When it probably isn’t
The launch material doesn’t call out anti-patterns directly, so treat this as my read rather than AWS’s guidance:
If your workload is stateless per-invocation and short, regular Lambda is still simpler and cheaper reasoning-wise.
If you need more than 8 GB of memory or 4 vCPUs per instance, you’re outside the MicroVM baseline envelope and you should look at Fargate or EC2.
If your sessions genuinely need to outlive eight hours without a snapshot/resume, this isn’t your primitive either.
Security notes worth reading twice
The isolation story is genuinely strong — hardware-level, one guest per user or job, which is the whole point of using Firecracker as a primitive rather than just a container runtime. That’s what makes it defensible as a sandbox for untrusted or model-generated code.
Two things to design around, though.
First, outbound internet access is on by default. If you’re running untrusted code, you almost certainly want egress controls, and the launch material doesn’t spell out how granular those are — treat this as a question to answer before you go to production.
Second, private VPC connectivity is not a checkbox; it requires configuring an LNC. Factor that into your networking design early, not late.
Pricing
Lambda MicroVMs are priced per instance-second. Check the Lambda pricing page (MicroVMs tab) before you commit to a design, especially for long-lived sessions — an eight-hour idle MicroVM has very different economics from a 200 ms function invocation.
Horizontal scale advertised as “several hundred per minute” during spikes — quantify this against your own burst profile before betting on it.
Sessions in the “few minutes to 8 hours” range; the exact hard upper bound isn’t explicitly stated.
VPC access is opt-in via LNC.
Wrapping up
The most useful way to think about Lambda MicroVMs is that AWS unbundled Firecracker from Lambda’s request/response model and let you address it directly, while keeping the parts of Lambda that were actually pleasant — no capacity planning, no patching, no scheduler to write. If you’ve been building per-user sandboxes on top of ECS or bare EC2 and feeling like you were re-implementing Firecracker badly, this is the primitive to evaluate.
Start with the product page and the developer guide, and if snapshot-based startup is new to you, the older SnapStart post is worth a read for the mental model. Networking details live in the LNC docs, and quotas are on the service limits page. And refer to this AWS Blog which walk you through AWS MicroVms.