Category Archives: Cloud Services

Amazon CloudWatch Managed Prometheus Collectors: Retiring the Self-Managed OTel Collector

Ask anyone who has wired Prometheus metrics into CloudWatch what the real bottleneck was, and it was never Prometheus. It was the collector. You’d stand up an OpenTelemetry Collector as a Deployment or a DaemonSet, hand-tune its scrape config, guess at memory limits for cardinality you hadn’t measured yet, and then spend the next year treating it like one more piece of cluster infrastructure that needed patching, right-sizing, and on-call attention of its own.

On July 31, AWS announced managed Prometheus collectors for Amazon CloudWatch, a fully managed, agentless scraper for Amazon EKS, Amazon EC2, Amazon ECS, Amazon MSK, and Amazon OpenSearch Service. You supply a scrape configuration and a way to reach your resources, and CloudWatch takes it from there: provisioning the collector, scaling it, and keeping it running. What it scrapes arrives in OpenTelemetry format, queryable with PromQL right next to your AWS vended metrics.

That’s the announcement. The more useful question, if you’ve spent any time in AWS’s observability stack already, is what’s actually new here versus what’s just wearing a new badge.

The new part is the destination, not the collector

This isn’t new collector technology. AWS shipped this same agentless scraper (automatic target discovery, no in-cluster agent, multi-AZ, metrics that never leave your VPC) as the Amazon Managed Service for Prometheus collector back in November 2023, initially scoped to EKS and writing into an AMP workspace. That lineage is visible directly in the API today: you still create one of these with the CreateScraper operation, under the aws amp CLI namespace, not some new CloudWatch-specific command. What’s new in this announcement is the destination and the source list. Alongside an AMP workspace, the same managed collector can now write into a CloudWatch dataset, and the supported sources have grown from EKS-only to include EC2, ECS, MSK, and OpenSearch.

Here’s an EKS scraper creation, adapted from AWS’s setup docs:

aws amp create-scraper \
  --alias "eks-metrics-scraper" \
  --source eksConfiguration="{clusterArn='arn:aws:eks:us-west-2:111122223333:cluster/prod-cluster', \
    securityGroupIds=['sg-0123456789abcdef0'], \
    subnetIds=['subnet-0aaa111','subnet-0bbb222']}" \
  --scrape-configuration configurationBlob=$(cat eks-scrape-config.yaml | base64 -w 0) \
  --destination cloudWatchConfiguration="{datasetArn='arn:aws:cloudwatch:us-west-2:111122223333:dataset/default'}"

Swap that final --destination flag for an ampConfiguration with a workspace ARN and you’re back to the exact collector AWS shipped in 2023. If you’re already running AMP collectors for Grafana dashboards, that’s the useful takeaway: this is an incremental destination option on infrastructure you may already trust, not a new product to evaluate from a cold start.

It also builds on something CloudWatch shipped only six weeks earlier: native OTLP ingestion with PromQL querying, launched June 9. That release is what made CloudWatch a legitimate destination for Prometheus-shaped metrics at all, with per-GB pricing and curated Container Insights dashboards for EKS. Managed Prometheus collectors are the agentless front door to that same pipeline, extended past what Container Insights auto-instruments.

How a scrape actually reaches CloudWatch

The mechanics will feel familiar if you’ve used any managed AWS scraper before. You hand the collector a set of subnets, and it creates an Elastic Network Interface in each one. It scrapes your targets over those ENIs using OTLP, then delivers the result to your CloudWatch dataset through a VPC endpoint. Nothing crosses the public internet.

What isn’t obvious from the diagram is that target discovery differs meaningfully by source, and that’s what determines how much of an existing scrape config actually survives the move:

EKS uses Kubernetes service discovery against the cluster API, plus control-plane metrics (kube-scheduler and kube-controller-manager, exposed starting at Kubernetes 1.28).
ECS uses DNS-based discovery through AWS Cloud Map, so tasks register once and get picked up without a static IP list.
EC2 is static_configs against private IPs, no different from pointing a self-managed setup at a fixed target list.
MSK uses DNS-based discovery against the cluster’s own bootstrap DNS name, which resolves to every broker and survives broker replacement without reconfiguration.

MSK is worth a closer look because it needs a prerequisite step: enabling Open Monitoring on the cluster, which exposes a JMX Exporter on port 11001 and a Node Exporter on port 11002. A scrape config covering both looks like this:

global:
  scrape_interval: 60s
  external_labels:
    cluster_name: my-msk-cluster

scrape_configs:
  - job_name: 'msk-jmx'
    dns_sd_configs:
      - names: ['my-cluster.abc123.c4.kafka.us-west-2.amazonaws.com']
        type: A
        port: 11001
    relabel_configs:
      - source_labels: [__address__]
        target_label: instance

  - job_name: 'msk-node'
    dns_sd_configs:
      - names: ['my-cluster.abc123.c4.kafka.us-west-2.amazonaws.com']
        type: A
        port: 11002
    relabel_configs:
      - source_labels: [__address__]
        target_label: instance

That gets you broker-level Kafka metrics (topic throughput, consumer lag, under-replicated partitions) from the JMX exporter, and host-level CPU, memory, and disk metrics from Node Exporter, both queryable with PromQL once they land. Two things to check before you plan around this: MSK Serverless and MSK Express aren’t supported, and public access combined with KRaft metadata mode is excluded too.

The scrape config is Prometheus-compatible, not Prometheus

This is worth reading closely before you migrate anything, because “Prometheus-compatible YAML” undersells how much is actually missing. The supported configuration surface covers global settings, scrape_configs with static_configs and dns_sd_configs, and relabeling. What isn’t there matters more than what is:

Minimum scrape interval is 30 seconds. An SLO built on 10 or 15-second scrapes doesn’t fit here.
No file_sd_configs, and no Consul, Eureka, or other external service discovery. If your target list comes from anywhere other than Kubernetes, Cloud Map, a static list, or MSK’s broker DNS, this collector can’t reach it.
remote_write and remote_read don’t apply, because delivering to CloudWatch is the whole job. A fan-out to a second backend doesn’t carry over.
Configuration blobs cap out at 256 KB, base64-encoded.

None of this is really a flaw. It’s the standard trade-off of a managed service. AWS narrowed the surface to what it can operate reliably at scale, and the narrowing happens to remove exactly the parts of Prometheus configuration that are hardest to run safely inside someone else’s control plane. But “copy your existing prometheus.yml over” is rarely literally true. Diff your current scrape configs against this list before you commit to a cutover date, not after.

What ships automatically, and what doesn’t

EKS and MSK both get a curated dashboard the moment metrics start flowing — EKS OTel and MSK OTel in the CloudWatch console — plus attribute enrichment on every metric: account, region, an inferred unit, and for EKS specifically the cluster name and ARN. That enrichment is what makes PromQL filtering pleasant instead of an exercise in label archaeology. The EC2/ECS path doesn’t come with an automatic dashboard, which is a reasonable trade given how varied EC2/ECS metric shapes are in practice, but it’s worth knowing before cutover rather than after.

Operationally, a managed collector vends its own logs to CloudWatch Logs covering target discovery, scrape successes and failures, and configuration errors, and you manage the scraper itself with aws amp list-scrapers, describe-scraper, and delete-scraper. Deleting one tears down its ENIs, and that cleanup takes a few minutes, so don’t script an immediate re-create into the same subnet.

Cross-account guidance has shifted too. Instead of the role-chaining pattern AMP has historically used for cross-account scrapers, AWS’s current recommendation for both EKS and MSK is CloudWatch metric centralization: replicate the metrics to a monitoring account rather than granting the scraper cross-account reach in the first place.

What this actually costs

Pricing has two components that scale independently, and it’s worth keeping them separate. The collector itself is billed hourly, similar in shape to an ENI or a NAT gateway, though the specific per-hour rate isn’t broken out yet on AWS’s public pricing page as of this writing. What is confirmed is that everything the collector delivers rides on standard CloudWatch OpenTelemetry ingestion pricing: currently $0.50 per GB for general OTel metric publishing, a flat rate that bundles in fifteen months of storage with no separate per-metric or per-API charge. A typical data point with ten to fifteen attributes runs 300 to 600 bytes, for a sense of scale. Console PromQL queries (Query Studio, dashboards) are free; programmatic PromQL queries cost $0.01 per million samples scanned.

That per-GB model is a real shift in how you’d think about cost here. Classic CloudWatch metrics billed per unique metric-dimension combination, so cardinality was the thing to fear. Under OTel ingestion, cardinality by itself is free; what costs money is the serialized size of each data point, which grows with the number and length of the labels attached to it. Fifteen short labels on a metric barely move the needle regardless of series count. A handful of long, high-cardinality values move it fast: unbounded request IDs, verbose ARNs repeated on every point. If you’re porting scrape configs over from a self-managed setup, that’s the moment to prune labels you’ve been carrying out of habit rather than actual query need.

One more line item: scraping itself can rack up VPC data transfer charges between the collector and your targets, separate from CloudWatch ingestion. AWS’s own suggestion is to enable gzip on your /metrics endpoints, which cuts transfer cost without changing what actually gets ingested.

Where I’d actually reach for this

If your team is already CloudWatch-centric and has been putting off a real Prometheus setup mainly because nobody wanted to own collector infrastructure, this closes that gap cleanly. For EKS and MSK metrics specifically, it’s a solid default: alarms and dashboards in the same place as everything else, without standing up Grafana and an AMP workspace just to get there.

I’d still keep a self-managed collector, or the AMP-workspace version of this same scraper, in a handful of specific situations: sub-30-second scrape requirements, target discovery through Consul, Eureka, or file-based service discovery, or a remote_write fan-out to more than one backend because you’re mid-migration or running a deliberately multi-vendor setup on purpose. None of those are rare for a mature platform team, so this doesn’t replace the self-managed path outright. It’s a genuinely good default for the common case, which happens to be most of it.

Availability is broad but not universal — it’s live everywhere CloudWatch’s OTLP endpoint already exists, minus Asia Pacific (New Zealand) for now. Worth checking before you write the migration runbook, not after you’ve scheduled it.

Self-Managed S3 Buckets for Lambda Code: You Finally Own the Artifact

Leave a reply

Picture a platform team running a few hundred Lambda functions across a handful of accounts. Every function has several published versions kept around for safe rollbacks. One morning a deploy fails with a quota error, and the culprit isn’t the new function at all. It’s the 75 GB account-level limit on function and layer code storage, quietly filled up by copies of packages you thought lived in your own S3 bucket. If you’ve ever had to explain to a security reviewer why your deployment artifacts also sit in an internal bucket you can’t see, encrypt, or tag, this one is for you.

AWS has added a way to point Lambda at your own S3 bucket and have it read your code from there directly, with no hidden second copy.

What actually changed

There’s a new function configuration setting called S3ObjectStorageMode. It has two values.

The default is COPY, which behaves exactly like Lambda always has. If you don’t set the field at all, you get COPY, so nothing about your existing functions changes on its own.
The new value is REFERENCE. Set it when you create or update a function, and Lambda stops copying your .zip package into its internal storage. Instead it keeps a reference to your S3 object and reads the code from your bucket when it needs it. Your object becomes the one canonical artifact.

The feature works in all AWS standard regions where Lambda is available, and there’s no extra charge for it beyond the normal S3 storage and request costs you’d pay anyway.

How it works, old way and new

Under the old model (now COPY), the flow looks like this: you upload a .zip to your bucket, call CreateFunction or UpdateFunctionCode with the bucket name and object key, and Lambda pulls that artifact into a service-managed bucket. It builds the optimized, runnable version of your function from that internal copy. The catch is that this copy counts against your 75 GB account quota, and you have no say over how it’s stored.

With REFERENCE, that copy step disappears. Lambda records where your object lives and reads it directly. Two things fall out of that. First, your package no longer counts toward the 75 GB limit, because there’s no Lambda-side copy to count. Second, creating and updating functions gets faster, since Lambda skips the copy-into-internal-bucket step. AWS describes this as a faster time to first invoke for new functions and after updates.

Architecture

The diagram below contrasts the two modes and sketches the multi-account pattern that, in my experience, is the real reason to adopt this.

Two modes for S3 buckets of Lambda code.

In a COPY deployment, the artifact exists twice: once in your bucket, once inside Lambda. In REFERENCE mode there’s a single object, and your function holds a pointer to it. Extend that to an organization and the shape gets interesting: put every artifact in one bucket in a central “artifact” or shared-services account, then grant s3:GetObject to each workload account’s Lambda execution role through the bucket policy. Now one bucket is the source of truth for what’s deployed everywhere, with one place to enforce encryption, versioning, and retention.

When to reach for it

CI/CD and artifact management. Your pipeline uploads a package once, and the function references that same object. One set of lifecycle rules and access controls covers everything, and a rollback becomes “point the function at the previous S3 object version.”
Multi-account, multi-team setups. Centralize artifacts in one account, hand out cross-account s3:GetObject via bucket policies, and keep a single inventory of what code runs where.
Disaster recovery. Because you own the bucket, you can turn on Cross-Region Replication (CRR) or Same-Region Replication (SRR), and pair it with S3 Versioning and Object Lock to keep a tamper-resistant archive that survives an accidental delete or a corrupted deploy. See the S3 replication docs and S3 lifecycle examples.
Quota and compliance pressure. If you’re brushing up against 75 GB, or you need your own encryption, access logging, Object Lock, or compliance tags on the artifact, this gives you that control.

When to leave it alone

For plenty of workloads, COPY is fine and simpler. If you aren’t near the quota, don’t need custom encryption or tagging on the artifact, and have no DR requirement on your deployment packages, there’s little reason to change anything. The default exists for a reason.

Security notes

The trade in REFERENCE mode is control for responsibility. You now own the bucket’s posture: its encryption, access policies, lifecycle transitions, and audit trail are yours to configure and to get right.
For cross-account access, you grant s3:GetObject to the calling function’s execution role in the bucket policy.
Versioning plus Object Lock is the combination worth setting up early if you care about a durable, tamper-proof code archive.
The full IAM and bucket-policy details live in the Lambda developer guide.

Pricing

No additional charge for the feature. You pay the standard S3 storage and request costs for the object, which you were largely paying already if your artifacts lived in S3.

Limitations and open questions

It doesn’t state whether REFERENCE needs KMS decrypt permissions beyond s3:GetObject when you use customer-managed encryption (It should), exactly how a referenced object version is pinned for rollback, or what happens to a live function if the referenced object is later deleted or modified. Any latency effect from reading code directly at runtime isn’t discussed either. Confirm these against the Lambda console and the developer guide before you roll it out widely.

Bottom line

This is a small setting with an outsized effect for teams operating at scale. If the 75 GB quota or “we can’t audit that copy” has ever slowed you down, S3ObjectStorageMode: REFERENCE is worth a look. Start on a non-critical function, get your bucket policy and versioning right, and expand from there. Original announcement on the AWS Compute Blog.

Lambda MicroVMs: When Functions Aren’t Enough and EC2 Is Too Much

Leave a reply

Picture this: you’re building a browser-based notebook where data analysts paste in Python, load a 3 GB dataframe, generate a few charts, then wander off to a meeting. Ninety minutes later they come back and expect their kernel, their variables, and their half-finished plot to still be sitting there.

Now try to build that on what AWS gave you before June 2026. Lambda? Fifteen-minute execution ceiling, no persistent process between invocations, no way to hold that dataframe in memory across the analyst’s coffee break. ECS or EC2? Sure, but now you’re running per-user containers or VMs, paying for idle capacity, and writing your own scheduler to reap dead sessions. Fargate gets you closer, but you still own the isolation story when the code being run was generated by an LLM you can’t fully trust.

This is the gap Lambda MicroVMs is aimed at.

What actually launched

AWS Lambda MicroVMs is a new compute primitive that exposes the Firecracker virtualization layer (the same one that has always run underneath Lambda) as something you can address directly. You get per-instance hardware isolation, snapshot-based startup, and — this is the part that matters — the ability to keep a single execution environment alive for an entire working session rather than a single request.

Alongside it, AWS shipped a companion resource called the Lambda Network Connector (LNC), which is how you attach a MicroVM to a private VPC when you need it to reach a database or an internal API.

How it actually works

Two resource types, and once you understand these the rest falls into place:

MicroVM image: a versioned artifact you build from a Dockerfile. When you create one, the service runs your Dockerfile, boots your application inside a MicroVM, and takes a Firecracker snapshot of the memory and disk state. Think of it as a “warm” image — dependencies already imported, JIT already warmed, whatever init your app does already done.
MicroVM: an instance launched from that image. Because it’s restored from a snapshot rather than cold-booted, it comes up close to instantly.

Each MicroVM gets its own HTTPS endpoint, and that endpoint speaks to individual ports on the guest — plain HTTPS, WebSockets, and gRPC all work, so you connect to it the same way you’d connect to any container you were running yourself.

On sizing: the default baseline is 2 GB of memory and 1 vCPU. You can configure that up to 8 GB and 4 vCPUs at launch, with vCPU pinned to memory at a 2:1 ratio (memory in GB is double the vCPU count). From whatever baseline you pick, a MicroVM will auto-scale vertically up to 4x during peak demand. Horizontally, the service claims you can launch several hundred MicroVMs inside a minute.

Sessions can run from a few minutes up to eight hours. Egress to the public internet works out of the box; VPC access requires the LNC.

Architecture

See the diagram below. The pattern is the per-session model: user requests come into your control plane, the control plane looks up whether that user already has a live MicroVM, and either routes traffic to the existing HTTPS endpoint or launches a new instance from a MicroVM image. When the user goes idle, you decide the lifecycle — keep it warm, snapshot it, or let it go.

The interesting design question isn’t really “how do I launch a MicroVM” — it’s “who owns the session-to-endpoint mapping, and what’s my policy when the user disconnects?” You will end up building a small state machine. Plan for it.

When this is the right tool

Browser-based IDEs, notebooks, and the current wave of vibe-coding platforms — anywhere users bring their own code and expect their environment to feel persistent.
Analytics platforms running user- or LLM-generated queries where the working set is large and the session is long.
AI coding assistants that iterate on generated code and want to hold context between iterations, including RL-style loops that spin environments up and down to compare execution paths.
Security and vulnerability scanning where you need real isolation between scans and sometimes elevated OS privileges inside the guest.
CI/CD build and test runners where each job wants a clean, isolated box that starts fast.

When it probably isn’t

The launch material doesn’t call out anti-patterns directly, so treat this as my read rather than AWS’s guidance:

If your workload is stateless per-invocation and short, regular Lambda is still simpler and cheaper reasoning-wise.
If you need more than 8 GB of memory or 4 vCPUs per instance, you’re outside the MicroVM baseline envelope and you should look at Fargate or EC2.
If your sessions genuinely need to outlive eight hours without a snapshot/resume, this isn’t your primitive either.

Security notes worth reading twice

The isolation story is genuinely strong — hardware-level, one guest per user or job, which is the whole point of using Firecracker as a primitive rather than just a container runtime. That’s what makes it defensible as a sandbox for untrusted or model-generated code.

Two things to design around, though.

First, outbound internet access is on by default. If you’re running untrusted code, you almost certainly want egress controls, and the launch material doesn’t spell out how granular those are — treat this as a question to answer before you go to production.
Second, private VPC connectivity is not a checkbox; it requires configuring an LNC. Factor that into your networking design early, not late.

Pricing

Lambda MicroVMs are priced per instance-second. Check the Lambda pricing page (MicroVMs tab) before you commit to a design, especially for long-lived sessions — an eight-hour idle MicroVM has very different economics from a 200 ms function invocation.

Limitations to keep on the whiteboard

2 GB / 1 vCPU baseline, 8 GB / 4 vCPU max, 2:1 memory-to-vCPU ratio.
Vertical auto-scale capped at 4x baseline.
Horizontal scale advertised as “several hundred per minute” during spikes — quantify this against your own burst profile before betting on it.
Sessions in the “few minutes to 8 hours” range; the exact hard upper bound isn’t explicitly stated.
VPC access is opt-in via LNC.

Wrapping up

The most useful way to think about Lambda MicroVMs is that AWS unbundled Firecracker from Lambda’s request/response model and let you address it directly, while keeping the parts of Lambda that were actually pleasant — no capacity planning, no patching, no scheduler to write. If you’ve been building per-user sandboxes on top of ECS or bare EC2 and feeling like you were re-implementing Firecracker badly, this is the primitive to evaluate.

Start with the product page and the developer guide, and if snapshot-based startup is new to you, the older SnapStart post is worth a read for the mental model. Networking details live in the LNC docs, and quotas are on the service limits page. And refer to this AWS Blog which walk you through AWS MicroVms.

Know about Amazon GuardDuty Investigation: AI-powered security analysis

Leave a reply

Security teams routinely burn hours triaging a single GuardDuty finding. The workflow is familiar: pull the finding, pivot into CloudTrail, cross-reference VPC flow logs, chase the principal across accounts, map the behavior to a known technique, and finally decide whether the alert is a real incident or noise. Multiply that across an organization producing dozens or hundreds of findings a day, and investigation backlog becomes the bottleneck — not detection.

Amazon GuardDuty is now addressing that bottleneck directly. The GuardDuty investigation is available in public preview and performs on-demand, AI-powered investigations of GuardDuty findings, returning a structured assessment that a human analyst would otherwise assemble by hand.

What changed

The investigation agent adds a new capability to Amazon GuardDuty: rather than only surfacing findings, GuardDuty can now investigate them for you. Each investigation produces a structured output that includes a risk level, a confidence score, a natural-language summary, investigation details, MITRE ATT&CK technique mappings, resource mappings, and prioritized recommended actions — including specific AWS CLI commands the analyst can execute.

The agent is accessible through the AWS Management Console, the AWS CLI, AWS APIs, AWS SDKs, and the AWS MCP server — meaning it slots into both console-driven workflows and agentic security tooling built on top of the Agent Toolkit for AWS.

How it works

An investigation is initiated by an caller with appropriate permissions through the GuardDuty console, by calling the CreateInvestigation API, or via the AWS CLI. The target can be:

a specific GuardDuty finding
a member account
an entire AWS organization

CLI and API callers can supply a free-form natural-language trigger prompt of up to 2,048 characters to steer the investigation — for example, focusing the agent on a particular hypothesis or scope.

Under the hood, the agent correlates findings and evidence and returns a structured assessment. Results are retrieved with GetInvestigation, and prior investigations can be enumerated with ListInvestigations.

Processing uses cross-Region inference via the Cross-Region Inference Service (CRIS). Investigation data remains stored in the Region where the investigation was created, but inference and summary generation may occur in another Region within the same geography, transmitted over Amazon’s encrypted network.

Please go through this very detailed AWS blog that walks you through the whole investigation process – Amazon GuardDuty investigation agent: on-demand AI-powered threat assessment.

Architecture

The accompanying diagram shows the request flow: an administrator triggers an investigation from the console, CLI, SDK, API, or an MCP-connected agent; GuardDuty accepts the request in the origin Region; CRIS performs inference in-geography; and the structured assessment is returned to the caller while the underlying investigation data stays resident in the origin Region.

When to use it

The investigation agent fits naturally when you want to:

Triage a single GuardDuty finding without a manual pivot chase
Assess security posture across an entire AWS organization
Reduce manual correlation of evidence spread across CloudTrail, VPC flow logs, and other GuardDuty data sources
Wire GuardDuty investigations into AI-assisted security workflows through the AWS MCP server
Produce on-demand, structured threat assessments consumable by downstream automation

When not to use it

The agent is not the right tool if:

GuardDuty is not enabled in the account or organization
You are operating in a Region that does not support the preview
You need a member account to view another member’s or the administrator’s investigations, which is not permitted

Security considerations

Architecturally, three points matter for a review.

First, cross-Region inference. Investigation data is stored only in the origin Region, but processing and summary results may traverse another Region in the same geography. If your data-residency posture is scoped to a single Region rather than a geography, this is worth an explicit review before enabling the feature in regulated workloads. Please refer geography and their inference regions here.

Second, transport. Data moves across Amazon’s internal, encrypted network — but the residency point above still stands independently of encryption in transit.

Third, authorization. Investigations honor the existing GuardDuty authorization model. Callers can investigate only accounts they are already authorized to access. The three IAM actions to allow on principals that will drive investigations are:

guardduty:CreateInvestigation
guardduty:GetInvestigation
guardduty:ListInvestigations

Scope these on the administrator account role that runs the SOC’s investigation workflow, not broadly.

Pricing

In Preview mode, GuardDuty Investigations are made available at free of cost. Confirm current pricing in the GuardDuty product page before enabling at scale once its GA.

Limitations

The feature is in public preview.
GuardDuty must already be enabled, the account must be in a supported Region, and only administrator accounts can create investigations in admin as well as member accounts.
Member accounts cannot access investigations belonging to peer members or to the administrator.
Cross-Region inference behavior described above also applies.
During preview, 10 investigations/account/day with total limit of 100 investigations/account.Failed investigations do not count toward these quotas.
This feature is available only in the following 10 commercial AWS Regions: US East (N. Virginia), US East (Ohio), US West (Oregon), Canada (Central), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Europe (Stockholm), and Asia Pacific (Tokyo).
The trigger prompt is capped at 2,048 characters when invoked through API/CLI.

Verify latest limitations against the GuardDuty investigation documentation before committing to a design.

Conclusion

The investigation agent shifts GuardDuty from a detection surface to a triage surface. For teams whose incident response cost is dominated by evidence correlation rather than detection, that is the interesting move. Preview status means the mechanics — Regions, quotas, latency, retention — need to be validated against your environment before it lands in a runbook, but the shape of the workflow is clear enough to start piloting against a defined scope, most sensibly a single administrator account or a bounded set of high-signal findings. Details and getting-started guidance are on the AWS Security Blog and in the GuardDuty documentation.

AWS CloudFormation IaC Generator!

Leave a reply

AWS has recently introduced a Console-to-Code functionality, allowing users to automatically generate Infrastructure as Code (IaC) templates based on actions performed in the EC2 console. Additionally, AWS offers the CloudFormation IaC Generator, a compact tool embedded in the AWS CloudFormation console. This tool facilitates the creation of CloudFormation templates for pre-existing resources within your account. In the following post, we will walk through this tool and step-by-step instructions on how to use it.

*Exploring AWS CloudFormation IaC Generator*

What is the IaC generator?

The AWS CloudFormation IaC generator lets you generate a template for AWS resources that are already provisioned in your account and not being managed by CloudFormation. It does makes sense to exclude resources managed by CloudFormation since you already have a template for them!

Firstly, initiate a scan of your account under the IaC generator console to provide the tool with a comprehensive list of resources not governed by CloudFormation. There are a few quota limits on these scans which can be referred here.

Upon completion of the scan, the tool displays a list of resources that are not under CloudFormation management. Begin creating the template by selecting these resources, and the tool will then generate the template in either YAML or JSON format.

The powerful IaC generator capabilities of CloudFormation can also be accessed via the AWS CLI, making it a compelling choice for streamlined and efficient automation.

How to use the IaC generator?

Login to Cloudformation console.
Click on IaC generator link on the left hand side menu.

If you are using it for the first time, you might want to hit the Start a new scan button for running the first resources scan of this tool. This scan will identify all the AWS resources in your account that are not managed by CloudFormation.
Click on Create template button.
On a create wizard, provide the template details.

We are choosing to create a template from scratch. If you want to add a resource in exsiting CloudFormation stack then choose ‘Upadte the template for an existing stack‘. Make an appropriate choice for deletion and update replace policies.
On a next screen, choose resources to be included in the IaC template.

If the resource you are looking for is not in the list, probably you need to initiate the new scan since your scan inventory might be the old one.
Click on Next button and choose related resources if any.

The tool gathers the list of dependent or related resources from the resource you chose in the last step. Since we selected S3 bucket with no external dependency like KMS key, etc. we are not having any related resource.
Click on Next button for the final Review screen of the wizard.

Review the selections and click Create template button.

The teamplate will be genrated. You will have option to choose YAML or JSON format. You can copy or download it as well. Or you can import it into a CloudFormation Stack by clicking Import to stack button.
There is also AWS CDK application command in last tab AWS CDK

Importing the AWS resources in to the CloudFormation stack

It is always advisable to manage AWS resources through IaC templates, as this approach enhance resource management and minimizes the cloud clutter. One common challenge in the IaC journey is incorporating manually created existing resources into IaC. The IaC generator proves valuable in this scenario by assisting in the creation of IaC templates for your resources, facilitating a seamless transition.

Using IaC generator, you can generate the IaC template of existing resources that are not managed by Cloudformation. Once the template is generated, it also offers an option to import it in to the stack. Select that option (the last screesnhot in above section) and it will kickstart Cloudfomation import stack wizard.

Review all the information on the console, and then proceed to initiate the creation of a new stack that will import the chosen resources. I won’t go into a detailed, step-by-step procedure here, as it follows the standard CloudFormation stack process.

The CloudFormation stack will be created and you can see the selected resource is now imported and managed by CloudFormation.

This is a convenient method to scan for resources not governed by CloudFormation and subsequently import them into a CloudFormation stack, thereby fostering the adoption of Infrastructure as Code practices within your account and organization.

Conclusion

The IaC generator from AWS is a commendable initiative aimed at assisting customers in achieving 100% compliance with Infrastructure as Code for their infrastructure. It provides a seamless experience, effortlessly identifying non-IaC resources and smoothly importing them into CloudFormation stacks. Furthermore, it expedites the adoption of IaC by automatically generating code templates for you.

Exploring CloudFormation Git Sync!

Leave a reply

In late Nov 2023, amazon announced the new CloudFormation Git sync feature. Let’s explore this new feature; how it works, how it impacts CD patterns of Infrastructure as Code (IaC), etc.

What is the CloudFormation Git sync?

Recently announced CloudFormation Git sync feature lets customers deploy and sync the CloudFormation IaC code directly from remote Git repositories. This will be a game changer in the future as this feature might empower customers to omit the Continuous Deployment tools like Jenkins, GitHub Actions, etc. altogether and hence their maintenance.

In summary, the customer is required to establish a GitHub Connection with AWS and subsequently generate a CloudFormation stack using the relevant IaC template repository information. Once the stack is set up, CloudFormation continually monitors the template files in the remote Git repository. Any new commits in the remote repo, trigger the automatic deployment of the corresponding changes to AWS.

Pre-requisite for CloudFormation Git sync

A Git repository with a valid CloudFormation IaC template
A GitHub Connector configured for the target AWS account
IAM role for Git Sync operations. It should have –
- Access IAM policy
- Trust policy

IAM policy

For tighter security control, one can scope down the IAM policy for certain CloudFormation Stacks using a resource block in the SyncToCloudFormation statement.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "SyncToCloudFormation",
            "Effect": "Allow",
            "Action": [
                "cloudformation:CreateChangeSet",
                "cloudformation:DeleteChangeSet",
                "cloudformation:DescribeChangeSet",
                "cloudformation:DescribeStackEvents",
                "cloudformation:DescribeStacks",
                "cloudformation:ExecuteChangeSet",
                "cloudformation:GetTemplate",
                "cloudformation:ListChangeSets",
                "cloudformation:ListStacks",
                "cloudformation:ValidateTemplate"
            ]
        },
        {
            "Sid": "PolicyForManagedRules",
            "Effect": "Allow",
            "Action": [
                "events:PutRule",
                "events:PutTargets"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "events:ManagedBy": [
                        "cloudformation.sync.codeconnections.amazonaws.com"
                    ]
                }
            }
        },
        {
            "Sid": "PolicyForDescribingRule",
            "Effect": "Allow",
            "Action": "events:DescribeRule",
            "Resource": "*"
        }
    ]
}

Trust policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "TrustPolicy",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudformation.sync.codeconnections.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

How to create a stack using CloudFormation Git Sync?

Let’s look at the step-by-step procedure to deploy a new CloudFormation stack using Git Sync.

Environment setup considered here –

An AWS account configured with GitHub Connection in the Developer Tools Console.
A Private GitHub repository cfn-git-sync-poc that hosts a valid CloudFormation template at cloudformation/s3-stack.yaml
An IAM role was cfn-github-role created on the target AWS account with above stated policies.

Now, it’s time to get our hands dirty!

Log in to the AWS CloudFormation console. Click on the Create stack dropdown button and choose With new resources (standard)
On a Create stack page, select Template is ready and Sync from Git

Start providing the stack details like the stack name of your choice and select automatic Deployment file creation. If you wish to create a deployment file, please refer to this file format details.

Then, specify the Git repository information. Ensure that you have a configured and active GitHub connection to fill in the fields in this segment. Choose the repository containing the Infrastructure as Code (IaC), specify the deployment branch, and designate a Deployment file path where AWS will store the newly committed deployment file. Lastly, furnish the IAM role details that will be utilized for executing Git Sync operations.

Lastly, enter the file path for the CloudFormation IaC template inside the remote Git repo and define the parameters that are declared in the template. These details will be used to build the Deployment file.

Click the Next button and you will enter the Stack options page.
Stack options are the same as any other CloudFormation stack hence we will not discuss them here.
After verifying/modifying Stack options click Next button on the page.
Now, you will enter the review page where you can see the Deployment file constructed using the details provided and it will be committed to the remote Git repo on defined location

Review the rest of the configurations and click the Submit button.
Now, CloudFormation will commit and push the Deployment file to the remote Git repo by raising a Pull Request (PR). The message will be flagged on the page. Meanwhile, since the Deployment file is not found in the remote repo, the Provisioning status will be marked as Failed. Once PR is approved, Deployment file will be available in the remote repo.

*The First deploy using CloudFormation Git Sync*

Head over to GitHub and you should see a PR raised by Amazon to commit the deployment file into the remote repo.

*Deployment file creation by AWS Connector in PR*

Review, approve, and merge the PR.
Once PR is merged the code modification is detected by CloudFormation and it starts provisioning the stack.

Once all defined resources are created, stack creation will complete and CloudFormation keep monitoring the Git repository for any new changes.

In multiple environments such as development, testing, staging, and production, it’s possible to utilize distinct Stack Deployment files while leveraging a single CloudFormation IaC template file. These varied Deployment files can be organized into separate folders within the same repository for better segregation. By selecting the appropriate deployment file based on the environment where the CloudFormation stack is being established, the process becomes more streamlined. Something like –

cloudformation
├── dev
│   └── deployment.yaml
├── test
│   └── deployment.yaml
├── staging
│   └── deployment.yaml
├── production
│   └── deployment.yaml
└── template.yaml

By selecting the appropriate deployment file based on the environment where the CloudFormation stack is being established, the process becomes more streamlined. I anticipate that the Git Sync feature will evolve with additional capabilities in the future, potentially prompting customers to reconsider the necessity of separate Continuous Deployment (CD) software.

What do you think about this feature?

What are your thoughts on the CloudFormation Git Sync feature? Could it potentially revolutionize the game? Will it make several Continuous Deployment tools obsolete? It feels like an ArgoCD for Kubernetes, operating in a remarkably similar manner. For sure, it may not yet offer extensive control over commit-wide deployments, but the potential for future enhancements is exciting. This feature appears to transform the Infrastructure as Code (IaC) landscape for AWS customers, possibly luring some back from alternative IaC platforms. Witnessing its development and utilization in enterprise productions is going to be exciting!

How to add a GitHub connection from an AWS account?

Leave a reply

In this blog post, we will guide you through a step-by-step process to establish a GitHub connection in an AWS account.

What is a connection?

Firstly, let’s understand the concept of a connection in the AWS world. In AWS, a connection refers to a resource that is used for linking third-party source repositories to various AWS services. AWS provides a range of Developer tools, and when integration is required with third-party source repositories such as GitHub, GitLab, etc., the connection serves as a means to achieve this.

Adding a connection to connect GitHub with AWS

Let’s dive into the step-by-step procedure to add a connection that helps your AWS account to talk with your personal GitHub repositories.

Sign in to AWS Developer tools console.
Click on Create connection button.

*AWS Developer Tools Connection console*

On a wizard screen, select Github and name your connection.
Click on Connect to GitHub button

Now, AWS will try to connect to GitHub and access your account. Ensure you are already logged into GitHub and you should see below authorization screen. If not, you will need to login to GitHub first.

You can review the permissions being allowed to AWS on your account by clicking Learn more link on this screen.
Click on Authorize AWS Connector for GitHub
After authorizing the AWS connector, you should be back to the GitHub connection settings page.
At this point, AWS requires a GitHub Apps detail that will allow Amazon to access your GitHub repositories and make modifications to them.
AWS also offers to create a GitHub app on your behalf if it’s not created already. You can use the Install a new app button here to let AWS create the GitHub app in your account.
In that case, you need to verify the configuration (repo selection) and then click the Install button.

Once the App is created, the GitHub Apps ID will be populated in the wizard or manually enter the ID if the App is already created.

*GitHub Apps details for creating a connection*

Click on Connect button
You should be greeted with a success message with the new connection created!

Your GitHub connection is now ready. You can use this connection in compatible AWS services and let those services access your Github repositories.

Exploring the Latest AWS Console-to-Code Feature

Leave a reply

On November 2023 AWS announced the Preview going live for the new feature AWS Console-to-Code. Two months later, in this blog, we will explore this feature, learn about how to use it, what are the limitations, etc.

What is the AWS Console-to-Code feature?

It’s the latest feature from AWS made available in the EC2 console that leverages Generative AI to convert the actions performed on the AWS EC2 console into the IaC (infrastructure as Code) code! It’s a stepping stone towards IaC creation methods in the world of AWS cloud.

The actions carried out on the AWS console during the current session are monitored by the feature in the background. These recorded actions are then made available to the user to select up to 5 of these actions, along with their preferred language. AWS then utilizes its Generative AI capabilities to automatically generate code that replicates the objectives achieved through manual actions on the console.

It also generates the AWS CLI command alongside the IaC code.

The usefulness of the AWS Console-to-Code feature

With the current list of limitations and the preview stage, this feature might not be a game changer but it does have potential in the future. The AWS Console-to-Code feature will surely help developers and administrators to get the IaC skeleton quickly to start from and speed up the IaC coding with less effort.

This feature simplifies the process of generating AWS CLI commands, eliminating the need to constantly consult documentation and manually construct commands with the correct arguments. As a result, it accelerates automation deliveries with reduced effort.

By the way, there is no additional cost to use Console-to-Code so it doesn’t hurt to employ it for initial IaC drafting!

Limitation of AWS Console-to-Code feature

Currently, it’s in the ‘Preview’ phase.
Only available in North Virginia (us-east-1) region as of today.
It can generate IaC code in the listed types and languages only –
- CDK: Java
- CDK: Python
- CDK: TypeScript
- CloudFoprmation: JSON
- CloudFoprmation: YAML
It does not retain data across sessions. The actions that are performed in the current session are made available for Code Generation. Meaning if you refresh the browser page, it resets the action list and starts recording afresh.
Up to 5 actions can be selected to generate code.
Actions from the EC2 console only are recorded. However, I observed even a few actions like Security Group creation or Volume listing, etc. are not being recorded.

How to use the AWS Console-to-Code feature

Login to the EC2 console and select region N. Virginia (us-east-1)
On the left-hand side menu, ensure you have a Console-to-Code link.
Perform some actions in the EC2 console like launching an instance, etc.
Navigate to Console-to-Code by clicking on the link in the left-hand side menu.
It will present you with a list of recorded actions. Select one or a maximum of 5 actions for which you want to generate code. You can even filter the recorded actions as per their Type:
- Show read-only: Read-only events like Describe*
- Show mutating: Events that modified/created/deleted or altered the AWS resources.
Click on the drop-down menu and select the type and language for the code.

It should start generating code.
After code generation, you have an option to copy or download it. You can also copy the AWS CLI command on the same page.

*Python code generated by AWS Console-to-Code*

It also provides the generated code’s explanation at the bottom of the code.

Scaling with AWS PrivateLink

Leave a reply

In this article, we’ll discuss the scalability aspects of AWS PrivateLink. We’ll examine how the expansion of the service consumer VPC count impacts AWS PrivateLink implementation and its management. Additionally, we will delve into key considerations for designing a scalable solution using AWS PrivateLink.

AWS PrivateLink Primer

AWS PrivateLink provides a method for making your service accessible to other VPCs through a secure, private network connection over the AWS backbone network. This ensures that your data remains within the AWS network, thereby improving security and lowering data transfer expenses compared to when utilizing the public internet. The basic architecture of AWS PrivateLink is depicted as follows –

To set up the connection, you must establish an Endpoint Service within the service provider VPC, using a network/gateway load balancer. In the service consumer VPC, you should create a VPC endpoint that links to this Endpoint Service. The endpoint policies facilitate access control by specifying which principles are permitted to connect to the Endpoint Service. Please refer to this AWS documentation for more details.

Scalability aspect

Now, let’s discuss the scalability aspect concerning AWS PrivateLink. When we talk about scalability, we’re referring to the expansion of the number of VPCs acting as service consumers. In scenarios where you have critical or shared services hosted within the service provider VPC and made accessible through AWS PrivateLink for consumption by services located in different VPCs, it’s clear that the count of consumer VPCs will keep increasing. Therefore, it becomes essential to take scalability considerations into account.

Various VPC endpoints situated in different consumer VPCs can establish connections with a single endpoint service located in the service provider VPC. Hence, you can think of a high-level architecture as below –

Multiple VPC endpoints to one endpoint service

Furthermore, it’s important to note that AWS PrivateLink can enable communication to endpoints located in different AWS Regions through the use of Inter-Region VPC Peering.

I recommend reading this AWS blog, which outlines an architecture involving PrivateLink and Transit Gateway. This approach has the potential to significantly decrease the number of VPC endpoints, streamline the deployment of VPC endpoints, and offer cost optimization benefits, especially when implementing solutions at scale.

Scaling considerations

While it’s possible to configure many-to-one connectivity using AWS PrivateLink, there are several important factors to keep in mind when considering this type of scaling:

Cost and management: As you introduce new consumer VPCs to AWS PrivateLink, you’ll also be adding new VPC endpoints to your infrastructure, which can add to your billing and infrastructure management overhead.
AWS PrivateLink quotas: Be sure to take into account AWS PrivateLink quotas, as these define the limits for various aspects of your PrivateLink setup.
Network throughput: VPC endpoints support a maximum throughput of 100Gbps. This is an important consideration for applications that have high network demands when exposed through AWS PrivateLink.
LB quotas: Be considerate about network load balancer quotas/gateway load balancer quotas.
IP requirements: AWS PrivateLink consumes a certain number of IP addresses for Load Balancers and endpoints from your VPC’s IP address pool. Ensure that your VPCs can accommodate these IP requirements without causing IP address exhaustion.

Transit Gateway as an alternative?

Let’s look at Transit Gateway if it can be an alternative in a continually expanding VPC environment.

If unidirectional traffic is your primary requirement, AWS PrivateLink is the choice.
For a cost-efficient solution, AWS PrivateLink is certainly more economical than Transit Gateway.
It’s worth noting that Transit Gateway is not suitable when dealing with VPCs that have overlapping CIDRs.
In a nutshell, Transit Gateway becomes a viable alternative only when you are designing a highly scalable solution involving a significantly huge number of participating VPCs with non-overlapping CIDRs, and your solution prioritizes simplicity and reduced management overhead over cost considerations.

Understanding the basics of Lambda Function URLs

Leave a reply

In this guide, we’ll take you through the fundamental concepts of Lambda Function URLs. We’ll discuss their definition, explore their applications, and address security considerations, providing a comprehensive overview.

What is the Lambda Function URL?

It’s a dedicated, unique, and static URL for your Lambda function, enabling remote invocation of the backend Lambda function over the network call. This straightforward and budget-friendly method simplifies Lambda function invocation, bypassing the need for managing complex front-end infrastructure like API Gateway, Load Balancers, or CloudFront. However, this comes at the expense of advanced features provided by these services.

It follows the format:

https://<url-id>.lambda-url.<region>.on.aws

Why to use Lambda Fuction URL?

Creating them is quite straightforward and simple. The AuthType (security) is the only configuration you need to provide. CORS config is optional.
They come at no additional cost.
Once configured, they require minimal maintenance.
For straightforward use cases, they can replace the need for designing, managing, and incurring the costs of front-end infrastructure, such as API Gateway.
They are most appropriate for development scenarios where you can prioritize other aspects of applications/architecture over the complexity of Lambda invocation methods.

When to use Lambda Function URLs?

Lambda Function URLs serve a valuable role in accelerating the testing and development of the application, by prioritizing Lambda invocations in the application’s progress, while the method of invocation takes a backseat.

In production, they’re practical when your design doesn’t necessitate the advanced features provided by alternative invocation methods like API Gateway or Load Balancers, etc.

These URLs are also beneficial when dealing with a limited number of Lambdas, offering a simple, cost-effective, and maintenance-free approach to invocations.

How to secure Lambda Function URLs?

You can manage access to Lambda Function URLs by specifying the AuthType, which offers two configurable options:

AWS_IAM: This allows you to define AWS entities (users or roles) that are granted access to the function URL. You need to ensure a proper resource policy is in place allowing intended entities access to Action: lambda:InvokeFunctionUrl
NONE: Provides public, unauthenticated access. Use this option cautiously, as it allows unrestricted access. When you choose this option, Lambda automatically creates a resource-based policy with Principal: * and Action: lambda:InvokeFunctionUrl and attaches to function.

It’s important to remember that Lambda’s resource-based policy is always enforced in conjunction with the selected AuthType. Please read this AWS documentation for more details.

The Lambda resource policy can be configured at Lambda > Configuration > Permissions > Resource-based policy statements.

With the basics of Lambda Function URLs in mind, refer to how to create Lambda Function URL and kick-start your journey with them!