Tag Archives: cell-based-architecture

Cell-Based Architecture on AWS, Part 6: Shipping Cells with IaC and ArgoCD

Leave a reply

By this point in the series you’ve picked a partition key, chosen EKS or ECS as the substrate, worked out the networking, built in shuffle sharding and bake time, and made peace with what it all costs. What’s left is the least glamorous problem and the one that actually determines whether any of the above survives contact with a real team: how do you provision and update dozens of near-identical cells without dozens of near-identical ways for them to drift apart.

Shipping cells!

Infrastructure as code: the stamp pattern

The core idea is simple to state and easy to get subtly wrong in practice: write one module that describes a cell, and apply it once per cell with different inputs. A hands-on Terraform walkthrough of this pattern frames it as a blueprint you stamp out repeatedly, alongside a separately-built router layer that isn’t part of any individual cell’s module at all.

A minimal shape for that module looks something like this:

module "cell" {
  source   = "./modules/cell"
  for_each = var.cells

  cell_id       = each.key
  account_id    = each.value.account_id
  region        = each.value.region
  az            = each.value.az
  tenant_range  = each.value.tenant_range
}

variable "cells" {
  type = map(object({
    account_id   = string
    region       = string
    az           = string
    tenant_range = string
  }))
}

The detail that matters more than the module code itself is state isolation. Each cell’s Terraform state needs to be genuinely separate — a distinct backend key per cell, not just a shared workspace inside one state file — so that a plan or apply against cell-07 can’t accidentally touch cell-03. A shared state file for “all cells” quietly reintroduces exactly the coupling the whole architecture exists to avoid, just at the tooling layer instead of the runtime layer.

CDK works the same way conceptually, even though the syntax differs. AWS’s own reference implementation structures this as three separate stacks — one for shared ECR repositories, one for the routing components, and one for the cell blueprint, which gets deployed repeatedly, once per cell. Terraform, CDK, or CloudFormation, the stamp pattern is the same idea wearing different clothes.

A cell registry as the single source of truth

Once you’re past a handful of cells, you want one place that lists every cell that exists — its account, region, AZ, tenant range, and lifecycle status (active, draining, newly provisioned). This can be as simple as a cells.yaml checked into the same repo as your Terraform, or as robust as a DynamoDB table if other systems need to query it at runtime. What matters isn’t the storage mechanism, it’s that this registry becomes the one input that drives both your infrastructure provisioning and your application deployment generator — because the moment those two read from different sources of truth, you’ve created a gap where they can disagree about how many cells exist.

Cell registry

Deploying applications with ArgoCD

On the application side, ArgoCD’s ApplicationSet controller is built for exactly this fan-out. A list or git-file generator, sourced from the same cell registry, produces one child Application per cell (or per cell-and-app combination), rather than hand-maintaining one YAML file per cell per application.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: order-service-cells
  namespace: argocd
spec:
  generators:
    - git:
        repoURL: https://github.com/your-org/gitops-cells.git
        revision: main
        files:
          - path: "cells/*/registry.yaml"
  template:
    metadata:
      name: 'order-service-{{cell_id}}'
    spec:
      project: order-service
      destination:
        server: '{{cluster_endpoint}}'
        namespace: order-service
      source:
        repoURL: https://github.com/your-org/gitops-cells.git
        targetRevision: main
        path: 'apps/order-service/overlays/{{cell_id}}'

Platform-wide baseline components — an ingress controller, cert-manager, the monitoring agents every cell needs regardless of tenant — are a good fit for a separate App-of-Apps hierarchy, kept distinct from the per-cell, per-tenant applications so a change to the shared baseline doesn’t get tangled up with a single cell’s release train.

Sync waves are worth using deliberately here: router configuration and any cell-registration step should land before the cell’s own workloads sync, not after, or you risk a cell coming online with traffic already routed to it before its services exist. And each cell’s rollout should carry its own canary and bake time — tying back to the deployment discipline from earlier in this series — rather than one canary standing in for the whole fleet. A fleet-wide canary defeats the purpose of having cells in the first place; the point was that a bad release only ever touches one of them at a time.

The habit that actually keeps this from falling apart

None of this tooling prevents drift on its own. What prevents drift is treating the cell registry as a change that goes through the same pull request and review path as any other production change, and making sure both the Terraform plan and the ArgoCD ApplicationSet read from that same file. When someone adds cell-12 to the registry, that one commit should be the trigger for both the infrastructure to get stamped out and the applications to start deploying into it — not two separate manual steps that someone has to remember to keep in sync.

That’s really the throughline of this whole series. Cell-based architecture starts as a resiliency idea — smaller blast radius, contained failure — but by the time it’s running in production it’s mostly a delivery-engineering discipline. The isolation only holds up if your pipeline can safely stamp out cell N+1 in an afternoon, deploy into it with the same confidence as cell N, and tear it down just as cleanly if it turns out you didn’t need it. If your tooling can’t do that yet, the resilience story is still theoretical — and that’s a good place to start the next iteration of the design, rather than the last one.

Cell-Based Architecture on AWS, Part 5: Well-Architected and the Real Cost of Cells

Leave a reply

Cell-based architecture gets pitched almost entirely as a reliability pattern, and it is one — the AWS Well-Architected Framework files it directly under the Reliability pillar’s bulkhead guidance. But it touches all six pillars once you actually build it, and at least two of them — cost and sustainability — tend to get discovered the hard way, after the architecture is already in production and the bill has arrived.

Well architected and cost in Cell Based Arch!

Running it through all six pillars

Reliability is the pillar cells were designed for, and it’s the easy one: smaller, isolated fault domains, contained blast radius, and a natural unit for fractional deployment.

Operational excellence cuts both ways. Cells give you a clean unit for canary releases and bake time, which is a genuine operational win — but every cell you add is another entry in the on-call runbook, another set of dashboards, another thing that can silently drift out of configuration with its siblings if your automation isn’t keeping up.

Security benefits in the same shape reliability does: a leaked credential or a misconfigured IAM policy inside one cell doesn’t automatically hand over the rest of the estate. The cost is that you now have N sets of IAM roles, N certificates, and N network boundaries to keep patched and rotated instead of one.

Cost optimization is where the trade-off gets sharp, and it’s covered in more detail below.

Performance efficiency tends to improve with smaller cells, because a noisy neighbor problem shrinks to the size of one cell instead of the whole platform. AWS’s ad-serving case is a clean illustration in the other direction: when tenants shared in-memory state on common infrastructure, one tenant with a large dataset could trigger memory pressure that degraded everyone sharing that heap, which is precisely the failure mode dedicated per-tenant compute was brought in to solve.

Sustainability is the pillar cell-based architecture can quietly work against. Fixed-cost resources replicated per cell — a NAT gateway sized for peak, a node pool that never scales below some minimum — sit idle a lot of the time by design, since isolation means you can’t pack unrelated tenants’ spare capacity together. Idle reserved capacity is wasted energy whether or not the AWS bill makes that obvious.

The cost math that catches people off guard

A few numbers are worth having in your head before you commit to a cell count.

An EKS control plane costs roughly $0.10 an hour, which works out to around $73 a month — and that’s before a single pod runs. Ten cell-per-cluster EKS deployments means roughly $730 a month in control-plane fees alone, on top of whatever compute those clusters actually run. ECS doesn’t have this problem in the same way — a cluster is a free logical construct, and billing starts at the task — which is a real reason cluster-per-tenant is a more forgiving pattern on ECS than on EKS.

AWS PrivateLink interface endpoints run close to $7.30 a month each plus data transfer, which is trivial shared once across a tier of cells and a real line item if you accidentally provision one per cell instead of pre-wiring it once at the tier level — the difference between those two choices is exactly what drove an 80 percent cut in network configuration overhead in AWS’s own ad-serving redesign.

And the starkest number in that same case study isn’t a unit price at all — it’s utilization. Before the redesign, the isolated-per-tenant fleet ran at roughly 3 percent average CPU and 19 percent average memory, with servers idle more than 98 percent of the time. That’s the cost of strict isolation taken to its logical extreme: every tenant’s capacity has to be sized for that tenant’s peak, and peaks, by definition, don’t happen most of the time.

Karpenter and the bin-packing tension

If you’re on EKS, Karpenter earns its cost savings mainly through consolidation — packing workloads onto fewer, better-utilized nodes and leaning on Spot capacity where it’s safe to. Karpenter documentation and the practical write-ups from teams running it at scale are consistent on this: it works best with a large pool of mixed workloads to bin-pack across dozens of nodes with varied sizes, not a handful of strictly-separated node pools each sized for one tenant.

Cell isolation deliberately works against that. If every cell gets its own NodePool to preserve the isolation boundary, you’re intentionally giving up some of Karpenter’s consolidation opportunity in exchange for the guarantee that one cell’s workload can’t crowd out another’s. That’s a legitimate, conscious trade — not a mistake — but it’s worth naming explicitly rather than discovering it as an unexplained line item during a cost review.

What this means for how you size cells

Everything above points back to the same conclusion from earlier in this series: start with fewer, larger cells, and shrink them as your tooling matures. A few practical habits follow directly from the cost picture:

  • Bundle genuinely fixed-cost resources — a VPC, a NAT gateway — across a batch of cells where the blast radius trade-off allows it, rather than one per cell.
  • Tier tenants by traffic profile instead of mapping every tenant to its own cell 1:1; group similar-sized tenants together and reserve full dedicated cells for the ones that actually need them.
  • Revisit your cell count and size against the AWS Well-Architected Tool and the SaaS Lens periodically — this isn’t a decision you make once and forget, it’s one that should move as your tenant mix and automation both change.

There’s no version of cell-based architecture that’s free. The honest framing is that you’re buying blast-radius containment with a mix of duplicated fixed costs, reduced bin-packing efficiency, and extra operational surface — and the job of a good design is making sure you’re paying for isolation you actually need, not isolation you inherited from copying someone else’s cell count. The last post in this series covers the part that makes all of the above sustainable day to day: provisioning cells with infrastructure as code and shipping application changes into them with ArgoCD.

Cell-Based Architecture on AWS, Part 4: Making Cells Actually Resilient

Leave a reply

Splitting a system into cells doesn’t automatically make it resilient. If you still push a bad deployment to every cell at once, or if the same unlucky customer keeps landing on the same overloaded set of cells as everyone else, the isolation on your architecture diagram isn’t doing much for you in production. The pattern only pays off once you pair it with two more things: shuffle sharding, and real discipline about how changes roll out.

Resiliency in Cell Based Arch!

Shuffle sharding: spreading the overlap thin

Cells alone reduce blast radius by dividing customers into groups. Shuffle sharding goes a step further by giving each customer (or request) a near-unique combination of cells or nodes, so that any two customers only partially overlap, and the odds of two customers sharing the exact same full set of resources drop fast as your pool of cells grows. AWS describes this in more depth in Shuffle Sharding: Massive and Magical Fault Isolation, and the cell-based architecture FAQ is direct about the boundary: shuffle sharding works within a cell, but by definition a cell shouldn’t share state across cell lines, so don’t let shuffle sharding become an excuse to blur the boundary you just built.

The AZ-as-cell-boundary example from the Journey to Cloud-Native series makes the payoff concrete. With customers shuffle-sharded across three AZ-aligned EKS cells, a black-swan event that would have hit 100 percent of the application without cells is capped at roughly a third of capacity — and shuffle sharding also gives you a scaling lever for a single customer whose traffic spikes unexpectedly, since their load doesn’t have to be absorbed by one fixed cell.

Bake time and fractional deployment

The other half of real resilience is how you ship changes. A cell-based system gives you a natural unit for fractional deployment — roll a change to one cell, watch it, then the next, rather than everywhere at once. What actually makes this work is bake time: a deliberate pause after each incremental step, long enough to know whether the change caused trouble before promoting it further. Depending on the system, that might be fifteen minutes or several hours — the right duration is whatever it takes for you to trust the signal, not a fixed number copied from someone else’s runbook.

The discipline that goes with it is equally important: roll back at the first sign the release is destabilizing things, rather than pushing forward hoping it stabilizes on its own. A fractional failure contained to one cell is a good outcome. A full rollout that later needs a full rollback is the outcome cells were supposed to prevent. Stateful changes deserve extra caution here specifically because they’re harder to walk back — a schema migration that can’t be un-run is a one-way door, cell-based or not.

Release Flow

AWS’s own hyperscale teams lean on progressive delivery tooling for exactly this — Argo Rollouts alongside EKS, watching response time and error rate as the promotion signal, described in the same Journey to Cloud-Native post. We’ll come back to wiring this into ArgoCD directly in the last post of this series.

Monitoring: per-cell, and in aggregate

Splitting one system into many cells multiplies your monitoring surface by the same factor. Each cell needs its own health signal — white-box metrics from inside the application, black-box checks from outside it, and business metrics that catch problems the infrastructure metrics miss entirely, like a quiet drop in successful checkouts. On top of that, you need an aggregate view that rolls individual cells up into “how many cells are healthy right now,” because a dashboard with forty individual cell panels and no summary is not a dashboard anyone can act on during an incident.

Tag everything with a cell identifier from the start — logs, traces, and metrics alike — so that when something does go wrong, correlating the failure back to a specific cell is a query, not an investigation. On EKS, Karpenter’s topology spread support and AWS’s Zonal Shift capability are worth pairing with AZ-aligned cells specifically, since they let you push pods away from an impaired zone and cordon affected nodes without hand-rolling that logic yourselves.

Multi-cell transactions: avoid them if you can

Sooner or later someone will ask for a request or a report that needs data from more than one cell. Handle this as a deliberate, external coordination step — a map-reduce-style service that calls each cell through its normal public interface, never by reaching into a cell’s internals directly — and treat every one of these you add as a real complexity cost, not a free feature. Each cross-cell dependency is a small crack in the isolation boundary you built the rest of this architecture to protect. Some are unavoidable. Most are worth pushing back on.

Resilience, in a cell-based system, isn’t a property you get from the architecture diagram. It’s the sum of a routing layer that fails safely, a shuffle-sharding strategy that keeps blast radius genuinely small, and a deployment process that treats bake time as non-negotiable. Get the diagram right and skip the process, and you’ve built an expensive way to still take a full outage. Next in the series: what all of this actually costs, and how it maps onto the Well-Architected Framework beyond just reliability.

Cell-Based Architecture on AWS, Part 3: Networking a Multi-Cell Estate

Leave a reply

Every cell you design is, by intent, isolated from every other cell. The one piece of the system that can’t fully follow that rule is the network path that decides which cell a piece of traffic reaches in the first place. Get the routing layer wrong and you’ve rebuilt the single point of failure you were trying to eliminate — just one layer higher up.

Networking in Cell Based Arch!

VPC topology: per-cell, shared, or AZ-as-boundary

There’s no single correct way to lay out VPCs against cells, and the right answer depends on which boundary you’re actually defending. A VPC per cell gives the cleanest isolation — a NACL misconfiguration or route table mistake in one VPC can’t reach another — but it multiplies NAT gateways, VPC endpoints, and IP address planning by the number of cells you run. A shared VPC with dedicated subnets per cell is cheaper to run and easier on IP address management, at the cost of some shared blast radius at the VPC level itself.

A third option skips the VPC question and uses the Availability Zone as the cell boundary directly. AWS’s own Journey to Cloud-Native architecture write-up describes exactly this: pods aligned into cell groups on EKS, spread across three AZs, with only the routing layer — Route 53, DynamoDB, and the load balancer — shared across the whole system. The payoff is concrete: without cells, an initial bad event could take out the entire application; spread across three AZ-aligned cells, the worst case for that same initial hit caps at roughly a third of total capacity.

Whether you also want an AWS account as an isolation boundary is a separate, harder call. Multiple accounts contain the damage from leaked credentials or an account hitting a service quota, but account sprawl brings its own tax — every account needs to be onboarded into billing, monitoring, and security tooling, and that overhead doesn’t disappear just because Organizations makes account creation easy.

Route 53 weighted routing, worked through

For AWS-native cell routing, Route 53 weighted routing does a lot of the heavy lifting cheaply. Each cell (or infra group within a cell) gets a weighted alias record behind one stable DNS name, and adding capacity is just adding another weighted record — existing clients never need a DNS change. A record for a new cell might look like this:

{
  "Action": "CREATE",
  "ResourceRecordSet": {
    "Name": "orders.us-east-1.example.com",
    "Type": "A",
    "SetIdentifier": "cell-04",
    "Weight": 50,
    "AliasTarget": {
      "HostedZoneId": "Z35SXDOTRQ7X7K",
      "DNSName": "cell-04-alb.us-east-1.elb.amazonaws.com",
      "EvaluateTargetHealth": true
    }
  }
}

Turning on health evaluation matters here — it’s what keeps Route 53 from sending traffic to a cell whose ALB has gone unhealthy. A hosted zone supports up to 10,000 weighted records, so this scales to a lot of cells before you need to think about it again.

The ALB quotas that quietly cap your design

Two unglamorous Application Load Balancer quotas end up shaping infra-group size before any architecture diagram does: a maximum of 100 target groups per load balancer, and a maximum of 5 target groups per listener rule. Work through the arithmetic and one ALB, configured with reasonably granular listener rules, comfortably serves on the order of 50 tenant or cell targets before you need a second ALB — which is exactly the trigger AWS Ads used to decide when to add a new infra group rather than scale the existing one vertically. Knowing this number before you design the routing layer saves a redesign later.

PrivateLink for shared dependencies — and its cost

Cells rarely live in complete isolation from the rest of your platform; they usually need to reach a handful of centrally-owned services — a shared cache, an identity provider, a billing system. AWS PrivateLink is the standard way to expose those without opening them to the public internet or wiring up VPC peering per cell. The trick that actually pays off is pre-wiring the interface endpoints once, at tier or platform creation time, rather than per tenant — AWS Ads reports an 80 percent cut in network configuration overhead from doing exactly that. Each interface endpoint runs a modest, predictable cost — roughly $7.30 a month plus data transfer — which is close to noise once it’s shared across dozens of cells, and a line item worth tracking once it’s multiplied per cell instead.

Where a Transit Gateway shows up to connect layers of cells to each other, treat it carefully. A single TGW sitting at the center of every cell-to-cell path becomes exactly the kind of shared point of failure the whole architecture was designed to avoid. If cross-layer traffic is limited to a small, known set of backend cells, bilateral connections are often the safer default over a hub-and-spoke TGW.

Where VPC Lattice fits

Amazon VPC Lattice is worth knowing about for the service-to-service leg of this problem specifically. It gives services a logical, IAM-authorized identity instead of an IP address, and it works across EC2, ECS, EKS, and Lambda targets in different accounts and VPCs without hand-built peering or Transit Gateway routes. That makes it a good fit for the case where a cell’s workloads need to reach a small set of centrally-owned services elsewhere in the organization — but it’s not a cell router. It solves reachability between services, not the “which cell does this customer belong to” assignment problem, which still needs the DynamoDB-and-DNS pattern from the routing layer.

Certificates: the boring single point of failure

One detail that’s easy to skip and expensive to skip: give each cell its own TLS certificate with its own expiration date rather than sharing one certificate estate-wide. A single shared certificate that expires or gets misconfigured is a shared blast radius, no different in kind from a shared VPC route table — it just doesn’t look like one until it fails.

Networking is the layer of a cell-based system you actually want to be a little boring — predictable quotas, pre-wired connectivity, health-checked routing, and nothing clever sitting in the critical path that doesn’t need to be there. Boring networking is what makes the resiliency claims in the next post actually hold up under a real failure, rather than just on the architecture diagram.

Cell-Based Architecture on AWS, Part 2: EKS or ECS for Your Cells, and the Services That Hold Them Together

Leave a reply

Once you’ve picked a partition key, the next decision is what actually runs inside a cell. For containerized workloads that’s an EKS-or-ECS question, and it’s worth more thought than “we already know Kubernetes.” The substrate decision determines how expensive isolation is per cell, and cost per cell is a number you’re about to multiply by every cell you’ll ever run.

When you have a choice to make!

EKS cells: namespace, or full cluster?

Inside EKS, a cell can live at two different depths. A namespace-per-cell model, with resource quotas and network policies enforcing the boundary, is cheap — one control plane serves every cell, and you’re mostly paying for the compute the pods consume. The catch is that the control plane itself becomes a shared blast radius domain again. A CRD conflict, an API server issue, or a cluster-wide add-on misbehaving can still touch every cell in that cluster, which undermines a chunk of the reason you wanted cells in the first place.

Cluster-per-cell removes that shared surface entirely, at a real price: an EKS control plane fee for every cluster, a separate node group or Karpenter deployment per cluster, and its own set of add-ons, IAM roles, and networking to keep patched. AWS’s own containerized cell-based reference design uses this model — pods are grouped into cell groups inside EKS, with topology-aware routing hints helping keep traffic within a cell’s Availability Zone rather than crossing between them. That Journey to Cloud-Native architecture post is worth reading end to end if you’re leaning EKS.

ECS cells: cluster-per-tenant is nearly free, until it isn’t

ECS clusters cost nothing to create — the meter starts when a task runs, which makes cluster-per-tenant a genuinely cheap way to get hard compute isolation without touching a Kubernetes control plane at all. It’s a well-worn SaaS isolation pattern for exactly that reason.

It’s also a pattern that can quietly become the problem. Amazon’s ad-serving platform ran an early cellular design that allocated a dedicated AWS account, Application Load Balancer, and ECS cluster to each tenant. Documented in AWS’s write-up on building hybrid multi-tenant architecture for stateful services, that setup hit real limits at a surprisingly small scale: just 18 clients spread across four AWS Regions already needed 181 separate targets to manage, onboarding a new client took roughly 52 days end to end, and average CPU utilization across the fleet sat around 3 percent. Strict isolation had bought accuracy, but the servers spent over 98 percent of their time waiting on requests that never came, because each tenant’s dedicated capacity had to be sized for its peak, not its average.

The fix wasn’t to abandon per-tenant isolation — it was to stop mapping tenants one-to-one onto cells. AWS Ads moved to a three-level hierarchy: a tier groups tenants with similar traffic profiles, a cell is the AWS account boundary within a tier, and an infra group — one VPC, one ALB, a set of per-tenant ECS clusters — is the reusable unit inside a cell. Onboarding became a configuration change against pre-wired infrastructure instead of a multi-week build, dropping to about 7 days. The lesson generalizes past ECS: your isolation boundary and your scaling unit don’t have to be the same thing.

The building blocks that show up in almost every cell design

Regardless of EKS or ECS, the same handful of AWS services tend to appear in the architecture diagram:

PurposeTypical service
Container computeAmazon EKS or Amazon ECS, often on Fargate or with Karpenter-managed EC2
Traffic routing to cellsAmazon Route 53 weighted or DNS-based routing, Application/Network Load Balancer
Cross-account, cross-compute service reachabilityAmazon VPC Lattice, AWS PrivateLink
Cell-assignment stateAmazon DynamoDB — fast, highly available, a natural fit for “which cell does this customer belong to” lookups
Container imagesAmazon ECR
Observability per cellCloudWatch Container Insights, Amazon Managed Service for Prometheus, Amazon Managed Grafana
Cost visibility per cell/tenantKubecost against Amazon Managed Prometheus for multi-cluster cost monitoring
DeploymentInfrastructure as code (Terraform or CDK) plus ArgoCD for the app layer

The DynamoDB piece deserves a callout. AWS’s own hyperscale teams have mapped customers to cells with a hash function whose result is stored in DynamoDB, precisely because that table needs to answer “which cell?” reliably under load without becoming a bottleneck itself. It’s a small, boring piece of infrastructure carrying an outsized amount of trust.

Router patterns, briefly

Every design needs something that decides which cell a given request goes to, and there are three broad ways to build it, each with a different failure profile. A load-balancer-style router sits in the request path permanently, forwarding every packet — simple for clients, but now in the critical path for every transaction. A hand-off router authenticates the client once and returns the cell’s address, after which the client talks to the cell directly; this removes the router from the ongoing request path but pushes some logic into the client. A DNS-based router leans entirely on the DNS system’s own reliability, at the cost of DNS-level control over routing decisions. None of these is strictly better — which one fits depends on how much control you need over routing versus how simple you want the client side to be. We’ll dig into this properly, including where a control-plane/data-plane split helps, in the networking post.

Which one would I reach for

If I’m starting fresh on containers and the workload doesn’t need OS-level tuning, I lean ECS cluster-per-cell for the early cells. It’s cheaper to stand up and tear down, and the limits you bump into are AWS service quotas rather than Kubernetes control-plane behavior you have to reason about separately. I’ll reach for EKS cluster-per-cell once the team already runs Kubernetes elsewhere and wants one operational model across cell and non-cell workloads — the consistency is worth the extra control-plane bill at that point. Namespace-per-cell inside a single EKS cluster is the one I’d be most cautious about recommending for anything claiming strong isolation; it’s cheap, but you’re still betting the whole cluster’s control plane won’t be the thing that takes every cell down at once.

The substrate decision sets your cost floor. The networking layer sets your failure floor — which is where we’re headed next.

Cell-Based Architecture on AWS, Part 1: The Thought Process Before You Draw a Single Box

Leave a reply

A dropped table. A malformed feature flag. A load balancer configuration that fails open instead of closed. None of these need a regional outage to hurt you. They only need a blast radius big enough to reach every one of your customers at once. Multi-AZ and multi-Region deployments protect you from AWS having a bad day. They do nothing for the day your own deployment pipeline has one.

Where it begins!

That gap is what cell-based architecture is built to close, and it’s the reason AWS has used the pattern internally for over a decade before writing it down as public guidance. Before touching a Terraform file or an EKS console, the job is mostly a thinking exercise: deciding what a “cell” is for your system, what goes inside one, and how big is too big. Get that wrong and no amount of Kubernetes or containers expertise will save you later.

What a cell actually is

A cell-based architecture partitions a system into a set of independent, self-contained replicas, each serving only a slice of the overall client base. AWS’s guidance on reducing scope of impact with cell-based architecture frames this as an extension of the same fault isolation AWS already applies at the Availability Zone and Region level, just moved down into your own workload.

The two things that make a cell a cell, rather than just another replica behind a load balancer, are isolation and partitioned state. Every cell runs standalone, with no runtime dependency on any other cell, and the data it owns isn’t replicated elsewhere. A thin routing layer is the only thing that knows all the cells exist — clients (or their traffic) get assigned to exactly one cell and stay there. AWS’s cell-based architecture guidance on GitHub puts it plainly: if a bad actor or a bug wipes a database inside one cell holding a tenth of your users, you’ve lost a tenth of your data, not all of it, and you can restore a tenth of a database a lot faster than the whole thing.

This is also why it sits inside the Reliability pillar of the AWS Well-Architected Framework, specifically as an expanded form of the bulkhead architecture best practice. It’s explicitly called out as guidance for workloads that need extreme levels of resilience, not a default starting point for every workload you own.

Decision flow for Cell Based Arch!

Where cell boundaries come from

The first real decision — before compute, before networking — is the partition key. What is the unit of the system that gets isolated? Tenant ID, customer account, geography, and traffic tier are the usual candidates, and the choice cascades into everything downstream: how routing state is modeled, how the data layer gets sharded, and later, how your Terraform and GitOps repos get structured. Pick this wrong and you’ll be re-partitioning a live system later, which is exactly the kind of one-way-door change the whole pattern is meant to avoid.

The second decision is where the isolation boundary actually sits. An AWS Availability Zone can be the cell boundary, as AWS’s own architecture teams have done for containerized workloads. An AWS account can be the boundary, which limits blast radius from compromised credentials or account-level service quotas, at the cost of more accounts to govern and bill. Or the boundary can sit lower, at the Kubernetes namespace or ECS cluster level, inside a shared account. None of these is universally correct — it depends on what failure you’re actually trying to contain.

Sizing cells: the trade-off nobody gets to skip

Smaller cells reduce blast radius, since each one carries fewer customers. They’re also easier to test, easier to reason about, and individually simpler to operate. But more cells means more of everything else — more routing entries, more monitoring dashboards, more deployment pipelines running in parallel — and that operational surface area doesn’t shrink just because each cell is small.

Larger cells go the other way. Fewer moving parts, better economics per customer, less to monitor — and a bigger blast radius when something does go wrong inside one. There’s no formula that spits out the “right” cell size; it’s a genuine trade-off, and the common pattern is to start with a small number of large cells and shrink them over time as your automation and tooling mature enough to absorb the extra operational load. Jumping straight to hundreds of tiny cells before your deployment tooling can handle it just moves the failure mode from “blast radius” to “operational chaos.”

Is this even the right tool for your workload

Here’s the opinion part: most teams don’t need this. If you’re running a single EKS cluster for an internal tool with a handful of customers, cell-based architecture will cost you more in engineering time than any outage it prevents. It earns its place when a single failure hitting all customers simultaneously is genuinely unacceptable — hyperscale SaaS, regulated industries with strict blast-radius requirements, or platforms where a black-swan event (a sudden traffic spike from one tenant, a bad config push, a compromised credential) has to be contained by design, not by hoping your canary process catches it in time.

A planning checklist before you open the AWS console

  • Identify the partition key your cells will be organized around.
  • Decide the isolation boundary: namespace, cluster, Availability Zone, or AWS account.
  • Decide where cell-assignment state lives and who owns the routing layer.
  • Decide which resources are centralized (accepting shared blast radius) versus replicated per cell (accepting cost).
  • Set an initial target cell size, and the trigger condition for splitting or adding another one.

That last point matters more than it looks. Without an explicit trigger — a tenant count, a request-rate threshold, an AWS service quota you’re approaching — cell sizing decisions get made reactively, usually during an incident, which is the worst possible time to be making them.

The rest of this series works through the pieces that hang off these decisions: which AWS container services and building blocks actually implement a cell on EKS or ECS, how the networking layer routes traffic without becoming a single point of failure itself, how shuffle sharding and deployment discipline make the isolation real rather than theoretical, what this costs against the Well-Architected pillars, and how to provision and deploy dozens of these things without losing your mind. Next up: choosing between EKS and ECS as your cell’s compute substrate.