Does Amazon MSK Serverless remove the fixed cost floor?

No, and this is the most commonly missed point in cost comparisons. MSK Serverless bills $0.75 per cluster-hour, which alone is roughly $540 a month before a single partition is created or any data flows, plus $0.0015 per partition-hour and per-GB data in/out charges. That is a materially different economic shape from Kinesis On-Demand, which has no comparable per-cluster charge, only $0.04 per hour per active stream plus per-GB pricing. Serverless removes broker sizing, not the cost floor.

How much does the minimum viable MSK cluster actually cost?

A 3-broker Provisioned Standard cluster on kafka.m5.large instances runs about $0.63/hr in broker cost alone, roughly $453/month before storage, data transfer, or Kafka Connect worker capacity. Three brokers is not an arbitrary choice: it matches Kafka's default replication factor, so a smaller cluster forfeits standard durability guarantees. A single Kinesis shard, by contrast, costs $0.015/hr, about $11/month.

What is the shared-throttle gotcha in Kinesis that catches teams in production?

Kinesis shards allow 5 GetRecords calls per second, and that quota is shared across every consumer reading the shard without Enhanced Fan-Out, not granted per consumer. Add a third or fourth service polling the same stream and each gets under 2 calls per second, triggering throttling that client libraries often retry silently, so consumer lag builds for hours before anyone notices. Enhanced Fan-Out fixes it with a dedicated 2 MB/s per consumer, but costs an extra $0.015 per shard-hour per registered consumer, on top of the base shard cost.

Can existing Kafka tooling run against Amazon Kinesis?

No. Kinesis does not implement the Kafka wire protocol, so Kafka Connect connectors, ksqlDB, Kafka Streams, and Debezium CDC pipelines cannot target it without a full rewrite against the Kinesis SDK, the Kinesis Producer Library, or the Kinesis Client Library. If Kafka ecosystem compatibility is a requirement, MSK is the only AWS-managed option that satisfies it directly.

How does message retention compare between the two?

Kinesis defaults to 24 hours and caps standard retention at 7 days; extending to the maximum of 365 days adds long-term-retention storage and retrieval fees on top of the base shard cost. MSK's retention is bounded only by the storage you provision on the brokers, commonly configured well past 7 days at a flat $0.10 per GB-month, with tiered storage available to push older data to cheaper S3-backed storage without a hard retention ceiling.

How difficult is migrating between Amazon Kinesis and Amazon MSK?

It is a full rewrite in either direction, not an incremental port. The Kinesis Producer Library and Kafka's producer client have different batching models and error-handling contracts, and consumer logic built on the Kinesis Client Library with DynamoDB checkpointing has no equivalent in Kafka consumer groups. Teams typically run both systems in parallel during the transition, which roughly doubles streaming infrastructure cost for the migration window.

What are the per-shard and per-partition throughput limits?

A Kinesis shard is hard-capped at 1 MB/s or 1,000 records/second on write, and 2 MB/s on read without Enhanced Fan-Out. MSK Serverless caps an entire cluster at 200 MB/s write and 400 MB/s read, with partitions themselves billed hourly, which makes partition count a cost lever, not just a throughput lever. Provisioned MSK has no fixed per-partition ceiling from the service itself; throughput scales with how many brokers you run and how you size them.

Tech Duel

Amazon MSK vs Amazon Kinesis

Q: What is the main difference between Amazon MSK and Amazon Kinesis?

Amazon MSK is managed Apache Kafka: the full Kafka API, consumer groups, and the Kafka connector ecosystem on AWS-managed brokers starting at $0.21/hr per kafka.m5.large broker. Amazon Kinesis is a proprietary AWS streaming service billed per shard-hour ($0.015) or per GB in On-Demand mode, integrating natively with Lambda and Firehose but with zero Kafka protocol compatibility. MSK fits teams with an existing Kafka investment or Kafka-ecosystem requirements. Kinesis fits AWS-native pipelines built from scratch.

Amazon MSK gives you real Apache Kafka on AWS-managed brokers, with the ecosystem and the fixed costs that come with running Kafka. Amazon Kinesis is AWS's own proprietary streaming service, billed by the shard or by the byte, with no Kafka compatibility at all. The right pick depends on what you already run, how many consumers read each stream, and whether you can absorb a five-hundred-dollar-a-month floor before any data moves.

Last reviewed: June 2026

Quick verdict: Amazon MSK vs Amazon Kinesis

Choose Amazon MSK if you're migrating an existing Kafka workload, have a team that already knows Kafka internals, or need consumer-group semantics, retention past 7 days, or Kafka-ecosystem tooling.

Choose Amazon Kinesis if you're building a greenfield AWS-native pipeline, want zero broker management, and your traffic is light enough that a per-shard or per-GB bill beats a cluster's fixed floor.

Both keep records in strict order within a partition or shard, so ordering guarantees aren't what should decide this one.

When to choose Amazon MSK vs Amazon Kinesis

Choose Amazon MSK when…

You're migrating an existing Kafka workload and rewriting every producer and consumer isn't on the roadmap
You need Kafka consumer groups, or ecosystem tooling like Kafka Connect, ksqlDB, or Debezium CDC connectors
Retention needs to run past 7 days without paying Kinesis's separate long-term-retention storage and retrieval fees
More than a handful of independent services need to read the same stream at full throughput without per-consumer surcharges
Your team already runs Kafka operationally, or is large enough to absorb the learning curve

Choose Amazon Kinesis when…

You're building a new AWS-native pipeline with no existing Kafka investment to protect
You want Lambda, Firehose, or Kinesis Data Analytics wired up as a native integration, not through a Kafka connector
Your team is small and standing up a managed Kafka cluster is more operational surface than you want
Traffic is bursty or modest enough that a per-shard or per-GB bill beats a fixed per-cluster floor
You don't need Kafka protocol compatibility, and going through the Kinesis SDK is fine

That's the generic picture. Consumer count and retention window are what usually tip it in practice. ↓

Amazon MSK vs Amazon Kinesis: at a glance

Dimension	Amazon MSK	Amazon Kinesis
Pricing model	Per broker-hour + storage (Provisioned), or per cluster-hour + per partition-hour (Serverless)	Per shard-hour (Provisioned) or per GB in/out (On-Demand)
Entry-level cost	~$453/mo minimum: 3× kafka.m5.large brokers at $0.21/hr, matching Kafka's default replication factor	~$11/mo for one Provisioned shard, or $0.04/hr per active stream on On-Demand with no shard math
Serverless cost floor	$0.75/cluster-hour (~$540/mo) plus $0.0015/partition-hour, before any data flows	No comparable per-service floor; On-Demand bills per GB and per active stream only
Protocol	Native Apache Kafka API	Proprietary AWS API (SDK / KCL / KPL)
Client compatibility	Any existing Kafka client, Kafka Connect, ksqlDB, Debezium	Kinesis-specific SDKs only; no Kafka client runs unmodified
Default / max retention	Bounded by provisioned storage; tiered storage extends cheaply well past 30 days	24 hours by default, up to 365 days at extra long-term-retention cost
Per-shard/partition throughput	Scales with broker count/instance size; no fixed per-partition ceiling from the service itself	1 MB/s write, 2 MB/s read per shard (1,000 records/sec write)
Concurrent consumer reads	Kafka consumer groups scale reads at no extra charge	5 GetRecords calls/sec shared across all non-Enhanced-Fan-Out consumers per shard
Dedicated consumer read throughput	Native via consumer groups	Requires Enhanced Fan-Out at $0.015/shard-hour per registered consumer
Ops model	You own broker sizing and patch policy (Provisioned); Serverless drops broker sizing but not partition planning	Fully managed; On-Demand mode auto-scales without shard math

Amazon MSK vs Amazon Kinesis: what "serverless" doesn't fix about the cost floor

The comparison most articles run is $0.21/hr per MSK broker against $0.015/hr per Kinesis shard, and Kinesis wins that comparison every time. What it misses is that neither service lets you buy a single unit. AWS's own guidance runs Kafka at a replication factor of 3, so the smallest sane Provisioned MSK cluster is three kafka.m5.large brokers: about $0.63/hr, roughly $453 a month, before a single byte is written or a GB of storage provisioned.

MSK Serverless is marketed as the fix for that floor, and it removes broker sizing, but it doesn't remove the floor itself. Serverless bills $0.75 per cluster-hour, which alone comes to roughly $540 a month, plus $0.0015 per partition-hour on top and per-GB data in/out charges. A team that reads "serverless" and expects Kinesis-like elasticity is comparing the wrong shape of bill: Kinesis On-Demand has no cluster-level charge at all, only $0.04 per hour per active stream plus data volume.

Where MSK earns that floor back is consumer fan-out. Kafka consumer groups read a topic at no additional charge no matter how many services subscribe. Kinesis's per-shard GetRecords quota is shared across consumers, so scaling past a couple of independent readers means paying $0.015 per shard-hour per consumer for Enhanced Fan-Out. Past a handful of consumers or a sustained high-volume workload, MSK's fixed cost can be cheaper in total than a Kinesis bill stacking multiple Enhanced Fan-Out registrations.

If your team can't yet estimate consumer count or sustained throughput, that uncertainty itself is an argument for Kinesis: its bill scales down as easily as it scales up.

Amazon MSK vs Amazon Kinesis: what you give up without the Kafka protocol

MSK's biggest structural advantage isn't a feature, it's a protocol. Because it's real Apache Kafka, anything already built for Kafka runs against it unmodified: Kafka Connect connectors for every major data store, ksqlDB for stream processing, Kafka Streams applications, Debezium for change-data-capture. If your organization has any existing Kafka investment, that compatibility is close to free.

Kinesis has no equivalent ecosystem to inherit from, because there's no protocol to be compatible with. Producers go through the Kinesis SDK or the Kinesis Producer Library; consumers go through the Kinesis Client Library or a Lambda event source mapping. Both are well-documented and genuinely fast to stand up for a new pipeline, but neither is a drop-in replacement for a Kafka client, and nothing written against Kafka's client libraries will run against Kinesis without a rewrite.

That asymmetry makes the migration direction matter. Moving from Kinesis to MSK later, once a pipeline outgrows shard limits or needs Kafka tooling, means rewriting both producers and consumers, not extending them. Teams that might eventually need Kafka-ecosystem tooling but aren't sure yet face a real tradeoff: start on Kinesis for speed and accept a rewrite later, or start on MSK and carry its cost floor while the requirement is still hypothetical.

If any part of your stack already speaks Kafka, MSK protocol compatibility is worth more than the price difference suggests.

Amazon MSK vs Amazon Kinesis: the shared-throttle gotcha most teams find in production

Kinesis allows 5 GetRecords calls per second per shard, and that quota is shared across every consumer reading that shard without Enhanced Fan-Out, not granted per consumer. A pipeline that works fine with one consumer starts throttling the moment a second or third service subscribes to the same stream: each now gets under 2 calls per second, and client libraries frequently retry the resulting throttling errors silently, so consumer lag builds for hours before anyone notices the pipeline is behind.

The fix is Enhanced Fan-Out, which gives each registered consumer a dedicated 2 MB/s read channel, but it costs $0.015 per shard-hour per consumer, stacking on top of the base shard cost. A 20-shard stream with 4 fan-out consumers adds $1.20/hr, about $876/month, purely to work around a limitation that doesn't exist in Kafka's consumer-group model.

MSK Serverless has its own version of this tradeoff, just shaped differently: partitions are a direct billing line at $0.0015/partition-hour, and the cluster caps out at 200 MB/s write and 400 MB/s read regardless of how many partitions you create. That turns partition count into a cost lever as much as a throughput lever, which is easy to miss when comparing it only against Provisioned MSK's per-broker pricing.

Before committing to either service, estimate your real consumer count. It changes the Kinesis bill more than almost any other variable.

Common questions about Amazon MSK vs Amazon Kinesis

What is the main difference between Amazon MSK and Amazon Kinesis?

MSK is managed Apache Kafka: full Kafka API, consumer groups, and ecosystem tooling on AWS-managed brokers starting at $0.21/hr per broker. Kinesis is a proprietary AWS streaming service with a shard model at $0.015/shard-hr that integrates natively with Lambda and Firehose but has zero Kafka protocol compatibility. MSK is for teams who need the Kafka ecosystem or are migrating existing Kafka workloads. Kinesis is for teams building AWS-native pipelines from scratch who want to avoid Kafka's operational complexity.

Does MSK Serverless remove the fixed cost floor?

No, and it's the most commonly missed point in cost comparisons. MSK Serverless bills $0.75 per cluster-hour, roughly $540/month before any data flows, plus $0.0015 per partition-hour and per-GB in/out charges. Kinesis On-Demand has no comparable per-cluster charge, only $0.04/hour per active stream. Serverless removes broker sizing, not the cost floor.

What is the hidden production failure mode in Kinesis that most teams discover too late?

The 5 GetRecords API calls per second limit is shared across ALL consumers on a single shard. Add a third or fourth microservice reading the same stream and each gets fewer than 2 calls per second, triggering throttling errors that client libraries often retry silently. Your pipeline looks healthy but consumer lag climbs for hours before anyone notices. The fix is Enhanced Fan-Out, but it costs an extra $0.015/shard-hr per consumer. On a 20-shard stream with 4 consumers that is $1.20/hr added to your bill, purely to avoid a limitation that does not exist in Kafka.

Can I use Kafka clients with Amazon Kinesis?

No. Kinesis uses a proprietary protocol and does not support the Kafka client API. Producers must use the Kinesis Producer Library or AWS SDK calls; consumers must use the Kinesis Client Library or Lambda event source mappings. None of your existing Kafka producer or consumer code will work against Kinesis without a full rewrite. If you need Kafka protocol compatibility on AWS, MSK is the only AWS-managed option.

How long does message retention last in Kinesis vs MSK?

Kinesis defaults to 24 hours of retention and supports a maximum of 7 days on the standard tier. Extended retention to 365 days is available at additional cost. MSK gives you configurable retention up to your available broker disk capacity, with tiered storage extending it further at low cost. If your compliance or reprocessing requirements demand retention beyond 7 days, MSK is the more direct answer.

What team size is each service actually suited for?

Kinesis or MSK Serverless works best for small teams without dedicated Kafka operations experience: setup is faster and the day-to-day operational burden is lower. Provisioned MSK pays off for teams large enough to justify its ~$453/month floor against sustained throughput or existing Kafka expertise. There's no fixed headcount threshold, but the fixed cost is easier to justify once it's a small fraction of your total infrastructure spend rather than the majority of it.