Skip to content

Glossary · Updated Oct 31, 2026

Apache Kafka

Apache Kafka is an open-source event streaming platform that stores messages in an ordered, replayable log, so many systems can read the same stream independently.

Kafka organizes messages into topics, splits each topic into partitions for scale, and keeps messages for a set retention period after they are read. Each consumer tracks its own position in the log, which is what lets a new system start from the beginning and replay history. A queue, by contrast, normally removes a message once a worker has handled it.

The common mistake is adopting Kafka before there is a need for it. Running it well takes real operational skill, and most companies of 20 to 200 people are better served by a managed queue or topic until they need replay or very high throughput. Managed options such as Amazon MSK reduce the work without removing it.

See how an idea becomes a working solution.

Explore how we scope, build and hand over AI projects, with clear decisions at each stage.