Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
100+ Lines of Code to Implement Kafka on S3 (github.com/automq)
7 points by TopRainbowT on July 16, 2024 | hide | past | favorite | 2 comments


Just like warpstream or confluent freight cluster, but AutoMQ is source available, you may find the key s3 wal implementation here: https://github.com/AutoMQ/automq/blob/main/s3stream/src/main...


AutoMQ claims this is the best design for streaming storage on the cloud: Shared WAL + Shared Object. This new shared storage architecture for streaming systems can leverage the advantages of different cloud storage services. They use only a few lines of code to implement features like building a Kafka service on S3. This blog shows how they achieve this. What do you think?




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: