A lot of good insights here. I am also wandering if they can just simply put different jobs (unclaimed, in-progress, deleted/done) into different directory/prefix, and rely on atomic object rename primitive [1][2][3] to solve the problem more gracefully (group commit can still be used if needed).
well I checked the code in their repo，but could not find any snippets as mentioned in the post，which made the claimed perf numbers very suspicious. There was also no cluster configration details in their claim，but you can verify our performance numbers by simply following the deploy steps in the readme.
FractalBits is an S3-compatible object storage system designed for high performance and low latency. Using our custom-built fractal ART metadata engine, it delivers up to 1 million 4K read IOPS for single bucket with p99 latency ~5ms, at significantly lower cost than AWS S3 Express One Zone. Unlike standard S3, FractalBits provides native atomic rename support for both objects and directories. We are happy to open source our API server implementation (which support both actix and axum framework) today and welcome any feedback!
[1] https://docs.cloud.google.com/storage/docs/samples/storage-m... [2] https://docs.aws.amazon.com/AmazonS3/latest/API/API_RenameOb... [3] https://fractalbits.com/blog/why-we-built-another-object-sto...