Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I've been looking at Python tasks queues recently. Does anyone have experience on how Celery and rq stack up?

Rq is a lot smaller, more than 10x by line count. So if it works just as well, I'd go with the simpler implementation.



I used both, ended with Rq. Freedom if choice can be good, but when you able to make decision. Variety of backends, storages force you to understand how each component really work and when you dig into details you find that they all not equivalent. But you just need something f--kng working and you don't want to pay another guy to maintain zoo of different products.

That is why I decided to use Rq, it is better to know limitations of something simple then know possibilities but not able to make choice.


That's very helpful, thanks!


There are many differences, but most notably rq spawns one process per task. Line count is a stupid metric, e.g recently our line count doubled because of our new coding style, also the majority of the source code is tests.


rq is like a luger pistol, light, simple, gets the job done.

celery is like a .50 caliber machine gun, industrial strength, lots of options, used for a variety of completely different use cases.

For simple stuff, use rq, but celery + rabbitmq work better if you have dozens and dozens plus worker nodes (ie: different servers), whereas with rq, you use redis, which could potentially be a SPOF, even with redis sentinel.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: