Can someone do an eli5 please? I've read the statements of the people involved but hardly contextualized, how this is involving rust if so?

Except sqs has a limit for unacked messages of 12 hours

If I were designing a system with tasks that took that long, I'd probably want to find a way to checkpoint along the way and break the task down into smaller steps. I'd hate to lose 11 hours and 59 seconds of work if the task fails near the end. You can then use multiple queues to pass the task along between steps.

Not experienced here but as a genuine interest can you tell what problems airflow solves that can't be handled by celery and rabbitmq?

I have not used celery + rabbitmq but I assume that combo is like sidekiq + redis, or any other job queue + worker system.

Airflow packages those things together and adds some additional features - UI with Graph, gantt, logs and other views of the workflow - Users and permissions - Places to store config - Mechanisms for passing small data between tasks - Various "sensors" for triggering workflows - Various operators that interact with common data-oriented systems (bigquery, snowflake, s3, you name it). These are basically libraries that expose a config-forward API.

Probably the main selling point is the pre-made operators, but in short it is a complete solution with bells and whistles that aligns itself with the data ecosystem.

An analogy is "can you tell what problems Django solves that can't be handled by wsgi and psycopg?" Nothing fundamentally different, but life is a whole lot easier with Django. Honestly if you're doing data engineering and you haven't spent time with a good DAG runner, you're doing yourself a real disservice.

My sibling comment did a good job explaining, but the UI + configurable storage + configurable triggers all out of the box make life a lot easier.

Django is easier when you want to do things only the "Django" way. However once you need something done differently it quickly shows its truly rigid and brittle self, and you'll find yourself fighting a great and challenging battle.

Perhaps unwittingly, you've hit upon people's exact frustrations with Airflow! :)

Expressing the problem you are trying to solve as a DAG is idiomatic in Airflow, but expressing your problem in terms of queue processing is idiomatic in Celery.

    a b c d vs. a (bc) d

They make different design decisions about what to surface via UX and what to make easy as a consequence of thinking of the problems in terms of different data structures.

Airflow with a celery backend is a pretty sweet combination. In that instance, airflow just gives you a nice scheduler to manage all the celery jobs.

Mt3 for me has a really good angle, expecially having small hands..but the top profile ( the side of the O on the keycap ) of the cap is too sharp and tiring after some time.

So what

Given how well the narco traffic is doing I would actually buy that phone

They know that if they actually capture them all they would be left without work. Instead their goal is to say "look they are even bigger than before we need even more funds!!"

The right immune system here would be to legalize the drugs and regulate them, just as we do with ethanol. Then you can remove the funding from war on drugs and related entities. Financing an army of bureaucrats and militants who feast on county's resources and undermine the democracy is not a terminal goal of a society.

We have a really small web agency (5 people) and I agree that we don't need to be in the office, that we share with other people of the same company of another division, to get good work done. We need to be together to boost ideas more. Technology and remote tools stigma don't allow you to substitute that for now.

Banks and insurance companies and IBM mainframe emulation will ensure Cobol will outlast Java.