> you usually really want 'at least once delivery with acks and idempotent processing on the other side'.
Why? I'm pretty sure I really want (the illusion of) exactly-once delivery, and it seems to me that I can implement that pretty easily given at-least-once delivery. Why would I not want that?
But not 100%. At some point, a counter move on the delivery has to be stored... -somewhere-.
And sure you -can- make it very very close to EOD and for some subsets you can totally do EOD, but you are, realistically, better off with ALOD+Ack once it makes its way into a system. There's always that 'moving the counter' problem.
The upshot is, things tend to get faster, easier to code review, and simpler to test.
Pragmatically speaking, I've found devs are better able to handle ALOD+ACK than "Exactly once but because reality you might get a message that's doubled because you couldn't persist the ack".
And I'll note I'm possibly extra pedantic about this because I've had a month and a half of dealing with the fallout of people trusting low-code salesmen alongside gartner reports leading to a 'you people thought this was exactly once and it was not' sort of problem.
> I'm pretty sure I really want (the illusion of) exactly-once delivery
Do you know what idempotency is? This is exactly what he described.
Idempotency is important to prevent unwanted behaviour for duplicate actions. If you have "exactly-once", and accidentally execute the action twice that could cause problems.
Uh, what exactly do you think is "agreed" here? My claim is that idempotent processing can produce exactly-once delivery, and so the original claim that you "cannot have exactly-once delivery" is false.
Change your words from "delivery" to "processing" and you in alignment with reality.
Idempotent processing is exactly once processing. This is true. Delivery is from the sender's point of view, not the recipient. How the recipient processes the information, idempotently or not, is not the concern of the sender.
"Cannot have exactly-once delivery" is true and is as true as not dividing by zero. Read about the two general's problem. It is, quite literally, impossible to have exactly once delivery. Like, your friend is in the other room, and you shout "someone is at the door" - how do you _know_ your friend heard you? If you shout again, you are attempting another delivery. What do you do if your friend doesn't respond? In exactly once delivery, you guarantee they heard you in the other room. In exactly once processing, you can shout a couple of times until you hear them acknowledge.
You may think that this is not material and at the end of the day, as long as one thing is processed, then who cares? Well, you have to understand how delivery can fail otherwise you will handle the failure incorrectly. Is it safe to try again? If I say "transfer money from me to you", and I don't hear back, is it safe to again say "transfer money from me to you" again? Will I be double charged?
> Change your words from "delivery" to "processing" and you in alignment with reality.
You are not the first to say this, but so far no one has been able to explain what the difference between "delivery" and "processing" is. How do you do "delivery" (on a computer) without also doing (at least some) "processing"?
> Delivery is from the sender's point of view, not the recipient.
I don't see what difference that makes. In fact, I don't see why a "point of view" should enter into it at all. Whether a message has been "delivered" or not (whatever that actually turns out to mean) is (it seems to me) a property of the system, independent of anyone's point of view.
I gave two concrete examples. What about each of those examples is not landing?
One of shouting at your friend. You want to make sure your friend knows someone is at the door. Two: you tell your computer to transfer money and you don't want to be doubled charged.
More in depth: you click send money on a computer. The computer connects to another computer and sends data to it over an unreliable network. Computer A sends data over the network to B, just like you shouting to your friend you think is in the other room. Data can be lost / your friend might not hear you. Usually, the other computer says "acknowledged, I got your message" - and that is how know that B is moving money or your friend is getting the door. If A never hears back, should A try again? If B gets two requests to move $25, should that be deduplicated or was there two actual requests and $50 should be moved? To know how to solve that, you have to first admit that you might get 0, 1, or multiple messages delivered to B when A wants to send 1 message.
Read the two general's problem. It is the gateway to distributed computing.
Neither of which was on point because they both ignored how exactly-once delivery can be done.
> Read the two general's problem.
The 2GP is not on point because it's about achieving consensus, not exactly-once delivery. Achieving consensus is indeed impossible with unreliable messaging, but that has nothing to do with exactly-once delivery.
Why? I'm pretty sure I really want (the illusion of) exactly-once delivery, and it seems to me that I can implement that pretty easily given at-least-once delivery. Why would I not want that?
> The difference is subtle but important
Why?