I thought UDP is to TCP as C++ is to Java with regards to memory management. Meaning, TCP will do whatever it can to ensure packet integrity, while UDP pretty much leaves packet integrity up to you.
Is that correct? If so, then regardless of UDP actually appearing somewhat reliable, the spec makes no guarantees that it will be consistently reliable.
There's a triplet that loosely describes protocols: { Reliable, Stream, Connection} Each can be true or false. So TCP is { True, True/False, True } and UDP is { False, False, False }. (TCP can be stream or datagram)
There are (were) other protocols possible but these are about all that's supported/tested these days.
Also there's fragmentation/reassembly issues with UDP. If you send a 30K UDP datagram packet, it gets sent as many Ethernet-sized chunks. The receiver is supposed to put them back together. If one is lost, the entire thing is lost. And to boot, Linux didn't do UDP reassembly at all until recently (last year?)
Linux has always supported UDP reassembly. Quoting a manpage:
"By default, Linux UDP does path MTU (Maximum Transmission Unit) discovery. This means the kernel will keep track of the MTU to a specific target IP address and return EMSGSIZE when a UDP packet write exceeds it. When this happens, the application should decrease the packet size. Path MTU discovery can be also turned off using the IP_MTU_DISCOVER socket option or the /proc/sys/net/ipv4/ip_no_pmtu_disc file; see ip(7) for details. When turned off, UDP will fragment outgoing UDP packets that exceed the interface MTU. However, disabling it is not recommended for performance and reliability reasons."
And they ain't kidding, because there are plenty of layer 3 devices out there who play fast and loose with fragments of UDP packets; it used to be you could assume they'd drop anything that doesn't have a header, so by default UDP on linux doesn't encourage it.
It could be that distributions have been bundling sysctl.conf files with ip_no_pmtu_disc set to true, being that most modern layer 3 devices no longer mistreat UDP so badly; this may be what you're experiencing in the last year.
It was on Hacker News; UDP reassembly had never worked in Linux. Maybe I'm hallucinating that?
{ edit } It might have had something to do with UDP fragment reordering in Linux?
Maybe what it was (quoting from memory) is that the timeout for re-ordering/assembly started at 30 seconds (not 120 per the RFC) and only increased up to a limit of 180 when the kernel thought it wasn't something bursty like DNS; some kind of heuristic probably introduced to make NFS pre v4 work better. Maybe the queues were too small when MTUs could be 1500 bytes or less?
Not at all! There's no way to ask for delivery of a datagram of unknown size (which is pretty much the definition of a datagram-oriented protocol), as in UDP, RDP, SCTP, and other datagram-oriented protocols.
(No, not even the PSH bit guarantees this!)
This has implications when implementing application-layer protocols. In stream-oriented protocols like HTTP, which intersperse headers with payload, it is impossible to read the header from the OS's socket without also (possibly) reading part of the payload (unless you're reading a single octet at a time). Your application is thus forced to implement buffering on top of what the OS provides (if the OS does not itself provide a general pushback mechanism).
With an (ordered) datagram-oriented protocol, this is not a problem, as you can ask the OS to return a single datagram corresponding exactly to the header, and process it before retrieving any of the payload.
And to boot, Linux didn't do UDP reassembly at all until recently (last year?)
"Packet integrity" usually means something different than how you're using it: generally, it refers to whether the data in the packet arrives unchanged. By this definition, UDP has identical packet integrity as TCP (they use the same checksum, albeit optionally in the case of UDP), assuming the packet gets there at all.
What UDP doesn't guarantee is ordering and arrival of packets. Usually "channel reliability/integrity" or simply "reliability" is used to refer to this.
Is that correct? If so, then regardless of UDP actually appearing somewhat reliable, the spec makes no guarantees that it will be consistently reliable.