> TimescaleDB relies on PostgreSQL’s WAL mechanism, which puts data into WAL buffers in RAM and periodically flushes them to WAL file. This means that the the data from unflushed WAL buffers is lost on power loss or on process crash.
That links to the manpage which says "The contents of the WAL buffers are written out to disk at every transaction commit". Maybe there's a missing "TransactionDB only commits periodically" that makes the quote above true, but any suggestion that PostgreSQL does not guarantee durability of committed transactions out of the box is incorrect.
A broader reason is: it talks about how WALs may be "lost / corrupted" before fsync. Then how the "write directly to SSTable" approach just loses recently added data, and "IMHO, recently written data loss on process crash has lower severity comparing to data corruption". But in general, I'd expect these databases to have a mechanism by which they don't apply a corrupted WAL (typically involving a strong checksum on WAL entries). So ultimately these are two ways of describing the same thing. If those databases really do apply corrupt/half-written/unflushed WAL entries and thus corrupt the previously committed data, yes, that's very interesting, but the smoking gun is missing. The article is either wrong or incomplete.
One specific reason:
> TimescaleDB relies on PostgreSQL’s WAL mechanism, which puts data into WAL buffers in RAM and periodically flushes them to WAL file. This means that the the data from unflushed WAL buffers is lost on power loss or on process crash.
That links to the manpage which says "The contents of the WAL buffers are written out to disk at every transaction commit". Maybe there's a missing "TransactionDB only commits periodically" that makes the quote above true, but any suggestion that PostgreSQL does not guarantee durability of committed transactions out of the box is incorrect.
A broader reason is: it talks about how WALs may be "lost / corrupted" before fsync. Then how the "write directly to SSTable" approach just loses recently added data, and "IMHO, recently written data loss on process crash has lower severity comparing to data corruption". But in general, I'd expect these databases to have a mechanism by which they don't apply a corrupted WAL (typically involving a strong checksum on WAL entries). So ultimately these are two ways of describing the same thing. If those databases really do apply corrupt/half-written/unflushed WAL entries and thus corrupt the previously committed data, yes, that's very interesting, but the smoking gun is missing. The article is either wrong or incomplete.