IME -- and I've just replaced a Postgres-only unlogged cache table with Redis -- it's not about the storage or caching, but about the locking. Postgres needs to acquire (or at least check) a lock for reading as well as writing. Although some optimizations have been done for mostly-reading-workloads (search for postgres fast-path locking), you'll still run into lock contention problems relatively quickly.