I’d heard similar complaints re: TB networking latency & jitter. Did some investigations and tuning on a pair of machines with USB4 ports connected via short TB5-rated cables. Eventually got the thunderbolt links to consistently beat the ether ones on both latency and jitter. And not just switched Ethernet either - even a direct Ethernet P2P link lost out to TB, though the difference there was small.