Semantic versioning is an illusion. It's a human-managed attempt to convey things about the surfaces and behaviors of software systems. Best case, it isn't completely misleading and a waster of everyone's time.
There is no perfection here, but the correct way to reason about this is to have schema-based systems where the surfaces and state machines are in high level representations and changes can be analyzed automatically without some human bumping the numbers.
Not entirely, there are other reasons too
But we should respect semantic versioning. Python is a dreadful sinner in that respect.