In an ideal world where your OS has no vulnerabilities, your application stack has no vulnerabilities and you never make configuration errors I would agree.

However, as far as computer security goes it is very far from an ideal world.

See ghshephard's comments about "Defense in Depth" above.