You are aware that the chromecast is really nothing more then a web browser in a stick? It has no storage for any thing other then the firmware to run the device.
A local network is more than capable of serving up the chromecast app. It's a restriction they've added to the device that it won't load it from a local network.
I think "restriction" is a poor choice: it's more like an architectural decision - that is exactly how they designed it.
I could equally say your computer is more than capable of running a self-hosted version of GMail, but "it's a restriction [Google] added" for them to run it on their servers.
I run a Plex media server on my laptop, which is already serving up content to my local network over http. It would be very simple for them to host their Chromecast app on the same local server and for the cast button to send that URL.
The Chromecast checks the URLs against a whitelist. Presumably as a security measure, which is a perfectly valid reason, but it's still a "restriction".
To allow people to run a self-hosted GMail, huge changes would need to be made to the code. To allow me to serve my own Chromecast apps, it would need to remove a whitelist check.