Plus, couldn't you have multiple layers of SVG, and have the javascript respond by hiding/showing detail layers? SVG is more javascript-friendly than PNG so there's opportunity to have one SVG that gets tweaked by your js instead of multiple raster files.
Correct. But PNG works and is widely deployed today. Considering SVG fails at one of the primary use cases where it could have an edge over PNG, it's no surprise few care for it.