I always remove them. They're like referer headers. Where the visitor came from is just like any other info that might be useful to the site operator, but is really not any of their business unless the visitor voluntarily discloses it.