Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You know, I'm somewhat suprised Google isn't actually crawling tor. Or maybe they are, who knows?


There are a few cases I've seen of Google actually picking up onion.to links but they tend to rank terribly, maybe for their name


Yeah, there are lots of onion.to sites indexed. Maybe the ranking is to do with the fact they experience so much downtime?


And they're slow! Google has stated that page speed is a ranking factor which means that hidden services will be hit hard (if they were actually crawled) and hidden service proxies would get that and then some.


I recall the engine once actually picked up a genuine .onion domain (it was the Hidden Wiki, IIRC).

It's safe to assume they do crawl the Tor space.


If they just pick up the .onion domain, they will try to crawl it and determine that the URL is incorrect, as .onion domains do not exist in the standard DNS stack. The only we for them to crawl Tor is if they went out of their way to crawl it (in which case they may or may use domains found on the standard web), or if they crawl domains like .onion.to, which behave like normal sites.


onion.to is not "crawling tor sites". It's crawling someone's public web proxy's copies.


.onion crawling was a 20% time project in 2009/2010 (IIRC). I'm sure you could find the announcement if you searched for it. Google does crawl and index.


https://encrypted.google.com/search?hl=en&q=site%3A.onion

No results here. If they are crawling, the results are not shown.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: