Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Google did index it, but only with an extra URL parameter: http://www.webmasterworld.com/robots.txt?view=producecode


Google has it in its pages - http://www.google.com/search?q=brett+tabke+blog and with a nice meta description

Brett Tabke experiments with writing a weblog in a text file usually read only by robots. Commentary on the world of search engine marketing. www.webmasterworld.com/robots.txt - 2k

It dawns on me today - the web is made unnecessarily heavy.





Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: