Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It's not uncommon for lightly trafficked sites to have 90% or more of their resources taken up by bots of various kinds. Remember Google isn't the only search engine and there are many crawlers that aren't search engines at all.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: