Merit Network
Can't find what you're looking for? Search the Mail Archives.
  About Merit   Services   Network   Resources & Support   Network Research   News   Events   Home

Discussion Communities: Merit Network Email List Archives

North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

RE: Google Crawler

  • From: Andy Ellifson
  • Date: Fri Jan 03 12:12:11 2003


Thank you!

--- Mike Damm <MikeD@irwinresearch.com> wrote:
> 
> http://www.google.com/bot.html for issues with the crawler.
> mailto:googlebot@google.com will get you a human bean to talk to.
> Normally
> when there is a problem with their robot, they are pretty responsive.
> 
> 	-Mike
> 
> ---
> Michael Damm, MIS Department, Irwin Research & Development
> V: 509.457.5080 x298 F: 509.577.0301 E: miked@irwinresearch.com
> 
> 
> -----Original Message-----
> From: Andy Ellifson [mailto:andy@ellifson.com] 
> Sent: Friday, January 03, 2003 8:45 AM
> To: nanog@merit.edu
> Subject: Google Crawler
> 
> 
> We are a domain registrar and we host/park over 750,000 domain names.
> 
> Every now and then the Google Crawler decides to bury the machines
> that
> host our 'parked' domain pages.  We use robots.txt but that doesn't
> help under these circumstances.  I have tried sending a message to
> Google using their web site.  They don't have a NOC entry on
> puck.nether.net either.  Our only alternative right now is to block
> the
> crawler at the router level.
> 
> Does anbody have a contact at Google or is anyone at Google
> listening?
> 
> Thanks!
> Andy Ellifson





Discussion Communities


About Merit | Services | Network | Resources & Support | Network Research
News | Events | Contact | Site Map | Merit Network Home


Merit Network, Inc.