Merit Network
Can't find what you're looking for? Search the Mail Archives.
  About Merit   Services   Network   Resources & Support   Network Research   News   Events   Home

Discussion Communities: Merit Network Email List Archives

North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Spammer web harvesting tool countermeasures

  • From: Jon Stevens
  • Date: Thu Oct 30 22:02:35 1997

"Deepak Jain" <deepak@jain.com> said the following at 10/30/97 6:56 PM:

>And wouldn't we, in turn, see some kind of problems arise with legitimate 
>search engines because of this?

If you downloaded it and looked at it, you would have noticed that it 
follows search engine guidelines by adding the appropriate <META> tag to 
the HTML as well as the fact, that you can also use the robots.txt file 
to block it.

Of course this also breaks down if spammer robots actually follow the 
rules...but how many of those do you think that there are? ;-)

-jon

Jon (no h) S. Stevens
Web Engineer
j@clearink.com
Clear Ink and The Internet Weather Report
<http://www.clearink.com/> | <http://www.internetweather.com/>





Discussion Communities


About Merit | Services | Network | Resources & Support | Network Research
News | Events | Contact | Site Map | Merit Network Home


Merit Network, Inc.