Archive of UserLand's first discussion group, started October 5, 1998.

---

Re: It's about editing

Author:	Jim Correia
Posted:	9/12/2000; 8:26:47 AM
Topic:	Discuss by email?
Msg #:	21232 (In response to 21229)
Prev/Next:	21231 / 21233

A "bot" that's monitoring changes here might catch an unedited message and broadcast it.

A search engine like Google might also pick up these changes.

Why not just add /robots.txt with the contents:

User-agent: *
Disallow: /

All robots are now disallowed from looking at anything on the site. If someone uses a robot at that point then they are either explicitly going against your wishes, don't adhere to the Robots.txt standard, or both. In any case, they have little excuse for having done so, and your wishes and policy are now clear.

Jim

---

There are responses to this message:

---

This page was archived on 6/13/2001; 4:56:39 PM.

© Copyright 1998-2001 UserLand Software, Inc.