I am working on a new Manila Plugin that allows Managing Editors to set up robots.txt responses for their sites. Several years ago I added a robots.txt responder to my development server. It provided a generic "go away" message. I don't want search engines linking to applications that I am currently developing.
But now that I am hosting sites, I realize that my clients may want a visit by search engines. So, I need to enhance my responder to provide clients with the ability to set thier own robots.txt responses.
I got the basics working this afternoon. Now I need to develop the UI that lets ME's specify the robots.txt response for different paths on a site.
4:24:15 PM 
|
