Peter on Robots Txt

Wed, 25 Nov 2009 00:49:45 +0000

The plugin file doesn't actually affect a robots.txt file, although it does send some robots instructions as follows: Since anti-spam words expire (to mitigate abuse by bots), search engines would index the audio file portion for expired words. Therefore, to prevent search engines from indexing a bunch of “That anti-spam number is no longer valid” pages, those pages tell the search engines not to index them. If you’d like to turn that off, you can edit the variable $cas_displaytext['not_valid'] in the plugin file itself.

sammy on Robots Txt

Tue, 24 Nov 2009 06:34:56 +0000

Hello

We seem to be having almost 3000 robot txt instructions created by the plugin? Is this normal is there a way to halt this?

Peter's Useful Crap - Topic: Robots Txt

Peter on Robots Txt

sammy on Robots Txt