# Format of this file is specified at www.robotstxt.org # # Disallow works like a match-string on the URL. # # If it ends in a '/' then it will disallow an entire directory # because all subdirectories and files will contain those chars # within their URL. # # If it does not end in a '/', then it will match anything, # file or directory that contains those chars within their URL. # User-agent: * Disallow: /cgi-bin/ Disallow: /cp/ Disallow: /images/ Disallow: /modlogan/ Disallow: /webalizer/ Disallow: /downloads/ # The following line will disallow the entire forum directory. #Disallow: /forum/ # The following disallows directories or files that start with these chars. # In this case, since there are no subdirectories with these matches, then # it is disallowing access to these particular php files. # Disallow: /forum/mcp Disallow: /forum/memberlist Disallow: /forum/posting Disallow: /forum/search Disallow: /forum/ucp