## robots.txt for http://pan.stanford.edu/index.html/## $Id: robots.txt, 2003/04/25  william akey Exp $## applies to all robots and disallows files that have an expiration date in the meta tagUser-agent: * Disallow: /Out-Of-Date# applies to all robots and disallows indexing of files in various folders User-agent: *Disallow: /devDisallow: /_privateDisallow: /archiveDisallow: /templates