# Tells Scanning Robots Where They Are And Are Not Welcome # # User-agent: can also specify by name; "*" is for everyone # Disallow: disallow if this matches first part of requested path # # For now disallow all we can modify this as needed to allow certain crawlers. # User-agent: * Disallow: /clients Disallow: /cproj Disallow: /danielkmedia Disallow: /data Disallow: /Al Disallow: /cgi-bin Disallow: /db Disallow: /ebay Disallow: /Elle Disallow: /FutureData Disallow: /futuredata Disallow: /gathering Disallow: /IFMG Disallow: /jd Disallow: /jedman Disallow: /kdemail Disallow: /kdesvr Disallow: /lux Disallow: /mail Disallow: /mms Disallow: /personal Disallow: /RADO_images Disallow: /scgi-bin Disallow: /Take2 Disallow: /tdelete Disallow: /tko Disallow: /tonyhg Disallow: /vendors Disallow: /ZiffData Disallow: /ziffdata