added robots.txt
2007-08-09 @ 19:08#
i just added a simple robots.txt file to the root of my site. nothing big, just disallow entries for some of the scripts, images, etc. that use on the site. i'll start trolling my logs files and consider disallowing more (including whole bots, if necessary) and time goes on.
i'll also start digging up the details on generating a sitemap and adding google tracker to the pages, too. all in good time.