四大机器人的代码

googlebot/2.1+(+http://www.google.com/bot.html)
mediapartners-google/2.1

google

baiduspider+(+http://www.baidu.com/search/spider.htm)

baidu

+yahoo!+slurp;+http://help.yahoo.com/help/us/ysearch/slurp
yahoo

sohu-search

sohu