List of All User Agents for Top Search Engines

Here is a working list of all user agents for the major, top search engines. I use this information frequently for my plugins such as Blackhole for Bad Bots and BBQ Pro, so I figured it would be useful to post the information online for the benefit of others. Having the user agents for these popular bots all in one place helps to streamline my development process. Each search engine includes references and a regex pattern to match all known user agents.

Search Engines

(In alphabetical order)

• AOL.com
• Baidu
• Bingbot/MSN
• DuckDuckGo
• Google
• Teoma
• Yahoo!
• Yandex

Note: References/links provided at the end of the post.

AOL.com

Mozilla/5.0 (compatible; MSIE 9.0; AOL 9.7; AOLBuild 4343.19; Windows NT 6.1; WOW64; Trident/5.0; FunWebProducts)

Mozilla/4.0 (compatible; MSIE 8.0; AOL 9.7; AOLBuild 4343.27; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)

Mozilla/4.0 (compatible; MSIE 8.0; AOL 9.7; AOLBuild 4343.21; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E)

Mozilla/4.0 (compatible; MSIE 8.0; AOL 9.7; AOLBuild 4343.19; Windows NT 5.1; Trident/4.0; GTB7.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)

Mozilla/4.0 (compatible; MSIE 8.0; AOL 9.7; AOLBuild 4343.19; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E)

Mozilla/4.0 (compatible; MSIE 7.0; AOL 9.7; AOLBuild 4343.19; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E)

Plus many more older versions/variations (see links for details)..

Regex to match all of these user-agent strings: aolbuild

Baidu

Baiduspider
Baidu Web Search	
Baidu Image Search	
Baiduspider-image
Baidu Mobile Search	
Baiduspider-mobile
Baidu Video Search	
Baiduspider-video
Baidu News Search	
Baiduspider-news
Baidu Bookmark Search	
Baiduspider-favo
Baidu Union Search	
Baiduspider-cpro
Baidu Business Search	
Baiduspider-ads

Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Baiduspider+(+http://www.baidu.com/search/spider_jp.html)

Baiduspider+(+http://www.baidu.com/search/spider.htm)

Regex to match all of these user-agent strings: baidu

Bingbot/MSN

Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)

Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)

Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 530) like Gecko (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)

Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)

msnbot/2.0b (+http://search.msn.com/msnbot.htm)

msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)

Mozilla/5.0 (compatible; adidxbot/2.0; +http://www.bing.com/bingbot.htm)

Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; adidxbot/2.0; +http://www.bing.com/bingbot.htm)

Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 530) like Gecko (compatible; adidxbot/2.0; +http://www.bing.com/bingbot.htm)

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b

Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 530) like Gecko BingPreview/1.0b

Regex to match all of these user-agent strings: bingbot, bingpreview, msnbot

DuckDuckGo

DuckDuckBot/1.0; (+http://duckduckgo.com/duckduckbot.html)

Regex to match this user-agent string: duckduckgo

Google

Googlebot/2.1 (+http://www.googlebot.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Googlebot/2.1 (+http://www.google.com/bot.html)

Googlebot-News

Googlebot-Image/1.0

Googlebot-Video/1.0

SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

[various mobile device types] (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html)

Mediapartners-Google

AdsBot-Google (+http://www.google.com/adsbot.html)

Regex to match all of these user-agent strings: adsbot-google, googlebot, mediapartners-google

Teoma

Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html)

Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)

Mozilla/2.0 (compatible; Ask Jeeves/Teoma)

Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)

Regex to match all of these user-agent strings: teoma

Yahoo!

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

Regex to match this user-agent string: slurp

Yandex

Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)

Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexBot/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexAccessibilityBot/3.0; +http://yandex.com/bots)

Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexMobileBot/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexDirectDyn/1.0; +http://yandex.com/bots

Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexVideo/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexMedia/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexBlogs/0.99; robot; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexFavicons/1.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexWebmaster/2.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexPagechecker/1.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexImageResizer/2.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YaDirectFetcher/1.0; Dyatel; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexCalendar/1.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexSitelinks; Dyatel; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexMetrika/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexAntivirus/2.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexVertis/3.0; +http://yandex.com/bots)

Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)

Regex to match all of these user-agent strings: yandex

More?

If you know of a search engine that you feel deserves to be included in this list, feel free to leave a comment. Thanks! :)

References

AOL

• http://www.useragentstring.com/pages/useragentstring.php?name=AOL

Baidu

• https://help.baidu.com/question?prod_id=99&class=0&id=3001
• http://www.useragentstring.com/pages/useragentstring.php?name=Baiduspider

Bing/MSN

• https://www.bing.com/webmaster/help/which-crawlers-does-bing-use-8c184ec0
• http://www.useragentstring.com/pages/useragentstring.php?name=Bingbot
• http://www.useragentstring.com/pages/useragentstring.php?name=msnbot

DuckDuckGo

• http://www.user-agents.org/cgi-bin/csv2html.pl?data=allagents.csv&template=detail.html&match=%5Cbid_a_f_290308_2%5Cb

Google

• http://www.useragentstring.com/pages/useragentstring.php?name=Googlebot
• https://developers.whatismybrowser.com/useragents/explore/software_name/googlebot/

Teoma

• http://www.useragentstring.com/pages/useragentstring.php?name=Teoma

Yahoo

• https://help.yahoo.com/kb/SLN22600.html
• http://www.useragentstring.com/pages/useragentstring.php?name=Yahoo%21+Slurp

Yandex

• https://yandex.com/support/webmaster/robot-workings/check-yandex-robots.html
• http://www.useragentstring.com/pages/useragentstring.php?name=YandexBot

About the Author

Jeff Starr = Fullstack Developer. Book Author. Teacher. Human Being.