1 |
1 |
1 |
1 |
1 |
2 |
2 |
1 |
3 |
|||||||||||||||
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| None |
| 1 | *.crawl.commoncrawl.org | |
| *.crawl.sogou.com | ||
| 3 | *.mj12bot.com | |
| 101.33.66.34 | ||
| *.ahrefs.net | ||
| *.googlebot.com | ||
| 43.130.141.193 | ||
| 49.51.233.95 | ||
| 182.42.105.144 | ||
| 43.130.57.46 | ||
| 119.28.89.249 |
| 1 | Mozilla/5 | |
| 2 | Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07) | |
| CCBot/2.0 (https://commoncrawl.org/faq/) |
| 1 | Etc. | |
| 2 | Linux |