ROBOTS 90
Robots.txt Guest on 13th May 2020 12:57:13 PM
  1.  
  2. User-agent: Googlebot
  3.  
  4. User-agent: Yahoo-slurp
  5.  
  6. User-agent: Msnbot
  7.  
  8. User-agent: bingbot
  9.  
  10. User-agent: Slurp
  11.  
  12.  
  13. Disallow: /cgi-bin/
  14. Disallow: /wp-admin/
  15. Disallow: /wp-includes/
  16. Disallow: /wp-content/
  17. Disallow: /trackback/
  18. Disallow: /index.php
  19. Disallow: /xmlrpc.com
  20. Disallow: /wp-login.php
  21. Disallow: /wp-content/plugins/
  22. Disallow: /comments/feed/
  23.  
  24. Disallow: /administrator/
  25. Disallow: /bin/
  26. Disallow: /cache/
  27. Disallow: /cli/
  28. Disallow: /components/
  29. Disallow: /includes/
  30. Disallow: /installation/
  31. Disallow: /language/
  32. Disallow: /layouts/
  33. Disallow: /libraries/
  34. Disallow: /logs/
  35. Disallow: /modules/
  36. Disallow: /plugins/
  37. Disallow: /tmp/

Paste is for source code and general debugging text.

Login or Register to edit, delete and keep track of your pastes and more.

Raw Paste

Login or Register to edit or fork this paste. It's free.