Bladeren bron

Tell search engine bots not to crawl our git mirrors.

git-svn-id: file:///srv/caca.zoy.org/var/lib/svn/web/trunk@4307 92316355-f0b4-4df1-b90c-862c8a59935f
master
sam 16 jaren geleden
bovenliggende
commit
912397749e
1 gewijzigde bestanden met toevoegingen van 4 en 0 verwijderingen
  1. +4
    -0
      www/robots.txt

+ 4
- 0
www/robots.txt Bestand weergeven

@@ -9,6 +9,10 @@ Disallow: .svn
Disallow: /cgi-bin/trac.cgi
Disallow: /log

# Don’t crawl git repos
Disallow: /git/*.git/*
Disallow: /git/*.git.broken/*

# "This robot collects content from the Internet for the sole purpose of
# helping educational institutions prevent plagiarism. [...] we compare
# student papers against the content we find on the Internet to see if we


Laden…
Annuleren
Opslaan