Przeglądaj źródła

Tell search engine bots not to crawl our git mirrors.

git-svn-id: file:///srv/caca.zoy.org/var/lib/svn/web/trunk@4307 92316355-f0b4-4df1-b90c-862c8a59935f
master
sam 16 lat temu
rodzic
commit
912397749e
1 zmienionych plików z 4 dodań i 0 usunięć
  1. +4
    -0
      www/robots.txt

+ 4
- 0
www/robots.txt Wyświetl plik

@@ -9,6 +9,10 @@ Disallow: .svn
Disallow: /cgi-bin/trac.cgi
Disallow: /log

# Don’t crawl git repos
Disallow: /git/*.git/*
Disallow: /git/*.git.broken/*

# "This robot collects content from the Internet for the sole purpose of
# helping educational institutions prevent plagiarism. [...] we compare
# student papers against the content we find on the Internet to see if we


Ładowanie…
Anuluj
Zapisz