You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
yacy_search_server/source/de
theli 9a4375b115
*) robots.txt: adding support for crawl-delay
18 years ago
..
anomic *) robots.txt: adding support for crawl-delay 18 years ago