You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

orbiter e81be7d4f2 added many missing user-agent declarations for yacy http client connections. ... the most important fix was the addition of the yacybot user-agent for robots.txt loading, because web masters look for that access to see if the crawler behaves correctly. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4968 6c8d7289-2bf4-0310-a012-ef5d649a1542		17 years ago
..
bzip	…
doc	…
gzip	…
mimeType	…
odt	added many missing user-agent declarations for yacy http client connections.	17 years ago
pdf	…
ppt	…
ps	…
rpm	added many missing user-agent declarations for yacy http client connections.	17 years ago
rss	- organize imports	17 years ago
rtf	…
sevenzip	…
swf	…
tar	- organize imports	17 years ago
vcf	added many missing user-agent declarations for yacy http client connections.	17 years ago
xls	…
zip	- organize imports	17 years ago
AbstractParser.java	…
Parser.java	…
ParserException.java	…
ParserInfo.java	…