allo
28f890ee3e
Important SECURITY FIX!
...
The UserDB Passwordcheck was buggy since 1138, any password allows access with the rights of
the user!
USE SVN or disable all Users(take away all Rights)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1246 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
0f1212feb9
userDB.hasAdminrights to check adminRights.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1245 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
b3dca06bb1
added location column to network pages.
...
The location is computed from the userAgent string of connecting peers.
Therefore this information is not available right after start-up.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1241 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
5b3e01bd3c
avoid division by zero when importing very small indexes (<100 entries)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1238 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
d0c2c67f4c
Update YaWoStat version.
...
See http://www.yacy-forum.de/viewtopic.php?p=14215#14215 for possible use.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1236 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
60fc34d9e8
iterartion bug:
...
http://www.yacy-forum.de/viewtopic.php?p=14182#14182
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1235 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
6c91921e9b
*) yacyWiki: more versatile distinction between real external links/links that point to DATA/HTDOC
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1232 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
b7f9adc2c9
new filters added
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1231 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
9b617bcb65
*)compression of -domlist now optional (-format zip
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1230 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
2bd4a66133
*)-domlist now creates a zipped txt-file.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1229 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
45ebe477ee
Bugfix for http://www.yacy-forum.de/viewtopic.php?p=14175
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1224 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
eb0f511767
*) yacyWiki: easy way to link to files on local peers (external links)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1223 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
79667a172e
*) Bugfix for additional parser problem
...
See: http://www.yacy-forum.de/viewtopic.php?p=14146#14146
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1221 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
b6be828d15
*) Bugfix: Share subdirectory couldn't be views because of LinkageErrors
...
See: http://www.yacy-forum.de/viewtopic.php?t=1634
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1218 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
c4dd39e8f2
fixing/changing wiki image inline code
...
for local images hosted within DATA/HTDOCS
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1217 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
80b3e85356
*) yacyWiki: Images in a share on the same peer can be addressed by path and name only, no complete URL needed anymore. (As requested by theli in http://www.yacy-forum.de/viewtopic.php?t=809 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1215 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
8c594841a8
*) Bugfix for incorrectly indexing of URLs that were requested with Cookies in the
...
Request header
See: http://www.yacy-forum.de/viewtopic.php?p=14077
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1214 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
b5d02d649a
fixed bug caused strange search result behaviour
...
(results from remote peers had not been saved propery after search)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1213 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
4500506735
fixed some bugs concerning url entry retrieval and intexControl interface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1212 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
83a34b838d
* added Object allocation monitor on performanceMemory page
...
* added some final statements
* changed shutdown sequence order
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1211 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
4ff3d219e8
increased delay for cacheScan start and slowed down scan process
...
to provide more time to other tasks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1210 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3031903d50
re-design of RAM cache flush into assortment cluster
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1209 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
676e384b45
changed synchronization to fix 'contains loops' bug in kelondroTree
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1208 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3cc02fe749
enhanced buffered write by combination of several chunks into one chunk
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1206 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0c762daf4b
better startup failure handling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1205 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f27f9ecf15
* activated write buffer for databases.
...
This should increase IO performance and reduce HD activity
* bugfixes for new exception-on-failure policy
* bugfixes for new IOChunks
* new Object pool for database write-buffer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1204 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
c59d1b2f5e
- Tests with write buffer (new class kelondroBufferedIOChunks, not yet active)
...
- minor bugfixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1203 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ca7e60279d
introduced IOChunks.
...
This is an abstraction of chunked IO-processes.
It will help to synchronize access to IO-ports.
Furthermore this is a preparation for upcoming chunk-oriented write buffers.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1201 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bb79fb5d91
- changed handling of error cases retrieving urls from database
...
(no more NULL values are returned, instead, an IOException is thrown)
- removed ugly damagedURLS implementation from plasmaCrawlLURL.java
(this inserted a static value into the Object which is not really a good style)
- re-coded damagedURLS collection in yacy.java by catching an exception and evaluating the exception message
to do:
- the urldbcleanup feature must be re-tested
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1200 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
fed92d364b
introduced USAGE object for counter synchronization in kelondroRecords
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1199 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
35cf6712b2
*) fixes for httpd
...
- don't send Body on HEAD requests
- don't send a Last-modified: date, that is later then Date:
- Use Cache-control instead of Pragma with HTTP/1.1
- don't send header with HTTP/0.9
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1198 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
5a627a690f
*) Extending hydrox urlDbCleanup function
...
- now the function tries to correct the URL first
- if the url can not be corrected it will be deleted
See: http://www.yacy-forum.de/viewtopic.php?p=13898
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1197 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
e7d16ef831
*) Corrections in jMimeMagic MagicRule-file to detect some special rss feeds
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1196 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3086e38bb1
added getRemoved method on demand from theli for migration purpose
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1195 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
386d9e45d8
*) Bugfix for code cleanup
...
- Code must be in finally block, otherwise it does not work if an error occurs!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1193 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
356231b033
changed only for testing with new hastable implementation.
...
Core funktions are unchanged.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1192 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
5a1d45715d
*) Bugfix for parser configuration bug
...
- it was not possible to disable all parsers
See: http://www.yacy-forum.de/viewtopic.php?t=1579
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1191 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1041f4946a
experiments with locks to remove synchronization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1189 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
288b2c353e
debugging of kelondroRecords ... possibly synchronization bugs found
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1188 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
a1061495d4
Fixed some spelling mistakes and added some text which (should) make it easier to understand the options.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1187 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0cdc58aaea
fixed indexing of local domains.
...
see http://www.yacy-forum.de/viewtopic.php?p=13680#13680
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1186 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
76b97e4d2a
integrated geo-snap DBStressTest.java in dbtest
...
this is still beta.
It uses serverInstantThreads instead of Java 1.5 code for multiple threads
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1185 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
ec1202edbe
*) Fixes for httpd
...
- Fix for local timezone in http header
See: http://www.yacy-forum.de/viewtopic.php?t=836
- Allow static content to be cached by browser
See: http://www.yacy-forum.de/viewtopic.php?t=1311
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1184 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
e1c2d8ec5f
*) Speedup "removed from queue"
...
See: http://www.yacy-forum.de/viewtopic.php?p=13442#12188
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1183 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
96930f0d2b
*)added function to removed malformed URLs from urlHash.db
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1182 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
d007d14905
re-insert of migrateSwitchConfigSettings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1180 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
8862b6ba4b
*) Corrections for code cleanup 1175
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1179 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0e88ba997e
* added option to generate url-lists as plain text file or in html
...
* modified generation of dom-lists so that they can be also generated as html
these options can be called as:
java -classpath classes yacy -domlist -format html
java -classpath classes yacy -domlist -format html .
java -classpath classes yacy -domlist -format text .
java -classpath classes yacy -urllist -format html .
java -classpath classes yacy -urllist -format text .
the -format <type> can be ommitted. The text is default
a home path can be asserted or omitted at the end of the parameters
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1178 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
13fdebc50d
added authentication for link deletion in search result
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1177 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
37f88b4017
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1176 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ec2b39c1ce
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1175 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
76618442e0
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1173 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
8f1f2daa5e
implemented interactive link deletion of search results.
...
next steps: attach voting and restrict to administrator
to see the deletion button, move the mouse pointer to the left of a search result
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1172 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
6d0f7e6988
*) Adding missing file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1171 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
44fa94ac52
*) Modifications for dbImport functionality
...
- dbImporter threads are now shutdown by the switchboard on server shutdown
- adding possibility to pause a importer thread via GUI
- Bugfix for abort function
See: http://www.yacy-forum.de/viewtopic.php?p=13363#13363
*) Modification of content parser configuration
- now it's possible to configure which parsers should be enabled for the proxy,
crawler, icap, etc. separately
-
*) htmlFilterContentScraper.java
- adding regular expression to normalize URLs containing /../ and /./ parts
*) httpc.java
- adding functionality to unzip gzipped content
- requested by roland: should be used later to allow gzipped seed lists
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1170 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
28ddba8813
added vote-button images
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1169 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
dc778659fb
fixed problem with time-out during result joint which caused OR behavior instead of AND beahvior
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1167 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3d8a5ae652
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1166 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
ae6a4650bc
reenabling debugMode (60 Seconds timeout for *all* http connections)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1165 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
bb1f73ec15
*) Bugfix for code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1164 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
7920e1547d
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1163 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
972b8ddd2a
*) minor changes: using constants properly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1162 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
1c3750de57
*) Bugfix for code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1161 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
64478b1f02
*) Adding possibility to delete crawler queue entries using regular expressions
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1160 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1d6a6d1f85
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1159 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a04930f025
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1158 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
90b0eb144e
just a typo...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1155 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
129b15f3e1
*) Correcting logging output of db importer thread
...
See: http://www.yacy-forum.de/viewtopic.php?t=1555
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1154 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
7028412c54
*) Bugfix for NullpointerException in Java Finalizer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1153 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
420d56ce79
extended db-testing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1152 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ecf765ec33
temporary fix to make jrpm extension compilable with my netbeans environment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1151 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
8c0d982191
1.) Fix from Martin (he ist not at home...)
...
2.) Search button now gets blocked if clicked and JS active.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1150 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
b604654c25
*) Adding possibility to do a settings migration on yacy startup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1149 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
8ed0aaae8d
*) Adding content Parser for RPM Files
...
- at the moment only the metadata is extracted
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1147 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
0f769215b5
*) urlRedirector now uses http head requests to determine the mimetype
...
of a resource before it checks if a URL has to be rejected or not.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1146 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
b62c36d2ab
*) Bugfix for urlRedirector
...
- media extension was not parsed correctly
*) Displaying urlRedirector connections in Connections_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1145 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
8f4259f1e3
md5Auth
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1144 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
818d37ce44
*) Removing getSimpleName
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1143 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
b35c5a48bf
*) First version of urlRedirector.pl script
...
- with this script it's possible to pass URLs from squid
to yacy via the squid redirector interface
- this URLs are then used by YaCy to feed the crawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1141 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
bdf30117c1
*) Redesign of parser configuration
...
- restructuring of mimeTypes based on the parsers
- displaying parser usage count
- displaying human readably parser names
- displaying parser version information
*) httpdFileHandler.java
- adding possibility to support "streaming" servlets
which are special servlets that can communicate with
the client via the connection streams autonomous
- the name of these new servlet types must end with the
file extension .stream
- this feature will be needed by the yacy ScreenSaver
class to fetch statistic data from the peer without the
need to reconnect to the server all the time
*) Adding human readable names and version information for
all supported parsers
*) plasmaParser.java
- adding new structure to store parser statistic data
*) Adding openDocument parser
- can be used to parse odt files
*) jmimemagic
- adding rules to detect openDocument formats properly
*) serverLog.java
- adding functions that can be used to query if a given
logging level is enabled or not.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1140 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
b86d1085e2
passwordAuth
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1138 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
5bf70e6e14
*) Bugfix for serverClassLoader.java
...
- Classloading didn't work properly if there are multiple classes with the same name
- This could occure because the yacy servlets have no package name defined and therefore
are all in the same (default) package.
*) Bugfix for Duplicated Class Error
See: http://www.yacy-forum.de/viewtopic.php?t=1341
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1135 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
d4ac3e25b1
*) Bugfix for file system link bug during detection of invalid URLs
...
See: http://www.yacy-forum.de/viewtopic.php?p=13301
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1134 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
adf75bc9fa
better logging for invalid file path detection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1133 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
40621a5663
anhancements in ranking preparation and fixed problem with parser/mime recognition
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1132 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
c650b112ea
*) Bugfix for relative URL Bug in Crawler
...
See: http://www.yacy-forum.de/viewtopic.php?p=13266#13266
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1130 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
7e670894d9
*) Suppressing stackTraces in proxyError message for "connect timed out" errors
...
See: http://www.yacy-forum.de/viewtopic.php?t=1504
*) Increasing default http client timeout
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1129 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
4e73035aef
*) Bugfix for "too many open files" during index distribution
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1128 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
d8afe60e07
Bugfix for last Bugfix ;-).
...
host/port were set to originaladdress instead of the correct values for the new Url.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1126 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1b656f6b31
correction of bug from svn 1123
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1125 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f57e2d67f5
shortened network overview (less columns fit easier on page)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1124 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
24d15eb0e8
moving the redirector code
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1123 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
787c368696
synchronized redirectors and using the port.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1122 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
85282b1d98
enhanced YBR recognition and search result heuristics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1121 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
4776f3f815
squid like redirctors
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1120 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
b9cc9029e3
added ybr selection for remote search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1119 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0e25020f51
added first generation and usage of YBR index-files. Enhanced overall ranking of search results.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1118 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
52a0237bf2
using Filetemplates for #[metas]# and other static includes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1116 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
90d6c6223b
*) Adding color codes to network graphic legend
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1114 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bfe51c7228
added generation of domain-list
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1112 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0ec54d9c5f
enhanced CR-file handling and added first RCI-evaluation tests
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1110 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
99fb26e499
*) Suppressing stackTraces in proxyError message for harmless errors
...
See: http://www.yacy-forum.de/viewtopic.php?t=1504
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1108 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
c2fe3a1670
*) Updating jMimeMagic Ruleset
...
- to detect some special formated html documents correctly
- adding rule to detect vCards
*) plasmaParser now supports parsing of files that have a supported fileExtension
but a unsupported mimeType because the webserver has set it incorrectly to text/plain
*) Adding vCard new Parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1107 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
88e3234393
fine-tuning of rci-generation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1105 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a12759c1bf
first try to implement a rci-computation from cr-files
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1103 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
4a8e8f269e
refactoring of cr-processing; new kelondro class to handle the attribute file format
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1100 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
24dc0e0760
implemented cr-file processing and further transmission steps
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1099 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
5cd1e9cef4
*) fixed some dirty code, idea analog to bit stuffing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1098 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
022530df7e
small bugfix in kelondroTree
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1097 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
9d9a87f445
limited htcache storage length
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1096 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
8e308cf50e
*) Possibility to change the server port on-the-fly.
...
- Now it's possible to change the server port without the need to restart the whole server.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1089 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
d0dfccdb77
*) Making CrawlStacker pool configurable via GUI and config file
...
See: http://www.yacy-forum.de/viewtopic.php?t=1448
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1087 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
3631cb1f6d
*) deleting empty entities during index selection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1086 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
ca26aab9b1
*) More debugging output for migrateWords
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1085 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
9b35ae9027
*) Correcting wrong % values on IndexTransfer_p page
...
See: http://www.yacy-forum.de/viewtopic.php?p=12646
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1084 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
e6bf9d90a5
*) Fixing Problems with MalformedURLs during Word Selection
...
- removing (lurl.toString() == null) comparison because toString() is never null
- adding (lurl.url() == null) condition because url() is null if we have selected a word entry with
a malformed URL
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1083 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
86a9210264
*) indexing queue slots are now configurable via config file
...
See: http://www.yacy-forum.de/viewtopic.php?t=1480
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1081 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
3c11d7b81c
*) Bugfix for minimizeUrlDB
...
- function didn't work correctly because of new url hash structure
See: http://www.yacy-forum.de/viewtopic.php?p=12753#12753
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1080 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
9913049009
fixed outOfMemory bug caused by loops in kelondroTree during enumeration
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1079 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
f8f9d509d5
removed dead Code
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1078 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
5918d3985e
removed Debug Statements
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1076 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
bbb936b9ea
*) Bugfix for not human readable content of PDFs while viewing the URL Content via GUI
...
- This Bug also affects the snippet generation on non html/text documents
See: http://www.yacy-forum.de/viewtopic.php?t=1472
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1075 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
445e3a620f
*) Avoid rejecting of html content by the crawler when the file extension is not set properly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1074 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a3fd0069f5
fixed bug in kelondroTree node iteration (rotation did not work)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1072 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
fd58d5f8e6
*) Adding possibility to specify the interface / IP-Address where YaCy should bind to.
...
- e.g. Port = 192.168.0.1:8080
Port = #eth0:8080
Port = 8080
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1071 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
889de6686c
Migration in yacyVersion
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1070 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
444a5a9368
*) Bugfix for Entries with null url in GlobalQueue
...
See: http://www.yacy-forum.de/viewtopic.php?p=12675#12675
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1069 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
3bbb932fa2
Bugfix for nullpointerexception.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1067 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
4976f97956
Logout Function on User.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1065 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
b74e990c74
better Logout function
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1064 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
ebac51df52
restore defaultRemoteProfile
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1063 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
4daa4b00e9
logout function
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1062 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
5778428455
move cutUrlText to nxTools,
...
max length from URLs(title) on searchpage now 120 chars
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1060 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
9158845c3b
bugfix for snippet text null bytes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1059 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
6e81f2580d
try to fix bug with storage of settings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1058 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f763923e0a
added missing files for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1057 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
9649d08171
*) More tolerant robots parser
...
- converting tabs to spaces
- cutting of '*' in the disallow section
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1056 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
79818a320f
introduced citation-rank transmission protocol and activate transport for anonymisation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1055 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
6f8d7d3bcd
*) Adding first version of YaCy bookmarklet
...
- this can be used to easily crawl a webpage which is currently opened in the browser
- to get the bookmarklet javascript simply call http://localhost:8000/QuickCrawlLink_p.html
and drag and drop the link shown to your Browsers Toolbar/Link-Bar.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1053 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
7e0647f692
*) Bugfix for userDB usage during authentication
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1052 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
93cadb47b9
*) More tolerant robots parser for robots-files which missing empty lines between rule blocks
...
See: http://www.yacy-forum.de/viewtopic.php?p=12471
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1048 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
02f8013013
auto-delete of corrupted word files during word-migration
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1047 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
d2731418bf
added creation of global ranking files and changed url normal form usage
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1046 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
6f9f8ed8f8
*) Automatic Reset of Stack Crawler DB on startup errors
...
See: http://www.yacy-forum.de/viewtopic.php?t=1432
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1045 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
fb766413d1
*) Changes on httpc dns caching
...
- Bugfix: old dns cache did not handle case insensitive hostnames correctly.
- adding a possibility to set domain name patterns defining hostnames that should not be cached by the httpc dns cache
e.g. borg-300.dyndns.org
This can be done by setting the new httpc.nameCacheNoCachingPatterns property
- using httpc.dnsResolve wherever possible within the sourcecode
[httpd.java,plasmaCrawlStacker.java]
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1044 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bc420c62f6
fixed htcache path generation (never change a running system)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1041 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
795f488222
new urlNormalform version
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1040 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
c86d801b0f
removed dyndns domains from dns caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1039 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
6dc42a2392
detecting of loops in kelondroTree during last/first-Node search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1038 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
17d2830394
see: http://www.yacy-forum.de/viewtopic.php?t=1416
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1037 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago