lotus
73f233bb11
* set resource observer to 1000MB
...
* transparent favicon
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5160 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
5fbccfd75e
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1366&p=9348#p9348
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5155 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
a28faabfd2
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1351&p=9242#p9242
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5154 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
7b63c66a08
- bugfix in bookmarksDB.Tag.hasPublicItems()
...
- this anoying little bug prevented display of public items without admin login for /xml/bookmarks/...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5151 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1fb1665e71
increased dht interval to avoid peer selection failure
...
(maybe too less peers available to fill the big gaps)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5143 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1eb813bd43
shifted index deletion-on-exit rule to the class where the errors are produced
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5141 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
ba76995d2c
* fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1415
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5140 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
bea6c13139
* with r5137 robotParser didn't work at all -> fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5139 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
3ded1efe84
kelondroExceptionCounter didn't work
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5138 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
ae677e1738
* fix problem in robotparser, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1421&p=9742
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5137 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
383d89481e
count errors before deleting collection.index
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5136 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
0bb4fbc403
delete corrupted collecion.index on exit for rebuild on next start
...
see http://forum.yacy-websuche.de/viewtopic.php?p=9725#p9725
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5135 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
b68d06a6e8
performance settings based on network's remote crawl speed
...
removed some _pro values from config
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5134 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
d60b2b198d
proxy fixed 'not modified' http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1419
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5133 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
bd0318ba81
* YaCy only supports gzip-encoding, so remove any other encoding from request
...
* fixes http://www.yacy-forum.org/viewtopic.php?f=2&t=163
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5132 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
bb5c898441
enhancements to localsearch behavior
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5131 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
42e2d195ac
added hint from http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1294
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5130 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
39964e88fa
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1329#p9121
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5129 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3f3673b6e5
extended balancer:
...
- added automatic time delay in case that a large number of urls come from the same domain
- added additional time delay in case that an url is a dynamic (CGI) url. This shall cause less IO on targets
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5128 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3c6e8d2015
set default ppm when network is switched
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5127 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3288c19c1a
reduce remote crawl PPM for fresh peers in freeworld to 6 PPM
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5124 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
5ce9a100bb
fix(2) for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1416
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5122 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
cf29ca19d4
possible fix for POST character encoding http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1374
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5121 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
a2eeb6138c
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1416
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5120 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d09ddabd09
corrected a design mistake (5-byte hashes not necessary)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5119 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
c97d0fcee7
modified the domain list export function:
...
- used the new superfast domain list generation from the domain statistics
- better interactive behavior
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5118 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
77ee0765a4
- added domain statistic generation to IndexControlURLs_p.html servlet
...
- added 'delete all' button to all results of such a domain statistic output which causes that all urls to this domain are deleted
- extended stack cleaner to clean also the statistics: they are not completely destroyed, only the smallest counting domains are removed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5117 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
80a7bc93d6
- added statistical evaluation about domains that appear during crawling
...
- added tables that show this statistics in CrawlResults web pages
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5113 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4fbee21cea
- added fetch-ahead again (had been removed in last commit)
...
- reverted default query mode to verify=false
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5111 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
423a89ebe8
* fix if yacy was installed to a path with whitespace
...
* show nice dots when waiting for restart/update
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5110 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
fc03b0437a
fixed a error case where a second search after a first search with a different search word failed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5109 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
eca171ba2e
fix for case where javascript was not filtered by the html parser
...
see http://forum.yacy-websuche.de/viewtopic.php?p=9667#p9667
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5108 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
e645bae29f
display table in log
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5106 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ead39064c5
fixed problem with wrong result number calculation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5105 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
2437beb96c
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1360&p=9321#p9321
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5104 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7b12e77a63
fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1393&hilit=&p=9655#p9655
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5103 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
05dbba4bab
added logging conditions to all fine and finest log line calls
...
this will prevent an overhead for the generation of the log lines in case that they then are not printed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5102 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d3d41e2ee4
- fixed problem with searching with quotes (still not complete, but not as bad as before)
...
- fixed parsing of crawl-delay statements when seconds were given with float numbers
- enhanced performance of profiling (not too many loggings; not more than one per second)
- removed some debug output
- fixed wrong return type in logging
- added a logging condition in httpd to prevent that logging statements are generated when they are not written (should be added everywhere!)
- fixed wrong word distance computation in RWI management
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5101 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
3fbfd5a78b
* fix for non-changing offset on new search term
...
* dht-heap doesn't has to be deleted (5097), we simply write a new one on exit
* do not install YaCy in startup because a Windows-shutdown might corrupt something. Installing YaCy as a service would solve this.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5099 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
219b93df6a
- fixed internal error after receiving chunked POST
...
- removed debug output
- added info for "501 Unknown" messages
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5098 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
c245c7a45e
delete index.dhtin/out.heap if restore fails
...
see http://forum.yacy-websuche.de/viewtopic.php?p=9613#p9613
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5097 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
cd19d0aee6
- added warnings for failed transferRWI (dht-in)
...
- fixed parseMultipart (uncompress gzipped body) (dht-in)
- fixed parseMultipart (using content-length only if uncompressed)
- better gzipped POST (chunked instead of content-length) (dht-out)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5096 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
df4ff423c4
added additional properties to query id's to distinguish search events better
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5093 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
d6d9b0f14a
fixed transferRWI.html 'Read timed out'
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5092 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
e503158527
Proxy: fix for never ending loading after POST
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5091 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
1a1d57e449
Proxy: added binary passthrough for POST
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5089 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
aa6ae77e5e
- autoReCrawl: fix for filter settings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5088 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
8ae29bad57
- fix to previous change of Crawl Profile Names
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5087 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
434104e4a0
- change Crawl profile name for autoreCrawl
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5085 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
9ff4fc11da
partial fix (images,audio,video) for proxy and content-type problem http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1374
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5084 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
0df2e47012
changed auto recrawl to comply with new date format
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5083 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
d9d9c522a1
addendum to last commit
...
moved recrawl times for standard profiles to constants
calculate new specific dates in cleanup job
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5082 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
480497f7c9
changed recrawl
...
use a specific date to define old documents
this solves an unwanted recrawl-loop during a running crawl
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5081 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
da1b0b2fc6
added two new classes that will be used for the new htcache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5080 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
536e77e8b7
modifications towards a single database operation to read/write http header and cached file at once:
...
- removed distinction between header file types for http and ftp; ftp is simulated by using http properties
- removed all old resourceInfo classes that handled this distinction
- introduced a new distinction between http request and http response objects
- unified new response objects with two other object types that had been introduced elsewhere
- changed all servlet call methods to use the new http request header object type
- divided static object keys for http header properties into request and response types
- refactoring here and there (a large number of type changes and many methods merged/moved)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5079 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
08cdf6db8a
fix for wrong "VegaYacyB" peers
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5077 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
4d937f6b21
fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1396
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5073 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
bd931a82f7
- added dynamic filters to autoReCrawl.conf
...
- Restrict to sub-path: sub
- Restrict to start-domain: dom
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5070 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
b3fc5e96a3
- removed unused import from bookmarksDB
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5067 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
bc048db7b6
- bugfix for bookmarksDB's rebuildDates()
...
- dates are now saved as String.valueOf(TimeStamp)
- it might be a good idea to delete (backup) bookmarkDates.db and restart YaCy to rebuild it
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5066 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
3c68905540
remove redundant null checks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5065 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
753a1ae430
- changed default browser from netscape to firefox
...
- fixed "Inefficient use of keySet iterator instead of entrySet iterator" [WMI_WRONG_MAP_ITERATOR, FindBugs]
- fixed some possible null pointer accesses
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5063 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7989335ed6
Preparations to replace the HTCache with a new storage data structure:
...
- refactoring of the HTCache (separation of cache entry)
- added new storage class for BLOBs. (not used yet, this is half-way to a new structure)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5062 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
be28af50f5
- fixed "yacy2yacy no proxy"-problem
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5058 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
f99c307eff
* correct debian build dependencies
...
* add huge mem page detection in general initscript
* disable logging completely in jmimemagic-library
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5056 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
bdae051d9a
- extended new performance graph (better timing)
...
- added paths for new libraries in classpath for eclipse
- refactoring to remove compiler warnings (static access to finals variables)
- removed some unused import
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5055 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
d9cea5ff23
removed annotations which broke the build with java 1.5
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5054 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
a087090bbb
fixed starting crawl results in "No parser available to parse mimetype 'application/octet-stream'"
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5047 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
7e7e6a099a
undo 5044
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5046 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
f2d0bd7790
fix for NPE in JakartaHttpClient.setProxy
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5045 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
bb6a6fc233
fixed 'FileUploadException Stream ended unexpectedly'
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5044 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
8422ee5ec4
- fixed UnsupportedEncoding (in proxy) using defaultCharset if no characterEncoding can be determined
...
- serverFileUtils.copy* use now Charset instead of String
- added some warnings for ignored exceptions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5043 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
3ac1988059
Add some sanity checks for invalid seeds
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5042 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
cff4393f0c
Fix HTCache so oldest Files get deleted first
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5041 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
31d97f2b9f
replaced httpd.parseMultipart() by a 'right' implementation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5040 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
621b473b18
* removed some warnings of findbugs ( http://findbugs.sf.net )
...
- removed unnecessary code (unused variables, String.toString)
- corrected some calculations (cast int to double or long ;)
- improved little performance (using Integer.valueOf() instead of new Integer)
- log if some File-actions fail (mkdir(), delete(), ...) and some ignored exceptions
- finalized some (more) fields
- finally close some streams
- made inner classes static if not using environment
- generalized some equals (from specificClass to Object)
- fixed some potential nullpointer accesses
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5039 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
0500b1179e
added a 2 min start up delay to serverBusyThread autoReCrawl to avoid a Null Pointer Exception...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5035 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
e1574fe02e
- added autoReCrawl folders to bookmarks (DATA/SETTINGS/autoReCrawl.conf)
...
- the serverBusyThread checks folders every 60 min. (==> autoReCrawl_idlesleep in yacy.conf)
- added option to create bookmarks from CrawlStart URL
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5033 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ebb40d324b
enhanced memory chart: shows now also the size of the word cache as third vector.
...
The PPM is now shown without a scale, but with a new anotation at the chart entry.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5032 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
17b7845eb5
* refactoring
...
- moved constants from plasmaSwitchboard to own class (all 232 ;)
- moved remoteProxy-Methods to httpRemoteProxyConfig, better names
- removed some unnecessary code (else-statements)
* formatting (correct indentation)
* minor bugfixes (due to findbugs.sf.net)
* hopefully fixed "missing quote" (announcing StringParts as UTF-8)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5031 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
3bb870bfcd
added final where possible
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5030 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
7e92484400
fix for open browser on windows 2000
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5029 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
b0724e5ec0
* add config option to disable cookie monitoring (disabled by default)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5028 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
0b2f67577e
Index Transfer:
...
- fix for chunk size calculation
- fix: if chunk size was 1, an infinite selection loop ran because no entries were found. if chunk size fails <=3 it will be set back to 500
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5023 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
694084c570
fix for NPE on shutdown
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5021 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
5f77f55ed7
possible fix for negative speed values
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5019 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
50ef5c406f
- refactoring of robots parser (removed opaque Objects[] result vector)
...
- added Allow-component to robots result object
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5016 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
7913bdb75b
Flextable: filename in errormessage if inconsistent
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5015 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
d42eae25f8
yacyTray:
...
fix for unproper shutdown
some messages
installer:
start shortcuts minimized
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5014 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
c3d461d191
- removed superfluous copyright statement
...
- updated my email address
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5011 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3ca98fee42
removed superfluous copyright statement
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5010 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
c049d80fbd
fixed login problem with yacy as proxy (POST and Cookies)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5009 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
62afea0c9f
some improvements for yacyTray
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5008 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
7c110e07f0
removed debug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5006 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
eadc204130
gzip POST wiederholbar gemacht (macht transferURL stabiler)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5004 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
28c39e2aa4
fix for new starter files
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5002 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
fa695c2d9f
tray is now only shown on Windows and doesn't block on linux
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4997 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
d77ed28e2f
temporary disabled tray because of flaws on only-shell-linux
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4996 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
f8a1e3175e
new yacyTray
...
this will make a YaCy icon in the tray area on supported platforms
enabled by default
the search page will open on double click
used JDIC 0.9.4 from https://jdic.dev.java.net/
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4992 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
05c26d58d9
fixed missing remove operation in balancer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4990 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
606b323a2d
fixed bug that appeared when a new crawl ist started
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4989 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
38eb5bd1ee
fixed a bug in kelondroBLOBHeap. The following files are probably inconsistent and should be deleted:
...
DATA/HTCACHE/responseHeader.heap
DATA/PLASMADB/crawlRobotsTxt.heap
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4988 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
28d5703f8a
- fixed a bug in Robots.txt loader which could have caused that robots.txt files had been loaded from the same domain more than once
...
- patch in BLOBHeap to prevent OOM during startup
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4987 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7b1c9e6aee
discovered and removed a (possibly large) memory leak:
...
many classes used the kelondroMapDataMining (was: kelondroMapObjects) which adds statistical
functions to the kelondroMap (was: kelondroObjects), but these functions were not used by these
classes. Especially the HTCACHE and robots.txt database allocate a very large number of objects
for statistical use, but never used them. By replacing the kelondroMapDataMining with the
kelondroMap object for these classes now less memory is allocated.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4986 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0f5fe8cc53
refactoring of method calling for objects from kelondroMapDataMining
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4985 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
01d1ae6676
patch for negative time in case that the time of the computer is changed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4984 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4acf0a61cd
refactoring of kelondroObjects (mainly renaming to kelondroMap)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4982 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
441e9c861e
fix for npe in HTCache cleaning process
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4981 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f7aaeb3fad
created new main menu entry 'Customization and Integration'
...
- moved some already existing servlets to this menu
- renamed the skin servlet to appearance
- added a set-to-default-button to the search page appearance setting
- removed the peer profile servlet which is now replaced by a field in the new appearance servlet
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4980 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
5488543b8f
disabled disk usage logpoints
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4979 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1e6d12f146
Major update to BLOB data structures:
...
- introduced a new BLOB file format: kelondroBLOBHeap. This is a flat file with an index in RAM.
very similar to the eco-tables, but with flexible value sizes. It will replace the kelondroBLOBTree,
which is based on a kelondroTree, a file-AVL-based index data structure.
- the HTCACHE header file was replaced by the new blob heap file structure
- the robots.txt file was replaced by the new blob heap file structure
- the robots parser was enhanced (bugfixing for double-loading of the same robots.txt)
- other BLOB-dependent data structures were prepared to use also the new BLOB heap
- fixed a bug in the snippet fetch process: the file header was not written to the header index
There should now be less IO during snippet fetch and during crawling
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4978 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
81f75f5056
- removed unnecessary classes (these objects are much easier to handle using generics)
...
- generalized BLOB referencing. This is the preparation to use another BLOB class, the kelondroHeap
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4977 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
b38f467e3c
better SRU compliance
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4976 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7052f2f61f
- added copyright header of ResourceObserver
...
- commented/removed some code to eliminate code warnings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4974 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1400cdc91e
- refactoring of resourceObserver (moved it to crawler)
...
- partly redesign of diskUsage: little bit more functional behavior, less side effects, better error case handling
- the resourceObserver can now show a error message if the diskUsage is 'out of order'
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4973 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
b6301a54fa
* added class ListDirs to provoid generic listing of directories in systemdirectories and jar-files
...
* yacy runs, when classes are in a jar-file (->build-jar ant-target)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4971 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
f2e2d09916
- fix for index transfer
...
- imported a random startpoint function from plasmaDHTChunk
in case there was already a gap at the beginning of the index, the transfer process was endless selecting from first startpoint
tested & working on my index
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4970 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
a6719dfd2b
- refactoring of robots parser
...
- no more keep-order parameter in remove (it was not possible to make this strict, and not useful)
- some small enhancements in balancer
- robots parser without references in switchboard
- changes synchronization in robots
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4969 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e81be7d4f2
added many missing user-agent declarations for yacy http client connections.
...
the most important fix was the addition of the yacybot user-agent for robots.txt loading,
because web masters look for that access to see if the crawler behaves correctly.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4968 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
474e29ce4a
added options to configure the 'corporate identity'-icons, the home page link and the greeting line from
...
the skin menue. Additionally an example is given there how to integrate a search page with an iframe.
Please see the skin menu.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4967 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
474659a71f
- modified and enhanced the crawl balancer: better list export, fixing of damaged crawl queue at start-up, re-sorting at start-up to enhance domain order
...
- added option to set minimum crawl delta for domains in balancer
- added default values to crawl deltas in yacy.init
- added configuration for these deltas in performance queues
- enhanced performance setting computation (more time for indexing queue for a faster flush
- remote crawling is now enabled during local crawling if indexer has space and time for more links
- added database stub for new distributed file system
- refactoring of time computation to get an abstraction level that will be used by a TTL rule in new distributed file system
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4966 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
080cda97ef
added another peer selection rule:
...
- select also non-robinson (dht-) peers if their peer tags match with search words
- the peer tag '*' can now act as catch-all rule: shall be selected always
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4963 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d37fd064f9
changed peer selection for search targets:
...
- less dht targets are selected
- more other peers are selected: all robinson peers with more than one million urls
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4962 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
69aac0d74c
modified the diskUsage class regarding the following two aspects:
...
1. The usage and dependency of the plasmaSwitchboad was used many times in the past but this was
a bad mistake. The classes should be independent from the switchboard to support a better abstraction. Therefore the object was removed. The parameters from the switchboard are computed outside and then handed over.
2. the class is considered as a tightly connected to hardware resources. Classes which handle data that cannot be replicated because it would need to replicate hadware should not support dynamic object allocation, but should be coded as collection of private static methods. Therefore all class objects had been transformed into static private objects.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4961 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
da917cf4b1
undo reduced menu
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4960 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
0c1dc703e4
- set staticIP at startUp
...
- added setting for reduced menu (simpleMenu)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4959 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
f7f9ceb967
diskUsage: replaced blocking sleep with semaphore
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4957 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
4a53649ee7
fixed dht-urls and ranking distribution log statistics
...
* NOTE: please have in mind that there can be whitespaces in pathnames
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4956 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
8d83185cb4
fixed dht-chunks/protocol log statistics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4955 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
63eadfdf84
fixed unlimited FileSizeLimit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4954 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
2dc7c00c1c
fixed indexing log statistics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4953 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
dba7ba079e
fixed NPE seen with queues_p.xml (serverClassLoader finds already loaded classes)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4952 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
273fb01142
revert last fix; was wrong
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4950 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
b6f50851fa
fix memory requirement calculation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4949 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
ac85c52bae
better readability for MIN_FREE_DISK_SPACE
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4948 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
54a73b58cf
fixed restart on Windows when directory had spaces in it's name
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4947 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
609aaf0df3
rework of the windows part
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4943 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
1a4f26ba30
exclude HTDOCS from recursiv scan
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4942 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
6c07e894d9
add needed sleep
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4941 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
d742cc080c
Fix for RAMCache not flushing
...
see: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1255
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4940 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
6b7e873962
resourceObserver refactoring and some synchronisation for console output
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4939 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
6bdd99e065
- more asserts to solve the ooB-problem
...
- better caching (?), lets see how it behaves
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4937 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
b928ae492a
some code-cleanup and possible speed enhancements in different core methods
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4935 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
6a9cc29cdd
workaround for IndexOutOfBoundsException in ResultURLs.getExecutorHash() seen @ CrawlResults.html?process=4
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4934 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
c998dc6556
- added security functions to flush url and search caches in case that memory is full
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4933 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f4ae8082c3
- better error analysis for ooRange Exception in kelondroBase64Ordering
...
- quadcore support for kelondroRowSet array ordering
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4932 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
84cbe75005
more asserts
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4930 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e269c12710
small changes in partition routine
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4929 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
31efb8fbee
- fix for LOG path generation when the DATA/LOG does not exists (fix for bug introduced in SVN 4923)
...
- some more/better asserts
- slight performance enhancements in remove method in index management. Works for all who do not run using asserts (the majority)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4928 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
877299cc74
better installer on Windows Vista
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4927 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
726218dd4a
fixed logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4926 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
21c87c36e3
added a log line
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4925 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
68c38c2d34
- WatchCrawler shows status without JavaScript
...
- Performance can be scaled + DHT-profile
- names for pool-threads
- some small refactorings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4923 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
fc79f013c4
better solution to update shortcut
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4920 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
c0dfe49743
also exclude collection.0028.commons and RANKING at startup check
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4919 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
11656741f1
exclude LOCALE and RELEASE at startup check
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4917 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
48edbef5c7
* fix: display proper port on 1st startup
...
* new message on portchange
* first implementation of external link-update for search page (still inactive)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4915 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
0727bb1e63
rework of console message handling; add of debugging output
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4914 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
43c47218ef
fox for open browser on Windows
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4912 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f5ef7f222e
- fixed a bug in parser (directory paths had not been recognized)
...
- no access check when a search is made only local without snippet fetch
- added comment and status message in resourceObserver (this takes very long at startup time!)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4911 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
ed24eab737
small fix for windows in resource observer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4909 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
75b4a5ced4
reinstate old timeout values for transferRWI and transferURL
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4903 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3330181aa0
refactoring:
...
find a better way to store BLOBs; generalize current BLOG data structure (kelondroDyn)
and prepare it to replace it with something better. The best candidate is the kelondroHeap,
which will become the kelondroBLOBHeap;
removed also some never-used classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4902 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9a9737a54e
fix for "no more elements available" exception
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4901 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
260553c3a5
better messages
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4897 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
6afeb535cd
another bugfix for the windows drive check
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4896 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
b416af7568
bugfix for the windows drive check
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4895 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
4b71912e76
fixed wrong class name
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4894 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
7feae906aa
- organize imports
...
- removed potential null pointer accesses
- removed unnecessary casts
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4893 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
f597185026
Initial import of the resource observer framework
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4892 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
c10eaf9bdb
- fix for pop-up page upon first start
...
- added comments in opensearchdescription to explain fast mode
- release 0.59
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4890 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
d3037c2950
Accept all SSL-certificates (not only valid and self-signed), but put a warning into log file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4888 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e0e7f86f82
some bugfixes for the peer-ping process
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4885 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
cbe722c480
small code cleanUp
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4884 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
b21598bdd0
- enhanced handling of own IP address inside seed
...
- prevention of false information of own IP address
- enabled searching before an own IP address is assigned (before first ping happened)
- removed warning about limited search function
- added better time-out settings for peer-ping process (10 seconds complete, 5 seconds for back-ping)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4883 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
40d7f485f3
- fixed several NPE bugs
...
- fixed loosing of own seed hash (hopefully)
- fixed a bug with crawl start s beginning with (bookmark) files
- added better IP recognition during hello process
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4882 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
2f381b8d7a
- fixed at least two causes for a NPE after a use case switch.
...
A large refactoring was neccessary
- added another crawl start option: automatic restriction to sub-path
- removed crawlStartSimple and renamed crawl start expert
to crawl start (without expert)
- some changes to texts in crawl start
- added some more deletions when an web index is deleted:
delete also queues and robots cache
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4881 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
2a604b7402
added superfast search result computation which can be obtained for local search when snippet fetching is disabled. An example search for the rss interface would be:
...
http://localhost:8080/yacysearch.rss?query=yacy&Enter=Search&contentdom=text&count=10&resource=local&verify=false
(just add "&verify=false")
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4878 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9bef20b537
- added cleanup for unused server loggings: they are removed after the client had not been seen since one hour
...
- removed configBasic popup trigger when no password is set
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4875 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
5f35ba6f05
added a list for content of the brute-force registration hashtable in access tracker
...
the purpose of this list is not only to view attempts from read attacks, but also to show if the
yacy-yacy protocol is working with respect to the bug that was fixed with SVN 4869
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4873 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
74e3a547db
more logging for mySeed loading error cases
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4871 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
6f1a3fce05
BF Bugfix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4869 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
b09af53643
proper links to directories in repository dirlisting
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4867 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
ba4091c5b2
proxy sends status code now
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4860 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1a1841392c
small fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4859 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
698293ef32
fix for lonely peers in networks with only one peer, especially intranet peers
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4858 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
8a0e401320
- fix for bad code in peer actions
...
- fix for bad images in basic config
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4854 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0b52ef3e4b
- update of grafics
...
- check for startsWith("127.") instead of equals("127.0.0.0")
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4853 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
18ad12eceb
added another fix for localhost addresses in seeds
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4852 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
11e00a0849
- refactoring of seedURL handling
...
- additional check for seedURL pointing to localhost: deny such peers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4851 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e91bf4c8cc
- fix for bad reset of index / bad index location after deletion
...
- some modification of rssTerminal window location and size
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4850 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
f284386b63
update deploy improvements for windows - ready for release now :-)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4849 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
fd8bd5d0d1
* fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1176 (encoding issue)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4848 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
25192e0d36
added a deletion button to indexControlRWIs that deletes the complete web index
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4847 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
01b3e9431a
- fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1140&p=7626#p7626
...
- less dots for ppm bar in watchcrawler (one dot for each 10 ppm)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4846 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
8be462986e
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1174&p=7841#p7841
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4845 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f8b015949c
fix for bug in html scraper that appears if opening and closing tag are not both in same case
...
see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1173&p=7836#p7836
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4844 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d8277e6af1
- added parsing of numeric html entities for crawler
...
- fixed a bug in search response
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4843 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
f06c046f6c
*) refactoring (several instances of Indetificator can be created and used parallel now)
...
*) fixed a potential NPE
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4841 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0c173821fd
more access security regarding database access and snippet retrieval: restrict number of results for not-authorized searchers
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4838 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago