orbiter
eb12e15738
moved all Double values to Float values because of
...
http://www.exploringbinary.com/java-hangs-when-converting-2-2250738585072012e-308/
YaCy does not really need double-precision floating point computation anywhere, so this should not affect any feature
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7460 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
f1ori
982aa689ef
* fix StringIndexOutOfBoundException in WebStructureGraph
...
* add better escaping to saveMap and loadMap
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7458 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
88773e4daa
changed the default port from 8080 to 8090
...
see also: http://forum.yacy-websuche.de/viewtopic.php?p=21683#p21683
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7454 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
991b92f4ae
enhanced network graphic
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7446 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
f1ori
a321c7673d
* adminAccountForLocalhost only for localhost
...
* yacy crawls local domains also, if no password is set (the interface is already protected)
* it's not required anymore, to set a password in intranet mode
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7436 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
hermens
930cb412dd
Let SHORT_MILSEC_FORMATTER make a new formatted String every millisecond
...
see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3103
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7434 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
090c73e32e
catch a OOM in HeapReader iteration
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7433 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
low012
48463c4507
*) General private License? ;-)
...
*) minor code changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7432 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
6c1b14c8e1
- more control in access tracker: count number of returned search results (not only info how much is in the index)
...
- extended query params for this
- enhanced cora
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7430 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
low012
9f38c0023d
*) Minor changes, mainly cleaning up a little bit, no functional changes.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7428 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
54e77e6255
refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7426 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
feefe17568
npe assert fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7424 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
733903f2c9
fix for http://forum.yacy-websuche.de/viewtopic.php?p=21489#p21489
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7422 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
24e4126eee
added JSON parser code from json.org (added generics to it)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7421 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
10ae8d961b
- cora package has now no dependencies to other yacy packages and becomes a 'base' package (refactoring)
...
- cleaned up (removed special code and documentation for 27c3)
- added remote search functions to be used within cora
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7420 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
lotus
0e54233408
UPnP: map port again if we are not reachable (e.g. when router rebooted)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7419 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
lotus
b1484299b2
same units for memory observer configuration (MiB)
...
old setting for DHT (RAM) will be lost after update
can be set on /Performance_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7418 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
387db84087
maybe found bug in non-working index dumper
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7414 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
a4c9d27287
- moved some variables from Stwitchboard to new class AccessTracker
...
- added a limitation in access tracking to delete queries which are older than 10 minutes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7410 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
f1ori
e4aabaa1c3
* fix negative filelength for files >2G
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7408 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
cdfe8afe3f
fix for really bad table iteration implementation: reduction of IO
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7407 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
low012
9eae33f886
*) Ooops...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7406 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
low012
a001e8075c
*) minor enhancements
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7405 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
low012
11ea966f9e
*) added SID file (Commodore 64) sound file parser
...
*) minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7403 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
b2ed4cfaf8
more small bugfixes and light refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7401 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
3ca06d6290
patch for http://forum.yacy-websuche.de/viewtopic.php?p=21460#p21460
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7399 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
903c824c2c
- allow only scanned resourced with granted status
...
- increased time-out when scanning an ip range
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7398 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
low012
936e976c23
*) added FreeMind ( http://freemind.sourceforge.net/ ) mindmap parser
...
*) minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7397 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
low012
3d95981f7d
*) cleaning up the code a little bit
...
*) minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7396 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
low012
2a6499364d
*) minor changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7395 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
low012
c0274bd123
*) minor changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7394 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
fe46536f6e
enhanced network scanner (less name resolving during scanning and no name resolving during search)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7392 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
e753027c43
fix for http://forum.yacy-websuche.de/viewtopic.php?p=21439#p21439
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7390 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
bf4ef1513e
- fix for map view
...
- remove some UNRESOLVED PATTERN
- maybe a fix for non-flushing cache
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7389 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
6b70393d1d
- new java version 1.6
...
- replaced old gif animator by java 1.6 gif animator
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7388 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
e88c428008
fix to ftp loader
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7387 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
59b70a5a92
another fix to the ftp crawler: now correct directory listings according to rfc2640 (path with spaces) and better title names for such files
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7386 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
9b25a33fd9
- fixed numerous bugs
...
- better document names
- fixed problem with ftp crawling
- added automatic removal of search results from services that are not online according to the latest network scan: this does not delete the index but just does not show them. after the next network scan when the server is available again, the results are again showed.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7385 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
7bdb13bf7f
more fixes to smb crawling: better file names
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7384 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
94c48500cc
several fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7383 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
0ac7311a62
fix for token parser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7382 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
58b59f9bc8
- a collection of bug fixes and some redesign of the Scanner class
...
- fixed smb crawling
- added smbget to download script generation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7381 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
c288fcf634
redesigned CrawlStartScanner user interface and added more features:
...
- multiple hosts for environment scans can be given (comma-separated)
- each service (ftp, smb, http, https) for the scan can be selected
- the scan result can be accumulated or refreshed each time a network scan is made
- a scheduler was added to repeat a scan and add all found urls to the indexer automatically
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7378 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
f1ori
9d2159582f
* fix system update if urls are in blacklist (for example for very general blacklists like *.de)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7375 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
56264dcc17
- added CamelCase parser to MultiProtocolURI: generate better to-be-indexed words from urls
...
- integrated new parser into loader processes: enrich document parser
- fixed a concurrent modification exception in kelondro iterator
- hand-over of document size from crawler to indexer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7374 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
99a7fe87f9
- removed old intranet scanner (the generic scanner now completely subsumes the old one)
...
- added information about granted access
- enhanced servlet design
- added submit-feedback (because it is a long-running task)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7372 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
acab6801d9
added new network scanner
...
- you can scan any ip or host in the internet for services
- this replaces the intranet scanner
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7371 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
14e4fae8e9
fixes to ftp client
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7369 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
a563b05b60
enhanced crawler:
...
- added a new queue 'noload' which can be filled with urls where it is already known that the content cannot be loaded. This may be because there is no parser available or the file is too big
- the noload queue is emptied with the parser process which indexes the file names only
- the 'start from file' functionality now also reads from ftp crawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7368 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
orbiter
c36da90261
added a very fast ftp file list generator to site crawler:
...
- when a site-crawl for ftp sites is now started, then a special directory-tree harvester gets the complete directory structure of a ftp server at once
- the harvester runs concurrently and feeds into the normal crawl queue
also in this:
- fixed the 'start from file' crawl function
- added a link detector for the html parser. The html parser can now also extract links that are not included in <a> tags.
- this causes that a crawl start is now also possible from clear text link files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7367 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago