orbiter
7fdf59a77f
misc NPE check
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6630 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a512aef6ad
fix for http://forum.yacy-websuche.de/viewtopic.php?p=18918#p18918
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6629 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
38a3d55afd
added more possible php extensions for html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6621 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4403304957
bugfix for list()
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6616 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3889438db6
fix for bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6615 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
23bcca07a3
removed directly linked servlets that had been there to test memory failures that appeared in that servlets
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6612 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
69c29acb6e
no exception thread dump if parser cannot parse becuase that mime-type/extension is in the deny-set
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6611 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
0098e6e859
bugfix for heap iterator
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6610 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
db19a941cf
added new image index storage classes (not integrated yet)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6608 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
c8aece34a4
update to yacy/ai (just more testing)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6607 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
8ce936bcdd
added an api recording function: it shall be possible to record
...
all operations on YaCy in a database that should make it possible
1) to re-create a setting on fresh peers
2) to transmit a setting from one peer to another
3) to re-create crawl starts after a complete deletion of the index
This functionality will also support
4) scheduled re-crawls (new implementation)
To implement this, a new database structure has been crated that stores maps into blob heaps. to encode maps the b-encoding technique was used (this is the same encoding that torrent files use)
- added a b-encoder
- enhanced the b-decoder
- added a b-encoded map heap data structure
- added a table organisation based on b-encoded heaps
- added a servlet to maintain such tables (see Tables_p.html)
- integrated the servlet into the Advanced Settings menu
- added an api recording based on the new tables
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6606 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
56e0d9bd01
- testings with image parser
...
- added image size as part of parsed text in images
- avoid unnecessary error messages if parsing of documents failed but one succeeded
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6597 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
e80e060ca6
- increased thread priority for server threads
...
- decreased thread priority for crawler threads
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6596 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
234f733a3d
- relocation of seed db is better for network switch than re-initialization because of the embedding of the peers object in other objects
...
- small refactoring of blacklist interface code to remove PMD warnings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6593 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
473b11033d
fixed network switch process - crawling did not work after a switch before this fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6592 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
fd7b348973
some fixes for the network switch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6591 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
7d400b17d0
html parser support for .cfm files
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6590 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
f6731c6240
more logging etc.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6589 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
007f8297de
added php3 as extension type for html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6588 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4f1f4863c4
fix for deadlock when initializing a SplitTable with a file of size 0, see also:
...
http://forum.yacy-websuche.de/viewtopic.php?p=18594#p18594
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6587 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
cc5dcf69ff
missing change for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6585 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ca1ef9a079
fix for http://forum.yacy-websuche.de/viewtopic.php?p=18584#p18584
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6584 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d9169cc6c3
increased proxy load time-out from 30000 to 60000 milliseconds
...
according to http://forum.yacy-websuche.de/viewtopic.php?p=17782#p17782
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6583 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
938e806182
tried to fix date problem that may have prevented that foreign peers stay in the network
...
- removed unused code
- removed possibly wrong utc difference correction
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6581 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
bd05e57d3b
fix for http://forum.yacy-websuche.de/viewtopic.php?p=18563#p18563
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6580 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
5df628a2a4
- added BEncoder class
...
- added BEncodedHeap class that encodes B data structures and stores that to a heap
- refactoring of MapView, this is now named MapHeap to fit into the naming scheme of the BEncodedHeap
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6579 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
82f57f79e5
more PMD enhancements
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6576 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
5d930c96f0
more fixes to search result page navigation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6575 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
8c520f128d
reverted a change in ranking process committed this afternoon
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6573 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a06f7ddb33
more PMD recommendations
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6572 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
eb79ceb3ff
update to kelondro data structures
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6571 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
18172451a0
better search computation:
...
- increased sort limit, now 3000 entries, before: 1000
this should cause that more results can be shown in case
of strong limitating constraints, like domain navigation
- enhanced the sort process
- check against domain navigator bugs
- fix in sort stack
- showing now all naviagtion pages at first search (not only next page)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6569 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d126d6c1b5
renamed the servlet WatchCrawler_p to Crawler_p
...
this was done because that servlet may be used for wget/cronjob
triggered crawl starts and it appears to be confusing that the
name of the crawl start servlet looks like a pure monitoring tool.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6568 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
66c0a8e849
more PMD recommendations
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6567 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
909a4f91c7
added a logging output for crawl starts that shows the URL that can be used to start the crawl again
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6566 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
bc96d74813
- clean-up of robots.txt parser
...
- added 'yacybot' as key to recognize robots.txt entries for YaCy
- removed unused method to get robots.txt from database
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6565 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2113fcd7e5
- fixed usage of isEmpty() which is not available in java 1.5
...
- increased visibility of some methods
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6564 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
dd459281c8
applied code changes that are recommended by PMD
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6563 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
eac2daf2e8
* reenable DHT if yet enough memory is available
...
* reset treshold on reconfiguratoin
(thanks to sixcooler)
* display status message in web interface
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6562 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
0752634b8b
log YaCy version on startup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6561 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d77a8f3b3e
added some modifications recommended by PMD for better performance
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6560 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d1973bae2a
code cleanup: removed unused code and unused methods
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6559 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a3b8b7b5c5
some redesign of the main menu structure:
...
- moved all index generation servlets to it's own main menu item, including proxy indexing
- removed external index import because this operation is not recommended any more. Joining an index can simply be done by moving the index files from one peer to the other peer; they will be merged automatically
- fix to prevent endless loops when disconnecting http sessions
- fix to prevent application of bad blacklist entries that can cause a 'Dangling meta character' exception
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6558 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
ab3cf60dbe
fix for npe
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6557 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
7f20963b41
add-on to last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6556 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
eeca2ded92
fix for http://forum.yacy-websuche.de/viewtopic.php?p=18500#p18500
...
- catch uncatched OOM
- less wasting of memory
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6555 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
32972139af
added nice configuration for the resource observer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6554 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
bb2e03761c
- fix for deadlock with 100% CPU during search
...
- fix for failure of ranking because of a ConcurrentModificationException
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6553 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3f771d2a16
fix for rss parser: be lazy when rss is not well-formed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6552 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
dff4f95c78
some patches to get the torrent parser working
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6551 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago