all operations on YaCy in a database that should make it possible
1) to re-create a setting on fresh peers
2) to transmit a setting from one peer to another
3) to re-create crawl starts after a complete deletion of the index
This functionality will also support
4) scheduled re-crawls (new implementation)
To implement this, a new database structure has been crated that stores maps into blob heaps. to encode maps the b-encoding technique was used (this is the same encoding that torrent files use)
- added a b-encoder
- enhanced the b-decoder
- added a b-encoded map heap data structure
- added a table organisation based on b-encoded heaps
- added a servlet to maintain such tables (see Tables_p.html)
- integrated the servlet into the Advanced Settings menu
- added an api recording based on the new tables
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6606 6c8d7289-2bf4-0310-a012-ef5d649a1542
- Fixed some end tags (#-----------------------------) for syntax highlighting to properly detect the end
- Updated translation on pages in Search & Browse category
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6603 6c8d7289-2bf4-0310-a012-ef5d649a1542
- Updated changed lines in CrawlProfileEditor_p.html and WikiHelp.html
- Updated new navigation menu structure and some submenus
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6601 6c8d7289-2bf4-0310-a012-ef5d649a1542
- added image size as part of parsed text in images
- avoid unnecessary error messages if parsing of documents failed but one succeeded
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6597 6c8d7289-2bf4-0310-a012-ef5d649a1542
- added BEncodedHeap class that encodes B data structures and stores that to a heap
- refactoring of MapView, this is now named MapHeap to fit into the naming scheme of the BEncodedHeap
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6579 6c8d7289-2bf4-0310-a012-ef5d649a1542
this is a work-around for the problem that the search widget
does not load a second page if the first page did not fill up
the window with enough lines such that a scrollbar ist visible.
Because the scrollbar triggers loading of following pages, this
must be enforced with the trick that more result lines must be
shown after the first search.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6570 6c8d7289-2bf4-0310-a012-ef5d649a1542
- increased sort limit, now 3000 entries, before: 1000
this should cause that more results can be shown in case
of strong limitating constraints, like domain navigation
- enhanced the sort process
- check against domain navigator bugs
- fix in sort stack
- showing now all naviagtion pages at first search (not only next page)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6569 6c8d7289-2bf4-0310-a012-ef5d649a1542
this was done because that servlet may be used for wget/cronjob
triggered crawl starts and it appears to be confusing that the
name of the crawl start servlet looks like a pure monitoring tool.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6568 6c8d7289-2bf4-0310-a012-ef5d649a1542
- added 'yacybot' as key to recognize robots.txt entries for YaCy
- removed unused method to get robots.txt from database
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6565 6c8d7289-2bf4-0310-a012-ef5d649a1542
orbiter