karlchenofhell
f05ca43780
- the wiki-parser works for remote wiki-code now, not displaying links anymore as if they were local (ViewProfile comment)
...
- fixed wrong link to CrawlStart on Status-page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3816 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
30c3d909b1
- fixed charset problem in ConfigProfil_p.html (use accept-charset="UTF-8" in forms)
...
- fixed wrong XML output if no peers are known in Network.xml
- simplified parsing of table properties in wikiCode and ZTableToken
- reimplemented GC heuristics. They are needed to constantly ensure that an amount of free memory is available which is higher than Java's max. limit for performing a Full GC (please use serverMemory.request(long, boolean) rather than serverMemory.available(long, boolean) to provide data for averaging over the last GCs)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3793 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
4392ee0c51
BugFix for typo and wrong include
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3789 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
d1e1580223
Surftips Blacklist
...
Blacklists List Hardcoded instead of only updated on firststart / migration.java
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3788 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
44bac7dea1
*) blog-comments can now be moderated
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3778 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
957a25afff
getRight(rightName) instead of get...Right()
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3774 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
a0149317ac
*) fixed bug where headlines were added to directory of a wiki page multiple times ( http://www.yacy-forum.de/viewtopic.php?t=4034 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3762 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
baa9402b97
- wiki-parser is now configurable via the config setting wikiParser.class which holds the class-name for the parser to use
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3742 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
601fc7d1c5
- added source to J7Zip-modifed.jar and it's license (changelog is still to come)
...
- moved HTML-*replace-methods from wikiCode to de.anomic.data.htmlTools
- prepared use of different wiki parsers as suggested here: http://www.yacy-forum.de/viewtopic.php?p=34444#34444
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3741 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
b1680ab71f
*) bugfix for ArrayIndexOutOfBoundsException in robots-parser (thanks to low012)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3739 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
9a4375b115
*) robots.txt: adding support for crawl-delay
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3737 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
65a8a9fc58
fix for nullpointer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3726 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
139c59ebbd
- fixed dht selction problem: the seed tables used a wrong ordering
...
- cleaned some code
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3693 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
cb43ae11ba
*) Bugfix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3668 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
0b5fc3c28c
*) moving date functions to serverDate class
...
*) Sitemap-parser
- logging added
- parsing of modDate added
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3667 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
6f46245a51
*) Bookmarks: Ajax icon is displayed while loading title
...
*) First version of a sitemap parser added
- currently only autodetection of sitemap files is supported
*) DB-Import restructured
- pause/resume should work again now
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3666 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e48189c710
enhanced cluster routing
...
- cluster definitions can now contain an addition for local ip addresses
- cluster-cluster communication uses the local ip address instead the global address, if one is given
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3624 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
2399ed817c
*) robots.txt parser now extracts the sitemap-URL (will be used later)
...
*) some javadoc added
*) junit testclass for robots.txt parser added
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3602 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
rramthun
e6fb6426a3
*) Some cosmetical changes and corrections
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3582 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
40c14a4f0e
- better implementation of search query properties
...
- basic protection against start-up problems when database files are corrupted
- auto-delete of not-critical databases during startup when load error occurs
- on-the-fly reset option for all database tables
- automatic on-the-fly reset for seed tables during enumeration exceptions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3547 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
f4af360f7c
bugfix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3494 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
9b5fb3908d
*) a peer-message are now created when a blog-comment is written
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3480 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
6ad39bae1e
fixed shutdown problem
...
this fixes the 'inconsistency' messages during start-up
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3457 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
264a82eec8
- fix for http://www.yacy-forum.de/viewtopic.php?t=3657
...
- fix for http://www.yacy-forum.de/viewtopic.php?p=32758#32758
- Diff takes any objects now, not only strings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3455 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
d755a8026d
- better OOM protection
...
- better memory allocation for FlexTable indexes
- splitting between static index and dynamic index (only the dynamic part must grow)
- to enable a merge-iteration of new splittet index, a huge number of classes needed to be adopted for new iterator classes
- added new iterator classes that support cloneable iterators
- adopted all iterator classes to implement cloneable itarators
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3453 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
1cba31de43
redesigned ram organization for database caches
...
- each cache can now allocate as much memory as is available
- no more fixed limits
- replaced old performance memory monitor by new one
- added supervision methods as static functions into the classes that provide cache functionality
- steering of ram allocation is done with two simple limits that are ram availability-relative
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3434 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
bd03c6b874
*) bugfix in bookmarksDB:
...
- NullpointerException when trying to get an unknown bookmark
- bookmarks can either start with http or https
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3427 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
9623bf7bbe
- removed call of java 1.5 method
...
- added config servlet for local robots.txt
- removed YPStats_p as it is of no use anymore
- supertemplates use XHTML now
- quick-fix for http://www.yacy-forum.de/viewtopic.php?p=32296#32296
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3422 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
a1d68fe092
- use .class rather than Class.forName for classes in class-path
...
- added Bost's patch for Diff.findDiagonale() from: http://www.yacy-forum.de//files/patch_685.txt
- fixed minor bugs in Blog
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3416 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
54fef3574f
*) missing files for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3406 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
cb89c74d52
*) added blog-comments
...
*) removed debug-output when deleting news
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3405 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
6fbe31425a
- some code-cleanup (no more syntax-warnings here)
...
- added deletion from loadedURLs of URLs to be blacklisted in IndexControl_p
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3404 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e3480d4ad3
fix for warning in crawl balancer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3402 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
39a2000d8b
- added support for [[Bookmark:$bookmarkTag|description]]-link-listings (requested by theli) to wiki-parser
...
- added support for <pre>-tags to wiki-parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3393 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
619653c054
- fix for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3392 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
a5a36d9252
- hopefully last fix fo 1.5 methods (sorry for that, eclipse isn't that helpful in identifying those methods)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3387 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
e97b6f0458
- we still use Java 1.4 ...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3386 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
0c7b8cf632
- added first version of new wiki-parser
...
- added blacklist support to manual URLFetcher stack fill
- fix for NPE: http://www.yacy-forum.de/viewtopic.php?t=3559
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3385 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
801eea8849
*) Fixed bug where pairReplace() got caught in infinite recursion. ( http://www.yacy-forum.de/viewtopic.php?t=3466 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3383 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
d114a0136e
- crawl profile: don't add null-values
...
- added some settings and statistics for url-fetcher 'server'-mode
- added own stack for fetchable URLs
- added possibility to fill stack via shift from peer's queues, via POST (addurls=$count and url$num=$url) or via file-upload
- added "htroot" to classpath of linux start-script
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3370 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
c464157a6e
replaced some toString()
...
see http://www.yacy-forum.de/viewtopic.php?p=31151#31151
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3345 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
(no author)
e218940293
The copyright sign "\u00A9" is already replaced by "©". String "(C)" is not a unicode sequence!
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3334 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
1bc4d8d470
*) If there is more than one pair of patterns in a line, all of them (and not only one pair) will be replaced.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3333 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
ea7a8cf7aa
*) <hr> and <br> tags are XHTML compliant now.
...
*) Avoid superflous trailing blank in non-proportional sections.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3332 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
f2e6f19b90
- added versioning to Wiki
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3327 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
02a73dce87
- added Diff-class for wiki-versioning (forthcoming, first need suitable serverObjects.put() for it)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3325 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e4910f03d1
tag storage fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3302 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
991182b29b
more space for bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3299 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
88fa764b64
implemented new kelondroObjects into bookmarkDB
...
- Bookmark-Objects are stored inside the kelondroObjects cache
- removed superfluous classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3298 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
9c05e2a820
re-design ob kelondroMap
...
- this class is replaced by an object that can hold any type of object
- this object must be defined as a class that implements kelondroObjectsEntry
- the kelodroMap is now implemented as kelondroMapObjects
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3297 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago